How Distillation Makes AI Models Smaller and Cheaper - Quanta Magazine

Fundamental technique lets researchers use a big, expensive “teacher” model to train a “student” model for less.