The process of training a smaller, faster AI model to replicate the performance of a larger one, producing a model that is cheaper to run and quicker to respond while retaining most of the capability that matters for a specific task.
Also known as knowledge distillation, model compression
Model distillation is a training technique in which a smaller model (the student) is trained to replicate the behavior of a larger model (the teacher). Rather than training the student model from scratch on raw data, the student learns from the teacher’s outputs: the teacher processes a dataset and generates responses, and the student is trained to match those responses as closely as possible. The result is a smaller, faster, cheaper model that captures much of the teacher’s task-specific capability.
The intuition behind distillation is that large frontier models contain a great deal of general knowledge that is not relevant to any specific task. A model trained to do everything carries the computational cost of that general capability even when it is being used for something narrow. Distillation extracts the capability relevant to the specific task and packages it in a model sized appropriately for that task. The student model runs inference faster, costs less per query, and can often be deployed locally rather than requiring calls to a hosted API.
The quality tradeoff depends on how narrow the target task is. For broad, open-ended tasks that require general reasoning and diverse knowledge, a distilled model will show meaningful degradation relative to the teacher. For narrow, well-defined tasks, a well-executed distillation can produce a student model that matches or exceeds the teacher’s performance on the target task while operating at a fraction of the cost. The practical question for any distillation project is whether the performance degradation on the target task is acceptable given the cost reduction achieved.
Agencies evaluating AI vendors encounter distilled models constantly, often without knowing it. A vendor offering a fast, cheap model for a specific task like ad copy generation, brief summarization, or taxonomy tagging is often selling a distilled version of a larger frontier model fine-tuned on task-specific data. Understanding distillation helps agency teams evaluate those models accurately: the right question is not “is this as capable as the latest frontier model?” but “is it capable enough for this specific task at this cost?”
Custom distillation is increasingly accessible for agencies with well-defined, high-volume repeatable tasks. If an agency processes thousands of creative briefs, tags thousands of assets, or classifies thousands of pieces of content per month using a large frontier model, distilling a smaller task-specific model from that large model’s outputs can reduce inference costs by 80 to 90 percent. The upfront investment is a distillation training run. The ongoing return is a lower cost per inference on a task that runs at volume.
The brand voice application is direct. An agency that uses a large frontier model to generate on-brand copy can distill a smaller, faster model specifically for that task by using the large model’s approved outputs as training data for the student. The distilled model produces similar output quality for the brand voice task at much lower cost and can run at a scale that would be cost-prohibitive using the full frontier model.
The vendor evaluation implication is practical. When a vendor’s model performs well on a benchmark but underperforms on a specific agency task, distillation may explain why: the model was trained on the teacher’s general outputs rather than on examples specific to that task. Task-specific fine-tuning on top of a distilled base is the fix, not a different frontier model.
An agency has been using a large frontier model to classify incoming client creative assets into a 40-category taxonomy for campaign reporting. The classification task is well-defined and consistent, but the frontier model charges per token and the volume of assets is high enough that the monthly API cost has become significant. The agency uses the frontier model to label a dataset of 10,000 historical assets, then trains a distilled model on those labeled examples. The distilled model achieves 94 percent agreement with the frontier model on the taxonomy task, runs at a fraction of the cost, and returns results in under 200 milliseconds per asset compared to 2 to 3 seconds for the frontier model API call. The frontier model stays in use for tasks where broad reasoning is required. The distilled model handles the high-volume classification task at production scale, and the cost per month drops to a level where scaling the program to additional clients is straightforward.
The automations and agents module of the workshop teaches you how to build AI workflows that compress the busywork without taking the craft out of the studio.