The same scene shows up in almost every audit: the team knows a good share of its traffic would work on a cheaper model, and does not switch because it has no way to prove quality holds. The saving is identified and blocked by the lack of a measurement.
The answer is not an academic benchmark. It is a small evaluation, built on your own data, that you can repeat in minutes.
At a glance
| Problem | Nobody drops to a cheaper model without proof it does not get worse |
| Answer | A minimal evaluation on your own cases |
| Size | 50 to 200 examples per task |
| Effect | Turns a leap of faith into a decision with a number |
What a minimal evaluation is
A set of real cases from your operation, with the answer you consider correct, and an automatic way to compare the model’s answer against that reference. Nothing more. With 50 to 200 examples per task you can already tell a model that works from one that does not.
The reference answer does not have to be perfect. It is enough that it is what your system produces in production today and that a human has validated it. You are measuring whether the change makes things worse, not whether it reaches perfection.
How each task gets scored
It depends on the type. If the task is field extraction or classification, the comparison is exact and the result is a hit rate. If the task is writing or summarizing, you need a rubric: three or four explicit criteria (faithfulness to the document, tone, length, format) scored one by one.
What matters is setting the threshold beforehand. “I accept the change if accuracy does not drop more than two points” is a decision you make with a cool head, before seeing the results.
The experiment worth running
Take one task, not all of them. The most repeated one, which is usually also the simplest. Run your 100 examples through the current model and through two or three cheaper candidates. Compare accuracy and cost per task.
Almost always the same thing happens: the most repeated task did not need the large model, and the one that does need it is a minority of the volume. From there, routing per task stops being a bet, as in picking the model per task.
What the evaluation gives you beyond the saving
An evaluation you can repeat is worth far more than a one-off switch. It earns its keep when the provider updates the model underneath you, when someone edits the prompt, when a new model ships. Without it, each of those events is a leap in the dark; with it, it is a twenty-minute check. And it fits alongside monitoring AI spend: quality and cost measured in the same table.
Frequently asked questions
How many examples do I need? Between 50 and 200 per task is enough to decide. More examples sharpen the number but rarely change the conclusion.
Can I use one model to score another? Yes, that is standard for writing tasks, as long as the rubric is explicit and you have checked by hand that the judge agrees with you on a sample.
What if quality really does drop? Then you have saved yourself the unpleasant surprise in production. Try a mid-tier model or route only the easy part of the task.
At e-ficient we measure the cost per task of companies that already run AI in production and tell you how much you would save with each lever at your real volume. The first audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.