The provider charges for processing tokens, not for getting it right. If a request fails halfway, if the timeout fires while the answer was already being generated, or if your code discards the output because it does not match the format, that work is billed all the same and has to be done again.
It is the part of the bill that shows up on no dashboard, because dashboards count requests, not useful requests.
At a glance
| What happens | Failed work is billed like good work |
| Sources | Retries, timeouts, failed validation, duplicate answers |
| How to see it | Compare billed requests against completed tasks |
| The lever | Fail earlier, retry smarter, validate cheaper |
Where the repeats come from
Automatic retries are the first suspect. Many libraries retry by default on a network error or a rate limit, and they do it three times without saying so. If the failure was on the model side and generation had already started, you paid for three answers to use one.
Badly set timeouts are the second. A timeout shorter than the long answer the model sometimes produces cuts the connection but not the billing: the provider has already done the work.
The third one is yours: post-hoc validation. If you ask for a format and your code rejects the output when it does not fit, every rejection is a request paid for and thrown away.
How to see it in your numbers
Compare two figures from the same period: requests billed by the provider and tasks completed by your system. The gap is the waste. If you do not have the second figure, that is the first thing to instrument, and it goes hand in hand with monitoring AI spend.
In agents the effect multiplies, because a failed step drags the whole accumulated history with it. It is one of the reasons an agent costs more than it looks.
What to change
Retry only what is worth retrying. A network error, yes. An answer the model already generated and you do not like, no: that is not fixed by repeating, it is fixed by changing the prompt.
Set the timeout against the long answer, not the average one. And if you really need to cut, cap the output tokens instead of cutting the connection.
Validate cheaply. If the format fails often, use the provider’s structured output instead of asking for it in the prompt and checking afterwards.
Log the reason for every repeat. Without that label the waste is invisible and nobody fixes it.
Frequently asked questions
Am I billed for a request that timed out? Generally yes, if the model already started processing. The cut is on your side, not theirs.
How much does this usually weigh? It varies a lot, but on uninstrumented systems it is common to find a share of the spend that buys nothing but discarded work.
Is this a cost problem or a quality problem? Both. Every repeat is money and also latency for the user who is waiting.
At e-ficient we measure the cost per task of companies that already run AI in production and tell you how much you would save with each lever at your real volume. The first audit is free if you start a plan with a three-month commitment and returns a diagnosis within 72 hours.