The Assistant That Asked Twice

01Five dialogues, ten renders

A customer of one of my products, a voice tool that people can use from inside ChatGPT, wrote in on 6 November with their usage numbers, which looked too high. They were right. They had been turning an audiobook into audio through the assistant, six dialogues with several voices each, and five of the six had been rendered twice and billed twice. Of 119,160 characters charged that day, 55,026 were duplicates. Assistants retry what feels slow, so every tool they call has to be safe to call twice.

Not a double click

The log ruled out the obvious explanations. The copies were not simultaneous, which would suggest a double submit. Each one was a fresh request with exactly the same text that started after the first had already finished, between 16 seconds and almost two hours later. And each original render took between 68 and 120 seconds. The most likely story is simple: the assistant waited, decided the call had failed, and sent it again. From its side the first answer never arrived in time. From ours, it arrived complete, and then the work began again.

02A person waits; a model retries

Patience is a human feature

A person who starts a long render on a website sees a progress bar and waits. An assistant calling a tool has a time budget, and when the budget runs out it does what any careful program does: it tries again, or tries a slightly different request. In a server for AI agents I counted far more connections than tool calls, and assumed most of the noise was harmless. A retry on a cheap call is harmless. A retry on a two minute render that the customer pays for is not. An agent does not know your job is still running unless you tell it, and most tool replies do not.

The customer cannot see it either

This customer never visited our website while it happened. Like most people who use a product through an assistant, the pattern described in the users who never open your website, they only saw the answers in the chat and a usage number later. They could not have spotted the retries. Only the provider can, and only by looking at its own log for identical requests close together. When the customer lives inside an assistant, the provider is the only one who can see the assistant misbehave.

03A repeat guard, and what it costs

How it works

The fix shipped the next day. Every programmatic request now gets a fingerprint made from the account, the endpoint, the plan and the exact request body; the text itself is not stored. If the same fingerprint arrives within three hours and the earlier audio still exists, the product returns that audio again, free, marked as a repeat. If the copy arrives while the first render is still running, it waits for that render instead of starting another. A replay takes about half a second against 3.3 seconds for a fresh render in testing; six test requests produced three charges, exactly the three distinct ones. The duplicated characters were credited back to the customer. A usage number that jumps with no explanation reads like a mistake on the provider's side, so the explanation matters as much as the credit. Idempotency is not an optimisation for agent traffic; it is the billing policy.

Where it does not apply

The website's regenerate button is untouched on purpose. A person who presses it wants a different take, and treating that as a repeat would be wrong. The guard covers only callers that send requests through keys and connectors, where an identical body almost always means a retry. That boundary matters more than the code: decide which of your callers repeat by accident and which repeat on purpose, and guard only the first. The same thinking applies to anything expensive an agent can trigger, from exports to payments. The four OpenAI agents in four OpenAI agents each behave differently, and the one that acts for a user is the one whose retries cost money.

Tomorrow: this blog, four weeks after counting its own impressions.