Survive a rate limit instead of losing the task to it

[tarekziade]

Why

One 429 ended the deepseek_vl task on 2026-08-18 after it had already spent 1.13M input tokens (transformers#48050):

LLM endpoint returned 429 Too Many Requests: {"error":"Rate limit exceeded"}

Two causes, both fixed here.

A 429 shared the error budget. attempts = 3 covered 5xx, dropped connections and rate limits, with 2s/4s backoff — about six seconds of patience before throwing away half an hour of finished work. But they are not alike: a server error may never clear, while a rate limit clears by waiting. Rate limits now get their own, larger budget; a persistent 5xx still fails fast at 3.

Retrying the rejected call isn't enough. The loop that tripped the limit is about to issue the same burst of tool turns, so the next call gets refused too. The client now paces itself.

The numbers come from the server, not from me

• the retry waits what Retry-After says; failing that, what the x-ratelimit-reset family says; exponential backoff is only the last resort for an endpoint that says nothing;
• that same value sets the spacing for the calls that follow;
• on a successful response carrying x-ratelimit-remaining-requests, the spacing spreads the remaining allowance over the window it resets in — so the loop stays under the limit instead of discovering it by being refused. Only headroom the server reports as tight (<10 requests left) slows anything down.

Reset values are parsed in the forms providers actually send: plain seconds, 6m0s / 300ms durations, and absolute epochs (a deadline, not a 1.7-billion-second wait).

Guard rails

• Spacing decays on success, so a loop that hit one transient 429 returns to full speed rather than limping forever.
• An endpoint that never rate-limits us is never paced — asserted by a test.
• Every wait stays bounded by the existing per-wait ceiling: a server is allowed to be wrong or hostile, and no single wait should pin a worker.
• Pacing is per-client, i.e. per task pod. Cross-pod coordination would need shared state; this fixes the burst within a loop, which is the shape that failed.

675 tests pass (11 new), make format clean.

🤖 Generated with Claude Code