fix(auth): cap upstream 429 Retry-After so quota resets are picked up - #4725
Open
artile wants to merge 1 commit into
Open
fix(auth): cap upstream 429 Retry-After so quota resets are picked up#4725artile wants to merge 1 commit into
artile wants to merge 1 commit into
Conversation
Upstreams occasionally return multi-day reset hints (e.g. a weekly ChatGPT quota). Honoring them verbatim parks every credential in cooldown for days, and nothing re-probes before the window expires, so the proxy keeps 429ing until a service restart clears the in-memory cooldown. Cap the honored Retry-After at quotaBackoffMax (30 min), the same ceiling the proxy already uses for its own quota backoff, in both 429 cooldown paths (per-model MarkResult and applyAuthFailureState). Hints at or below the cap are honored unchanged. Fixes the scenario in router-for-me#4724.
|
This pull request targeted The base branch has been automatically changed to |
|
Clean fix — no conflict with the antigravity recovery engine (commit d5b5787). The recovery engine's |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4724.
Problem
A single upstream 429
Retry-Aftercan be huge — ChatGPT returned561635(~156h, a weekly quota window) on our instance. Both 429 cooldown paths honored that hint verbatim:MarkResultpath (conductor_cooldown.go)applyAuthFailureState(conductor_cooldown.go)That parked every credential for days, and nothing re-probes a credential while its cooldown window is open. So when the account was reset upstream (manual reset-credit, or the window rolling over), the proxy kept returning
429 model_cooldownuntil asystemctl restart cliproxyapicleared the in-memory cooldown.The proxy already caps its own quota backoff at
quotaBackoffMax = 30 * time.Minute(conductor_refresh.go), so re-probing at that cadence is exactly what a healthy setup already does.Fix
Cap the honored upstream
Retry-AfteratquotaBackoffMax(30 min) in both 429 cooldown paths. Hints at or below the cap are honored unchanged; the credential now re-probes within half an hour, so an upstream reset is picked up without a restart.Notes
POST /v0/management/reset-quota(v7.2.100+) remains the immediate operational escape hatch for a manual reset.Retry-Aftervalues still work through the retry/wait path (conductor_selection.go), which is already bounded bymaxWait.Tests
TestCappedUpstreamRetryAfter— helper edges (zero, negative, below-cap, at-cap, above-cap).TestApplyAuthFailureStateCapsUpstreamRetryAfter— 7-day hint → capped at 30 min, and auto-recovers after the capped deadline.TestMarkResultCapsUpstreamRetryAfter— per-model path: 7-day hint → capped at 30 min, auto-recovers.go build ./...andgo test ./sdk/cliproxy/auth/...pass.