feat(routing): session-affinity max-requests with per-model/provider rules

[dangxia]

Summary

Add session-affinity max-requests so sticky session→auth bindings can rebind after a request cap, with optional per-provider / per-model overrides.

Motivation

Session affinity keeps a client on the same credential for continuity, but long-lived sessions can pin traffic to one account indefinitely (until TTL expires or the auth becomes unavailable). Operators need a way to:

• Soft-rotate credentials after N sticky picks
• Apply different caps for specific models/providers (e.g. expensive models vs cheap ones)

without turning off session affinity entirely.

Changes

Config

• routing.session-affinity-max-requests — global default (<=0 / omit / -1 = unlimited)
• routing.session-affinity-rules[] — optional overrides:
  • provider (optional)
  • model (optional; thinking-suffix stripped for matching)
  • max-requests

Match specificity (most specific wins):
provider+model > model-only > provider-only > global default

Equal-specificity rules keep declaration order (first match wins at equal score).

Runtime

• Session cache tracks hit count per binding (shared across session aliases)
• On sticky cache hit, if hits > max-requests, invalidate the binding and re-pick via the fallback selector, then rebind
• Rebind also still occurs when the bound auth is unavailable (existing behavior)
• Hot-reload: routing state equality includes max-requests + compiled rules key so config changes rebuild the selector

Docs / example

• config.example.yaml documents the new knobs and the fill-first + max-requests interaction

Behavior notes / caveats

Rebind uses the same routing.strategy selector.
With fill-first, rebind still returns the first available credential, so max-requests rarely rotates accounts unless the current one is unavailable (cooldown / quota / disabled).
Prefer round-robin or weighted-round-robin when using max-requests for intentional rotation.

Only effective when routing.session-affinity: true.

Example

routing:
  strategy: round-robin
  session-affinity: true
  session-affinity-ttl: "1h"
  session-affinity-max-requests: -1   # unlimited global default
  session-affinity-rules:
    - model: "grok-4.5"
      provider: "xai"       # optional
      max-requests: 20

Test plan

• Unit: max-requests rebinds under round-robin after N sticky picks
• Unit: fill-first does not rotate while first auth remains available (documented intentional)
• Unit: per-model / per-provider rule resolution specificity
• Unit: rules key stable across thinking-suffix / case normalization; order matters at equal specificity
• Manual: enable session-affinity + max-requests with RR/WRR, confirm log lines for rebind (max-requests reached) and hit counters on cache hits
• Manual: hot-reload change of max-requests / rules rebuilds routing without restart

Files

• internal/config/config_types.go — config types
• config.example.yaml — example + caveats
• sdk/cliproxy/auth/session_cache.go — hit counting + alias-group invalidate
• sdk/cliproxy/auth/session_affinity_rules.go — compile / resolve
• sdk/cliproxy/auth/selector.go — max-requests rebind path
• sdk/cliproxy/service_config.go — wire config → selector + hot-reload equality
• tests for rules, max-requests, and routing state

[github-actions]

This pull request targeted main.

The base branch has been automatically changed to dev.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6065dc79fb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[dangxia]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5a3268d8a4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Resolve affinity rules against the selected provider

When session affinity is used on a multi-provider route, pickNextMixedLegacy invokes this selector with provider == "mixed", so resolveSessionAffinityMaxRequests can only match model-only/global rules and never matches a rule configured for the actual execution provider such as provider: "xai". In that setup, a provider-specific session-affinity-rules max-requests override is silently ignored for mixed routing, leaving those sessions sticky for the global limit instead of the configured provider limit.

Useful? React with 👍 / 👎.

[dangxia]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 2293abb868

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".