Skip to content

feat: replace custom rate limiter with governor (GCRA, lock-free) - #110

Merged
nfvelten merged 1 commit into
masterfrom
feat/governor-rate-limiter
Apr 7, 2026
Merged

feat: replace custom rate limiter with governor (GCRA, lock-free)#110
nfvelten merged 1 commit into
masterfrom
feat/governor-rate-limiter

Conversation

@nfvelten

@nfvelten nfvelten commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replaces Mutex<HashMap<String, Vec<Instant>>> sliding-window rate limiter with the governor crate (GCRA algorithm)
  • Enforcement is now lock-free (atomics) and O(1) per check — no Vec::retain on every request, no background cleanup task
  • Fixes the root cause of the TOCTOU race tracked in test: large payload and deeply nested JSON rejection #82
  • Adds optional rate_limit_burst field on AgentPolicy — defaults to rate_limit for full backward compatibility

Config (new optional field)

agents:
  cursor:
    rate_limit: 30         # requests/min (existing)
    rate_limit_burst: 5    # burst allowance (new, defaults to rate_limit)

Design

  • Per-key LimiterEntry (one per agent / tool / IP) stored in RwLock<HashMap> — read lock to clone the Arc, write lock only when inserting a new key; the governor check itself never takes a lock
  • AtomicI64 remaining counter + AtomicU64 window-start give O(1) X-RateLimit-Remaining headers via CAS reset every 60s
  • Config hot-reload detection: if a stored entry's limit differs from the current policy, it is atomically replaced

Test plan

  • cargo fmt --check passes
  • cargo clippy -- -D warnings passes
  • cargo test --lib — 429 tests pass (includes new burst_config_respected test)
  • All 11 existing rate-limit tests pass unchanged

Closes #98

🤖 Generated with Claude Code

…oses #98)

Replace the Mutex<HashMap<String, Vec<Instant>>> sliding-window implementation
with the governor crate (GCRA algorithm):

- Lock-free enforcement via atomics — eliminates the TOCTOU race from #82
- O(1) per check — no Vec::retain, no background cleanup task
- Configurable burst allowance via new optional `rate_limit_burst` field on
  AgentPolicy (defaults to rate_limit for full backward compatibility)
- Per-key limiters stored in RwLock<HashMap> — read lock for lookup (clones Arc),
  write lock only when inserting a new key; governor check itself is lock-free
- Separate AtomicI64 + AtomicU64 window counter gives O(1) remaining-count for
  X-RateLimit-Remaining headers without any mutex
- All 11 existing rate-limit unit tests pass; adds burst_config_respected test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nfvelten
nfvelten merged commit 2cbd178 into master Apr 7, 2026
3 checks passed
@nfvelten
nfvelten deleted the feat/governor-rate-limiter branch April 7, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: replace custom rate limiter with governor (GCRA, lock-free, burst support)

1 participant