CMR-11195: As a developer, I want incoming CMR search requests classified and rate limited by compute cost so that heavy requests cannot cascade into backend failures - #2489
Conversation
|
|
||
| ### `GET /health` | ||
|
|
||
| Informational health check. Always returns HTTP 200 — dependencies report their status but do not affect the top-level `ok?`. The result is cached for 5 seconds (unhealthy results are not cached so recovery is visible immediately). |
There was a problem hiding this comment.
It always returns 200? So what's the point of checking the health if it always returned 200?
There was a problem hiding this comment.
So that the ELB health check doesn't bounce the tasks because redis is down. I can take another look and see if this is still needed.
There was a problem hiding this comment.
what was the conclusion to this?
There was a problem hiding this comment.
yea we decided that we shouldn't have the search-proxy task fail and restart if its connection to redis or elastic is bad, or they have what would otherwise be a bad status.
|
|
||
| class TestLoadShedding: | ||
| async def test_sheds_when_full(self, tight_lanes): | ||
| async with tight_lanes.acquire("heavy"): |
There was a problem hiding this comment.
I think every test should have comment of "When... Then..." for easier reading. Although the naming of the python tests are clear enough in this case. So I would categorize this request as 'nice to have', if we can be consistent with tests having comments.
There was a problem hiding this comment.
Yea, I think that is reasonable. Can we move it to a separate ticket?
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2489 +/- ##
==========================================
- Coverage 57.96% 57.96% -0.01%
==========================================
Files 1074 1074
Lines 74707 74707
Branches 2193 2189 -4
==========================================
- Hits 43307 43303 -4
- Misses 29354 29356 +2
- Partials 2046 2048 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
e3d2ad3 to
9fd41f8
Compare
5 participants
Overview
What is the objective?
Introduce a traffic-shaping proxy in front of CMR search that classifies requests into priority lanes (express/standard/heavy), applies distributed rate limiting via Redis semaphores, and caches responses to reduce backend load.
What are the changes?
New search-proxy FastAPI service with three traffic lanes, each with configurable concurrency permits and response cache TTLs
Redis-backed distributed semaphores for per-lane in-flight request counting across instances
Response cache keyed on method, path, query, auth token, Accept header, cmr-search-after cursor, and POST body hash
Request classifier routing traffic to lanes based on spatial complexity, facets, temporal params, and other signals
Feature toggles for bypass, caching, load shedding, and classification via ECS environment variables
AWS infrastructure: ECS task definition, ECR repository, ALB listener rules (priority 83 bypass rule to prevent routing loop, priority 84 proxy rule)
Health endpoint exposing Redis, backend, and per-lane utilization status
What areas of the application does this impact?
List impacted applications
All CMR search traffic (collections and granules endpoints) in environments where enable_search_proxy = true
Internal ALB routing rules
ElastiCache Redis (new usage for semaphores and cache)
Required Checklist
Additional Checklist