Skip to content

fix: raise RAG proxy timeout to cover /v1/studies cold-cache path - #31

Merged
man4ish merged 1 commit into
mainfrom
fix/rag-proxy-timeout
Aug 9, 2026
Merged

fix: raise RAG proxy timeout to cover /v1/studies cold-cache path#31
man4ish merged 1 commit into
mainfrom
fix/rag-proxy-timeout

Conversation

@man4ish

@man4ish man4ish commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

The 10s httpx timeout in routes_rag_proxy.py was failing /rag/studies outright with 503 ReadTimeout on every request — the Admin Console's RAG/PubMed tab could never load.

Root cause was upstream, not here: omnibioai-rag's GET /v1/studies rescanned ~75.4M abstract files across 213 study directories from disk on every call, taking ~11s — it was never able to answer within this timeout at all. That presented as a proxy misconfiguration, but the proxy was doing the right thing against a genuinely too-slow upstream.

That's fixed properly on the RAG side (companion PR — the listing is now Redis-cached with a 1h TTL and its scan runs off the event loop), which makes the slow path rare rather than universal: a warm call returns in ~18ms.

But a cache miss (first call, or once per TTL window) still takes RAG's full ~11s, and 10s would keep failing exactly those calls a second short of the answer. 20s gives that cold path margin while still bounding the request — deliberately a modest bump to cover a known ~11s worst case, not to mask an unresponsive upstream.

Verification

Before After
Cold 503 ReadTimeout 200 in 11.06s
Warm 503 ReadTimeout 200 in 18ms

🤖 Generated with Claude Code

The 10s httpx timeout here was failing /rag/studies outright with a
503 ReadTimeout on every single request -- the Admin Console's
RAG/PubMed tab could never load. Root cause was upstream, not here:
omnibioai-rag's GET /v1/studies rescanned ~75.4M abstract files across
213 study directories from disk on every call, taking ~11s, i.e. it
was never able to answer within this timeout at all.

That's fixed properly on the RAG side (companion PR: the listing is now
Redis-cached with a 1h TTL, and its scan runs off the event loop), which
makes the slow path rare rather than universal -- a warm call now
returns in ~18ms. But a cache miss (first call, or once per TTL window)
still takes RAG's full ~11s, and 10s would keep failing exactly those
calls a second short of the answer.

20s gives that cold path real margin while still bounding the request
-- deliberately not a large increase, since the point is to cover a
known ~11s worst case, not to mask an unresponsive upstream.

Verified live end-to-end: cold call 200 in 11.06s (was 503), warm call
200 in 18ms.
@man4ish
man4ish merged commit 0e7f43f into main Aug 9, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant