fix: raise RAG proxy timeout to cover /v1/studies cold-cache path - #31
Merged
Conversation
The 10s httpx timeout here was failing /rag/studies outright with a 503 ReadTimeout on every single request -- the Admin Console's RAG/PubMed tab could never load. Root cause was upstream, not here: omnibioai-rag's GET /v1/studies rescanned ~75.4M abstract files across 213 study directories from disk on every call, taking ~11s, i.e. it was never able to answer within this timeout at all. That's fixed properly on the RAG side (companion PR: the listing is now Redis-cached with a 1h TTL, and its scan runs off the event loop), which makes the slow path rare rather than universal -- a warm call now returns in ~18ms. But a cache miss (first call, or once per TTL window) still takes RAG's full ~11s, and 10s would keep failing exactly those calls a second short of the answer. 20s gives that cold path real margin while still bounding the request -- deliberately not a large increase, since the point is to cover a known ~11s worst case, not to mask an unresponsive upstream. Verified live end-to-end: cold call 200 in 11.06s (was 503), warm call 200 in 18ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The 10s httpx timeout in
routes_rag_proxy.pywas failing/rag/studiesoutright with503 ReadTimeouton every request — the Admin Console's RAG/PubMed tab could never load.Root cause was upstream, not here:
omnibioai-rag'sGET /v1/studiesrescanned ~75.4M abstract files across 213 study directories from disk on every call, taking ~11s — it was never able to answer within this timeout at all. That presented as a proxy misconfiguration, but the proxy was doing the right thing against a genuinely too-slow upstream.That's fixed properly on the RAG side (companion PR — the listing is now Redis-cached with a 1h TTL and its scan runs off the event loop), which makes the slow path rare rather than universal: a warm call returns in ~18ms.
But a cache miss (first call, or once per TTL window) still takes RAG's full ~11s, and 10s would keep failing exactly those calls a second short of the answer. 20s gives that cold path margin while still bounding the request — deliberately a modest bump to cover a known ~11s worst case, not to mask an unresponsive upstream.
Verification
503 ReadTimeout200in 11.06s503 ReadTimeout200in 18ms🤖 Generated with Claude Code