Follow-up from #1304, split out rather than widening that PR.
Problem
_read_video_analysis_sync in src/youtube_extension/backend/real_api_endpoints.py is dispatched via asyncio.to_thread. That successfully removes filesystem latency from the event loop, but not the parse:
open()/read() release the GIL → genuinely off-loopjson.load() is CPU-bound C code that holds the GIL → still stalls the loop for its full duration
Measured on CPython 3.12 / macOS arm64 by counting loop ticks during the work:
| Work |
Loop ticks |
to_thread(time.sleep, 0.30) |
22,847 |
to_thread(open + json.load) |
179 |
to_thread(json.loads) alone |
3 |
Three ticks across the whole parse — the stall is effectively the entire wall time. Scales linearly at roughly 3 ms/MB: 1.58 MB → 4.61 ms, 8.02 MB → 23.07 ms, 24.42 MB → 68.02 ms.
Why it matters
Payload size is unbounded and grows with video duration. services/real_video_processor.py:294 persists the full transcript object into the cache entry, so entry size tracks video length — roughly 0.15 MB for an hour, ~1.6 MB long-form. Nothing caps it. A long enough video produces a proportionally long loop stall affecting every concurrent request.
There is also a regression case already disclosed in #1304: on a warm page cache with a small (0.10 MB) payload, the executor dispatch hop (~0.5 ms) costs more than the 0.4 ms parse it defers — 0.40 ms → 0.94 ms. The offload is a net negative in exactly the case where the filesystem is fast.
Options
- Cap entry size — cheapest. Reject or truncate cache entries above a threshold; bounds the stall by construction. Needs a decision on what to do with oversized entries.
- Stop storing the full transcript — attacks the cause rather than the symptom. Store a reference or a trimmed projection. Largest win, largest blast radius.
- Streaming/incremental parse — yields between chunks so the loop can interleave. Adds a dependency or hand-rolled parsing.
- Process pool — genuinely parallel, sidesteps the GIL, but adds IPC serialisation cost that may exceed the parse for small payloads.
- Size-conditional dispatch — parse inline below a threshold, offload above it. Fixes the warm-cache regression directly; needs the threshold measured, not guessed.
Options 1 and 5 are complementary and both cheap. Option 2 is the real fix.
Acceptance
Not in scope
Anything touching real_video_processor.py while PR #1237 holds it.
Follow-up from #1304, split out rather than widening that PR.
Problem
_read_video_analysis_syncinsrc/youtube_extension/backend/real_api_endpoints.pyis dispatched viaasyncio.to_thread. That successfully removes filesystem latency from the event loop, but not the parse:open()/read()release the GIL → genuinely off-loopjson.load()is CPU-bound C code that holds the GIL → still stalls the loop for its full durationMeasured on CPython 3.12 / macOS arm64 by counting loop ticks during the work:
to_thread(time.sleep, 0.30)to_thread(open + json.load)to_thread(json.loads)aloneThree ticks across the whole parse — the stall is effectively the entire wall time. Scales linearly at roughly 3 ms/MB: 1.58 MB → 4.61 ms, 8.02 MB → 23.07 ms, 24.42 MB → 68.02 ms.
Why it matters
Payload size is unbounded and grows with video duration.
services/real_video_processor.py:294persists the full transcript object into the cache entry, so entry size tracks video length — roughly 0.15 MB for an hour, ~1.6 MB long-form. Nothing caps it. A long enough video produces a proportionally long loop stall affecting every concurrent request.There is also a regression case already disclosed in #1304: on a warm page cache with a small (0.10 MB) payload, the executor dispatch hop (~0.5 ms) costs more than the 0.4 ms parse it defers — 0.40 ms → 0.94 ms. The offload is a net negative in exactly the case where the filesystem is fast.
Options
Options 1 and 5 are complementary and both cheap. Option 2 is the real fix.
Acceptance
test_parse_still_stalls_the_loop_in_proportion_to_payload) updated — its failure message already flags that the documented guarantee must change if the parse becomes incremental_read_video_analysis_syncwidened to match whatever is actually deliveredNot in scope
Anything touching
real_video_processor.pywhile PR #1237 holds it.