Problem
GET /api/v2/videos/{video_id} performs its entire cache lookup inline on the
event loop:
cache_path = processor._get_cache_path(video_id)
if cache_path.exists():
with open(cache_path, 'r', encoding='utf-8') as f:
video_data = json.load(f)
return video_data
raise HTTPException(status_code=404, ...)
Three blocking operations run before the coroutine can yield:
Path.exists() — a stat syscall
open() — a second syscall on the same path
json.load() — a full parse of the stored analysis payload
Only the first two are bounded. The parse cost scales with the size of the
stored analysis, which is written by the processor and is not capped by this
endpoint. While it runs, every other in-flight request on the same worker is
stalled, including health checks and requests that touch no filesystem at all.
This is the same defect class already fixed for the sibling list endpoint in
#1287 / #1288 (/api/v2/videos/list), which globbed the cache directory and
parsed every entry on the loop. That fix left the single-video read behind.
Secondary issue: check-then-open race
exists() followed by open() is a time-of-check/time-of-use window. If the
entry is removed between the two calls, open() raises FileNotFoundError,
which the handler's broad except Exception converts into a 500 — even
though the correct answer is plainly 404. Two syscalls are being spent to
produce a worse answer than one.
Proposed fix
Follow the idiom this repository already uses for exactly this shape:
- add a module-level
_read_video_analysis_sync(cache_path) helper that opens
directly and treats FileNotFoundError as the miss, returning None
- have the handler
await asyncio.to_thread(...) it, and raise the 404 on
None
- leave
_get_cache_path() on the loop — it is pure string arithmetic and
touches no filesystem
RealVideoProcessor._read_cache_file already uses the
open() / except FileNotFoundError: return None / with handle as f:
sequence, and its own docstring notes that keeping the sequence in a single
call avoids a stat/read race across separate thread hops.
Explicitly not reusing _read_cache_file
_read_cache_file applies a 24-hour TTL (_CACHE_TTL_SECONDS = 86400) and
returns None for anything older. This endpoint has never had a TTL — it
serves a cached analysis regardless of age. Reusing that helper here would
silently turn every analysis older than a day into a 404. That is a behaviour
regression wearing the costume of a refactor, so the new helper deliberately
omits the age check.
Acceptance criteria
Problem
GET /api/v2/videos/{video_id}performs its entire cache lookup inline on theevent loop:
Three blocking operations run before the coroutine can yield:
Path.exists()— astatsyscallopen()— a second syscall on the same pathjson.load()— a full parse of the stored analysis payloadOnly the first two are bounded. The parse cost scales with the size of the
stored analysis, which is written by the processor and is not capped by this
endpoint. While it runs, every other in-flight request on the same worker is
stalled, including health checks and requests that touch no filesystem at all.
This is the same defect class already fixed for the sibling list endpoint in
#1287 / #1288 (
/api/v2/videos/list), which globbed the cache directory andparsed every entry on the loop. That fix left the single-video read behind.
Secondary issue: check-then-open race
exists()followed byopen()is a time-of-check/time-of-use window. If theentry is removed between the two calls,
open()raisesFileNotFoundError,which the handler's broad
except Exceptionconverts into a 500 — eventhough the correct answer is plainly 404. Two syscalls are being spent to
produce a worse answer than one.
Proposed fix
Follow the idiom this repository already uses for exactly this shape:
_read_video_analysis_sync(cache_path)helper that opensdirectly and treats
FileNotFoundErroras the miss, returningNoneawait asyncio.to_thread(...)it, and raise the 404 onNone_get_cache_path()on the loop — it is pure string arithmetic andtouches no filesystem
RealVideoProcessor._read_cache_filealready uses theopen()/except FileNotFoundError: return None/with handle as f:sequence, and its own docstring notes that keeping the sequence in a single
call avoids a stat/read race across separate thread hops.
Explicitly not reusing
_read_cache_file_read_cache_fileapplies a 24-hour TTL (_CACHE_TTL_SECONDS = 86400) andreturns
Nonefor anything older. This endpoint has never had a TTL — itserves a cached analysis regardless of age. Reusing that helper here would
silently turn every analysis older than a day into a 404. That is a behaviour
regression wearing the costume of a refactor, so the new helper deliberately
omits the age check.
Acceptance criteria
identical payload
false 404
src/youtube_extension/backend/services/real_video_processor.pyis notmodified (it is claimed by open PR perf: offload cache-directory scan off the event loop (#1231) #1237)