Skip to content

perf: read single-video cache entry off the event loop #1303

Description

@groupthinking

Problem

GET /api/v2/videos/{video_id} performs its entire cache lookup inline on the
event loop:

cache_path = processor._get_cache_path(video_id)

if cache_path.exists():
    with open(cache_path, 'r', encoding='utf-8') as f:
        video_data = json.load(f)
    return video_data

raise HTTPException(status_code=404, ...)

Three blocking operations run before the coroutine can yield:

  1. Path.exists() — a stat syscall
  2. open() — a second syscall on the same path
  3. json.load() — a full parse of the stored analysis payload

Only the first two are bounded. The parse cost scales with the size of the
stored analysis, which is written by the processor and is not capped by this
endpoint. While it runs, every other in-flight request on the same worker is
stalled
, including health checks and requests that touch no filesystem at all.

This is the same defect class already fixed for the sibling list endpoint in
#1287 / #1288 (/api/v2/videos/list), which globbed the cache directory and
parsed every entry on the loop. That fix left the single-video read behind.

Secondary issue: check-then-open race

exists() followed by open() is a time-of-check/time-of-use window. If the
entry is removed between the two calls, open() raises FileNotFoundError,
which the handler's broad except Exception converts into a 500 — even
though the correct answer is plainly 404. Two syscalls are being spent to
produce a worse answer than one.

Proposed fix

Follow the idiom this repository already uses for exactly this shape:

  • add a module-level _read_video_analysis_sync(cache_path) helper that opens
    directly and treats FileNotFoundError as the miss, returning None
  • have the handler await asyncio.to_thread(...) it, and raise the 404 on
    None
  • leave _get_cache_path() on the loop — it is pure string arithmetic and
    touches no filesystem

RealVideoProcessor._read_cache_file already uses the
open() / except FileNotFoundError: return None / with handle as f:
sequence, and its own docstring notes that keeping the sequence in a single
call avoids a stat/read race across separate thread hops.

Explicitly not reusing _read_cache_file

_read_cache_file applies a 24-hour TTL (_CACHE_TTL_SECONDS = 86400) and
returns None for anything older. This endpoint has never had a TTL — it
serves a cached analysis regardless of age. Reusing that helper here would
silently turn every analysis older than a day into a 404. That is a behaviour
regression wearing the costume of a refactor, so the new helper deliberately
omits the age check.

Acceptance criteria

  • the read runs on a worker thread, not the loop thread
  • the loop stays responsive for the duration of a slow read
  • missing entry still returns 404; present entry still returns 200 with an
    identical payload
  • corrupt JSON and unreadable paths still surface as 500, not as a
    false 404
  • entries older than 24 hours still return 200
  • src/youtube_extension/backend/services/real_video_processor.py is not
    modified (it is claimed by open PR perf: offload cache-directory scan off the event loop (#1231) #1237)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions