video: qwen38-caption.py — whole-video captioning + timestamped timelines (PoC)

[davanstrien]

What

video/qwen38-caption.py: caption whole videos with no chunking using Qwen/Qwen3.8-27B (27B dense VLM, Apache 2.0, ungated) served on vLLM in-job, driven by saturate (crash-safe parquet, exact resume) — same shape as marlin-caption.py, minus the chunking machinery Marlin needs.

• --timestamps: per-video <start - end> event timeline in global seconds. Times are read off per-frame time tags, not estimated — verified to ~2s against extracted frames on an 11-min 1935 film (96 events, on-screen text quoted verbatim, title-card text word-for-word).
• Thinking off by default (chat_template_kwargs); --thinking re-enables with model-card sampling params + larger budget.
• README: new script section + honest Marlin-vs-Qwen3.8 guidance (bulk/cheap → Marlin; timelines/text/long-form → Qwen3.8).

Status: PoC — shared before merge

Being shared as work-in-progress (branch raw URL runs directly via hf jobs uv run). Merge once we're confident; no Hub sync until then.

Evidence

• Caption mode Jobs-tested on a100-large (vllm/vllm-openai:latest image): 3/3 videos incl. the full 11-min film in one request, 0 failed, 947.7 tok/s aggregate (job 6a7f586f).
• --timestamps mode: verification re-run in flight (first run's requests all succeeded but the output flush hit a concurrent-bucket-mount collision from the test setup, not the script — job 6a7f5871); result will be posted here.
• Timestamp accuracy evidence from the probe runs: 8/8 then 8/10-exact frame checks across the full film (jobs 6a7f4d0c, 6a7f5185).

Known limits

• Both engines cap video sampling at ~13K prompt tokens regardless of --fps (Qwen3-VL processor frame budget) — fine to ±2s; raising the budget for tighter accuracy is future work.
• bf16 weights ~52 GB → a100-large minimum.

🤖 Generated with Claude Code

https://claude.ai/code/session_012xKNwUAGVdWrLgjLacx25k

[davanstrien]

--timestamps verification complete (job 6a7f5e11, after 655e060's thinking-default + parser fix):

video	duration	events	coverage
raw_AboutBan1935.mp4	664s	92	0.0–664.0s (full)
JoanAvoi1947.mp4	60s	13	0.0–60.5s
Sleepfor1950.mp4	60s	15	0.0–60.5s

3/3 ok, 0 failed, 128.8 tok/s aggregate. For contrast, the pre-fix non-thinking run (job 6a7f5b21) answered the same film's timeline request with a 27-token summary — the measured basis for --timestamps defaulting to thinking mode.

Both smoke modes now pass: caption 3/3 (947.7 tok/s, job 6a7f586f) and timestamps 3/3. Ready for review/merge whenever confidence feels right; sync to uv-scripts/video fires on merge.