perf: stop the search index from freezing the event loop

[NovakPAai]

Summary

buildSearchIndex re-parsed every session with detail — a findSessionFile lookup plus a full detail load (sync fs read + JSON.parse per line) each — in one synchronous tick, on whichever request happened to miss the 60s cache. Measured on a 900-session history: a 3.6s hard stall of the event loop, which also stalls every other API call and the terminal WebSocket data pump. The analytics job already solved this exact problem with chunk+yield (_scheduleAnalyticsRecompute); search never got the same treatment.

Chunking per session alone wasn't enough. Real histories have a heavy tail — median session ~0.1MB on my machine, but codex transcripts up to 78MB — and readLines slurps the whole file into a string before splitting, so one such session blocks for seconds regardless of the outer chunk size.

• buildSearchIndex is async, processes sessions in small chunks (8 — each item is far heavier than the computeSessionCost calls analytics batches 80-at-a-time) and yields via setImmediate between them.
• JSONL sessions over SEARCH_STREAM_THRESHOLD (4MB) are read line-by-line off a stream that yields every 2000 lines, so one huge transcript can't block either. Nothing is truncated — this only changes when the work happens, not what gets indexed.
• getSearchIndex is stale-while-revalidate (mirroring getCostAnalytics): a >60s-old index is still overwhelmingly accurate for search, so it's served instantly while the refresh runs in the background. Only a genuine cold start awaits.
• Concurrent rebuilds dedupe into one in-flight job, so a burst of searches during a rebuild no longer queues N full-history scans.
• Cleanup in the same function while I was in it: six near-identical per-format if/else branches collapsed into a SEARCH_DETAIL_LOADERS lookup table (they differed only by loader name, so a fix to one silently missed the other five); the two JSONL readers now share one per-line parser so they can't drift; the repeated 500 snippet cap is a named constant.
• searchFullText is now async; /api/search and the codbash search CLI command updated accordingly.

Measured before/after (same 900-session history)

	before	after
worst event-loop stall	3622ms	60ms
cold index build	3.6s	3.0s
warm search	~14ms	~14ms

Test plan

• node --test "test/**/*.test.js" — 268 passed, 1 skipped (win32-only); 11 new contract tests in test/search-index-chunking.test.js
• Results verified byte-identical to origin/main across 6 queries / 496 result rows, via a differential run against a git worktree of main — the refactor and the streaming path change nothing about what search returns
• Event-loop stall measured with a 20ms heartbeat probe on both versions (numbers above)
• Live server: cold /api/search 5.8s → warm 13ms; /api/version stayed at ~0.16s throughout the cold build instead of hanging
• codbash search <query> CLI verified for both hit and no-results paths (it's top-level CJS, so the now-async call runs in an IIFE with an error path)

🤖 Generated with Claude Code