OBS-004: alert when a universe silently loses mapped symbols

[DoRmAmMu1997]

Found during the September 2026 tech-debt audit (see the register section in #118).

Problem

A symbol that Dhan's instrument master no longer contains is marked missing_security_id (backend/universe_builder.py:561) and then silently filtered out of every scan by mapped_only(). Nothing outside the interactive Streamlit sidebar ever reports it:

• universe_status() / all_universe_statuses() (backend/universe_loader.py:67) have exactly one non-test consumer: ui/status_panel.py:83.
• backend/jobs/run_daily_scan.py imports load_universe but emits no mapping-health signal in EVENT_DAILY_JOB_STARTED / EVENT_DAILY_JOB_COMPLETED.
• backend/notifications/ never references mapping_status.

So in production — where render.yaml runs refresh_universe_files() and then run_daily_scan with no human watching a sidebar — a universe can quietly shrink and nobody finds out.

Measured drift

Universe	Unmapped in git HEAD	Unmapped on disk
hemant_good_200	6 / 262	8 / 262
hemant_super_good_200_union	9 / 294	11 / 294
hemant_good_45 / hemant_super_45	—	3 / 43 · 1 / 43
nifty_100 / nifty_500	—	0

~3% of the Good 200 universe is currently not being scanned, and the number only ever grows.

Root cause of the two most recent drop-outs — confirmed, and NOT a code bug

JBCHEPHARM and GUJGASLTD are absent from Dependencies/all_instrument 2026-08-24.csv by symbol and by company name (zero hits for GUJARAT GAS or JB CHEM). The snapshot itself is healthy — 213,213 rows, 9,874 NSE EQUITY rows, final line intact — so this is a genuine vendor/corporate-action change rather than a truncated download.

For contrast, SHRIPISTON was merely renamed to "SPR Auto Technologies" and mapped fine. That is why a recent diff shows a benign rename sitting beside two real losses — and why a human skimming it would likely miss them.

Suggested work

1. Emit per-universe rows / mapped / unmapped as a structured event from run_daily_scan and the prefetch, reusing the existing log_event / EVENT_* vocabulary in backend/observability/.
2. Alert through backend/notifications/ when the unmapped count increases versus the previous run — a delta, not a threshold, so an already-known unmappable symbol doesn't nag forever.
3. Reuse universe_status() rather than adding a parallel mechanism.

Touches the notifications contract, which is why it was kept out of the audit's own PRs.

Acceptance

• A universe losing a symbol produces exactly one alert on the run where it happens.
• A steady-state universe with pre-existing unmapped symbols produces none.
• The daily job's completion event carries the per-universe counts.

🤖 Generated with Claude Code