fix(intake): three event-normalization bugs — coords transposed, salience inverted, timestamps always now()#4
Draft
WCJR-2029 wants to merge 5 commits into
Conversation
added 3 commits
July 14, 2026 15:03
The single coords branch applied the GeoJSON convention to both
geometry.coordinates (correctly [lng,lat]) and the Osiris 'coords'
field, which is [lat,lng] — see KEYWORD_COORDS in the news route
('iran': [32.427, 53.688]). Every located news event came out
transposed: Iran stories plotted at 53.7N 32.4E (western Russia).
Split the branches per source convention and add a range sanity
guard (|lat|<=90, |lng|<=180).
rs / (100 if rs > 1 else 1) assumed risk is 0-1 or 0-100, but the news route's scoreRisk() emits int 1-10. Result: risk 1 (noise) divided by 1 -> salience 1.0, risk 9 (missile attack) divided by 100 -> 0.09, floored to the 0.4 base. Ranking was upside down and min_salience filtering selected FOR spam. Normalize per scale: >10 -> /100, 1-10 int -> /10, 0-1 float as-is. Guard bool (isinstance(True, int) is True).
WorldEvent.ts always defaulted to now_ms() — the engine had no notion of when an event actually happened. Measured live: 20/29 news events served by /agent/events were >72h old (some 17 days) while stamped minutes-fresh. Consequences: the 'since' filter is meaningless for feeds with publish dates, and the swarm reasons over week-old posts as if they were breaking. Add _event_ts(): parse published/pubDate/date_added/updated/time (ISO 8601 or epoch s/ms) into ts, keeping now_ms() as the fallback for feeds that carry no time of their own.
WCJR-2029
marked this pull request as draft
July 14, 2026 20:28
added 2 commits
July 14, 2026 16:30
The previous fix still guessed the scale from the value, which cannot work: 8 means 8/10 from the news route but 8/100 from country-risk. 'int 1-10 -> /10' therefore INVERTED any 0-100 feed below 10 — a country scoring 8/100 got salience 0.8 instead of 0.08. Same inversion class as the bug being fixed, moved to a different boundary; latent today only because country-risk currently emits no events (separate bug: its items have no title key, so _to_event drops all 20). Replace the heuristic with a source-keyed RISK_SCALE map. An unlisted source is ignored rather than guessed — dropping a signal is recoverable, inverting one is not.
Osiris country-risk emits {code, risk_score, risk_level, tags} — no name,
no coords — so _to_event found no title and returned None for every item.
The feed was registered in FEEDS and contributed ZERO events, which is
indistinguishable from a quiet feed. Nothing logged.
Add _country_risk_events(): map the ISO-2 codes osiris actually serves to
a name + approximate centroid so they become locatable events. Salience is
capped at 0.5 and derived from the score alone — this is a hand-maintained
baseline (osiris marks it static:true), background context for the oracle,
never live signal. A 90/100 baseline is context; a missile landing is news.
Drop 'risk' from RISK_SCALE: with a dedicated handler it never reaches
_salience, so listing it would be dead config that reads as live.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three independent bugs in
engine/osiris_intake.py, found while consuming/agent/eventsand the MCP tools against a live stack (2026-07-14). One commit per fix; each is one function with a clear repro.1.
_coord()transposes Osiriscoords(lat/lng swapped)The single coords branch applies the GeoJSON
[lng, lat]convention to bothgeometry.coordinates(correct) and the Osiriscoordsfield — which is[lat, lng], perKEYWORD_COORDSin the news route ('iran': [32.427, 53.688],'moscow': [55.755, 37.617]).Repro: any located news item. Iran stories returned
lat=53.688, lng=32.427— plotted in western Russia.Fix: split the branches per source convention; add a
|lat| ≤ 90, |lng| ≤ 180sanity guard.2.
_salience()inverts the news risk scaleassumes risk is 0-1 or 0-100, but the news route's
scoreRisk()emits int 1-10 (starts at 1, +2 per keyword, capped 10). Sors=1(noise) → salience 1.0 whilers=9(missile attack) → 0.09, floored to the 0.4 base.Repro (live feed, same minute): a Telegram channel-auction spam post scored 1.0; "Kuwait's air defenses dealing with another wave of drone and missile attacks" scored 0.4.
min_saliencefiltering selects for spam, and the swarm's auto-scan sees noise as the top signal.Fix: normalize per scale (
>10 → /100, int 1-10 →/10, 0-1 float as-is), guardbool.3. Events are stamped with fetch time, not event time
Nothing in the engine reads
published/pubDate/etc. —WorldEvent.tsalways defaults tonow_ms().Repro: 20/29 news events served by
/agent/eventson a live stack were >72h old (several 17 days, some feeds carry 4-year-old posts) while stamped minutes-fresh. Thesincefilter is meaningless for dated feeds, and the oracle reasons over week-old posts as breaking news.Fix:
_event_ts()parsespublished/pubDate/pub_date/date_added/date/datetime/updated/time(ISO 8601, or epoch s/ms) intots, keepingnow_ms()as the fallback for feeds with no time of their own.Verification
All three verified against a live stack pre/post engine restart:
lat=32.427, lng=53.688(was transposed); Yemen items at15.552, 48.516; GeoJSON feeds (NWS alerts, USGS) unaffected — confirmed/api/newsis the only feed emitting barecoords.get_events(min_salience=0.9)returns exactly the high-risk events.Happy to split into three PRs if you prefer.