Seed script for content.fetch-policy — no supported way to exercise the consumer before watcher#245

[gregoryfoster]

Why

#19 shipped the content.fetch-policy consumer. The producer half is issuer-side
(CannObserv/watcher#245) and has not landed, so today there is no supported way to put a policy
on the stream — which means there is no supported way to exercise the consumer end to end
outside the test suite.

scripts/seed_fetch.py exists for exactly this reason on the command stream: hand-rolling wire
fields at a REPL is how a frame ends up on the wrong stream, or on production's, or with an
envelope that does not match what the real producer emits. The policy stream now has the same
gap and one extra hazard — it is read without a consumer group, so a bad frame is not
dead-lettered anywhere. It is skipped past and gone.

What

A --policy mode on seed_fetch.py, or a sibling seed_policy.py. Either is fine; the
existing script's argument surface is the thing worth copying, not its file.

Requirements, all inherited from seed_fetch.py for the same reasons:

• Publishes through co-core's to_wire and FetchPolicyState, never a hand-built field
  map. A FetchPolicyEmit (extra="forbid") on the emit path would catch a typo'd field, which
  is what that strict variant is for.
• --redis-url and --topic required explicitly, with --production additionally required
  for the one combination the live worker consumes (db 0 and content.fetch-policy). A
  frame there changes how the running service paces real origins.
• --revoke publishes a tombstone (revoked=True, no interval) rather than making the
  caller reason about which fields to omit — the model rejects a live policy with no interval,
  and getting that wrong at a REPL is the likely first attempt.
• --watch tails the stream with a plain XREAD and never joins a group. Same rule as
  seed_fetch.py --watch: a group left behind by an operator tool accumulates a PEL nothing
  drains, and on this stream that is doubly wrong because the design has no group at all.

Not in scope

• Anything that looks like a policy store. The numbers belong to the issuer; this is a test
  harness for the consumer, and it should stay obviously that. If it starts holding a set of
  hosts it becomes the thing docs/contracts/replicator-boundaries.md says Replicator must not
  become.

Refs

• Consume content.fetch-policy — replace the single env default with Watcher's per-host numbers (cannobserv#285) #19 — the consumer half (found during its code review)
• Phase 4 cutover blocker: per-domain rate limiting stops working the moment Watcher stops fetching watcher#245 — the real producer
• CannObserv/cannobserv#285 — the stream contract
• scripts/seed_fetch.py — the precedent, including the --production guard and the groupless --watch

[gregoryfoster]

The real producer landed: watcher#245 is closed — Watcher publishes its Domain politeness numbers (full set + tombstones, every 5 min and on API domain mutations) to content.fetch-policy, verified consumed by the live policy reader on 2026-08-06 (applied a host fetch policy × 3, ~3 ms after publish). A seed/scratch tool may still be worth having for consumer testing without touching Watcher's Domain table, but the "no supported way to exercise the consumer" gap this issue opens with is closed.

[gregoryfoster]

Closing — the premise this was filed on is gone, and what remains does not justify the
tool.

Verified live, not inferred from watcher#245's closure

On the production broker, just now:

XLEN content.fetch-policy            → 237
XREVRANGE ... COUNT 1                → 1786053000550-1
  key      lcb.wa.gov:2026-08-06T21:50:00.548551+00:00
  payload  {"schema_version":1,"event_type":"fetch_policy",
            "occurred_at":"2026-08-06T21:50:00.548551Z","host":"lcb.wa.gov",
            "min_interval_seconds":1.0,"revoked":false}

and the live worker consuming them:

{"logger": "src.worker.policy", "message": "applied a host fetch policy",
 "host": "lcb.wa.gov", "min_interval_seconds": 1.0, "default_interval_seconds": 1.0,
 "timestamp": "2026-08-06T21:50:00.552069+00:00"}

238 such lines in the last 24 h across three hosts, arriving ~1 ms after each publish, on
Watcher's 5-minute full-set republish. The consumer is exercised end to end, continuously,
by the real producer emitting real frames — which is strictly better than what a seed script
could have demonstrated, since it also proves the envelope matches what the producer
actually writes rather than what we thought it would.

Why not keep it for scratch testing

The residual case — exercise the consumer without touching Watcher's Domain table — is
real but thin, and it is already covered from both ends:

• tests/worker/test_policy.py covers the map's semantics, tombstones included
  (test_a_revoked_host_drops_out_of_the_map, branch-on-revoked-first, and the
  full-set-republish boot case), and tests/worker/test_policy_integration.py covers the
  groupless reader against a live broker on scratch streams.
• Reading the live stream needs no tool at all: XREAD on content.fetch-policy, no
  group — which is exactly what this issue's --watch requirement was going to wrap.

That leaves publishing a hand-made policy frame as the only thing not otherwise possible,
and it is the half with the production hazard the --production guard existed to manage. A
tool whose remaining purpose is to write to the stream that governs how the live worker paces
real origins is worth having only if something needs it, and nothing does.

There is also a shape argument the issue makes against itself. Its Not in scope section
warns that anything holding a set of hosts becomes what the boundaries charter says
Replicator must not be. That risk was acceptable while the tool was the only producer; with
a real producer publishing full sets every five minutes, a second thing in this repo that
knows how to write policies is a strictly worse trade.

The one thing that did get harder, recorded rather than fixed

The tombstone path is now only reachable through Watcher — a Domain delete writes
fetch_policy_tombstones, which merges into the next full-set republish. So exercising a
revocation against the live consumer means deleting a Watcher domain, which is not something
to do casually, and the unit tests are what stand in for it. If a consumer-side change ever
needs to observe a real revocation arriving over the wire, that is the moment to file a
narrow tool for it — with a concrete reason, rather than in anticipation of one.

Refs: #19 (the consumer), CannObserv/watcher#245 (the producer, closed), cannobserv#285
(the stream contract).