Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions .socraticodecontextartifacts.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,17 @@
{
"name": "design-plans",
"path": "./docs/plans/",
"description": "Founding MVP design (2026-06-25, adopted). The command/fact contracts as shipped in co-core v0.7.0 — ContentFetchCommand (URL-addressed, command_id as the wire idempotency key) and BlobAvailableEvent (content_fingerprint, blob_uri, size_bytes, media_type, url, optional command_id) — the stream taxonomy and DLQ naming, the temp-storage interface, the two-level idempotency argument, why fingerprint parity dissolves when Replicator is the sole fetcher, explicit MVP scope cuts, and the build sequence. The founding plan's three open questions were settled 2026-07-31 in a companion doc: blob_uri is a file:// URI over a two-level sharded, content-addressed path (<blob_dir>/<ab>/<cd>/<sha256>.bin, constant .bin extension, temp-file + os.replace so presence means complete); the MVP command issuer is an in-repo scripts/seed_fetch.py rather than Watcher or Archiver; and the MVP boundary is stop-at-fact, with archiver-writeback reclassified from MVP+ to won't-do — Archiver consumes blob_available and writes the SourceRevision itself (archiver#118, which also carries the info_source_id contract gap). Blob retention is out of MVP scope and tracked in #5."
"description": "Founding MVP design (2026-06-25, adopted). The command/fact contracts as shipped in co-core v0.7.0 \u2014 ContentFetchCommand (URL-addressed, command_id as the wire idempotency key) and BlobAvailableEvent (content_fingerprint, blob_uri, size_bytes, media_type, url, optional command_id) \u2014 the stream taxonomy and DLQ naming, the temp-storage interface, the two-level idempotency argument, why fingerprint parity dissolves when Replicator is the sole fetcher, explicit MVP scope cuts, and the build sequence. The founding plan's three open questions were settled 2026-07-31 in a companion doc: blob_uri is a file:// URI over a two-level sharded, content-addressed path (<blob_dir>/<ab>/<cd>/<sha256>.bin, constant .bin extension, temp-file + os.replace so presence means complete); the MVP command issuer is an in-repo scripts/seed_fetch.py rather than Watcher or Archiver; and the MVP boundary is stop-at-fact, with archiver-writeback reclassified from MVP+ to won't-do \u2014 Archiver consumes blob_available and writes the SourceRevision itself (archiver#118, which also carries the info_source_id contract gap). Blob retention is out of MVP scope and tracked in #5."
},
{
"name": "issuer-contract",
"path": "./docs/contracts/content-fetch-issuer-contract.md",
"description": "Normative contract for any service publishing content.fetch commands (seed_fetch.py today, Watcher from Phase 4) — and its permanent home, linked to rather than copied by issuer-side repos. Opens with the frame shape, because getting it wrong is the likeliest and quietest mistake: what lands on the stream is a co-core envelope produced by to_wire (key, payload as JSON, event_type, schema_version, occurred_at, content_type), never the model's fields flattened, and an XADD of top-level command_id/url fails from_wire into content.fetch.dlq silently. key is derived by to_wire (command_id for a command, content_fingerprint for a fact) and is not what Replicator dedupes on — that is the decoded payload.command_id — but it is what makes a DLQ entry correlatable, so an issuer can tail content.fetch.dlq with a plain XREAD and close a pending entry deterministically instead of waiting out a timeout. Then why the wire carries no info_source_id and what that pushes onto the issuer: mint a fresh ULID command_id per fetch occasion (a resource-stable id is silently deduped away for 24h by replicator:cmd:<id>); url is one-to-many onto InfoSources and therefore not a correlation key; persist the command_id -> domain map before publishing, outbox-style; correlation must be idempotent because a crash between publish and the dedupe SET emits a second fact under the same command_id; consumers must not dedupe their inbox on content_fingerprint, which is storage identity rather than correlation identity; there is no fetch_failed fact, so silence is the only failure signal and the issuer needs its own pending-entry reaper that re-issues rather than concluding loss; blob_uri is a host-local file:// URI whose 7-day TTL runs from last fetch-reference, not last consumer read. Carries the full failure taxonomy (which conditions dead-letter to content.fetch.dlq silently, which retry indefinitely, which park in the PEL under the byte ceiling), what Replicator does and does not guarantee, the unauthenticated-capability trust posture resting on Archiver's localhost bus ACL, and the deferred fetch_failed fact as a co-core change. Written for #8."
"description": "Normative contract for any service publishing content.fetch commands (seed_fetch.py today, Watcher from Phase 4) \u2014 and its permanent home, linked to rather than copied by issuer-side repos. Opens with the frame shape, because getting it wrong is the likeliest and quietest mistake: what lands on the stream is a co-core envelope produced by to_wire (key, payload as JSON, event_type, schema_version, occurred_at, content_type), never the model's fields flattened, and an XADD of top-level command_id/url fails from_wire into content.fetch.dlq silently. key is derived by to_wire (command_id for a command, content_fingerprint for a fact) and is not what Replicator dedupes on \u2014 that is the decoded payload.command_id \u2014 but it is what makes a DLQ entry correlatable, so an issuer can tail content.fetch.dlq with a plain XREAD and close a pending entry deterministically instead of waiting out a timeout. Then why the wire carries no info_source_id and what that pushes onto the issuer: mint a fresh ULID command_id per fetch occasion (a resource-stable id is silently deduped away for 24h by replicator:cmd:<id>); url is one-to-many onto InfoSources and therefore not a correlation key; persist the command_id -> domain map before publishing, outbox-style; correlation must be idempotent because a crash between publish and the dedupe SET emits a second fact under the same command_id; consumers must not dedupe their inbox on content_fingerprint, which is storage identity rather than correlation identity; there is no fetch_failed fact, so silence is the only failure signal and the issuer needs its own pending-entry reaper that re-issues rather than concluding loss; blob_uri is a host-local file:// URI whose 7-day TTL runs from last fetch-reference, not last consumer read. Carries the full failure taxonomy (which conditions dead-letter to content.fetch.dlq silently, which retry indefinitely, which park in the PEL under the byte ceiling), what Replicator does and does not guarantee, the unauthenticated-capability trust posture resting on Archiver's localhost bus ACL, and the deferred fetch_failed fact as a co-core change. Written for #8."
},
{
"name": "boundaries-charter",
"path": "./docs/contracts/replicator-boundaries.md",
"description": "Normative charter for what Replicator is allowed to become \u2014 sibling to the issuer contract, which settles the wire while this settles the service. The rule: Replicator owns the mechanics of acquiring bytes from a network and holding them briefly, never why, when, or what they mean. Three tests for any proposed capability, field, or setting (durable per-resource history -> issuer; cross-command coordination over a resource only the fetcher can see -> Replicator; expressible without domain vocabulary or it is the issuer's), resolved when both fire by mechanism-to-Replicator / policy-to-the-issuer / config-over-the-bus. Worked examples: per-host politeness (the hard case, where the tests do not resolve to one side) and conditional GET (ETag/Last-Modified are durable per-resource history, so the fact returns them and the next command replays them as headers \u2014 conditional GET with zero per-URL state here). Config taxonomy of three channels (env for host facts, command for the occasion, a policy stream for cluster policy needing cross-command state) with a fourth \u2014 an inbound admin HTTP API \u2014 rejected by name; ingress is read-only liveness and self-description, and the invariant that matters is asserted about the worker, which binds no port, not the dev-only FastAPI app. Specifies the agreed-but-unbuilt content.fetch.policy stream: last-write-wins per host, replayed from 0-0 at boot, MAXLEN-trimmed because periodic full republication onto an untrimmed stream is unbounded growth, a Redis hash rejected for having no schema_version and coupling to a key name, a conservative default for unknown hosts, and the enforcement idiom (park in the PEL, return via claim_stale) whose 60s granularity misses Watcher's 1.0s baseline by 60x \u2014 so the stream is a precondition of the Phase 4 cutover, not a follow-on (watcher#245). Records blob_uri's host-local file:// coupling as a tracked violation (#7) rather than omitting it. Enforced by tests/test_boundaries.py in CI: eight invariants including an AST scan of src/ for domain nouns in identifiers and string literals, a recursive route walk, no locally-defined wire models (payload shapes, not producer-owned token vocabularies like FailureReason), and REPLICATOR_-prefixed settings with BUILD_ID exempted by name. Written for #12."
},
{
"name": "commands",
Expand All @@ -38,17 +43,17 @@
{
"name": "wheelhouse-sync",
"path": "./scripts/sync_wheelhouse.py",
"description": "Mirrors the private cannobserv package index (gs://co-gcs-pypi/wheels/) into ./.wheelhouse, from which uv resolves co-core and co-core-aio via find-links. Must run before uv sync and must not import the project, since the project's deps are what it provides hence the isolated `uv run --no-project --with google-cloud-storage` invocation. ADC auth: the SA key on the VM, keyless WIF in CI."
"description": "Mirrors the private cannobserv package index (gs://co-gcs-pypi/wheels/) into ./.wheelhouse, from which uv resolves co-core and co-core-aio via find-links. Must run before uv sync and must not import the project, since the project's deps are what it provides \u2014 hence the isolated `uv run --no-project --with google-cloud-storage` invocation. ADC auth: the SA key on the VM, keyless WIF in CI."
},
{
"name": "redis-floor-guard",
"path": "./scripts/check_redis_floor.sh",
"description": "ExecStartPre guard asserting the Redis >=7.0 change-bus server floor. Replicator is the cluster's first user of AsyncBusConsumer.claim_stale, which reads XAUTOCLAIM's three-element reply the deleted-ids element added in Redis server 7.0 so a lower version breaks crash recovery. Soft on an unreachable broker (it may still be starting), fatal on a genuine downgrade."
"description": "ExecStartPre guard asserting the Redis >=7.0 change-bus server floor. Replicator is the cluster's first user of AsyncBusConsumer.claim_stale, which reads XAUTOCLAIM's three-element reply \u2014 the deleted-ids element added in Redis server 7.0 \u2014 so a lower version breaks crash recovery. Soft on an unreachable broker (it may still be starting), fatal on a genuine downgrade."
},
{
"name": "ci-pipeline",
"path": "./.github/workflows/ci.yml",
"description": "GitHub Actions CI. Keyless Workload Identity Federation auth to GCP for the wheelhouse mirror, a fail-fast guard asserting the GCP_WIF_PROVIDER org variable is visible to this repo, ruff check and format gates, a co-core extras import smoke proving [extract] and [bus] wire up in a clean environment, and pytest with the coverage gate. Deliberately no Postgres service and no alembic steps Replicator is DB-free."
"description": "GitHub Actions CI. Keyless Workload Identity Federation auth to GCP for the wheelhouse mirror, a fail-fast guard asserting the GCP_WIF_PROVIDER org variable is visible to this repo, ruff check and format gates, a co-core extras import smoke proving [extract] and [bus] wire up in a clean environment, and pytest with the coverage gate. Deliberately no Postgres service and no alembic steps \u2014 Replicator is DB-free."
},
{
"name": "dependency-contract",
Expand Down
Loading
Loading