Skip to content

Object-store blob backend (claim-check) — remove the shared-filesystem data-plane coupling #7

Description

@gregoryfoster

Problem

Replicator is already fully decoupled on the control/fact plane — commands in on content.fetch, facts out on content.blobs, no HTTP/SDK to any sibling (deps are co-core only). The one remaining coupling is the data plane: LocalBlobStore announces a file:// blob_uri, so a consumer of content.blobs must be on the same host with filesystem traversal into Replicator's blob directory to read the bytes.

The whole warn_if_unreachable / ancestor-permission-check surface in src/worker/main.py exists because of this — it's diagnosing a cross-service shared-filesystem dependency. That's a host/mount coupling, not a bus one.

Goal

Make Replicator completely decoupled: the bus carries the reference (already true — content_fingerprint + blob_uri + metadata), and the bytes live in shared storage infrastructure no service operates as a peer — the claim-check pattern, exactly analogous to Redis being Archiver-operated shared infra rather than a service anyone calls.

Putting bytes on the bus is explicitly rejected: 64 MiB blobs on Redis Streams is a memory/trim anti-pattern. The reference stays on the bus; the bytes move to a host-independent store.

Approach

Swap LocalBlobStore → an object-store BlobStore (GCS first, matching the cluster's existing GCS footprint) so blob_uri becomes gs://…, host-independent. The seam already exists and was designed for exactly this:

src/storage/base.py: "A Protocol rather than a base class … a GCS or GDrive backend added when durable replication lands satisfies it without importing anything from here. backend_uri is opaque to consumers."

co-core already ships co_core.effects.{gcs,gdrive} to build on. Consumers fetch via their own store credentials; the entire warn_if_unreachable + FS-traversal-permission surface dissolves.

Scope / considerations

  • New GcsBlobStore implementing the BlobStore Protocol (store/exists/open), content-addressed key = fingerprint, media_type recorded as object content-type. Select backend via config (REPLICATOR_BLOB_BACKEND=local|gcs), keeping local-FS as the dev/test default.
  • Retention (sweeper/retention.py) currently walks a local tree — an object-store backend needs the equivalent (object lifecycle policy or a store-aware sweep). The blob_max_total_bytes backpressure ceiling and BlobUsage accounting are FS-specific and will need rethinking against a store.
  • Temporal contract (blocking for "decoupled"): even with an object store, the fact contract has a temporal coupling — the blob must outlive the slowest consumer's read latency. blob_ttl_seconds already flags this as the consumer's to state (archiver#118). Settle a documented availability window (or a consumer fallback: re-issue content.fetch if the blob is gone) as part of this work.
  • The blob_uri-is-opaque contract must hold — no consumer parses it — so this is a pure backend swap from their side.

Out of scope

  • Durable/permanent replication per RepSpec provider (gcs/gdrive/ia) with path_template/credentials_alias — that's the separate later phase. This issue is only about re-homing the temp blob store off the shared filesystem.

Refs

  • Decoupling analysis: CannObserv/archiver#72 (Epic), Replicator MVP design docs/plans/2026-06-25-replicator-mvp-design.md.
  • Availability window: archiver#118.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions