Phase 3: enforce private visibility at the byte level (raw R2 bytes bypass the flag on public workspaces)

[zachdunn]

In plain terms

Phase 2 (#139, PR #142) lets a file be marked private: the /f/ page and GET /public/files/:ws/:key now 401 with auth_required, and members reach the bytes through the signed-URL resolver. But on any workspace with a publicBaseUrl custom domain, the raw object URL still serves the bytes to anyone — R2 public domains are all-or-nothing per bucket, so the flag gates only the curated endpoints, not storage itself. This is documented in the route (apps/api/src/routes/public-files.ts doc comment) and in PR #142's "Known limitation".

Phase 3 = make private mean private on public workspaces too.

Why the flag can't fix this alone

• publicBaseUrl exposes the whole bucket/prefix unsigned. No per-object opt-out exists at the R2 custom-domain layer.
• So a truly private object must not live under the publicly-served prefix at all: it needs a non-public home, served only via short-lived signed URLs (already wired: signedDownloadUrl in packages/storage, used by GET /me/workspaces/:name/file-url) or worker-streamed bytes.

Approach sketch (not committed)

• Private prefix/bucket: when visibility: private is set (upload header or toggle), store the object under a prefix excluded from the public domain (e.g. _private/<key> inside the workspace prefix, with the custom domain scoped to exclude it — needs verification of how the shared-bucket custom domain is mapped) or in a separate non-public bucket.
• Toggle = move: the PATCH /me/workspaces/:name/files/visibility rewrite in files-core.setObjectVisibility becomes a move between prefixes rather than a same-key metadata rewrite. Key identity as seen by the API/page stays stable; only the storage location changes. Listing/head must resolve across both locations (or the Rich upload provenance: object metadata and optional index #35 D1 object index records the location).
• Serving: members get bytes via the existing signed-URL ladder; nothing new needed on /public/files (it already 401s).
• Migration/consistency: existing private-flagged objects on public workspaces need a backfill move; interim state should fail closed on the endpoints (already does) while being honest that raw URLs leaked until moved.

Constraints & context for whoever picks this up

• files-sdk 2.1.0 (pinned patch) has no conditional writes (onlyIf/etag) — moves have the same last-write-wins window as the Phase 2 rewrite (documented on setObjectVisibility; CodeRabbit has a learning about it). Files.copy/move exist but take no metadata override.
• The two-endpoint trust split is a security invariant: no signing path may ever be added to unauthenticated /public/files.
• Public URLs previously handed out for an object that later goes private will keep working until the move lands and caches expire (upload cache-control is max-age=60 since PR fix(api): set Cache-Control: max-age=60 on uploads so overwrites propagate #10; BYO-domain edge caches are the wild card).
• Related: Rich upload provenance: object metadata and optional index #35 (D1 object index — natural home for per-object location/visibility), Decide: should uploads attach auto-create a gallery per PR? (cap + lifecycle) #92 (default share-link decision).

Definition of done

Marking a file private on a workspace WITH a publicBaseUrl makes its raw URL stop serving bytes (404/403) within a bounded window, while members can still fetch it via signed URL and the /f/ page's sign-in flow keeps working.

Refs: #139 (Phase 2, shipped in PR #142), #135/#136 (Phase 1), #123/#141 (resolver ladder).