Skip to content

Epics 12–17: public API, Inbox capture funnel, one-verb assign, export - #1

Merged
Seanathon merged 14 commits into
mainfrom
dev-epic-12-17
Jun 23, 2026
Merged

Epics 12–17: public API, Inbox capture funnel, one-verb assign, export#1
Seanathon merged 14 commits into
mainfrom
dev-epic-12-17

Conversation

@Seanathon

Copy link
Copy Markdown
Owner

Capture → Curate → Archive: public API + Inbox triage + export

This branch lands 8 stories from the v2 wave (Epics 12–17), turning board-oss from a single-surface app into one with a token-authed public API, a capture funnel (Inbox + bookmarklet), one-verb assignment with tiered enrichment, and data export. Every change is additive — the no-regression constraint (NFR-BC) is enforced by a dedicated test in each story, the single-FK item.board_id is preserved throughout, and existing boards/items/assets are untouched.

Tests: npm test423 pass / 0 fail (was 350 on main), 76 suites.

Stories included

# Story What it adds
12.1 Static bearer-token auth An encapsulated /api/v1 plugin guarded by BOARD_API_TOKEN (SHA-256 + timingSafeEqual, fail-closed) + scoped CORS
12.2 CRUD item + board API POST/GET/PATCH/DELETE /api/v1/items, GET /api/v1/boards — reuses existing helpers, no parallel write path
13.1 Inbox board + cheap capture A typeless Inbox seed board; capture runs cheap (no AI takeaway) — the expensive enrichment is earned on assignment
13.2 Bookmarklet A GET /bookmarklet help page → one-click save of the current tab to your Inbox
14.1 Cheap-vs-earned tier Formal tier contract on the enrichment pipeline (the seam 14.2 builds on)
14.2 Move/assign (the one verb) POST /api/v1/items/assign — single-FK move then earned enrichment against the target board
14.3 Inbox suggestion chip Read-only AI board suggestion + additive override-signal store + pure chip/picker renderer
17.1 Export POST /skills/export → full JSON (round-trippable) or a Netscape bookmark file

One new dependency: @fastify/cors@11.2.0 (passed the supply-chain score gate).


How the new features work

1. The token-authed API (/api/v1)

A new versioned surface, mounted as an encapsulated Fastify plugin so its bearer guard + CORS structurally cannot leak onto the existing SPA/legacy/collections routes. Set a token to enable it:

BOARD_API_TOKEN=$(openssl rand -hex 32)        # your secret; the server stores only its SHA-256 hash
BOARD_API_CORS_ORIGINS=https://your-extension  # optional; comma-separated; default = no cross-origin

With no token configured, the /api/v1 surface fails closed (401 on everything). The plaintext token is never logged, serialized, or written to board.db.

2. Capture funnel → Inbox

  • A typeless Inbox board is now seeded automatically (idempotent — existing DBs gain it on next boot, nothing else changes).
  • A POST /api/v1/items with no boardId lands the item in the Inbox.
  • Inbox capture is cheap: it fetches title/screenshot but does not spend an LLM call. The expensive descriptor-driven takeaway is earned when you assign the item to a typed board.

3. The one assign verb

POST /api/v1/items/assign {itemIds, boardId} is the single code path for promotion (the AI composer will reuse it too). It moves item.board_id first, then fires the earned enrichment against the target board's schema. It's idempotent (same-board re-assign is a no-op), reversible (assign back to Inbox is a safe no-op), and batch-capable.

4. AI suggestion + override signal

GET /api/v1/items/:id/suggestion returns a suggested home board (read-only; requires an LLM provider, else null → manual picker). POST /api/v1/suggestions/override records suggested-vs-chosen as a future-quality signal in a new additive table — never mutating item/board rows.

5. Export

POST /skills/export {format} serializes everything read-only: json (every board+descriptor, item, asset reference — re-ingestible via the importer) or netscape (a browser/linkding-compatible bookmark file). Binary assets are referenced by path+hash; copy your screenshots/ dir separately.


Manual testing: UI vs API

✅ Usable in the UI right now

  • Inbox board — start the app and open the collection switcher; Inbox appears as a third board. Anything captured without a target board shows up here (cheap metadata only).
  • Bookmarklet — visit http://localhost:3141/bookmarklet, paste your BOARD_API_TOKEN, and drag the "📥 Save to Board" button to your bookmarks bar. Click it on any web page → it saves that tab to your Inbox and shows a transient confirmation without navigating you away. (The token is filled in entirely in your browser; the page never sends it to the server.)

🔌 Usable via the API (curl / extension / PWA)

Run the server with a token, then (default port 3141):

TOKEN=your-token
BASE=http://localhost:3141

# auth probe
curl -s $BASE/api/v1/ping -H "Authorization: Bearer $TOKEN"            # {"ok":true}

# capture to the Inbox (no boardId)
curl -s -X POST $BASE/api/v1/items -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"url":"https://example.com"}'

# list (newest-first, filterable)
curl -s "$BASE/api/v1/items?board=inbox&limit=20" -H "Authorization: Bearer $TOKEN"

# board list for targeting
curl -s $BASE/api/v1/boards -H "Authorization: Bearer $TOKEN"

# assign (move + earned enrichment); use a real item id + target board
curl -s -X POST $BASE/api/v1/items/assign -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"itemIds":["<id>"],"boardId":"library"}'

# patch notes / favorite
curl -s -X PATCH $BASE/api/v1/items/<id> -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{"notes":"hello","favorite":true}'

# AI suggestion (null unless an LLM provider is configured)
curl -s $BASE/api/v1/items/<id>/suggestion -H "Authorization: Bearer $TOKEN"

# export (no auth — the generic skills route)
curl -s -X POST $BASE/skills/export -H 'Content-Type: application/json' -d '{"format":"json"}'
curl -s -X POST $BASE/skills/export -H 'Content-Type: application/json' -d '{"format":"netscape"}'

Try the guards too: omit the header → 401; junk ?limit=abc → falls back, no 500; DELETE an unknown id → 404.

🚧 Staged (not yet wired into the SPA)

The suggestion chip / manual board-picker in the Inbox list is delivered as a tested pure renderer + the two endpoints above, but the browser event-glue (tap → POST /api/v1/items/assign, and mounting the control into the Inbox list) is staged with the flat-JSON→SQLite SPA cutover — consistent with how Epic 8 staged its DOM wiring. So suggestion/assign are exercisable via the API today, not yet via a click in the Inbox. Same for an export button (export works via the endpoint).

🧭 Not in this branch (out of scope for this loop)

13.3 (PWA share-target), 13.4 (browser extension), and Epic 15 (AI composer / saved views) — Epic 15 stays planned pending confirmation of the view-definition design. 14.2's assignItems is the single path the composer will reuse.


Notes for review

  • Each story file in docs/bmad/stories/ is at Status: review with a full Dev Agent Record (decisions + the party-mode review findings that were fixed before commit).
  • Reviews caught + fixed: a Host-header reflected-XSS on the bookmarklet page, a NaN-param 500 on the list endpoint, a confounded cheap-tier test, the missing no-auto-assign regression, and an undeclared staged-DOM boundary.

Seanathon and others added 13 commits June 23, 2026 03:28
Enrichment now also returns an optional `title`, written to the title
system column (kept out of the field bag). Cleans a cluttered/wrong captured
page <title> on both add and refetch; an omitted title leaves the column
untouched. Prompt asks the model to keep a good title and fix a bad one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds describeProvider(config) (mirrors selectProvider precedence) and returns
it from /api/meta as { kind, agent, label } or null. Lets the UI label the add
button and list only the actually-configured provider instead of guessing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes a local .env (LLM_AGENT, PORT, etc.) persist across restarts, so the
'+ Add LLM' setup instructions actually take effect. Uses Node's native flag (no
new dependency); if-exists preserves the zero-config no-AI default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Library (and other non-inspiration boards) now honor the grid/list toggle:
  a screenshot-less tile grid (renderLibraryGrid), and the per-item menu hides
  'Replace screenshot' for non-visual boards.
- Add button reflects real provider state: 'Add' (no AI) vs 'Add with <provider>'
  from /api/meta; the caret menu always offers '+ Add LLM' (setup-directions
  modal) and lists only the configured provider (no phantom Codex).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an encapsulated Fastify plugin at prefix /api/v1 with a static
bearer-token onRequest guard (SHA-256 + crypto.timingSafeEqual, fail-closed)
and scoped @fastify/cors. Encapsulation structurally guarantees NFR-BC: the
guard/CORS cannot reach root routes (SPA, /api/bookmarks, /api/collections,
/skills), proven by no-auth-header regression tests.

- config: BOARD_API_TOKEN -> non-enumerable apiTokenHash (plaintext discarded,
  hash kept out of all serialization); BOARD_API_CORS_ORIGINS -> corsOrigins.
- buildServer: injectable apiToken/corsOrigins (defaults to config), falsy ->
  fail-closed.
- GET /api/v1/ping probe as the guarded test target (12.2 adds CRUD here).
- @fastify/cors@11.2.0 pinned (socket score: all thresholds pass).

Addressed party-mode review: non-enumerable hash, empty-token fail-closed,
case-insensitive Bearer scheme, OPTIONS-preflight + edge tests, non-vacuous
no-plaintext-log test.

353 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds POST/GET/PATCH/DELETE /api/v1/items and GET /api/v1/boards inside the
12.1 encapsulated plugin (behind the bearer guard + CORS). Reuses the
existing helpers verbatim — addItemSkill (optimistic pending create on the
single-writer path), patchItemFields (user-field allowlist), and
deleteItemWithAssets (row cascade + asset-file unlink) — so there is no
parallel write path and the orphaned-asset-file bug cannot reappear (NFR-BC).

- Only listItemsForApi (db/hydrate.ts) is new: cross-board, newest-first
  (idx_item_created_at), bounded limit, offset, since; page-scoped asset load.
- Create requires an explicit existing boardId (no Inbox default; that is 13.1).
- NFR-BC test proves v1 and the legacy collections path share one store.

Addressed party-mode review: NaN-param guard (no 500 / no silent-empty list),
pinned the shared-store reuse + unknown-board cause, and a tolerant v1-scoped
JSON parser so an empty-body DELETE with a json content-type returns 204.

366 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The linchpin of the capture->curate->archive wave. Additive only (NFR-BC):

- Seeds a typeless Inbox board (stable id 'inbox', view:'list', fields:[])
  via the unchanged idempotent seed() loop. item.board_id stays a NOT NULL
  single FK -- the Inbox is a board, not a global pool.
- Adds a tier ('cheap'|'earned') option to runCaptureEnrichJob, defaulting to
  'earned' so every existing board's capture->enrich is byte-for-byte
  unchanged. 'cheap' (Inbox) skips the enrich hop, so llm.complete is never
  called -- the AI takeaway is earned on assignment (Epic 14). Capture still
  populates a scannable title and the item reaches 'done'.
- captureTierForBoard() selects cheap for the Inbox; add-item uses it.
- POST /api/v1/items defaults an omitted/blank boardId to the Inbox; a provided
  unknown board still errors.

NFR-BC proven by a pre-wave boot/regression test (seed + re-seed, existing rows
byte-for-byte + routes serve unchanged with the Inbox added).

Addressed party-mode review: added a discriminating cheap-on-Inspiration test
(isolates the tier flag from the fields:[] early-return) and a
captureTierForBoard unit test, so the cheap-tier seam 14.1 builds on is
genuinely guarded.

372 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a pure buildBookmarklet({instanceUrl, token}) producing a javascript:
one-liner that POSTs the current tab to the authed /api/v1/items (12.2) with
no board, so it lands in the Inbox (13.1 default) with cheap enrichment. Never
navigates the user away.

A GET /bookmarklet help page serves a draggable bookmarklet. 12.1 reconcile:
the server holds only the token hash, never the plaintext, so the page ships a
TOKEN_PLACEHOLDER and the operator fills their own token client-side -- the
plaintext never touches the server.

Addressed party-mode review: fixed a Host-header reflected-XSS in the served
page (HTML-escape the instance URL + escape "<" in script-embedded strings so
"</script>" can't break out) with a regression test; documented the deliberate
delegation of the cheap-enrichment proof to 13.1's confound-free test.

377 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The production seam (runCaptureEnrichJob's tier param, default earned, cheap
skips runEnrichmentForItem) was delivered in 13.1 as the general pipeline
knob. This story adds the formal tier-contract test suite (enrichment/
tier.test.ts) that Story 14.2 (assign -> earned) depends on:

- cheap tier makes zero LLM calls on a board WITH fields (load-bearing: not
  confounded by the fields:[] early-return) and reaches done.
- earned tier calls the LLM once against the item's CURRENT board descriptor
  (prompt signature assertion); omitted tier defaults to earned (NFR-BC).
- single-item scope: an earned enrichment of one item never re-enriches a
  sibling already-enriched item on the same board.
- graceful no-LLM: earned + disabledLlm resolves to done, not error.

Addressed party-mode review: reframed the AC4 regression from a tautological
test (a cheap job can't touch a different row) into a load-bearing
single-item-scope assertion with an overwriting provider.

382 pass / 0 fail. No production change. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds assignItems (enrichment/assign.ts) -- the single assign code path both
the REST route and the composer (15.2) call. Phase 1 moves every item's
board_id (single-FK, never m2m; search_blob recomputed vs the target; fields/
assets preserved); Phase 2 fires the earned-tier enrich-only job per moved
item against the now-target descriptor. Idempotent (same-board re-assign
skipped, no LLM churn), reversible (assign-back-to-Inbox is a safe no-op via
the typeless early-return), batch-capable (allSettled, per-item resilient).

A thin POST /api/v1/items/assign route adapts the helper (validation + a
defensive 200-item cap; awaits enrichment so a manual assign returns the
enriched result). The bulk composer calls assignItems directly.

Addressed party-mode review: added the AC6 no-auto-assign NFR-BC regression and
a genuine enrich-failure-in-batch test; restructured to moves-first/enrich-
second so moves don't interleave with slow LLM jobs and a failed move can't
abort the batch; de-duped item ids.

393 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delivers the pure + backend layer of the Inbox triage UX:
- enrichment/suggest.ts: a READ-ONLY descriptor-driven LLM resolver that
  suggests a target home board (Inbox excluded), validated against a
  candidate-id allowlist (hallucinated/injected ids -> null), degrading to
  null on no-provider/error (-> manual picker). Never writes the item.
- Additive suggestion_override table + recordAssignmentChoice: captures a true
  override (suggestion existed AND chosen != suggested) as future-quality
  signal; confirms/manual-picks record nothing. CREATE TABLE IF NOT EXISTS so
  existing DBs gain it on boot (NFR-BC).
- descriptor/inbox-suggest.js (pure): assignControlMode + renderAssignControl
  (one-tap chip carrying the suggested board + change-picker, or manual picker)
  + renderInboxCount (no guilt-pile). XSS-safe.
- GET /api/v1/items/:id/suggestion (read-only) + POST /api/v1/suggestions/override.

Degradation keys off providerConfigured (not field-emptiness); the chip/picker
target the 14.2 assign verb (one assign path, no second mover).

DOM event-glue (tap -> fetch POST /items/assign, Inbox-list mount) is STAGED
with the SPA cutover, consistent with the 8.x precedent and now explicitly
declared in the story's Dev Agent Record (addressing the party-mode honesty
finding). Added AC5 count impl+test and an AC1 generic-hydrator test.

414 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds db/export.ts (read-only serializers) + a thin `export` skill invokable via
POST /skills/export:
- exportJson: every board (descriptor), item, and asset reference, grouped as
  per-board record arrays that re-ingest through importRecords where possible
  (dotted fields un-flattened to nested groups for inspiration, flat for
  library). Binary assets referenced by path+hash, never inlined.
- exportNetscape: a standards-conformant, browser/linkding-compatible bookmark
  file (DOCTYPE/DL/<A HREF ADD_DATE TAGS>), HTML-escaped, URL-less items skipped,
  tags resolved from the board's type:'tags' descriptor fields.

READ-ONLY by hard invariant (select() only, no ctx.queue); a zero-mutation test
asserts rows + FTS are unchanged after both formats. Round-trip verified by
feeding the export back through importRecords into a fresh DB.

Addressed party-mode review: !=null guard so epoch-0 createdAt isn't dropped;
strengthened AC1 assertions (status/analysis/added/asset dimensions); library
round-trip + empty-DB tests.

423 pass / 0 fail. No regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Seanathon Seanathon self-assigned this Jun 23, 2026
The "GET /api/bookmarks still serves" test asserted 200, but that route reads
the gitignored bookmarks.json via an unguarded readFileSync — present locally,
absent in CI, so loadCollection threw -> 500 and the test failed only in CI.

Snapshot/write-[]/restore the flat-JSON file around the assertion (the same
pattern server.test.ts uses for these gitignored personal-capture files), so
the test gets a real 200 whether or not the file exists. Verified green with
both bookmarks.json and library.json absent (full suite 423/0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Seanathon
Seanathon merged commit 2b10fc6 into main Jun 23, 2026
2 checks passed
@Seanathon
Seanathon deleted the dev-epic-12-17 branch June 23, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant