feat: migrate storage from SQLite to PostgreSQL#1793
Open
gsxdsm wants to merge 74 commits into
Open
Conversation
Contributor
|
Too many files changed for review. ( |
Contributor
|
Important Review skippedToo many files! This PR contains 937 files, which is 787 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (937)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
gsxdsm
commented
Jun 26, 2026
Collaborator
Author
Multi-agent code review — SQLite → PostgreSQL migration (3rd pass, HEAD
|
gsxdsm
added a commit
that referenced
this pull request
Jun 26, 2026
- Add workspaceWorktrees to PG_JSONB_TASK_COLUMNS (jsonb round-trip fix) - Guard InsightStore/ResearchStore/TodoStore/MissionStore in backend mode - Guard ingestIncidentSignal in PG backend mode - Guard OTel metrics export in backend mode - Await setCompletionHandoffAcceptedMarker in executor + self-healing - Fix concurrency cap overrun with conditional UPDATE (READ COMMITTED safe) - Re-throw on attachBackendLayer failure (fail startup vs silent degradation) - Fix tryClaimTask race with ON CONFLICT DO NOTHING - Fix pull_requests partial unique indexes (uniqueIndex.where) - Fix central_activity_log index names (idxCentralActivityLog*) - Replace search_vector GENERATED with trigger-based approach (write amplification) - Add mission hierarchy FK indexes (milestones, slices, mission_features) - Use rowToTask for proper jsonb deserialization in audit-ops - Fix getLastMessageForSessions with DISTINCT ON (O(sessions) not O(n*d)) - Add statement_timeout/idle_in_transaction_session_timeout to pool - Log healSchemaDrift ALTER failures instead of swallowing - Wrap schema baseline apply + bookkeeping INSERT in transaction
Collaborator
Author
Re-review of
|
Collaborator
Author
Re-review of
|
Collaborator
Author
Follow-up: PG-backend runtime crash + analytics 500 (commit af3d0db)Found while smoke-testing the embedded-Postgres default on a sandboxed instance: Fixes1. Fatal: agent-log flush killed the process. The agent-log buffer flush/append path dereferenced the SQLite-only 2. 3. Same class, found by sweep: Verification
Review notesRan a high-effort multi-agent code review on the diff. Addressed the confirmed correctness findings (silent deployments-catch now logs; redundant incidents COUNT dropped) and documented the one accepted tradeoff: in PG mode the agent-log flush skips the secondary deleted-task filter, but the primary purge happens at delete-time under lock (both backends) and |
Collaborator
Author
CI coverage for the regressed surfaces (commit 6b3d509)Correcting my earlier note: the PG tests do run in CI — the blocking gate provisions Postgres and runs a curated Added
Proven red→green: forcing the SQLite path in backend mode fails the flush assertion; the shipped guard passes. (Uses reserved task ids so per-task JSONL dirs don't collide under the harness's |
…QL backend Add scripts/lib/backend-db.mjs (boots the backend via core's startup factory, stages the baseline migration SQL into core dist) and port reimport-tasks, reconcile-leaked-soft-deletes, recover-stale-blocked-by, audit-task-id-collisions, and audit-branch-cross-contamination off node:sqlite / the sqlite3 CLI. Each script now splits pure planning (unit-tested with plain rows, no DB fixture) from thin PG I/O. reimport-tasks now rides TaskStore.reconcileOrphanedTaskDirs (FN-6783) instead of a raw INSERT. Delete obsolete SQLite-only one-offs: replace-seeded-workflow-prompts (its WORKFLOW_STEP_TEMPLATES source was removed by the graph-workflow cutover), replace-ws006-prompt, and backfill-fn-5696-feature-assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… planner-oversight FN-7596–FN-7600, intake-column board fix) Conflict: builtin-workflows.test.ts (deleted in the PG cutover test purge, modified on main). Resolved by keeping main's version and porting its 'store integration' block to the shared PostgreSQL harness (pgDescribe + createSharedPgTaskStoreTestHarness). That port surfaced two cutover regressions now fixed: the backend create paths dropped the task_workflow_selection row when workflowId and enabledWorkflowSteps were submitted together (restores the SQLite-side FNXC:WorkflowCreation 2026-06-28 fix in all three task-creation.ts copies), and backend hydration coerced an explicit empty enabledWorkflowSteps to undefined instead of []. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…aining PG test failures Two product regressions: backend listTasks excluded the log column in slim mode, silently disabling stalledReview badges and timedExecutionMs on board cards (the SQLite path fetched log, derived both, then stripped it); and the fresh-agent-log stall suppression (hasFreshAgentLogActivitySinceTaskUpdate, FNXC:WorkflowLifecycle 2026-07-01) was never ported into the split task-store read paths, so Stalled/Merge-stalled badges showed while a merger was streaming. Test fixes: two half-ported PG stall tests (missing h.store() binding, sync db.prepare remnant -> adminDb update), the pooler-warning test now collects all warnings (the external fixed-schema warning overwrote the capture), and the schema parity count applies the baseline without plugin hooks (81 core project tables). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st debt Rescue the five 2026-06-28 SQLite-cutover quarantine entries with root-cause fixes (no appeasement): mock stores expose getAsyncLayer(); real legacy AgentStore seeding replaced with prototype spies (VAL-REMOVAL-005 removed that runtime); project-store-resolver mocks the createTaskStoreForBackend seam; manual-backup parity tests stub runBackupCommand at the core seam (post-cutover it pg_dumps a live cluster). Remove ledger entries + vitest excludes together. Product fix found during the rescue: POST /api/agents/:id/runs built its AgentStore without the scoped store's AsyncDataLayer on the heartbeat-executor branch and threw in backend mode. CLI debt: project.test.ts mocks createTaskStoreForBackend so task-count and in-flight-agent assertions exercise the factory path; project-context.test.ts moves its three DB-dependent tests to a pgDescribe block with a PostgreSQL-backed CentralCore, and the unregistered-project resolve test points DATABASE_URL at the test cluster instead of leaking an embedded postgres subprocess. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, FN-7591 custom-column moves, FN-7601–7607, OAuth scope fixes) Conflict: store.ts moveTaskInternal (extracted to task-store/moves.ts on this branch, modified inline by main's FN-7591 fix). Resolved by keeping the branch delegate and porting main's fix into moves.ts: a non-legacy source column resolves its move targets from the task's own workflow adjacency instead of the legacy VALID_TRANSITIONS table. Porting main's regression test to the PG harness surfaced two more cutover gaps, both fixed: backend create paths ignored the workflow's manual intake column (resolvedEntryColumn — a Coding (Ideas) task landed in triage instead of ideas, with the wrong bootstrap prompt), and move validation resolved the task workflow IR via the sync resolver, which cannot read the selection row in backend mode and silently fell back to builtin:coding (rejecting every move out of a custom column). Moves now resolve the selection asynchronously. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…fixes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hat-session state, self-heal merged-PR in-review cards) Conflict: chat-project-services.ts — union of main's plugin-runner refresh on cache hit with this branch's backend AsyncDataLayer on the chat AgentStore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict resolutions: - packages/engine/src/agent-heartbeat.ts, executor.ts: adopt FN-7609 approvalDedupeKey param + payload comment from main on the createApprovalRequest signature, keeping HEAD's 'await' (PG await-conversion). Bodies auto-merged to main's approvalDedupeKey/command/cwd context. - packages/cli/src/__tests__/extension.test.ts: union the vitest/fs/path/os imports (HEAD's beforeAll/afterAll etc. for the PG harness + main's mkdir/mkdtemp/rm/tmpdir/relative for the FN-7619 worktree-boundary attach tests); drop unused dirname/fileURLToPath orphaned by the PG harness port. - packages/engine/src/__tests__/agent-tools.test.ts: honor deletion (HEAD decomposed into agent-tools-intake-column/delegation/etc.; main's FN-7611 one-line change is superseded by the dedicated intake-column suite). - packages/core/src/__tests__/mission-store.test.ts: honor deletion (sync SQLite Database body removed in PG cutover; the generated-fix methods live only in the sync store and are a documented unported gap in async-mission-store.ts). main's source fix (reconcileSupersededGeneratedFixFeatures) merged cleanly into mission-store.ts. Verified: pnpm typecheck (30 pkgs) + pnpm test:gate (engine-core 292, core pg-gate 95, ci-shape 63) all green.
Conflict resolution notes: - core/chat-store.ts: ported FN-7628 deleteMessagesFrom/updateMessageMetadata and FN-7631 searchSessionsByMessageContent to the dual-backend async pattern; PG counterparts added to async-chat-store.ts (createdAt+id tiebreak instead of sqlite rowid). - core/store.ts: kept the extracted task-store structure; ported FN-7641 Signature 1 (legacy recovery-rehome adjacency bypass) into task-store/moves.ts and Signature 2 (nodeId='end' finalize-on-proof-or-error pre-lock guard) into task-store/remaining-ops-6.ts. - dashboard/chat.ts + register-chat-routes.ts: awaited the async store surface in the new edit/rewind and content-search paths. - desktop/local-server.ts: kept the PG backend-boot factory, merged upstream's new plugin/provider-seeding imports. - Upstream sqlite-coupled tests deleted per the SqliteFinalRemoval precedent; new FN-7628/ FN-7631 coverage ported to postgres/chat-store-content-search-edit.pg.test.ts and a fake-store chat-manager-rewind-session.test.ts; FN-7641 pg tests fixed to use the shared harness store binding. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- extension.test.ts: seed the two nodeId='end' CLI regressions through the shared PG
harness store instead of a throwaway sqlite TaskStore (sqlite removed on this branch).
- register-task-workflow-routes.nodeid-finalize.test.ts: deleted — sqlite-bound route
slice; the FN-7641 finalize-on-proof contract is covered end-to-end against real PG
in core's task-node-override.pg.test.ts.
- TaskDetailModal.responsive-and-dependencies.test.tsx: neutralize braces inside CSS
comments before rule-block extraction so the FN-7634 comment quoting '.chat-input-row
{ … }' no longer truncates the extracted block (also red on upstream main).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict resolution notes:
- core/store.ts, db.ts, archive-db.ts: kept the extraction/sqlite-stub structure and
ported upstream logic into the branch's homes:
- FN-7658 (gate same-agent duplicate auto-archiving behind autoArchiveDuplicateTasksEnabled)
-> task-store/task-creation.ts.
- FN-7659 (paged newest-first Archived column read) -> new listArchivedTasksImpl in
task-store/archive-lifecycle-2.ts + listArchivedTaskEntriesPage in async-archive-db.ts
(archivedAt DESC, id DESC tie-break) + TaskStore.listArchivedTasks facade; sqlite
ArchiveDatabase gets throwing listPage stub.
- Upstream's sqlite usage_events retention / vacuum bodies stay out (Database is a stub;
PG operational-log pruning remains an explicit deferral in self-healing).
- index.gate.ts (new upstream FN-7667 gate barrel): dropped its db-migrate.js re-export,
which does not exist on this branch.
- Upstream sqlite-coupled tests: kept deleted per SqliteFinalRemoval precedent; ported
replacements: postgres/archive-pagination.pg.test.ts (FN-7659) and a PG-harness rewrite
of cli task-lineage-unlink.test.ts (FN-7661, seeds lineage via createTask source input).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Conflict resolution notes: - CLI commands (task/backup/mcp/memory-backup/settings-import/project/db/pr/branch-group/ research/agent/desktop + project-context): adopted upstream's FN-7704/7731/7734/7738/7740 lock-retry + close-store-on-exit structure, with every cwd-fallback store boot routed through the PG startup factory (createLocalStore/createTaskStoreForBackend) instead of the removed sqlite TaskStore; branch-only resolveAgentStoreBase kept for agent commands (upstream's unused getProjectPath dropped); db.ts keeps the branch's PG VACUUM/ANALYZE. - core: kept stub/extraction structure for db.ts/store.ts and ported upstream logic: FN-7736 pausedReason into pauseTaskImpl, FN-7720 bypassFailedPreMergeReviewStep onto the facade, FN-7791 attachment->artifact bridge + dual-mode artifact-row cleanup into remaining-ops-2.ts, FN-7786 token-usage model snapshot columns into BOTH sync+async workflow/team analytics fetch paths, plugin step-template thinkingLevel passthrough. - FN-7775 chat thinkingLevel: added thinking_level to the PG chat_sessions schema, baseline DDL, EXPECTED_PROJECT_COLUMNS drift self-heal, and async-chat-store create/read/update paths. - engine FN-7750: branch's getBranchGroup is async — made the shared-branch-group auto-merge exemption path async (isLiveSharedBranchGroupMember, allowInReviewMergeProcessing, enqueueEligibleInReviewTasks) and awaited all call sites. - Upstream sqlite-coupled tests deleted per SqliteFinalRemoval precedent; FN-7786 snapshot-pricing regression re-added as a PG test in command-center-remaining-analytics.pg.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e PostgreSQL-only branch - cli test mocks: add the branch's createLocalStore/resolveAgentStoreBase and upstream's asLocalProjectContext/resolveProjectPathOnly/closeProjectStore/createTaskStoreForBackend stubs so the merged command modules load; give research store mocks the ResearchStore prototype for the getSyncResearchStore instanceof gate. - deleted sqlite-bound reproductions with no PG equivalent: real locked-fusion.db retry tests (task-lock-retry real-store describe), fn db vacuum sqlite lock tests (branch uses PG VACUUM/ANALYZE), real-store leak tests (agent-export-lock-retry, git-lock-retry, agent-process-exit), FN-7764/7693 sqlite artifact-route/real-store tests, sqlite db-integrity timer test; FN-7730 durability test trimmed to its resolution seam. - ported coverage: FN-7791 attachment→artifact bridge PG test in store-attachments.pg.test.ts; FN-7786 snapshot pricing PG test (previous commit); store-bypass-review ported to the shared PG harness with the async run-audit reader. - dashboard: stub AWAITING_APPROVAL_PAUSE_REASON/THINKING_LEVELS in chat.test's core mock; fix routes-auth dedupe test stale vs FN-7745 codex seeding (assert no anthropic re-registration instead of zero registerProvider calls). - engine: FN-7787 runtime-config tests seeded via AgentStore prototype spies; removed the sqlite real-store artifact describe from agent-artifact-tools.test.ts; rebuilt stale grok plugin dist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ost writes, first-boot migration, backups) Blocker 1 — recoverStaleTransitionPending threw "SQLite Database is not available in backend mode" on every startup/maintenance sweep. Ported to the async layer (task-store/async-transition-pending.ts); backend-mode moveTaskInternal now also writes the crash-safe transitionPending marker inside the move transaction and clears it post-commit (capacity counting in PG already reads the marker, so it was load-bearing and always absent). Blocker 2 — lost task-field writes (triage's status:"planning" clear never sticking → cards permanently "unplanned", scheduler refuses dispatch). Backend-mode atomicWriteTaskJson/atomicWriteTaskJsonWithAudit previously full-row-upserted the caller's snapshot, clobbering any column another store instance committed in between (sqlite always wrote the changed-column subset). Both paths now compute getChangedTaskColumns against the row read inside the transaction and update only those columns. Data-loss trap — the startup factory documented "auto-migrate" as the upgrade path but never called the migrator, so switching an existing SQLite instance to PG booted an empty database. createTaskStoreForBackend now auto-migrates legacy fusion.db/archive.db/central sources into an EMPTY PostgreSQL database on first boot (loud failure aborts the boot; SQLite files kept as backups; second boot is a no-op). Backups — pg_dump/pg_restore are not bundled with embedded-postgres; PgBackupManager now resolves them from common Homebrew/apt/Postgres.app locations when absent from PATH before falling back to the bare name. Regression coverage: postgres/transition-pending-and-status-clear.pg.test.ts (sweep recovery, marker round-trip, triage status lifecycle, interleaved + cross-instance writers) and a first-boot auto-migration integration test in startup-factory-integration.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Single conflict: upstream's FUX-042 additions to the sqlite-coupled server.test.ts (deleted on this branch per SqliteFinalRemoval precedent) — kept deleted; the reliability-endpoint source changes auto-merged cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gent watch gate) Reviewed the TrinaryCompute/postgres-v057 fork findings against this branch: - (a) recoverStaleTransitionPending PG throw — already fixed here 2026-07-10 (async-transition-pending.ts port, plus marker write/clear in backend moves, which the fork did not cover). - (b) CONFIRMED and applied: createTaskStoreForBackend's projectId branch dropped options.rootDir and re-resolved via CentralCore from process.cwd(), rooting scoped stores at the dashboard dir. Explicit rootDir now wins (projectId-only callers keep the central resolution + its caching). - (c) CONFIRMED and applied: drainMergeQueue bound git cwd to config.workingDirectory (process.cwd() in in-process dashboards) so `git rev-parse refs/heads/fusion/<task>` ran in the wrong repo and merges aborted "branch missing — work appears lost". Now store.getRootDir() with the config fallback for test fakes. - Rough edge: AgentStore.startWatching called the sqlite-only db.getLastModified() gate in backend mode (caught, but noisy + degraded to the 60s sweep); the gate is now sqlite-only and backend mode diffs each poll tick. NOT applied here: the multi-project isolation gap (project.tasks has no project_id; single shared embedded cluster/DB at ~/.fusion/embedded-postgres/default). Confirmed real on this branch — the fork's Approach A (partition tasks/archived_tasks by project_id and thread the filter through claim/scheduler/merge/list/SSE/id-sequence) is the right shape and is in progress on their fork; adopting it should come as its own reviewed change, not a side effect of this commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nodes in PG mode connect to the same shared PostgreSQL database, so
replicating settings between nodes over the mesh is redundant and can only
introduce churn/clobber against the shared rows. In backend mode:
- POST /api/mesh/sync ignores inbound settings payloads (diagnostic emitted)
and never includes settings in its response.
- PeerExchangeService force-disables settings gossip regardless of options.
- GET/POST /nodes/:id/settings{,/push,/pull,/sync-status} and
POST /settings/sync-receive answer 409 with code
"settings-sync-disabled-postgres" and an explanatory message.
- useNodeSettingsSync treats that 409 as a quiet steady state: stops polling
the node, renders no sync chips, raises no error banner; explicit push/pull
clicks surface the server message.
Provider AUTH sync (/nodes/:id/auth/sync, /settings/auth-receive,
/settings/auth-export) is intentionally NOT gated: auth material lives in
per-machine auth.json files, not the shared database, so multi-node
credential propagation still needs it. The legacy SQLite topology keeps the
full sync path unchanged.
Covered by 5 new route tests in routes-system.test.ts. (The pre-existing
system-stats failure in that file predates this change — fails identically
at fdb1d88.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gaps Both files were red since the PG cutover for missing mock surface, not real regressions: MockStore lacked getAsyncLayer() (createServer's chat-layer resolution) and the mesh sync response path constructs a REAL AgentStore (sqlite init throws on this branch) — now stubbed; routes-system's createMockStore lacked getAsyncLayer() so system-stats' agent counting silently zeroed. mesh-routes 31/31 and routes-system 27/27 green for the first time since the cutover, which also puts real coverage behind the mesh-sync settings block gated in the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Store.ts ports into the extraction layout: - TaskDetailPromptResilience: best-effort PROMPT.md reads in getTask/ listTasks/searchTasks (reads.ts), archive prompt reader (remaining-ops-5), checkbox reset (remaining-ops-6), updateStep auto-init with cause attribution (merge-queue-ops); updateTask now writes an explicit prompt BEFORE the row commit (atomic) and treats the title/description heading sync as best-effort (task-update.ts) - Video attachments: mime allowlist + 100MB video cap + video artifact bridge (store.ts statics, remaining-ops-2) - updateArtifact: dual-backend port (async updateArtifactRow in async-comments-attachments.ts + updateArtifactImpl in remaining-ops-7), artifact:registered/updated added to TaskStoreEvents; PG regression test added to artifacts-documents-evals.pg.test.ts - mission-autopilot normalizeCompleteMissionAutopilotState ported async Kept ours: db.ts / sqlite-adapter.ts stubs (SqliteFinalRemoval — upstream's v141 sqlite migration and connection-reopen healer target the removed runtime); sqlite-coupled test files stay deleted per cutover precedent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…onversation cap) - readLiveTaskRows: column filters and pagination pushed into SQL — WHERE column [NOT] = ?, ORDER BY (created_at, numeric id suffix) matching the JS comparator, LIMIT/OFFSET — so paginated listTasks stops scanning and hydrating the entire live task table per board read; backend listTasks drops its client-side filter/slice. - getConversation (async + sqlite): capped to the most recent 200 messages by default (options.limit overrides), oldest-first order preserved — the CLI chat no longer loads unbounded history into AI context per exchange. - Regression tests: store-list.pg.test.ts pins SQL-page == old JS-page (numeric id tiebreak, offset windows, column-filter composition); satellite-db-injected-stores pins the cap window + ordering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
Closing the 2 open review findings (commit 6186a96)Both remaining perf items from the reconciliation review are now fixed:
Gates green: PG gate 23 files / 98 tests, engine-core 294, core typecheck clean. |
Conflicts were confined to the compound-engineering plugin's session store: upstream's plan-handoff claim (atomic ce_plan_handoff_claims transaction in create/delete) is kept and rebased onto this branch's nullable-db backend seam (this.db → syncDb() local). The two sqlite-harness plugin test files stay deleted per cutover precedent (upstream's claim-coverage additions rode on the removed makeHarness). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gsxdsm
added a commit
to TrinaryCompute/Fusion
that referenced
this pull request
Jul 12, 2026
…sync removal + main v0.57.x updates) Single conflict: readLiveTaskRows — combined the base's SQL-side column filter/ORDER BY/LIMIT/OFFSET (PR Runfusion#1793 perf fix) with this branch's taskProjectScope isolation operand; both now fold into the same WHERE. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uster (task project_id, Approach A) Squashed net change of TrinaryCompute/postgres-v057 rebased onto feature/postgres so the PR is a single linear commit (GitHub rebase-merge cannot replay the original merge-commit history). - project_id partition key on project.tasks / project.archived_tasks (schema, baseline DDL, EXPECTED_PROJECT_COLUMNS self-heal), stamped on every insert/upsert via the layer-bound projectId - taskProjectScope folded into every scan: readLiveTaskRows (composing with the SQL column filter + pagination), counts, merge-lease candidate scan, search, allocator - per-project config rows (project_id-keyed upserts) + test-harness seed - startup factory binds the AsyncDataLayer to options.projectId; first-boot SQLite auto-migration scopes its emptiness check per-project and stamps project_id on migrated rows - ProjectManager: live global-concurrency listener (concurrency:changed) applied to the shared semaphore immediately Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…2007 review P1) archive.archived_tasks is one shared table across every project on the embedded cluster, and the archived board read it unfiltered — project A's archived list/count surfaced project B's rows. Now: - project_id column on archive.archived_tasks (schema + baseline DDL + index), stamped by both archive writers from the bound layer projectId - listArchivedTaskEntriesPage / getArchivedRowCount / listArchivedTasks / filterArchived / searchArchivedTasks(+Like/Tsvector) take an optional projectId and filter to it; the archived-board impl threads layer.projectId; id-keyed get/delete stay unscoped (globally unique ids) - drift self-heal generalized to schema-qualified entries so existing databases gain archive.archived_tasks.project_id on boot - regression test in store-list.pg.test.ts (gate) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate storage from SQLite to PostgreSQL — full dashboard cutover
Migrates Fusion's storage layer to the embedded PostgreSQL
AsyncDataLayer(the default backend) and completes the satellite-store + feature cutover so every dashboard and Command Center surface works in PG mode.Status — every surface works in embedded-PG mode
Verified live against a running embedded-Postgres dashboard (all 200, zero 5xx) and gate-tested (23 files / 94 tests on embedded PG; core/engine/cli/dashboard typecheck clean).
Approach
Each satellite store gets an
Async<Store>wrapper exposing the sync store's method names over the existingasync-*-store.tshelpers;get<Store>Store()returns aSync | Asyncunion; consumersawait(harmless on sync), and engine/CLI paths that can't convert useinstanceof Syncgraceful fallback. Analytics aggregators branch on"ping" in dbOrLayerto run schema-qualified raw SQL overproject.*(snake_case) in PG. Executors/orchestrators/autopilot are await-converted to drive the union store; the async store wrappers extendEventEmitterso SSE live-push fires in both backends.Not-yet-ported capabilities degrade gracefully (never 500) and are individually called out in commits.
Rebase note
Branch is rebased onto v0.50.0 (latest release). A final rebase onto bleeding-edge
mainis deferred to integration time — the migration restructuredstore.ts(extracted intoremaining-ops-*modules) whilemainkeeps developing it inline, so the tip rebase needs a careful manualstore.tsmerge rather than an auto-resolve.Residual Review Findings
Multi-agent code review of the PostgreSQL satellite-store ports (U1–U5) applied 3 safe fixes (see
fix(review): apply autofix feedback). The following are real but gated — recorded here as follow-up work rather than auto-applied. All are SQLite→PostgreSQL concurrency/atomicity regressions: the sync stores were immune only by SQLite's single-writer, single-threaded-handler execution; the async ports open multi-await read-modify-write windows. Reachability is low today because the execution engines that generate concurrent same-run mutations (insight run executor, research orchestrator/dispatcher) areinstanceof-gated to sync mode in PG. No process-crash class survived (all engine fallbacks correctly guard the sync store).appendResearchEventdual-write is non-atomic (packages/core/src/async-research-store.ts, corroborated: adversarial + reliability). Theresearch_run_eventsinsert (own transaction) and therun.eventsjsonb update are separate writes — a crash between them, or two concurrent appends, splits the table count from the jsonb array. Fix: perform the seq-insert and the jsonb update in onelayer.transactionImmediate.async-research-store.tspersistResearchRun/updateResearchStatus). ConcurrentPATCH /runs/:id/status+POST /runs/:id/eventscan revert a terminal run torunningby overwriting the whole row, bypassing the transition guard. Fix: scoped columnUPDATEs with aWHERE status …guard, or optimistic version column.updateResearchRun/updateInsightRunread-then-write TOCTOU — concurrent PATCHes last-writer-wins on the lifecycle merge. Fix:SELECT … FOR UPDATE/ enclosing transaction.upsertRun/createRunOrThrowConflictcheck-then-create race (async-insight-store.ts) — two callers can each create an "active" run. Fix: partial unique index on(projectId, trigger) WHERE status IN ('pending','running').createResearchRetryRunreturn-value divergence — sync returns the pre-updatequeuedsnapshot; async returns the reloadedretry_waitingrun (persisted state is identical). Pick one side for cross-backend parity.getMissionWithHierarchy/getMissionHealthN+1 fan-out — O(milestones×slices) sequential round-trips hold one pool slot per request; can starve the pool for large hierarchies. Fix: batched/joined reads.MissionStore.Out of scope (deferred): AI run execution (insight/research) + mission autopilot + live SSE mission events remain sync-gated/degraded in PG mode.