From 12d123f52363fc3708727d0957e494d8d9420da1 Mon Sep 17 00:00:00 2001 From: Alex Barclay Date: Sat, 25 Jul 2026 12:07:04 -0600 Subject: [PATCH] docs(handoff): diagnose the e2e database-locked failure blocking v1.0.0 Brief for the successor picking up the only known functional defect, plus the STATUS.md state-summary rewrite and session-log entry that point at it. Records the two user decisions from this session: hold the tag and GitHub Release until e2e is green, and the successor works autonomously. Also flags that the local v1.0.0 tag points at ba82ca8, which predates the fresh-clone fix and must be moved before it is ever pushed. --- docs/STATUS.md | 14 +++- .../2026-07-25-e2e-database-locked.md | 78 +++++++++++++++++++ 2 files changed, 90 insertions(+), 2 deletions(-) create mode 100644 docs/handoffs/2026-07-25-e2e-database-locked.md diff --git a/docs/STATUS.md b/docs/STATUS.md index 59b6950..5713c88 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -7,11 +7,13 @@ ## State summary -**Version:** 1.0.0 · **Branch:** `main` · **Status: complete for now (feature-frozen).** +**Version:** 1.0.0 (untagged) · **Branch:** `main` @ `74cadcf` · **Status: feature-frozen, one open defect blocking the release.** + +> **Active handoff:** [`docs/handoffs/2026-07-25-e2e-database-locked.md`](handoffs/2026-07-25-e2e-database-locked.md) — the `e2e` job is red with `database is locked`. The release is deliberately held until it is green. Read that brief before picking this up. AppForge v1.0 is the finished form of what this project set out to prove: a **parallel, MCP-coordinated multi-agent orchestration engine** in which a real MCP state server and a pool of independent OS worker processes drive a product idea through a six-phase dependency graph (Clarify → Design → Code → Test → Deploy → Iterate), with human approval gates and automatic budget-driven model downgrade. -The engine is done, tested, documented, and published under MIT. There is no in-flight work and no next phase queued. Further work would be enhancement, not completion. +The engine is done, tested, documented, and published under MIT. No new capability is planned — but v1.0.0 is **not yet tagged or released**: the `e2e` CI job fails with `database is locked`, and the user's decision this session was to hold the release until that is resolved. The `v1.0.0` tag exists locally at `ba82ca8` and **points at the wrong commit** (it predates the fresh-clone fix `7bfa00d`); it must be moved before it is ever pushed. ### Readiness @@ -62,6 +64,14 @@ None required — the project is feature-frozen at 1.0. If it is picked up again ## Session log +### 2026-07-25 — handoff written; release held on the `e2e` defect + +- PR #11 merged; `main` @ `74cadcf`. `backend`, `frontend`, and `validate-config` are green — `e2e` is not. +- Wrote [`docs/handoffs/2026-07-25-e2e-database-locked.md`](handoffs/2026-07-25-e2e-database-locked.md) for whoever picks up the `database is locked` diagnosis. It carries the leading (unverified) hypothesis: `backend/main.py:99` passes a constant `data/web.db` into every `start_run`, so two concurrent runs put two OS processes on one SQLite file, which the store's in-process `asyncio.Lock` cannot serialise. +- **User decisions this session:** hold the tag and GitHub Release until `e2e` is green; the successor works autonomously. +- Repo hygiene: 8 stale agent worktrees and their branches removed, 9 merged local and 4 merged remote branches deleted. +- **State delta:** v1.0 work fully landed on `main`, with a single named blocker and a written brief standing between it and a tagged release. + ### 2026-07-25 — fix: the engine could not start on a fresh clone `Store.connect()` called `aiosqlite.connect(db_path)` without creating the directory holding the file. sqlite does not create missing intermediate directories, and the default paths (`data/engine.db`, `data/web.db`) live under `data/`, which is gitignored and therefore absent on any fresh clone or CI runner — so the engine died with `OperationalError: unable to open database file`. diff --git a/docs/handoffs/2026-07-25-e2e-database-locked.md b/docs/handoffs/2026-07-25-e2e-database-locked.md new file mode 100644 index 0000000..73d770c --- /dev/null +++ b/docs/handoffs/2026-07-25-e2e-database-locked.md @@ -0,0 +1,78 @@ +# Handoff — diagnose the `database is locked` failure keeping the `e2e` CI job red + +**Written:** 2026-07-25 · **Branch:** `main` @ `74cadcf` · **Session:** `e9c40800-fa25-4c60-ac39-6ccfa51a6080` + +## ⏳ In flight + +No long-running local operation. One remote job is running: + +- **What:** CI run [`30168784245`](https://github.com/adbarc92/appforge/actions/runs/30168784245) on `main` @ `74cadcf` (the PR #11 merge). +- **Check:** `gh run view 30168784245 --json status,conclusion,jobs` +- **Expected:** `backend (3.11)`, `backend (3.12)`, `frontend`, `validate-config` **pass**; **`e2e` fails** with `database is locked`. That is the known state, not a surprise — it is the work below. +- **If `backend` or `frontend` fails instead:** something regressed in the merge; that takes priority over e2e. + +## Goal + +Get the `e2e` job green so `v1.0.0` can be tagged and released. AppForge is otherwise feature-frozen at v1.0 — this is the **only known functional defect**. + +## State + +- **Status doc:** [`docs/STATUS.md`](../STATUS.md) — canonical and current. Read it first. +- **Done:** PRs #9, #10, #11 all merged. `main` @ `74cadcf` has the v1.0 rename, working `appforge` CLI, corrected dependencies, and the fresh-clone DB-directory fix (`7bfa00d`). +- **Working tree:** clean. +- **⚠️ The `v1.0.0` tag is LOCAL ONLY and points at the WRONG commit** — `ba82ca8`, which predates `7bfa00d` and therefore *cannot start on a fresh clone*. It must be moved to the release commit before pushing. Never push it as-is. + +## Successor autonomy + +**`autonomous`** — confirmed by the user this session. Diagnose, implement, and open a PR without checking in. Use judgment on the spec rewrite-vs-retire question below. + +## Successor's next action + +Reproduce locally, then diagnose: + +```bash +cd e2e && npx playwright test # boots both servers itself +``` + +Expected failure, seen in CI: + +``` +RuntimeError: claim_next_task failed: Error executing tool claim_next_task: database is locked +``` + +If it does **not** reproduce locally, it is timing-sensitive — CI runners are slower. Widen the window rather than dismissing it (more workers, artificial delay, or run under `--repeat-each`). + +### Leading hypothesis — UNVERIFIED, confirm before acting + +Every web run shares **one** database file. [`backend/main.py:99`](../../backend/main.py#L99) passes a constant `db_path` (`data/web.db`) into `start_run()` on *every* `start_project` event. Each `start_run` spawns its own state server plus 4 worker processes ([`backend/engine/run.py:50`](../../backend/engine/run.py#L50)). + +So if two runs are ever alive at once — two specs in sequence, or a previous run's server not yet torn down — **two separate OS processes write the same SQLite file**. The store's "single writer" guarantee ([`backend/engine/store.py:1`](../../backend/engine/store.py#L1)) is an `asyncio.Lock`, which serialises only *within* one process and does nothing across processes. + +Both `e2e/tests/phase3.spec.ts` and `e2e/tests/phase4.spec.ts` start a project, which fits. + +**Already ruled out:** a missing `busy_timeout` pragma. Python's `sqlite3.connect()` applies a 5s busy timeout by default and `aiosqlite` inherits it, so "no timeout configured" is *not* the explanation. Do not spend time there. + +Plausible directions if the hypothesis holds: give each run its own DB file (e.g. suffix `db_path` with the run id), or ensure a run's server is fully torn down before the next starts. Both are design calls — pick with the "single SQLite writer" claim in [`README.md`](../../README.md) in mind, since that claim is load-bearing for the project's story. + +### Second, independent problem + +The Playwright specs were last touched **2026-06-02**, *before* the engine rewrite, and still drive the retired LangGraph chat flow — they assert on `Clarifying question #N` and `Mock PRD`. Even once the lock contention is fixed, they may not pass. Expect two layers here and do not conflate them. + +## Live decisions (settled) + +- **Hold the release until `e2e` is green** — user-confirmed this session. Do not push `v1.0.0` or cut a GitHub Release before then. When it is green: move the tag to the release commit, push it, then create the Release. +- **Successor runs autonomously** — user-confirmed. +- `docs/superpowers/{plans,specs}` keep their `DevTeam.AI` naming on purpose: dated records that quote code verbatim. + +## ⚠️ Open questions (unresolved — do NOT settle implicitly) + +1. **Rewrite or retire the Playwright specs?** If they cannot be made meaningful against the engine's flow, retiring the `e2e` job may be more honest than patching assertions. Retiring a CI job is a user-facing call — surface it rather than deciding quietly. +2. **Two unmerged branches, undecided:** `feat/parallel-mcp-orchestration-engine` (one commit, `2f11513`, a lint/format pass that `main` already supersedes) and `windows-changes` (one commit, `6068628`, "Stash unknown changes" — contents unexamined). Both also exist locally; `windows-changes` is on the remote too. Delete or keep? +3. **Local branch `fix/create-db-parent-dir`** is merged and its remote is gone — safe to delete, not yet done. +4. **`CancelledError` traceback on CLI teardown** — cosmetic (exit 0), logged in `docs/STATUS.md`. Worth folding into this work, or leave for later? + +## Notes + +- Verify with `data/` **absent** (`mv data data.bak`) — it is gitignored, so a green local run can hide fresh-clone breakage. That is exactly how the `7bfa00d` bug reached `main`. +- Full check: `uv run pytest tests/` (155) · `cd frontend && npm test` (28) · `uv run ruff check backend/ tests/` · `uv run black --check backend/ tests/` +- Stale agent worktree scratch was archived to the session scratchpad at `worktree-archive/` (13 files); it ages out with the temp dir and nothing depends on it.