v4.1 Dashboard & Observability

.github/dependabot.yml

@@ -4,19 +4,28 @@ updates:
     directory: "/"
     schedule:
       interval: "weekly"
-    open-pull-requests-limit: 5
+      day: "monday"
+    open-pull-requests-limit: 10
+    labels:
+      - "dependencies"
+      - "python"
     groups:
       minor-and-patch:
         update-types:
           - "minor"
           - "patch"
+
   - package-ecosystem: "github-actions"
     directory: "/"
     schedule:
       interval: "weekly"
+      day: "monday"
     open-pull-requests-limit: 5
+    labels:
+      - "dependencies"
+      - "github-actions"
     groups:
       minor-and-patch:
         update-types:
           - "minor"
-          - "patch"
+          - "patch"

.github/workflows/ci.yml

@@ -22,8 +22,8 @@ jobs:
       PYTHON_KEYRING_BACKEND: keyring.backends.null.Keyring
 
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
         with:
           python-version: "3.13"
       - name: Install

.github/workflows/codeql-analysis.yml

@@ -20,19 +20,21 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        language: [ 'python' ]
+        include:
+          - language: python
+            build-mode: none
 
     steps:
     - name: Checkout repository
-      uses: actions/checkout@v2
+      uses: actions/checkout@v6
 
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v1
+      uses: github/codeql-action/init@v4
       with:
         languages: ${{ matrix.language }}
-
-    - name: Autobuild
-      uses: github/codeql-action/autobuild@v1
+        build-mode: ${{ matrix.build-mode }}
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v1
+      uses: github/codeql-action/analyze@v4
+      with:
+        category: "/language:${{matrix.language}}"

.github/workflows/gitleaks.yml

@@ -0,0 +1,30 @@
+# NOTE: requires gitleaks/gitleaks-action to be allowlisted under
+# Settings -> Actions -> General -> Actions permissions (repo currently
+# restricts to selected/GitHub-owned actions only, or "Allow all actions").
+# Otherwise the run ends in startup_failure with zero jobs.
+name: gitleaks
+
+on:
+  push:
+    branches: [master, dev]
+  pull_request:
+    branches: [master, dev]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  scan:
+    name: gitleaks
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+      - uses: gitleaks/gitleaks-action@v3
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/release-please.yml

@@ -0,0 +1,28 @@
+# NOTE: requires googleapis/release-please-action to be allowlisted under
+# Settings -> Actions -> General -> Actions permissions (repo currently
+# restricts to selected/GitHub-owned actions only, or "Allow all actions").
+# Otherwise the run ends in startup_failure with zero jobs.
+#
+# Operator note: this workflow authenticates with the default
+# secrets.GITHUB_TOKEN. GitHub suppresses pull_request-triggered checks
+# (CI/CodeQL) on PRs opened by the default GITHUB_TOKEN, so if branch
+# protection on master later requires those checks to pass, the
+# release-please PR will need a fine-grained PAT / GitHub App token
+# instead of GITHUB_TOKEN to trigger them normally.
+name: release-please
+
+on:
+  push:
+    branches: [master]
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  release-please:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: googleapis/release-please-action@v5
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}

.planning/.continue-here.md

@@ -1,79 +1,149 @@
 ---
-context: research
-milestone: v3.0 Resilience + Ecosystem
-workflow: new-milestone
-stage: research-synthesis-incomplete
+context: default
+phase: none (milestone v4.2 complete + archived; post-milestone UAT audit session)
+task: none
+total_tasks: 0
 status: paused
-last_updated: 2026-06-07T08:13:27.961Z
+last_updated: 2026-08-01T23:44:34.447Z
 ---
 
-# BLOCKING CONSTRAINTS — Read Before Anything Else
-
-> Discovered through failure this session. Acknowledge before proceeding.
-
-- [ ] CONSTRAINT: `.planning/research/SUMMARY.md` is STALE v1 content (Selenium→nodriver, Phase 1). The v3.0 synthesizer errored before writing it. Do NOT read SUMMARY.md as v3.0 research. Mitigation: regenerate it via `gsd-research-synthesizer` from the 4 fresh v3.0 files BEFORE defining requirements.
-
-**Do not proceed until the box is checked.**
-
 ## Critical Anti-Patterns
 
 | Pattern | Description | Severity | Prevention Mechanism |
 |---------|-------------|----------|----------------------|
-| Stale-SUMMARY consumption | SUMMARY.md on disk is the v1 summary, not v3.0. Reading it would ground v3.0 requirements in the wrong milestone. | blocking | First resume action = re-spawn synthesizer to overwrite SUMMARY.md; verify its first lines reference v3.0 / proxy-CAPTCHA-price before using it. |
+| `gsd-sdk query audit-uat` false all-clear | It scans `.planning/phases/`, which is EMPTY once milestones are archived. It reported `total_items: 0` while **80** real outstanding items sat in `.planning/milestones/*-phases/`. Following its "All Clear / stop here" branch would have closed 79 live items on a tooling blind spot. | blocking | Never trust the SDK's UAT all-clear after an archive. Glob `.planning/**/*UAT*.md` and `.planning/**/*VERIFICATION*.md` directly and check the archive dirs yourself. |
+| `rtk pytest` wrong interpreter | On this machine `rtk pytest` / bare `pytest` resolve to a stray Python 3.14 with a corrupted `nodriver` package → ~24 false collection errors | advisory | Always run `.venv\Scripts\python.exe -m pytest -q` (project venv, Python 3.13). `httpx` is now declared in `requirements.txt`, so the manual install is no longer needed. |
+| STATE.md schema drift breaks SDK verbs | STATE.md uses a custom prose schema; `gsd-sdk state.record-session` / `advance-plan` / `roadmap.update-plan-progress` silently no-op or error on it | advisory | Hand-edit STATE.md to match the existing convention when an SDK state verb fails. |
+| Module-scope test imports break under `sys.modules` purge | `tests/test_cli_no_fastapi.py:17-19` deletes `core.cli.web` from `sys.modules` to force its lazy-import guard. Any test file that imports from that module at module scope binds a DEAD module object: tests pass in isolation and FAIL in a full-suite run. Cost a real debug cycle. | advisory | In tests touching `core.cli.web` (or any purged module), resolve the module INSIDE each test body and monkeypatch the module object, not a dotted string path. See `tests/test_cli_web_port.py`. |
+| Uncommitted work in the session scratchpad | The scratchpad directory (`%TEMP%\claude\...\scratchpad`) was wiped mid-session by something outside the session, destroying two git worktrees and their uncommitted edits. | advisory | Never place a git worktree or any uncommitted work under the scratchpad. Use a durable path such as `E:\repos\ShopPyBot-wt-<name>`. |
+| Auto-mode classifier blocks remote mutations | `gh pr merge`, `gh api -X PUT .../branches/master/protection`, and `Write` outside the project dir are all DENIED even with explicit in-chat user approval. | advisory | Do not retry in a different shell to route around it. Hand the operator the exact command to run with the `!` prefix. Also: `gh pr comment --body-file -` (stdin) fails through rtk with "Body cannot be blank" — write the body to a file and pass its path. |
 
 <current_state>
-Mid `/gsd:new-milestone` for **v3.0 Resilience + Ecosystem**. Milestone is started: PROJECT.md has the Current Milestone v3.0 section, STATE.md switched to v3.0 (status=planning), both committed (e159f24). `.planning/phases/` is empty (cleared). `.planning/REQUIREMENTS.md` does NOT exist yet (removed at v2.0 close; new-milestone recreates it).
+Milestone **v4.2 is COMPLETE and archived**; there is no in-progress GSD phase work. This
+session was a post-milestone cross-phase UAT audit that turned up three real defects.
 
-Research-first was chosen. The 4 research agents (Stack/Features/Architecture/Pitfalls) finished and wrote solid v3.0 files. The synthesizer (SUMMARY.md) errored + was stopped — so SUMMARY.md is still the old v1 file.
+Working tree is CLEAN. Four commits are local-only on `chore/v4.0-milestone-close`
+(`d485710`, `0cebc9e`, `70f31c5`, `f883f13`) and have not been pushed.
+
+`master` has moved: PR #13 merged as `4123059`. **CI is green for the first time in the
+project's history** — 755 passed / 2 skipped, identical on `ubuntu-latest` and
+`windows-latest`. Branch protection now requires contexts that actually exist.
+
+PR #12 is rebased, all four checks green, `MERGEABLE` / `CLEAN`, awaiting an operator merge.
 </current_state>
 
 <completed_work>
 
-- v3.0 scope gathered: all four areas, FULL (anti-detection hardening, plugin ecosystem, price monitoring, stability/polish)
-- PROJECT.md updated (Current Milestone v3.0 + Active reqs) — committed e159f24
-- STATE.md switched to v3.0 — committed e159f24
-- Research: STACK.md, FEATURES.md, ARCHITECTURE.md, PITFALLS.md written (committed in this WIP)
+- Cross-phase UAT audit: 80 outstanding items across 66 files, found in the archive dirs
+- Claude-in-Chrome handoff doc authored, delivered, and executed
+- Live browser UAT sweep: **13 PASS / 1 FAIL / 5 BLOCKED / 2 PARTIAL**
+- SEED-003 planted (`d485710`): remote plugin manager, third-party disclaimer, extensibility
+- Port auto-select for `shoppybot web` + 18 tests (`0cebc9e`)
+- UAT verdicts recorded into 11 phase artifacts (`70f31c5`), +246/-40, frontmatter validated
+- STATE.md session record (`f883f13`)
+- **PR #13 MERGED** (`4123059`): CI compile fix + dependency install + `httpx`
+- **PR #12 open and green**: `_register_signals` main-thread guard + 4 regression tests
+- Operator closed: A1 PVR, A2 Actions allowlist, A4 CodeQL, A6 wiki, A7 sign-off, branch protection
+- `31-HV-1` (gitleaks) and `31-HV-2` (CodeQL) flipped to PASS — first greens ever for both
 </completed_work>
 
 <remaining_work>
 
-1. **Synthesize v3.0 SUMMARY.md** — re-spawn `gsd-research-synthesizer` (sonnet) over the 4 files; overwrite the stale SUMMARY.
-2. **Define requirements** (new-milestone.md step 9) — scope each area via AskUserQuestion multiselect; write `.planning/REQUIREMENTS.md` with new REQ-IDs (continue ANTI-04+; new categories e.g. PRICE-/ECO-/STAB-); commit.
-3. **Create roadmap** (step 10) — spawn `gsd-roadmapper`, **start phase numbering at 12**. Research build order: 12 anti-detect infra → 13 price (parallelizable) → 14 ecosystem/metadata → 15 BotService+front-ends → 16 stability. Approve, commit.
-4. **Link pending todos** (step 10.5), then done banner.
+1. Merge PR #12 (operator; classifier blocks agent merges)
+2. Push the 4 local commits on `chore/v4.0-milestone-close`
+3. Decide on PR #11 (264 commits)
+4. Unpause Dependabot, then re-check `31-HV-3` (7 alerts, #6-#12)
+5. Re-test `27-HV-2`, `28-HV-1`, `28-HV-2` running-half once PR #12 lands
+6. `10-HV-3` / `MC-4`: needs a `--host 0.0.0.0` restart
+7. `29-HV-5`: needs a DevTools source breakpoint
+8. `MC-1` / `MC-2`: Windows TTY checks, the only two items with zero prerequisites
+9. Repair 12 drifted UAT docs — three produce FALSE PASSES as written
+10. Fix the `docs/PLUGIN_REGISTRY.md` example row (`hard` vs the inherited `medium`)
 </remaining_work>
 
 <decisions_made>
 
-- v3.0 = all four areas, FULL (major release).
-- Research-first.
-- Only ONE new dep across the milestone: `2captcha-python 2.0.7` (AsyncTwoCaptcha, MIT). Proxy/fingerprint/price/wiki = zero new deps.
-- Walmart/Target use behavioral bot managers (HUMAN/Akamai), NOT CAPTCHA-solvable — keep manual-pause for those.
-- `core/browser_factory.py` = shared hook for proxy/fingerprint/CAPTCHA; ABC `setup()` default calls it; per-plugin migration, zero forced rewrites of the 7 plugins.
-- New secrets via CredentialStore SECRET_KEYS (TWOCAPTCHA_API_KEY, CAPSOLVER_API_KEY, PROXY_USERNAME, PROXY_PASSWORD); CAPTCHA opt-in default false (like SMS gate); price-drop dedup uses SEPARATE DB columns from stock dedup.
+- Wiki Plugin Registry published **headers-only**: `last-verified` means confirmed against a
+  live retail site, no plugin ever has been (`19-UAT-1`), so any date would be fabricated
+- **PVR is the sole security-report channel**; no maintainer email published (`32-HV-1`)
+- `_register_signals` fixed with a **main-thread guard**, not a widened `except`: signal
+  disposition is process-level and main-thread-only on both platforms, and the threaded path
+  already has `BotService.stop()` for cooperative cancellation
+- Branch protection requires `test (ubuntu-latest)`, `test (windows-latest)`, `CodeQL`.
+  Kept the stable `CodeQL` aggregate over `Analyze (python)`, which PR #11 renames to
+  `Analyze (python, none)` via `build-mode` — requiring the old name would permanently block
+- `httpx==0.28.1` added to `requirements.txt` rather than a new extra
 </decisions_made>
 
 <blockers>
-- v3.0 SUMMARY.md not generated (synthesizer errored). Workaround: regenerate first thing on resume.
+- **Agent cannot merge PRs or write branch protection** — operator runs them with `!`
+- **`31-HV-3` will not drain** — two causes: remediating bumps live on
+  `chore/v4.0-milestone-close` (needs PR #11), and Dependabot is repo-level PAUSED
+- **PR #11 `mergeStateStatus` reads UNKNOWN** — re-query after PR #12 lands
 </blockers>
 
 ## Required Reading (in order)
-1. `.planning/HANDOFF.json` — structured state mirror of this file.
-2. `.planning/research/STACK.md`, `FEATURES.md`, `ARCHITECTURE.md`, `PITFALLS.md` — the real v3.0 research (NOT SUMMARY.md until regenerated).
-3. `$HOME/.claude/get-shit-done/workflows/new-milestone.md` — resume at step 9.
 
-## Critical Anti-Patterns (do NOT repeat these)
-- Stale-SUMMARY consumption → regenerate SUMMARY.md before reading it (see blocking table above).
+1. `.planning/STATE.md` → "UAT Audit Session — 2026-08-01" — full verdict table, the three
+   defects, operator actions closed, outstanding items, and the documentation-drift list
+2. `.planning/HANDOFF.json` — machine-readable state for `/gsd:resume-work`
+3. `.planning/seeds/SEED-003-*.md` — will surface at the next `/gsd:new-milestone`
+4. Phase artifacts under `.planning/milestones/*/` — each now carries real verdicts with
+   evidence instead of `[pending]`
+
+## The three defects (context for anyone resuming)
+
+None were caught by any test or milestone review. All three were invisible for the same
+reason: nothing exercised them.
+
+1. **Dashboard Start Bot never worked.** `core/orchestrator.py:_register_signals` calls
+   `signal.signal()` off the main thread; `BotService.start()` runs `async_main` in a daemon
+   thread, so it raised `ValueError` at `async_main`'s fifth statement and the bot loop died
+   before plugin setup — while `POST /api/bot/start` still returned 200 and the UI showed
+   nothing. The CLI path masked it by running on the main thread. Fixed in PR #12.
+2. **CI never compiled.** `${{ runner.temp }}` in job-level `env:`, where the `runner`
+   context does not exist. 81 runs, 81 failures, zero jobs scheduled, no logs — it reads like
+   a flaky runner, not a syntax error. Introduced by `0e0e43f`, the only commit that ever
+   touched the file. Fixed in PR #13.
+3. **CI never installed dependencies.** The install step ran only `pip install -e .[web]`,
+   but `pyproject` declares just `platformdirs`; pytest and every runtime dep live in
+   `requirements.txt`, which CI never installed. Fixed in PR #13.
+
+**Consequence worth carrying forward:** v2.0 through v4.2 were all archived under a
+definition of done that included "CI-green." That was never true, and the suite had only
+ever run on one Windows machine. It does pass identically on Linux, so no code defect
+follows — but the claim was unearned, and the same audit trail should not be trusted for
+other "verified" assertions without a spot check.
 
 ## Infrastructure State
-- Branch master, ahead of origin by e159f24 + this WIP commit. Not pushed.
-- Bash tool is bash (use /dev/null, not $null). Commit via `gsd-sdk query commit`. No Claude co-author trailer (user CLAUDE.md).
-- Test baseline (v2.0): 354 passed, 2 skipped — run pytest from repo root.
+
+- **Dashboard server** may still be running in the background on `http://127.0.0.1:8001`
+  (started via `shoppybot web`, auto-incremented off 8000). Kill it when done.
+- **Port 8000** belongs to an unrelated project (`E:\repos\MMFManager` uvicorn backend). Do
+  not evict it; `shoppybot web` now auto-increments.
+- **Seeded test data** is in the real runtime DB at `C:\Users\brand\AppData\Local\shoppybot\`:
+  items "Seed Amazon Item" / "Seed BestBuy Item", one confirmed order
+  `112-3456789-0000001`, 24 Amazon price points, and ~612 seeded log lines in today's log.
+  **This is fixture data, not real activity.** Clear it before any production use.
+- **Worktree** at `E:\repos\ShopPyBot-wt-sig` (branch `fix/signal-handler-main-thread`).
+  `git worktree remove` it once PR #12 merges. `E:\repos\ShopPyBot-wt-cifix` can go now —
+  PR #13 is merged.
 
 <context>
-Everything is healthy except the one synthesizer error. The research is strong and converges on a clean 12→16 build order. Resume is mechanical: regenerate SUMMARY → requirements scoping (per-area multiselect) → roadmapper (start phase 12) → approve/commit.
+The session started as a routine "check UAT status" and the tool said there was nothing to
+do. That was wrong, and chasing why produced everything else. The pattern worth repeating:
+when an automated all-clear disagrees with a documented backlog, verify the tool's scope
+before believing it.
+
+Three of the defects found were only reachable by actually running things — the dashboard in
+a real browser, the workflow on a real runner. Static review and a green local test suite had
+been consistent with all three being fine for months.
 </context>
 
 <next_action>
-Start with: re-spawn `gsd-research-synthesizer` (model sonnet) to overwrite `.planning/research/SUMMARY.md` from STACK/FEATURES/ARCHITECTURE/PITFALLS, verify its content is v3.0 (mentions proxy/CAPTCHA/price), then continue new-milestone.md step 9 (Define Requirements).
+Ask the operator to run:
+  ! gh pr merge 12 --repo thezoid/ShopPyBot --squash --delete-branch
+
+Then offer to push the 4 local commits on `chore/v4.0-milestone-close`, and pull PR #11's
+merge-readiness now that branch protection requires satisfiable contexts.
 </next_action>