From 2a7a22007cf8fc5534c31e034a3a4560f93622a1 Mon Sep 17 00:00:00 2001 From: Sarthak Agrawal Date: Wed, 2 Sep 2026 21:51:59 +0530 Subject: [PATCH] chore: stage consolidated verification evidence 4/4 --- evidence/security/osv-baseline-2026-08-31.md | 70 ++++++ .../trivy-config-qualification-2026-08-31.md | 45 ++++ ...pple-container-qualification-2026-08-31.md | 60 +++++ .../cross-review-benchmark-2026-09-02.json | 93 ++++++++ ...native-agent-island-settings-2026-09-02.md | 58 +++++ .../native-application-service-2026-09-01.md | 54 +++++ .../native-cross-review-2026-09-02.md | 109 +++++++++ .../native-history-roots-2026-09-02.md | 45 ++++ .../native-hosted-qualification-2026-09-02.md | 110 +++++++++ .../native-memories-2026-09-02.md | 57 +++++ .../native-onboarding-2026-09-01.md | 56 +++++ .../native-ops-status-2026-09-02.md | 54 +++++ ...native-package-qualification-2026-09-01.md | 217 ++++++++++++++++++ .../native-qa-workspace-2026-09-01.md | 52 +++++ .../native-release-readiness-2026-09-02.json | 105 +++++++++ .../native-release-readiness-2026-09-02.md | 78 +++++++ .../native-repository-query-2026-09-01.md | 133 +++++++++++ ...ive-review-intent-diagnostic-2026-09-01.md | 58 +++++ .../native-surface-parity-2026-09-01.md | 75 ++++++ .../native-unpack-scan-parity-2026-09-01.md | 73 ++++++ .../native-usage-windows-2026-09-01.md | 57 +++++ ...e-watcher-live-qualification-2026-09-01.md | 63 +++++ .../stryker-accounting-oracle-2026-08-31.md | 30 +++ .../tool-collector-foundation-2026-08-31.md | 60 +++++ 24 files changed, 1812 insertions(+) create mode 100644 evidence/security/osv-baseline-2026-08-31.md create mode 100644 evidence/security/trivy-config-qualification-2026-08-31.md create mode 100644 evidence/verification/apple-container-qualification-2026-08-31.md create mode 100644 evidence/verification/cross-review-benchmark-2026-09-02.json create mode 100644 evidence/verification/native-agent-island-settings-2026-09-02.md create mode 100644 evidence/verification/native-application-service-2026-09-01.md create mode 100644 evidence/verification/native-cross-review-2026-09-02.md create mode 100644 evidence/verification/native-history-roots-2026-09-02.md create mode 100644 evidence/verification/native-hosted-qualification-2026-09-02.md create mode 100644 evidence/verification/native-memories-2026-09-02.md create mode 100644 evidence/verification/native-onboarding-2026-09-01.md create mode 100644 evidence/verification/native-ops-status-2026-09-02.md create mode 100644 evidence/verification/native-package-qualification-2026-09-01.md create mode 100644 evidence/verification/native-qa-workspace-2026-09-01.md create mode 100644 evidence/verification/native-release-readiness-2026-09-02.json create mode 100644 evidence/verification/native-release-readiness-2026-09-02.md create mode 100644 evidence/verification/native-repository-query-2026-09-01.md create mode 100644 evidence/verification/native-review-intent-diagnostic-2026-09-01.md create mode 100644 evidence/verification/native-surface-parity-2026-09-01.md create mode 100644 evidence/verification/native-unpack-scan-parity-2026-09-01.md create mode 100644 evidence/verification/native-usage-windows-2026-09-01.md create mode 100644 evidence/verification/native-watcher-live-qualification-2026-09-01.md create mode 100644 evidence/verification/stryker-accounting-oracle-2026-08-31.md create mode 100644 evidence/verification/tool-collector-foundation-2026-08-31.md diff --git a/evidence/security/osv-baseline-2026-08-31.md b/evidence/security/osv-baseline-2026-08-31.md new file mode 100644 index 00000000..de6cc638 --- /dev/null +++ b/evidence/security/osv-baseline-2026-08-31.md @@ -0,0 +1,70 @@ +# OSV offline baseline — 2026-08-31 + +This is a qualified repository-security baseline, not a product verdict and not +an allowlist. It records what OSV-Scanner observed so remediation can be +measured without hiding existing findings. + +## Reproduction + +- Source revision: `225480e5b6a018014bef1c190ed8ac3914a472e7` +- Scanner: OSV-Scanner `2.5.1` +- Command: `pnpm quality:vulnerabilities` +- Network during scan: disabled +- Warm scan duration: 8.942 seconds +- Machine output: ignored `artifacts/tooling/osv/results.sarif` +- Receipt: ignored `artifacts/tooling/osv/receipt.json` + +The command exits `0` when clean, `1` when findings exist, and `2` for an +operational failure. Database refresh is deliberately excluded from the scan. + +## Database identity + +| Ecosystem | Bytes | SHA-256 | +|---|---:|---| +| crates.io | 3,415,816 | `e276fb0061eefcb63ce5ba63b6888640ccc809dc195a698af7b7dbaaeaa9a02c` | +| Go | 11,481,601 | `49b2410b903ae009b3a89ab9be5245d531d0c176b6809931f2182b5ee6bc1204` | +| npm | 221,667,534 | `d89fbb49609224a0ecdb5cd14d6b874ab105197a5e83543f78450bab406cceb8` | + +## Findings + +- 35 affected package versions across two lockfiles +- 52 advisory/package matches +- 49 unique primary advisory IDs in OSV JSON +- 48 unique SARIF rules after alias normalization +- SARIF severity by unique rule: 0 critical, 11 high, 18 medium, 2 low, + 17 without a numeric severity +- Root `pnpm-lock.yaml` and the Go module were clean + +`docs-site/pnpm-lock.yaml` contains 17 affected package versions and 33 +advisory/package matches. They are transitive dependencies of the single direct +docs-site dependency, Blume `1.0.4`. Upgrading Blume is a production/build +dependency change and requires owner approval. + +`apps/desktop/src-tauri/Cargo.lock` contains 18 affected package versions and 19 +advisory/package matches. Most are unmaintained GTK3 bindings reached through +Tauri's Linux target graph. `event-listener` is reached through the Linux +notification stack. The `unic-*` crates are reached through +`urlpattern -> tauri-utils` and are not classified as Linux-only. These are +reachability notes, not suppressions. + +## Gate decision + +Do not add a second vulnerability scanner or silently baseline these findings. +First test the Blume update and bounded Cargo lockfile remediation, then repeat +this exact offline scan. CI enforcement should follow remediation so the gate +does not normalize known high-severity results. + +## Approved maintenance follow-up + +At exact revision `855202998b56c1658b9decda22298a1b63fb5caf`, after updating +Blume 1.0.4 to 1.5.3 and event-listener 5.4.1 to 5.4.2, the same database set +and offline runner completed in 9.745 seconds with 39 result instances and 36 +normalized SARIF rules. This is a reduction from 52 advisory/package matches, +not a clean result and not an allowlist. + +The independent docs-site production audit still reports 12 high, 7 moderate, +and 1 low advisory. Current Blume transitively retains those build/documentation +paths, and several have no resolution through the current direct version. The +repository-wide root production audit remains clean because the docs site has +its own lockfile; both facts must stay visible until issue #195 completes +reachability and upstream/remediation review. diff --git a/evidence/security/trivy-config-qualification-2026-08-31.md b/evidence/security/trivy-config-qualification-2026-08-31.md new file mode 100644 index 00000000..ea433ef7 --- /dev/null +++ b/evidence/security/trivy-config-qualification-2026-08-31.md @@ -0,0 +1,45 @@ +# Trivy configuration-scan qualification — 2026-08-31 + +## Scope + +This trial asks whether Trivy's embedded misconfiguration checks add a useful +repository lane without downloading databases or overlapping the offline OSV +scanner. It does not qualify Trivy vulnerability scanning or product bundling. + +## Tool and safety flags + +- Tool: Trivy 0.74.0 +- License: Apache-2.0 +- Command lane: `trivy config` +- Explicit controls: `--disable-telemetry`, `--skip-version-check`, and + `--skip-check-update` + +Debug output confirmed that the notification/version request was skipped, no +downloadable checks were loaded, and 563 embedded checks were used. This is +command-level evidence, not packet-capture proof of zero outbound traffic. + +## Results + +The unbounded repository scan found only a Dockerfile inside +`docs-site/node_modules/yaml-language-server` and reported two findings against +that dependency-owned file. After excluding dependency, build, target, and +artifact directories, Trivy detected one file: + +`apps/desktop/tests/fixtures/warm-verification/differential-runtime-qualification-current.json` + +It classified that product test fixture as CloudFormation, reported 24 passing +checks and zero failures, and found no first-party Dockerfile, Terraform, +Kubernetes, Helm, Ansible, Azure ARM, or CloudFormation surface to protect. + +The SARIF 2.1.0 envelope itself is structurally complete. In JSON debug output, +Trivy also includes repository URL, branch, commit, author, and committer +metadata, which is acceptable for this public-repository trial but must be +considered before any private-code product use. + +## Decision + +Trialled, not wired. A permanent lane would currently scan dependency-owned +files or misclassify a qualification fixture while protecting no supported +first-party infrastructure configuration. Re-evaluate when the repository adds +a real supported IaC or container surface. Keep all three network-suppression +flags mandatory in any future trial. diff --git a/evidence/verification/apple-container-qualification-2026-08-31.md b/evidence/verification/apple-container-qualification-2026-08-31.md new file mode 100644 index 00000000..c947b003 --- /dev/null +++ b/evidence/verification/apple-container-qualification-2026-08-31.md @@ -0,0 +1,60 @@ +# Apple Container qualification — 2026-08-31 + +## Scope + +This receipt qualifies Apple's `container` CLI as an external-prerequisite +sandbox candidate on one supported development host. It does not add a product +dependency, bundle the runtime, or approve a release architecture. + +## Identity + +| Item | Observed identity | +|---|---| +| Host | macOS 27.0 build 26A5421a, arm64 | +| Installer | `container-1.3.1-installer-signed.pkg` | +| Publisher SHA-256 | `a7c1b9d7927d30875f2f6c7bd1d0cb06c2daa6ca57ce9e90a5144e898fdf54a8` | +| Signature | Developer ID Installer: Apple Inc. - Containerization (UPBK2H6LZM); trusted, notarized, timestamped | +| CLI | 1.3.1, release commit `a9a62e2` | +| API server | 1.3.1, release commit `a9a62e28f6beb88940122a3d7b286f2d5ae8053a` | +| Default kernel | Official Kata kernel 3.32.0 | +| Trial image | `alpine:3.22`, observed Alpine 3.22.5, local digest prefix `14358309a308` | + +The package digest matched the release publisher value before installation. +`pkgutil --check-signature` and `spctl` both accepted the downloaded package. +Installation required the owner's administrator authorization. The first +service start separately downloaded and verified the 664.3 MB default kernel. + +## Measured trial + +- First image/init-image run: 20.56 seconds wall time, including registry fetch + and unpack; the kernel had already been provisioned. +- Warm cached no-op run: 0.61 seconds wall time. +- Limits exercised: 1 CPU and 256 MB memory. +- Isolation exercised: an internal network plus `--no-dns`, read-only root, + read-only bind mount, and all Linux capabilities dropped. +- The controlled network probe failed, host-only environment marker was absent, + `/Users` and `/root/.ssh` were absent, and writes to both the mounted workspace + and `/root` failed with a read-only-filesystem error. +- `--rm` teardown left zero containers and zero local volumes. The cached image + and init image occupied 1.45 GB and were intentionally retained. + +The trial network was deleted after use. No host credential, home, SSH, cloud, +or production path was mounted or inspected. + +## Failed requirement: caller-owned path containment + +The CLI accepted a controlled bind source containing `..` when that source +resolved to an existing sibling fixture. Apple Container therefore provides +mount mechanics, but it does not enforce CodeVetter's workspace-root policy. +Any adapter must canonicalize both the allowed root and requested source, +reject sources outside the allowed root before process launch, pass the +canonical source to the CLI, and cover symlink and time-of-check/time-of-use +cases. The product must not treat the CLI's mount validation as containment. + +## Decision + +Keep Apple Container as the measured external-prerequisite candidate, not a +bundled dependency. Its isolation controls and warm-start result are promising, +but path containment must be owned by CodeVetter. Issue #197 remains open for +the CLI-versus-Containerization-versus-libkrun architecture comparison, idle +resource measurement, signing/notarization analysis, and an adapter design. diff --git a/evidence/verification/cross-review-benchmark-2026-09-02.json b/evidence/verification/cross-review-benchmark-2026-09-02.json new file mode 100644 index 00000000..66bd3fdb --- /dev/null +++ b/evidence/verification/cross-review-benchmark-2026-09-02.json @@ -0,0 +1,93 @@ +{ + "schema_version": "codevetter.cross-review-benchmark-summary/v1", + "recorded_at": "2026-09-02T15:17:50.495Z", + "source": { + "runner_revision": "017985904f89ebcb4c73fe00147e0bb4fde53def", + "engine_revision": "f812ba1c8d5d349bc9a38e2ebde9f03d54509444", + "binary_sha256": "113e0a3fa419a4487af0cec8515d9f0fc9534d906c9f0f5b6ea127cd03e518e0", + "corpus": "benchmarks/public-catch-rate", + "cases": 27, + "expected_labels": 29, + "task": "Review this exact change for correctness, security, reliability, and maintainability defects." + }, + "scoring": { + "status": "human_reviewed", + "deterministic_proposal": "same source path, label line within five lines, and narrow defect-type keywords", + "human_correction": { + "case_id": "py-path-traversal", + "reviewer": "codex", + "finding": "Untrusted report name permits arbitrary local file disclosure", + "reason": "The core claim and source identify the labeled traversal defect even though the narrow mapper keyword was absent." + }, + "confirmed_single_reviewer_miss": { + "case_id": "ts-dead-code", + "reviewer": "codex", + "label": "unused-helper-function", + "severity": "low" + } + }, + "reviewers": { + "claude": { + "caught": 29, + "expected": 29, + "findings": 134, + "strict_precision": 0.21641791044776118, + "recall": 1.0, + "f1": 0.35582822085889565, + "total_duration_ms": 2387162, + "mean_duration_ms": 88413, + "usage_available_cases": 0 + }, + "codex": { + "caught": 28, + "expected": 29, + "findings": 46, + "strict_precision": 0.6086956521739131, + "recall": 0.9655172413793104, + "f1": 0.7466666666666666, + "total_duration_ms": 2675832, + "mean_duration_ms": 99105, + "usage_available_cases": 0 + }, + "cross": { + "caught": 29, + "expected": 29, + "findings": 99, + "strict_precision": 0.29292929292929293, + "recall": 1.0, + "f1": 0.45312500000000006, + "total_duration_ms": 5063091, + "mean_duration_ms": 187522, + "usage_available_cases": 0, + "finding_classes": { + "corroborated": 16, + "claude_only": 63, + "codex_only": 1, + "conflicting": 19, + "rejected": 0, + "stale": 0, + "unresolved": 0 + } + } + }, + "execution": { + "completed_cases": 27, + "scored_incomplete_receipts": 0, + "transient_incomplete_attempts": 1, + "retry_case": "py-insecure-deserialization", + "raw_artifact_bytes_approximate": 1153434, + "temporary_repositories_retained": false + }, + "decision": { + "default_strategy": "claude_unchanged_for_compatibility", + "efficiency_candidate": "codex", + "cross_review": "optional_high_recall", + "reason": "Cross-review recovered one low-severity label over Codex but added 53 findings and about 88 seconds per case." + }, + "limitations": [ + "The corpus is synthetic and single-file; it does not establish real-repository or real-PR quality.", + "Strict defect-only precision penalizes process findings and plausible additional defects outside the narrow ground truth.", + "Provider usage was absent from every pass summary, so observed cost is unavailable.", + "Reviewer agreement is review coverage and never executable proof." + ] +} diff --git a/evidence/verification/native-agent-island-settings-2026-09-02.md b/evidence/verification/native-agent-island-settings-2026-09-02.md new file mode 100644 index 00000000..cf3fe4be --- /dev/null +++ b/evidence/verification/native-agent-island-settings-2026-09-02.md @@ -0,0 +1,58 @@ +# Native Agent Island settings parity — 2026-09-02 + +## Verdict + +The Agent Island configuration contract is transferred. Rust, the +`codevetter settings` CLI, the native Settings desk, and the retained supervised +helper use the same 12 non-secret preference keys, defaults, and bounded option +sets. Agent Island remains opt-in and off by default. + +The live runtime is not transferred. The new Evidence Workbench does not launch +the helper, inspect live session content, speak updates, preview real provider +output, or action provider requests. Agent and MCP surfaces have no Agent Island +authority. The native desk therefore labels configuration as live and runtime +transfer as pending. + +## Canonical preference contract + +| Key | Default | Allowed values | +| --- | --- | --- | +| `native_agent_island_enabled` | `false` | boolean | +| `native_agent_island_speech_muted` | `false` | boolean | +| `native_agent_island_speak_completion` | `true` | boolean | +| `native_agent_island_speak_attention` | `true` | boolean | +| `native_agent_island_speak_failure` | `true` | boolean | +| `native_agent_island_speech_volume` | `0.8` | `0.5`, `0.8`, `1` | +| `native_agent_island_speech_rate` | `0.48` | `0.4`, `0.48`, `0.56` | +| `native_agent_island_speech_cooldown` | `30` | `15`, `30`, `60` seconds | +| `native_agent_island_quiet_start` | off | off, `20`, `21`, `22`, `23` | +| `native_agent_island_quiet_end` | off | off, `6`, `7`, `8`, `9` | +| `native_agent_island_codex_voice` | empty | at most 256 non-control characters | +| `native_agent_island_claude_voice` | empty | at most 256 non-control characters | + +The receipt schema remains `codevetter.native-settings/v1`. Unknown keys, +invalid options, duplicate keys, partial Agent Island contracts, and projected +`github_token` values fail closed. No session, prompt, output, command, path, +provider response, credential, or voice sample enters the preview. + +## Qualification + +- An isolated-app-data CLI smoke listed exactly 12 `agent_island` rows, saved + only `native_agent_island_enabled=true`, returned that exact `saved_key`, and + continued to declare `github_token` excluded. +- Rust unit coverage verifies the 12-row contract, save round trip, and the + 256-character voice-identifier bound. +- `pnpm test:native` passed 76 Swift package tests with no failures or skips in + 29.8 seconds; the native Debug app then compiled in 2.2 seconds. +- Swift rejects a schema-v1 receipt missing even one Agent Island preference. +- Deterministic 2560x1600 dark and light renders were inspected. The light pass + found and fixed inherited dark text inside the always-black status capsule. + +| Render | SHA-256 | +| --- | --- | +| `settings-agent-island.png` | `df36cb66ed120f7eac2e0387379484e54fdf67e99f1a68e0daa5a820227bf158` | +| `settings-agent-island-light.png` | `2073326d72cfd6fee7a1ed64074663388f2f1381d11c53ee8a5610c4398e0799` | + +No helper, installed application, foreground automation, provider process, +network listener, release, signing, notarization, or deployment action was +started by this qualification. diff --git a/evidence/verification/native-application-service-2026-09-01.md b/evidence/verification/native-application-service-2026-09-01.md new file mode 100644 index 00000000..4a6b5f89 --- /dev/null +++ b/evidence/verification/native-application-service-2026-09-01.md @@ -0,0 +1,54 @@ +# Native verification application-service qualification + +Date: 2026-09-01 +Scope: Review plan, execute, progress, cancellation, and terminal receipt transport + +## Result + +Native Review and `codevetter check` now enter one Tauri-independent Rust +application service instead of independently assembling the verification +lifecycle. One bounded request id correlates: + +- `codevetter.verification-command/v1` input; +- monotonic `codevetter.progress/v2` events; +- request-scoped `codevetter.verification-cancel/v1` termination; and +- the distinct canonical preflight or final local-check receipt. + +The Rust service still delegates source identity, target discovery, execution, +persistence, verdicts, and limitations to the existing authoritative engines. +Swift owns process supervision and rendering only. + +## Executable proof + +- a real clean-clone `codevetter check --preflight --request-id + native-service-live-smoke --json` returned `ready`, preserved the exact + request id, and resolved immutable base/head SHAs; +- the same command against the dirty migration worktree failed closed before + producing a receipt; +- Rust service tests cover bounded/generated request ids, progress ordering, + cancellation identity, and a real two-commit Git preflight through the + service; +- all 28 CLI tests cover parsing, progress-v2 serialization, shared-fixture + receipt/exit parity, and existing + output/exit semantics; +- Swift tests prove exact CLI arguments, matching progress and receipt + decoding, foreign-progress rejection, foreign-cancellation refusal, + matching cancellation without a receipt, mismatched-receipt rejection, + 1,000-event throughput, and worker crash recovery; +- the final native background gate passed 61 Swift tests and the macOS Debug + application build; +- the final all-target Rust regression passed 1,077 tests with 31 intentional + ignores and no failures; and +- the fresh Release host and package qualifier passed at + `artifacts/native-package/qualification-XrpqmY/CodeVetter.app`. + +No test failure is waived by this receipt. + +## Authority boundary + +MCP remains read-only and does not gain Review execution or cancellation +authority. It can now retrieve one already-persisted canonical local-check +receipt by bounded run id inside its authorized repository scope. Cancellation +is a supervised transport terminal action rather than a persisted engine +event. This contract does not imply a daemon, concurrent multi-run scheduler, +release authorization, or Tauri retirement. diff --git a/evidence/verification/native-cross-review-2026-09-02.md b/evidence/verification/native-cross-review-2026-09-02.md new file mode 100644 index 00000000..20d0a660 --- /dev/null +++ b/evidence/verification/native-cross-review-2026-09-02.md @@ -0,0 +1,109 @@ +# Native independent cross-review qualification — 2026-09-02 + +## Claim + +Unreleased CodeVetter source can request independent sequential Claude and +Codex review through native Review or `codevetter check --agent cross`, persist +one deterministic composite receipt, and inspect it through repository-scoped +read-only MCP. Agreement remains review coverage and never executable proof. + +## Contract evidence + +- `codevetter.cross-review/v1` binds both passes to the same immutable target. +- A separate SHA-256 coordinator policy binding covers the repository, exact + range, task, and original runtime context. A second SHA-256 identity covers + the ordered unit plan while deliberately excluding executor-specific unit + fingerprints. Any mismatch fails before a composite is persisted. +- Codex receives the original repository/task/runtime context, never Claude + output. +- Reconciliation keys only on exact path, positive line, and source anchor. + Similar titles at different sources remain separate. +- Unique, corroborated, and severity-conflicting qualified findings remain + explicit. Missing identity, target mismatch, missing executor, interruption, + or incomplete readiness discards every composite finding and reports an + incomplete receipt. +- Per-pass evidence retains reviewer identity, review identifier, duration, + qualified findings, readiness, manifest qualification diagnostics, and any + available usage. The current review contract does not expose provider raw + candidates or measured usage, and the receipt states that limit rather than + manufacturing values. +- MCP returns the persisted canonical receipt without review-start, + cancellation, provider, or credential authority. + +## Checks + +- Four Rust reconciliation tests cover corroborated/unique/conflicting output, + title non-merging, fail-closed missing anchors and partial execution, and + immutable-target mismatch. +- The CLI parser test preserves `--agent cross`; existing single-review + defaults remain Claude. +- The MCP lifecycle fixture preserves cross-review strategy and both reviewer + identities through the read-only projection. +- 83 Swift package tests pass, including exact cross-review command projection + and dark/light evidence rendering. +- The 35-state owner packet includes `review-cross-review.png` and + `review-cross-review-light.png`, with manifest-bound hashes and dimensions. +- Hosted XCUITest includes the strategy picker. The first isolated run exposed + an unreliable segmented-control `isSelected` assertion after the strategy + contract had already updated; the test now waits on the resulting contract + text and remains pending on the final-head draft-PR run. + +## Provider smoke + +One isolated current-binary smoke used the public `ts-sql-injection` synthetic +case with a clean temporary Git repository and separate temporary app data. +Claude and Codex both independently source-qualified the labeled injection at +line 13. The final receipt was `needs_attention`, carried policy binding +`4f8414c0ce6a19cd56ae15a0730087b047edf1ee69a6606b2982b77c95bb6fb3`, and +unit-plan identity +`a797e10aa0307e225585939f472eb94860f50e0d85376957a4102fa0e9f86fc5`. + +Claude completed in 64,891 ms with three qualified findings. Codex completed in +47,315 ms with one. Sequential cross-review completed in 112,209 ms and +reconciled one corroborated labeled issue plus one Claude-only missing-module +finding. Under the benchmark's strict one-label accounting, this single case is +100% recall and 50% precision for the composite, versus 100%/33% for Claude and +100%/100% for Codex. Provider usage was not reported, so cost is unavailable. +The source-only fixture had no executable correctness or performance target; +the overall receipt preserved those limitations rather than converting review +agreement into a pass. + +## Provider-backed corpus comparison + +The full 27-case public caught-bug corpus ran through the real `--agent cross` +path. Every case used a fresh temporary Git repository, separate app data, the +same generic task, and the exact `HEAD^..HEAD` change. Claude and Codex received +the original context independently. Temporary repositories were removed after +their canonical receipt was saved; the ignored raw run is 1.1 MiB. + +| Strategy | Labels caught | Findings | Strict precision | F1 | Mean review time | +| --- | ---: | ---: | ---: | ---: | ---: | +| Claude | 29/29 (100%) | 134 | 21.6% | 35.6% | 88.4 s | +| Codex | 28/29 (96.6%) | 46 | 60.9% | 74.7% | 99.1 s | +| Claude + Codex | 29/29 (100%) | 99 | 29.3% | 45.3% | 187.5 s | + +The deterministic mapper initially proposed two Codex misses. Human review +confirmed that `py-path-traversal` was caught by the finding “Untrusted report +name permits arbitrary local file disclosure”; its wording omitted the +mapper's narrow keyword despite matching the source and core claim. The one +confirmed Codex miss was the low-severity unused helper in `ts-dead-code`. +Claude and the union caught it. Thus cross-review adds one of 29 labels over +Codex, but adds 53 findings and about 88 seconds per case. All findings beyond +the unique labeled defects count against strict defect-only precision, +including process findings and additional plausible defects not represented in +the intentionally narrow ground truth. + +One `py-insecure-deserialization` attempt returned an incomplete two-pass +receipt and was not scored. A fresh isolated retry completed; the runner now +retains incomplete receipts, checkpoints after every case, retries once, and +can resume or rescore without repeating completed provider calls. Provider +usage was absent in all 54 completed pass summaries, so observed cost remains +unavailable. The committed summary is +[`cross-review-benchmark-2026-09-02.json`](./cross-review-benchmark-2026-09-02.json). + +Decision: preserve the existing Claude single-review default for compatibility; +the corpus identifies Codex as the stronger efficiency candidate for a separate +default-policy decision. Keep cross-review optional when an operator explicitly +values maximum recall over latency and review burden. This corpus does not +justify automatic dual review, and agreement still does not replace executable +correctness or performance evidence. diff --git a/evidence/verification/native-history-roots-2026-09-02.md b/evidence/verification/native-history-roots-2026-09-02.md new file mode 100644 index 00000000..f56749fd --- /dev/null +++ b/evidence/verification/native-history-roots-2026-09-02.md @@ -0,0 +1,45 @@ +# Native Codex history-root qualification + +Date: 2026-09-02 +Surface: native macOS Usage settings, CLI, and generated capability registry + +## Result + +Additional Codex history recovery now uses one Rust-owned +`codevetter.history-roots/v1` contract. Native Settings and +`codevetter history-roots` can list, add, and remove the same bounded roots. +Selecting a `sessions` or `archived_sessions` directory normalizes to its +canonical Codex home before persistence. + +Rust rejects unrelated directories, relative or malformed stored paths, more +than 16 roots, and combined add/remove requests. Duplicate canonical roots are +idempotent. Each receipt reports directory availability without reading +transcript content. Removing a root changes future discovery only and never +deletes provider files. + +The generated capability registry marks native UI and CLI read/execute +authority as available. MCP and agent authority remain unavailable because +local history-path mutation is not required for evidence inspection. + +## Executable proof + +- two focused Rust service tests pass normalization, deduplication, removal, + transcript non-disclosure, unrelated-directory rejection, and malformed + stored-path rejection; +- the focused CLI parser test passes read/add/remove exclusivity; +- two focused Swift tests pass exact CLI argument/schema validation and + offscreen native rendering; +- the complete headless native lane passes 80 Swift tests with zero failures + and the Debug macOS application build through XcodeBuildMCP; +- a temporary end-to-end CLI smoke normalized a selected `sessions` directory, + reported active and archived availability, and returned zero configured + roots after removal. + +## Boundaries + +- The active `CODEX_HOME` remains automatic and is not duplicated in this + additional-root receipt. +- Adding or removing a root does not start Usage reconciliation. +- No transcript body, credential, provider token, or secret enters the receipt. +- This qualification does not cover live provider telemetry, production + signing, installation, updater cutover, or Tauri retirement. diff --git a/evidence/verification/native-hosted-qualification-2026-09-02.md b/evidence/verification/native-hosted-qualification-2026-09-02.md new file mode 100644 index 00000000..8727abdc --- /dev/null +++ b/evidence/verification/native-hosted-qualification-2026-09-02.md @@ -0,0 +1,110 @@ +# Native macOS hosted qualification + +Date: 2026-09-02 + +## Verdict + +The native Evidence Workbench passed its first complete isolated GitHub-hosted +qualification at source commit +`824a9e8bb92feea0de834e876eec64217c68c254`. [GitHub Actions run +33609288529](https://github.com/Codevetter/codevetter/actions/runs/33609288529) +completed successfully on the repository's arm64 `xcode-27` runner path. + +This qualifies an unsigned preview candidate. It does not authorize or prove a +production release, replacement of the installed Tauri app, or owner visual +acceptance. + +## Hosted gates + +| Gate | Observed result | +| --- | --- | +| Existing product CI | Linux lint, code health, typecheck, unit and automation tests, CLI/MCP sidecars, CLI artifact, Vite desktop build, MCP safety, T-Rex contracts, and browser tests passed | +| Native behavior | 81 Swift tests passed in 26.294 seconds with zero failures | +| Native build | Debug app and coverage-free Release app built successfully | +| Native interaction | 9 XCUITests passed with zero failures in 113.009 seconds | +| Owner packet | 33 offscreen states rendered and passed manifest/gallery validation | +| Package | arm64 `CodeVetter.app` 1.11.0 (11100), ZIP, DMG, and dSYM produced | +| Package inspection | Deep signature, host identity, version, companions, Hardened Runtime, and execution authority passed | +| Readiness | 7 of 16 checks passed; 9 production gates correctly remained blocked | + +The hosted interaction suite opened the actual application and checked primary +workspace navigation, command-menu navigation, compact-window navigation, +appearance changes, all Settings destinations, Review-to-Testing handoff, +repository selection, and toolbar actions. It ran only on the isolated hosted +desktop; no local application was opened or installed. + +## Large-receipt performance + +The fixed decode gate is 25 ms p95 and the fixed native host-render gate is +150 ms p95. The hosted run retained each full canonical receipt while bounding +initially visible rows. + +| Surface | Fixture | Decode p95 | Render p95 | +| --- | --- | ---: | ---: | +| Repo Unpack | 100 snapshots, 700 graph nodes, 1,000 tree rows | 12.490 ms | 96.536 ms | +| Usage | 365 daily periods, 100 sessions | 15.467 ms | 105.896 ms | +| Performance | 100 observed evidence rows | 4.711 ms | 121.065 ms | +| Testing | 100 journeys, 100 changed paths | 0.966 ms | 55.087 ms | +| Runs | 100 runs, 100 selected responses | 4.818 ms | 64.132 ms | + +These measurements cover receipt decoding and native host rendering on the +hosted runner. They do not measure current-package launch time, window-server +frame pacing, CLI workload time, energy, long-session behavior, or settled RSS. +The retained matched native/Tauri launch and memory comparison therefore +remains historical evidence for its exact earlier packages. + +## Package identity + +The qualifier staged the preview identifier +`com.codevetter.desktop.native-preview` with the distinct +`CodeVetterNative` host executable. Hardened Runtime was enabled, App Sandbox +was intentionally disabled, and the ad-hoc preview disabled Library Validation +because its components have no shared Developer ID team. + +| Artifact | Bytes | SHA-256 | +| --- | ---: | --- | +| Native host | 5,686,272 | `68d73fd1fd7aa7b4c00166470189d2fe6ceafed1df92f0550512e9f02a0f6fd4` | +| `codevetter` | 43,871,088 | `70893c0220543aa82eac3f052b4d3f2ab8ac031a444c27f8ee818b9754c8974c` | +| `codevetter-mcp` | 7,524,752 | `527118074c3cc166b7cfd1e0d28b78caea2f1fc87d020e2975ad838eb1e0783a` | +| `ccusage` | 3,196,064 | `f6f24d12f17c282b04056801355bb46632913daaf67eb48eed0366259dbd7f11` | +| ZIP | 17,398,740 | `b1af8f4fd4b073f96d1ae095a18be71ac87ee7c0a956cfbad6c4eaa4536cc2ba` | +| DMG | 20,088,260 | `43d0586a9849cc875a26c2f6d424cfc8eea1726bde82e5545b218a1833f30638` | +| dSYM DWARF | 30,282,948 | `432cb1a7ff151ccd5314320d9c1f6c405b32c8794917b71b01b4db2bda30dba6` | + +The package receipt itself has SHA-256 +`8921e713b12ed2b863ba02d31f266b0bfcbe6ce1d40db9fd2bfd15de1afd62f0`. +CLI help, MCP bounded usage, ccusage 20.0.20, and the bundled runtime capsule all +returned their expected exit semantics. + +## Visual evidence boundary + +The hosted packet manifest has SHA-256 +`bb7c8bdb2a2a034094f17f8e6adc994f34d20b272300a719b3bd403f6ef5cce5`. +All 33 files match that manifest, and a spot check retained the true-black +canvas, restrained amber actions, dense evidence hierarchy, legible selected +Rubrics state, and bounded large-receipt layouts. + +The hosted PNG bytes are not identical to the earlier committed local packet; +the packets were rendered by different macOS/Xcode environments. This receipt +does not promote either environment to a cross-host pixel oracle. Owner visual +acceptance remains pending. + +## Remaining release gates + +The exact hosted readiness receipt has SHA-256 +`f48c719f7d0001568a2728a32345305dfa6dc4d5204b87eebc6f31265a0c090c` +and correctly reports `shipping_ready: false`. Production still requires: + +- owner-approved production bundle-identifier transfer; +- one Developer ID team for the host and every companion; +- Library Validation in that production-signed app; +- a real HTTPS Sparkle appcast and EdDSA public key; +- archive-bound notarization, stapling, and Gatekeeper acceptance; +- installed upgrade, relaunch, stable-data continuity, and rollback proof; +- exact-package runtime, workload, energy, and long-session comparison where + still required; and +- owner visual acceptance and the explicit Tauri retirement decision. + +The downloaded artifact occupied 140 MiB of ignored workspace storage. The +post-download read-only check reported 124 GiB free. No file was installed, +launched, signed for production, notarized, published, or deployed. diff --git a/evidence/verification/native-memories-2026-09-02.md b/evidence/verification/native-memories-2026-09-02.md new file mode 100644 index 00000000..91c97426 --- /dev/null +++ b/evidence/verification/native-memories-2026-09-02.md @@ -0,0 +1,57 @@ +--- +title: Native memory inspection qualification +description: Evidence for the bounded read-only Memories contract shared by Rust, CLI, and the native macOS client. +--- + +# Native memory inspection qualification + +## Verdict + +Qualified for bounded, read-only list, read, and Git-diff inspection through +the Rust core, `codevetter memories`, and native Settings. This does not grant +memory editing, arbitrary-path access, or agent/MCP content authority. + +## Shared contract + +- Schema: `codevetter.memories/v1`. +- Rust discovers known memory locations and returns only sources that exist. +- Sources are selected by deterministic opaque SHA-256 identity. Receipts do + not expose absolute filesystem paths. +- At most 128 sources, 512 KiB per read, and 120,000 output characters enter a + receipt. Truncation remains explicit. +- Content and Git-diff lines that look secret-bearing are redacted + heuristically. Displayed memory must still be treated as private. +- Swift validates schema, operation, source identity, bounds, display paths, + and mutually exclusive read/diff payloads before rendering. + +## Reproducible verification + +The command smoke used a repository-owned synthetic fixture with isolated +`HOME` and `CODEX_HOME`. It checked 79 supported candidate locations, returned +one existing fixture source, emitted no absolute path, and replaced the fixture +secret line with `[redacted secret-like line]`. No operator memory file was +read by the smoke. + +Focused Rust tests pass for opaque identity, path projection, and content/diff +redaction. The CLI parser test passes for separate list, read, and diff +operations. The current quiet native lane passes 80 Swift package tests with +zero failures plus the Debug macOS build; the suite covers exact CLI arguments, +receipt validation, bounded private rendering, filtering, copying, and the +read/diff boundary. + +## Visual evidence + +- Dark: `evidence/design/native-acceptance-2026-09-01/settings-memories.png`, + SHA-256 `8a39a2affa1e5471277717883752e88761363580bf97c215b5199b05dd8cafb8`. +- Light: `evidence/design/native-acceptance-2026-09-01/settings-memories-light.png`, + SHA-256 `73b4ef2b59a23acb98b1d75d5144cb16999df0763bdee613da06e4eb5afe5e88`. +- Both renders are 2560x1600 and are included in the deterministic 33-state + owner-review manifest. + +## Limits + +Redaction is defensive and heuristic, not a confidentiality proof. The native +surface does not create, edit, or delete memories. The agent and MCP surfaces +cannot read memory contents. Git diff is available only for an admitted source +already tracked by its containing repository. Final owner visual acceptance +and production release gates remain separate. diff --git a/evidence/verification/native-onboarding-2026-09-01.md b/evidence/verification/native-onboarding-2026-09-01.md new file mode 100644 index 00000000..9b44c411 --- /dev/null +++ b/evidence/verification/native-onboarding-2026-09-01.md @@ -0,0 +1,56 @@ +# Native onboarding receipt + +Date: 2026-09-01 +Scope: first-run state, tool readiness, default agent, and product orientation + +## Result + +Native macOS and `codevetter onboarding` now consume one Rust-owned +`codevetter.onboarding/v1` receipt. It reuses the incumbent +`onboarding_complete` preference, so an existing user does not receive a false +first-run experience after migration. Completion transactionally persists only +that shared flag and the already allowlisted `default_adapter` value. + +The receipt reports Codex, Claude Code, and GitHub CLI executable presence. It +does not execute those tools, inspect authentication, read credential values, +or expose resolved filesystem paths. Missing tools remain visible limitations +and never become inferred readiness. + +The native flow has four states: + +1. the execution-backed product standard; +2. bounded local-tool readiness; +3. Codex or Claude Code default-agent selection; and +4. the synchronized app, CLI, and scoped-agent operating model. + +The About desk can reopen the tour without clearing or rewriting the completion +flag. + +## Verification + +- Two Rust service tests prove legacy completion compatibility, credential + omission, declared-adapter validation, and transactional persistence. +- The CLI parser separates read-only inspection from explicit completion and + rejects incomplete or unknown-adapter requests. +- An isolated real CLI smoke observed incomplete, completed, and persisted + inspect receipts without using the live application database. +- Two focused Swift tests prove exact CLI arguments/schema/authority and render + all four states within the 760 by 600 point native window. +- Strict recursive Swift formatting and the background native package/build + gate pass. + +## Rendered evidence + +- Purpose: `evidence/design/native-acceptance-2026-09-01/onboarding-purpose.png` + (`1520x1200`, SHA-256 + `82daad81c0e6f67279aa41e16e181e6bf9e1a2697e196220f6ca5b0ee37ed688`). +- Agent boundary: + `evidence/design/native-acceptance-2026-09-01/onboarding-agent.png` + (`1520x1200`, SHA-256 + `4ae058543eda58a06e2001592647e0bba613eff7d97bf80cabcb9e4d2c3393ea`). + +## Remaining boundary + +This receipt does not inspect provider authentication, migrate credentials, +authorize an agent run, qualify production signing or updates, establish owner +visual acceptance, or permit Tauri retirement. diff --git a/evidence/verification/native-ops-status-2026-09-02.md b/evidence/verification/native-ops-status-2026-09-02.md new file mode 100644 index 00000000..a29e9e6f --- /dev/null +++ b/evidence/verification/native-ops-status-2026-09-02.md @@ -0,0 +1,54 @@ +# Native Ops status parity — 2026-09-02 + +## Verdict + +The read-only Ops status slice is transferred. Native Settings and +`codevetter ops` share the Rust-owned `codevetter.ops-status/v1` receipt for +fixed 7, 30, and 90 day windows. It reports only local configuration presence +and bounded aggregate run evidence. + +This is not full Ops configuration parity. The receipt never returns +credentials or webhook URLs, writes configuration, refreshes provider billing, +sends a webhook, or grants agent/MCP authority. Those operations remain with +the incumbent owner until a separate secure-native contract is qualified. + +## Shared contract + +- `billing` contains only Anthropic and OpenAI configured booleans. +- `webhook` contains only a configured boolean and one normalized + `slack`, `discord`, `generic`, or `unknown` flavor. +- `observability` contains at most eight aggregate task-type rows with + session counts, success/failure counts, rates, and p50/p95 durations. +- `excluded_sensitive_keys` must be exactly `anthropic_admin_key`, + `openai_admin_key`, and `notif_webhook_url`. +- An unavailable local database produces false configuration presence and no + aggregate rows; it cannot manufacture readiness. + +## Qualification + +- Rust tests inserted sentinel credential and webhook values into an isolated + SQLite database and proved neither value appeared in serialized output. +- Rust rejects windows outside 7, 30, and 90 days and normalizes undeclared + webhook flavors to `unknown`. +- An isolated-app-data CLI smoke returned an unavailable-store receipt for a + fresh directory, exactly the three excluded keys, no aggregate rows, and no + configured integrations. It did not create or read the operator's database. +- The current `qualification-5r7JG4` package contains that byte-identical + companion, so the isolated smoke and secret-exclusion proof bind to the same + executable identity. +- Swift validates schema, requested window, row bounds, non-negative counts, + 0--100 rates, webhook flavor, and the exact sensitive-key exclusion set. +- `pnpm test:native:background` passed 80 Swift package tests with no failures + or skips in 22.4 seconds; the native Debug app compiled in 2.5 seconds. +- Deterministic 2560x1600 dark and light renders were inspected. Both preserve + the true-black/warm-light hierarchy, readable aggregates, and explicit + authority-held-back limitations. + +| Render | SHA-256 | +| --- | --- | +| `settings-ops.png` | `309b449bf248c4536c31653a12ce0c38e7be098242d1414982adb0ec450af46c` | +| `settings-ops-light.png` | `cc81a20579d20c6d63d0a5c9e2b5e420f57bd9660591829f0af603e34db110ec` | + +No provider process, network listener, webhook, installed application, +foreground automation, release, signing, notarization, or deployment action +was started by this qualification. diff --git a/evidence/verification/native-package-qualification-2026-09-01.md b/evidence/verification/native-package-qualification-2026-09-01.md new file mode 100644 index 00000000..92372d93 --- /dev/null +++ b/evidence/verification/native-package-qualification-2026-09-01.md @@ -0,0 +1,217 @@ +--- +title: Native macOS package qualification +description: Local evidence for the hardened native bundle, companions, Sparkle embedding, archives, and launch survival. +--- + +# Native macOS package qualification + +## Verdict + +The native packaging mechanism is locally qualified. It is not production +signed, notarized, installed, update-enabled, or approved to replace Tauri. + +The final checked package-only run staged a fresh copy at +`artifacts/native-package/qualification-5r7JG4/CodeVetter.app`; the stable +reproduction command is: + +```bash +pnpm native:build:release +pnpm native:package:qualify +``` + +The repository-owned `pnpm native:build:release` command pins XcodeBuildMCP, +arm64, isolated DerivedData, and coverage-off settings at the workspace command +boundary so Swift package products cannot silently retain test instrumentation. +Its checked default invocation completed in 34.0 seconds, produced a +5,667,872-byte coverage-free host, and `pnpm native:package:qualify` consumes +that default output. The run never +opened, changed, stopped, or replaced +`/Applications/CodeVetter.app`. + +## Package evidence + +| Check | Observed result | +| --- | --- | +| Bundle | `CodeVetter.app`, version 1.11.0, build 11100, arm64 | +| Preview identity | `com.codevetter.desktop.native-preview` | +| Host executable | `CodeVetterNative`, distinct from lowercase `codevetter` on case-insensitive macOS | +| Release authority | Hardened Runtime, no App Sandbox | +| Embedded updater | Sparkle 2.9.6; no `SUFeedURL`; no `SUPublicEDKey`; updater disabled | +| Companions | `codevetter` 43,853,088 bytes; `codevetter-mcp` 7,506,752 bytes; `ccusage` 3,178,064 bytes | +| Runtime capsule | 32 non-test ESM modules under `Contents/Resources/runtime-failure-capsule` | +| Bundle size | 62,060 KiB after companions and Sparkle | +| ZIP | 17,427,235 bytes; SHA-256 `d5dc651c85ac064b932e233bb96d11e18e3f332a8c30693cae1866e7548f98ea` | +| DMG | 20,034,510 bytes; SHA-256 `63f5ad84ce3239b4b9c45ac67ceef6a6c677c498bcbf0c7223bf07f79584fb8a` | +| Deep signature check | Passed after preserving framework symlinks and signing nested Sparkle components before the outer app | +| Release instrumentation | Shipped host contains no LLVM coverage/profile sections, is postprocessed and stripped, and retains a matching adjacent 34,144 KiB dSYM in the isolated build output; the package gate rejects coverage-instrumented hosts | +| Smoke checks | CLI help exit 0; MCP bounded usage exit 1; ccusage 20.0.20 exit 0; runtime bounded usage receipt exit 2 | +| MCP contract qualification | Byte-identical packaged sidecar (SHA-256 `0f000679f481ae313f77fdead93ae79cb00e43f75389f5f221b92c4ca517aee1`); 50 starts and 200 workload rounds; 28 unique tools; strict schemas; read-only annotations; no TCP listeners; 8.52 ms cold-initialize p95, 30.11 MiB ending RSS, 2.64 MiB second-half growth, and all repository budgets passed | +| Fix parity | Packaged CLI exposes confirmed `execute`, `inspect`, and confirmed `discard` operations | +| Repo Unpack parity | Packaged CLI exposes `scan`, `compare`, `export`, `query`, and internal supervised `query-worker`; a package gate requires graph/history domain, rich mode, target, direction, depth, and causal-selector flags | +| Ops boundary | Packaged CLI returns `codevetter.ops-status/v1` from isolated app data, exposes the exact secret-exclusion set, and leaves the fresh directory empty | +| Earlier packaged launch | Five clean, accessibility-confirmed Performance launches; 117,424 KiB median settled process-tree RSS | + +The generated receipt contains SHA-256 identities for each companion and both +archives. It remains under ignored `artifacts/` because it records one local +run directory; this checked note preserves the stable claim and reproduction +boundary. + +## Failures caught by the gate + +The qualifier found and fixed three release-real defects before acceptance: + +1. `CodeVetter` and `codevetter` collide on the default case-insensitive + filesystem. The visible bundle remains CodeVetter while the host executable + is now `CodeVetterNative`. +2. Node recursive copy rewrote Sparkle's relative framework symlinks to source + absolute paths, invalidating its seal. Staging now uses `ditto`. +3. The host initially lacked `@executable_path/../Frameworks`, then ad-hoc + Sparkle embedding hit Library Validation because ad-hoc code has no shared + Team ID. The Release build now has the correct runpath. The artifact-scoped + ad-hoc preview disables Library Validation so local launch can be tested. + Debug uses the same local-only exception for XCUITest; the checked-in Release + entitlement remains empty. + +The last exception is not a shipping design. Production must sign the host, +Sparkle components, and companions with one Developer ID identity and prove +Library Validation remains enabled. + +## Visual inspection + +![Packaged native Performance workspace](../design/native-packaged-performance-dark.png) + +The earlier launch-qualified staged package rendered the intended true-black workbench, amber +evidence hierarchy, native toolbar, split-pane workflow, explicit admission +empty state, and green Rust authority marker. The screenshot is 2184 x 1504; +its SHA-256 is +`855d176aa5d2cb4432404dc997168b58479f3302f9c923ae6ff25f1a0f4e1ec3`. +The current-source `qualification-5r7JG4` rerun rebuilt the Release host after +the isolated fix-attempt engine, native Review receipt, native Repo Unpack scan, +comparison, and export boundaries, persistent macOS repository permission, and +explicit watcher Retry were added. It additionally includes the compact +repository-query worker, native explain/impact/path/causal-trace desk, and the +stdin-first cancellation repair qualified by the 80-test Swift package lane. +It also contains the final true-black hierarchy correction: the canvas and +chrome remain black while working planes use only 1--4% near-black separation, +and the standalone Review proof-map and intent captures now own an opaque +canvas. Both search-only and rich repository-query states are independently +reproducible in the 33-state owner packet, alongside current-tree light +counterparts for Review, Testing, Performance, Runs, history recovery, memory +inspection, Agent Island configuration, and read-only Ops status. Agent Island +configuration now has dark and light evidence +with all 12 shared preferences, a non-activating preview, and the live-config / +pending-runtime boundary. The query worker's shutdown path now +closes stdin, grants a bounded 200 ms termination grace, and uses a final kill +only for its exclusively owned read-only child; its cancellation test requires +settlement within one second. +At the supported 980-point minimum, compact navigation now keeps the selected +workspace label visible while inactive destinations remain icon-only. The +Testing and Performance dark/light rerenders fit without clipping or content +movement, closing the one P2 orientation finding from the 33-state pixel audit; +no P0 or P1 visual finding remains in the packet. +The native Ops desk and `codevetter ops` now share the bounded +`codevetter.ops-status/v1` receipt. It exposes fixed-window local aggregates +and configuration presence while excluding credentials, webhook URLs, provider +calls, webhook sends, writes, and agent/MCP authority. +The retained Tauri startup path now attempts the sanitized WebView-local rubric +handoff until Rust owns a canonical preference. Existing canonical state wins, +invalid legacy state writes nothing, and isolated frontend/Rust tests cover the +bridge; installed custom-pack qualification remains open. +The same Release candidate separates bright action-fill amber from an +appearance-aware evidence foreground and gives success, warning, and failure +states darker light-mode counterparts. The checked semantic tokens meet a +4.5:1 normal-text contrast floor against true black, the warm light canvas, and +white evidence planes. +Its embedded Rust-generated glossary also keeps external collectors honest: +collector execution is available in the CLI, the native workflow remains +planned, and agent authority is unavailable. A deterministic capabilities +render binds that projection to the 33-state owner packet. + +The exact candidate also applies Release-only fat LTO with one Rust codegen +unit and native postprocessing. Relative to `qualification-60JfB0`, the bundle +is 31.6% smaller, the host 77.0% smaller, the CLI 14.6% smaller, the MCP +sidecar 27.5% smaller, the ZIP 18.8% smaller, and the DMG 21.9% smaller. The +retained Tauri comparison is now a 62.4% smaller bundle and a 92.4% smaller +host. This costs materially slower Release linking: the measured CLI and MCP +links took 200 and 159 seconds, while Debug and test profiles remain +unchanged. The exact receipt is +`evidence/performance/native-release-optimization.json`. +The native Usage desk also gained aligned +1w, 30d, 90d, and all-time windows for its chart, totals, models, and sessions, +plus a separate indexed Devin projection that follows the selected window. +Its Usage settings also expose the bounded Rust history-root receipt: selected +Codex session directories normalize to their canonical home, unrelated +directories fail closed, and removal affects future discovery without reading +or deleting transcripts. The new dark and light owner captures caught an +appearance-dependent system button-label truncation; the final text-only amber +action style renders the complete label in both appearances. +Native Review now routes through the correlated verification-command, +progress-v2, request-scoped cancellation, and terminal-receipt application +service. The qualifier then repeated deep-signature, archive, companion, and +rich repository-query CLI smokes. Its separately invoked exact packaged +sidecar smoke proved the 28-tool contract, including read-only persisted +local-check receipt access, without opening a listener. It deliberately did +not launch the app on the operator's active desktop. The earlier matched Release +harness launched `qualification-jUbwnc` five times and confirmed the Performance +surface before every settled sample; the current package-only run does not +supersede that foreground launch evidence or the inspected screenshot. + +The exact current package also contains the read-only Memories parity slice. +Native Settings and `codevetter memories` consume the same bounded Rust receipt; +only existing sources are returned, source identities are opaque, absolute +paths are omitted, and secret-like content and Git-diff lines are redacted +heuristically. Memory editing and agent/MCP content authority remain absent. +It also contains the Agent Island configuration slice: native UI and CLI share +the retained helper's exact 12 non-secret preferences, while helper launch, +live session presentation, speech execution, and provider actions remain +incumbent authority and off by default. + +## Validation receipts + +- The feature-complete Rust lane passed 1,105 tests with 31 explicitly ignored, + zero failures, strict Clippy, and `cargo fmt --check`. +- The native lane passed 80 Swift tests and a fresh Debug application build. +- The retained frontend passed 680 unit tests with one intentional skip, its + separate 20-scenario live warm-verification qualification, package-scoped + TypeScript, and a production Vite build. +- Biome lint, Knip, changed-file complexity, import cycles, clone-regression, + capability sync, docs, package tests (6), release-inspector tests (7), and + the 33-state owner-gallery manifest-sync test passed. +- The cleanup audit correctly requires manual review because dependency files + are part of the migration. Manual inspection found no new direct npm package; + the Cargo change is Release-profile configuration, the feature package has no + external dependency, and the workspace lock pins the one native framework, + Sparkle 2.9.6. The npm audit has one low-severity transitive + `postcss-selector-parser` advisory and no moderate, high, or critical finding. + No dependency upgrade was applied during this qualification. + +## Storage observation + +The latest read-only measurement after the complete background regression run +reports 130 GiB available on the 926 GiB data volume. Ignored local artifacts +are about 6.4 GiB, the Rust target is about 46 GiB, and the machine-wide Xcode +DerivedData directory is about 11 GiB. The earlier bounded workspace report +classified the large findings as review-required and estimated zero +automatically safe releasable bytes. No cleanup or deletion was performed. + +## Remaining release gates + +The read-only +[native release-readiness receipt](native-release-readiness-2026-09-02.md) +now machine-checks these boundaries. The current preview passes 7 of 16 checks +and remains blocked by nine exact production gates: + +- production bundle identifier transfer and owner approval; +- Developer ID signing of the host and companions with one team; +- Library Validation in the production-signed application; +- Apple notarization, stapling, and Gatekeeper acceptance; +- real HTTPS appcast and canonical 32-byte Sparkle EdDSA public key; +- installed incumbent-to-native update, data migration, relaunch, rollback, and + updater-channel proof; +- workload-execution, energy, and long-session comparison beyond the qualified + startup-parity, settled-RSS, and package-footprint claims; +- complete retained feature, accessibility, keyboard, no-confidence, and owner + visual acceptance. + +No release, deployment, installation, production signing, notarization, or +ticket closure was performed. diff --git a/evidence/verification/native-qa-workspace-2026-09-01.md b/evidence/verification/native-qa-workspace-2026-09-01.md new file mode 100644 index 00000000..b39748be --- /dev/null +++ b/evidence/verification/native-qa-workspace-2026-09-01.md @@ -0,0 +1,52 @@ +# Native QA workspace receipt + +Date: 2026-09-01 +Scope: saved journey parity, Playwright spec discovery, and post-fix setup + +## Result + +Native Testing, `codevetter qa`, and the repository-scoped MCP server now +consume one Rust-owned `codevetter.qa-workspace/v1` receipt. The same service: + +- reads repository-scoped saved workflows and targets; +- projects legacy data into a separate native preference without rewriting the + incumbent keys; +- omits credential-bearing storage-state paths and scrubs invalid or embedded- + credential preview URLs; +- leaves arbitrary external-command workflows read-only and refuses to execute + them from native Testing; +- discovers at most 60 repository Playwright specs without running project + code; +- prepares the same pre-fix flow for a deterministic post-fix comparison; and +- never restores preview network consent or starts browser execution. + +Selecting a saved target passes its exact route and goal to `codevetter trex`. +The Rust T-REX core deduplicates that route against changed-path discovery, +keeps the required root smoke, bounds the route portfolio, and records the +selected goal in the executable journey receipt. + +The MCP `qa_workspace_inspect` tool is read-only. Workflow and target mutations +remain explicit native/CLI actions; browser execution remains an explicit +Testing action. + +## Verification + +- 4 focused Rust QA-workspace safety and compatibility tests pass. +- The selected-target route/goal contract test passes. +- The `codevetter qa` parser test passes. +- 6 focused MCP server tests and all 3 stdio boundary tests pass with 28 strict + read-only tools. +- 65 serialized Swift package tests pass, including exact target handoff, + post-fix stale-proof invalidation, and consent reset; the macOS Debug app + build passes. +- Strict recursive Swift formatting passes. +- Offscreen true-black render: + `evidence/design/native-acceptance-2026-09-01/qa-journey-workspace.png` + (`2360x1520`, SHA-256 + `dcb8f5d1ff4669af79e4685ec04c378e446f5f9f95beb478642ab128902574e6`). + +## Remaining boundary + +This receipt does not prove a real application post-fix rerun, authorize a +remote preview, qualify arbitrary external commands or credential migration, +approve the visual design, retire Tauri, or authorize release. diff --git a/evidence/verification/native-release-readiness-2026-09-02.json b/evidence/verification/native-release-readiness-2026-09-02.json new file mode 100644 index 00000000..9b0d9fce --- /dev/null +++ b/evidence/verification/native-release-readiness-2026-09-02.json @@ -0,0 +1,105 @@ +{ + "schema_version": "codevetter.native-release-readiness/v1", + "authority": "read_only_inspection", + "recorded_at": "2026-09-02T06:20:29.880Z", + "status": "blocked", + "shipping_ready": false, + "application": { + "path": "/Users/sarthak/Desktop/fleet/codevetter-native/artifacts/native-package/qualification-5r7JG4/CodeVetter.app", + "bundle_identifier": "com.codevetter.desktop.native-preview", + "version": "1.11.0", + "build": "11100", + "executable": "CodeVetterNative", + "signing": "ad_hoc", + "team_identifier": null + }, + "updater": { + "feed_url": null, + "public_key_configured": false + }, + "checks": [ + { + "id": "qualification", + "passed": true + }, + { + "id": "deep_signature", + "passed": true + }, + { + "id": "production_bundle", + "passed": false + }, + { + "id": "host_executable", + "passed": true + }, + { + "id": "version_identity", + "passed": true + }, + { + "id": "packaged_companions", + "passed": true + }, + { + "id": "hardened_runtime", + "passed": true + }, + { + "id": "developer_id_signature", + "passed": false + }, + { + "id": "consistent_developer_team", + "passed": false + }, + { + "id": "library_validation", + "passed": false + }, + { + "id": "execution_authority", + "passed": true + }, + { + "id": "https_appcast", + "passed": false + }, + { + "id": "sparkle_public_key", + "passed": false + }, + { + "id": "gatekeeper", + "passed": false + }, + { + "id": "notarization", + "passed": false + }, + { + "id": "installed_upgrade", + "passed": false + } + ], + "gatekeeper": { + "accepted": false, + "detail": "/Users/sarthak/Desktop/fleet/codevetter-native/artifacts/native-package/qualification-5r7JG4/CodeVetter.app: rejected" + }, + "blockers": [ + "The production bundle identifier has not transferred to the native app.", + "The application is not signed by a Developer ID Application identity.", + "The host and packaged companions do not share one Developer ID team.", + "Library Validation is disabled in the staged application.", + "A production HTTPS Sparkle appcast is not configured.", + "A production Sparkle EdDSA public key is not configured.", + "Gatekeeper does not accept the staged application.", + "No archive-bound accepted and stapled notarization proof was supplied.", + "No archive-bound production-identity upgrade, relaunch, stable-data continuity, and rollback proof was supplied." + ], + "limitations": [ + "This receipt only inspects supplied local artifacts and proof files.", + "It never signs, notarizes, installs, publishes, enumerates identities, or reads credentials." + ] +} diff --git a/evidence/verification/native-release-readiness-2026-09-02.md b/evidence/verification/native-release-readiness-2026-09-02.md new file mode 100644 index 00000000..8d8409d1 --- /dev/null +++ b/evidence/verification/native-release-readiness-2026-09-02.md @@ -0,0 +1,78 @@ +# Native macOS release-readiness inspection + +Date: 2026-09-02 + +## Result + +The current local native package is **blocked for shipping**. The versioned +[JSON receipt](native-release-readiness-2026-09-02.json) passes 7 of 16 checks +and names nine production-only blockers. This is a read-only preflight, not +release authorization. + +The following local boundaries pass: + +- the exact inspected application is bound to a successful local package + qualification receipt; +- deep strict code-signature verification; +- `CodeVetterNative` host identity and matching version/build metadata; +- the exact `ccusage`, `codevetter`, and `codevetter-mcp` companions; +- Hardened Runtime; and +- the required non-sandboxed local execution authority. + +The package is not shipping-ready because: + +1. it retains the preview bundle identifier; +2. it is ad-hoc rather than Developer ID signed; +3. the host and companions have no one shared Developer ID team; +4. the staged preview disables Library Validation; +5. no production HTTPS Sparkle appcast is configured; +6. no canonical 32-byte base64 Sparkle EdDSA public key is configured; +7. Gatekeeper rejects the staged preview; +8. no accepted, stapled, archive-bound notarization proof was supplied; and +9. no production-identity installed upgrade, relaunch, data-preservation, and + rollback proof was supplied. + +## Reproduction + +From the repository root: + +```bash +pnpm native:release:inspect -- \ + --app artifacts/native-package/qualification-5r7JG4/CodeVetter.app \ + --qualification artifacts/native-package/qualification-5r7JG4/qualification.json \ + --out evidence/verification/native-release-readiness-2026-09-02.json +pnpm test:native-release +``` + +The inspector requires a `codevetter.native-package-qualification/v1` receipt +whose application path is the exact app under inspection. A future +notarization proof must use `codevetter.native-notarization-proof/v1`, report +`accepted`, report a stapled ticket, and bind to the SHA-256 of one qualified +archive. A future installed-upgrade proof must use +`codevetter.native-installed-upgrade-proof/v1`; bind the production bundle, +current version/build, and one qualified archive SHA-256; and explicitly pass +upgrade, relaunch, and rollback. Its nested +`codevetter.native-data-continuity/v1` projection must identify the incumbent +`com.codevetter.desktop` Application Support root and `codevetter.db`, then +prove a non-empty stable-record fingerprint is unchanged after native relaunch +and after rollback. The projection contains counts and SHA-256 digests, not +user content. + +The repository-owned `pnpm native:data-continuity` probe now produces that +nested projection through three fully quiesced, read-only SQLite identity +captures. Its isolated fixture suite proves that new rows are allowed, missing +incumbent rows fail, empty baselines fail, and receipt output excludes message +and preference values. No production database or installed application was +used while qualifying the probe. + +The inspector verifies supplied proof structure and binding; the production +workflow must still establish trustworthy provenance for those proof files. It +does not enumerate signing identities, read credentials, sign, notarize, +install, publish, or alter the installed application. + +## Boundary + +This receipt advances release engineering without exercising release +authority. Owner visual acceptance, production signing/updater inputs, Apple +notarization, installed migration and rollback evidence, and the Tauri +retirement decision remain open. diff --git a/evidence/verification/native-repository-query-2026-09-01.md b/evidence/verification/native-repository-query-2026-09-01.md new file mode 100644 index 00000000..75bfbf49 --- /dev/null +++ b/evidence/verification/native-repository-query-2026-09-01.md @@ -0,0 +1,133 @@ +# Native repository query parity receipt + +Date: 2026-09-01 +Scope: read-only structural and temporal exploration in Repo Unpack +Schema: `codevetter.repo-query/v2` + +## Result + +The native Repo Unpack Graph desk and `codevetter unpack --operation query` +now call the canonical Rust structural-graph and history read services already +used by Tauri and MCP. Structural search, node explanation, bounded impact, +directed path, temporal search, and causal trace are available through the same +receipt. Swift owns input and presentation only. Ranking, traversal, causal +selection, freshness, trust, source identity, bounds, and unavailable coverage +remain Rust-owned. + +The native client now keeps one supervised, read-only JSON-lines worker after a +snapshot opens. Rust retains one canonical search projection and its existing +query index. The first rich graph query upgrades that snapshot in place with +compact traversal edges; only bounded result edges regain full evidence and +source anchors from SQLite. Every request rechecks the latest stored snapshot +identity and live Git freshness, and cancellation drops the process. A CLI +without the worker protocol falls back to the exact supervised one-shot query; +there is no alternate ranking or traversal implementation. + +The receipt carries one canonical repository path, query domain, normalized +query, mode-specific target/direction/depth or history selector, applied limit, +graph and history index status, and exactly one typed result. Both domains fail +closed with `status: unavailable` when their +canonical index is absent. A valid unavailable receipt exits successfully so +the native viewer can render the coverage state instead of replacing it with +a process error. + +## Cross-surface authority + +| Surface | Authority | Qualified behavior | +| --- | --- | --- | +| Rust core | Authoritative read | Reuses `StructuralGraphReadService` and `HistoryReadService`; validates domain/mode fields, one-line identities, depth, and a 1–100 result limit | +| CLI | Read-only projection | Emits JSON or a human summary through `unpack --operation query --query-domain graph|history --query-mode ...` | +| Native | Read-only projection | Validates schema, repository identity, domain, authority, status, and typed result before rendering | +| MCP | Read-only projection | Continues to expose the richer canonical graph and history tools over an explicitly enabled repository scope | + +Native exploration does not add build, backfill, mutation, cleanup, or network +authority. An unindexed repository is not silently queried through a weaker +fallback. Graph topology and qualified causal leads remain navigation evidence, +not executable proof. + +## Executable proof + +- Rust boundary tests cover normalized bounds and the unindexed temporal + fail-closed state. +- CLI parser tests cover the explicit operation, graph/history domain, five + modes, path target, impact direction/depth, causal selector, and rejected + argument combinations. +- Live read-only CLI smoke against the indexed incumbent repository returned + 12 bounded graph matches and one history match. Both receipts reported their + stored indexes as stale relative to the live checkout rather than presenting + the results as current. +- Five independent Release-sidecar runs established the cold-process baseline: + 2,170 ms median graph latency and 820 ms median history latency. The scoped + worker then measured 50.5 ms median graph latency and 36.55 ms median history + latency after preparation, reductions of 97.7% and 95.5%. Background graph + preparation measured 2.49 s. The search-only projection reduced retained + worker RSS from an observed 549.8 MiB full-snapshot prototype to 242.4 MiB. + The [cold benchmark](../performance/native-repository-query-benchmark.json) + and [worker benchmark](../performance/native-repository-query-worker-benchmark.json) + record every sample, method, lifecycle, fallback, and limitation. +- The exact-tree rich-worker qualification measured 36.07 ms graph search, + 35.15 ms explain, 116.55 ms impact, 68.38 ms one-hop path, 32.46 ms history + search, and 32.21 ms causal trace medians after preparation. Search-only RSS + measured 242.6 MiB; compact traversal settled at 307.6 MiB, 39.9% below the + rejected 511.9 MiB full interactive snapshot. Search preparation took 2.45 s + and first traversal upgrade 0.99 s. The + [rich benchmark](../performance/native-repository-query-rich-worker-benchmark.json) + preserves every sample and the empty populated-trace limitation. +- Live read-only CLI smoke against the unindexed migration worktree returned + explicit unavailable receipts for both domains and ran no fallback query. +- The full browser-feature Rust matrix passed: 1,092 tests passed and 31 + intentionally ignored; the two example contract tests also passed. +- The previously load-sensitive review-executor fixture now gives non-timeout + assertions a five-second process-start budget. Its production timeout path + is unchanged and remains covered separately. +- Swift supervised-runner coverage checks all six typed results, exact rich-mode + arguments, persistent reuse, cancellation without partial receipt acceptance, + and clean worker restart. +- Final background qualification exposed a cancellation fixture whose shell + remained blocked on stdin after `SIGTERM`. The supervisor now closes the + scoped worker input before termination; the focused regression passed in + 7.1 s and the complete package rerun left no fixture worker behind. +- XcodeBuildMCP 2.7.0 passed 69 Swift package tests and the native macOS Debug + build with preview bundle identifier `com.codevetter.desktop.native-preview`. + The final post-fix rerun completed the tests in 22.1 s and the build in 8.5 s. +- Repository lint, Knip, changed-file complexity, import cycles, duplication + regression, high-severity dependency audit, capability sync, and docs + validation passed. Two newly reported high-severity transitive Browserslist + advisories were resolved by pinning the existing dependency to patched + 4.28.7; dependency audit now reports one low-severity advisory below the + configured high-severity gate. + +## Visual evidence + +`evidence/design/native-acceptance-2026-09-01/repository-query-evidence-workbench.png` +is a 3040×1960 offscreen dark render from the current tree. Its SHA-256 is +`297d8dbe40f6455cd40b875d42903816bcfecd0eda8455687d853685bd58b442`. + +Visual inspection confirmed the true-black hierarchy, populated snapshot +ledger, amber-only selected domain and Query action, canonical freshness +status, result trust and source identity, node relationship counts, bounded +impact controls, path affordances, and retained snapshot topology. The generic +Fleet web-viewport detector remains inapplicable to this fixed-minimum macOS +surface; no mobile evidence was fabricated. + +## Storage observation + +The final read-only post-qualification measurement reports 162 GiB available +on the 926 GiB data volume (82% used). The Rust target is 39,644,076 KiB +(about 37.8 GiB), artifacts are 3,514,976 KiB (about 3.35 GiB), node_modules +is 655,956 KiB (about 641 MiB), the native Swift package build cache is +674,356 KiB (about 659 MiB), and the shared XcodeBuildMCP cache is 18,547,212 +KiB (about 17.7 GiB). Final Debug/Release qualification increased the Rust +target by about 0.83 GiB relative to the preceding recorded observation. +macOS/APFS reported 145 GiB available in the immediately preceding sample and +162 GiB in this final sample without a cleanup command, so free capacity is a +point-in-time value rather than an attributed product saving. No CodeVetter +app, repository-query worker, or cancellation fixture remained active, and no +manual cleanup or deletion was performed. + +## Remaining boundary + +This receipt does not qualify a populated causal trace against the stale live +index, native source lineage, model synthesis, cleanup, foreground owner +acceptance, signing/notarization, installed update/rollback, release, or Tauri +retirement. diff --git a/evidence/verification/native-review-intent-diagnostic-2026-09-01.md b/evidence/verification/native-review-intent-diagnostic-2026-09-01.md new file mode 100644 index 00000000..e5703bbb --- /dev/null +++ b/evidence/verification/native-review-intent-diagnostic-2026-09-01.md @@ -0,0 +1,58 @@ +# Native Review intent diagnostic receipt + +Date: 2026-09-01 +Scope: deterministic Review intent and recorded synthetic-QA projection + +## Result + +Completed Review evidence now includes +`codevetter.review-intent-diagnostic/v1`. The Rust core owns the projection; +native SwiftUI renders it without recomputing closure or changing the canonical +local-check JSON returned through the CLI and local-agent path. + +The receipt records: + +- the operator-supplied task intent and its source; +- deterministic changed-surface classifications; +- finding, high-risk finding, QA run, QA outcome, artifact, and review-coverage + counts; +- explicit evidence gaps and an ordered intent-to-disposition chronology; +- `evidence_conflict`, `insufficient_evidence`, or + `ready_for_human_disposition` rather than an automatic success claim; and +- the invariant that intent closure always requires a human disposition. + +Native Review adds a dedicated Intent desk and safe Finder reveal actions for +recorded QA artifacts. Repository-relative artifact paths must remain inside +the recorded checkout; absolute paths must still exist. Revealing an artifact +is an explicit user action and does not execute it. + +The Intent and proof-map desks also hand the exact repository and range or pull +request to native Testing. The handoff clears stale Testing proof and prior +network confirmation while preserving the operator's preview field. Testing +continues to own browser execution through the existing `codevetter trex` +contract; Review remains an evidence consumer. + +## Verification + +- 3 focused Rust diagnostic contract tests pass. +- The focused Swift proof-map/intent host-render test passes. +- The focused exact-change Review-to-Testing handoff test passes and proves + that execution consent is not carried across the boundary. +- The serialized 65-test background Swift package gate and macOS Debug build + pass; serial execution prevents AppKit and process-pipe performance gates + from measuring contention created by the test runner itself. +- The full Rust all-target suite passes 1,081 tests with 31 explicitly ignored. +- Strict Swift formatting passes. +- Offscreen true-black render: + `evidence/design/native-acceptance-2026-09-01/review-intent.png` + (`1520x1600`, SHA-256 + `0af6c8f18cb5d25f0649765624a458170a9104e5fbc9f9013c0d01785ba1f67a`). + +## Remaining boundary + +Saved presets/targets, spec discovery, and post-fix rerun setup are now +consolidated into native Testing through `codevetter.qa-workspace/v1`; Review +still only hands over exact change identity. A real saved-flow post-fix rerun, +complete interaction/accessibility qualification, and owner acceptance remain +open. A recorded legacy pass is evidence context, not revision-exact proof, and +cannot close intent by itself. diff --git a/evidence/verification/native-surface-parity-2026-09-01.md b/evidence/verification/native-surface-parity-2026-09-01.md new file mode 100644 index 00000000..02068cc5 --- /dev/null +++ b/evidence/verification/native-surface-parity-2026-09-01.md @@ -0,0 +1,75 @@ +# Native surface parity receipt + +Date: 2026-09-01 +Scope: evidence-scope discovery plus canonical local-check receipt semantics +Fixtures: `apps/desktop/src-tauri/tests/fixtures/surface-parity/evidence-scope-v1.json` +and `apps/desktop/src-tauri/tests/fixtures/surface-parity/local-check-v1.json` + +## Result + +The Rust core, `codevetter scope`, native Swift runner, and packaged MCP +`resolve_evidence_scope` tool consume one repository-owned fixture and preserve +the same schema-v1 request, candidate identity, target, confidence, readiness, +and limitation semantics. + +A second shared fixture preserves one `codevetter.local-check/v1` +`no_confidence` receipt, request/run identity, stage status, limitation, and +exit-code semantics through the authoritative Rust service, `codevetter check`, +and native supervised runner. The repository-scoped MCP server reads that same +persisted canonical receipt through `verification_get_receipt` after redaction; +it cannot start or cancel the run. + +This passes issue #201 task 9 for local-check and evidence-scope receipt parity. +It is not a claim that every native migration row is complete. + +## Authority boundary + +| Surface | Authority | Qualified behavior | +| --- | --- | --- | +| Rust | Authoritative resolver and service | Discovers scope and owns verification command, verdict, persistence, and receipt semantics | +| CLI | Supervised execution | Sends the exact request, returns the canonical Rust receipt, and maps `no_confidence` to exit 2 | +| Native | Supervised execution | Sends the same request and rejects request, schema, verdict, or exit-status disagreement before rendering | +| MCP | Read-only projection | Returns the same discovery semantics and one already-persisted canonical local-check receipt without execution or cancellation authority | + +Both fixtures record `mcp_may_execute: false`, and the MCP contract test also +requires every exposed tool to be read-only, non-destructive, bounded, and +closed-world. + +## Executable proof + +The following focused checks passed against the same fixture: + +- authoritative Rust resolver: 1/1; +- authoritative Rust local-check receipt contract: 1/1; +- CLI parser, serialization, and human projection: 1/1; +- CLI local-check request, receipt, limitation, and exit semantics: 1/1; +- MCP read-only schema gate: 1/1; +- real MCP protocol lifecycle, scoped discovery, canonical receipt read, and + absolute-path redaction: 1/1; +- Swift supervised-runner discovery plus local-check request, schema, identity, + limitation, exit validation, and mismatch rejection: 2/2; +- packaged release MCP smoke: 28 unique strict read-only tools and no TCP + listeners. + +Commands: + +```bash +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml authoritative_resolver_matches_the_shared_surface_parity_fixture -- --nocapture +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml authoritative_service_owns_the_shared_local_check_receipt_contract -- --nocapture +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --features browser-agent --bin codevetter scope_cli_projects_the_shared_surface_parity_fixture_without_schema_drift -- --nocapture +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --features browser-agent --bin codevetter check_cli_preserves_the_shared_local_check_receipt_and_exit_semantics -- --nocapture +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml every_tool_is_explicitly_read_only_and_schema_bounded -- --nocapture +cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml protocol_lifecycle_is_scoped_structured_and_live_revocable -- --nocapture +npx -y xcodebuildmcp@2.7.0 swift-package test --package-path apps/macos/CodeVetterPackage --filter supervisedEvidenceScopeRunnerPreservesTheSharedDiscoveryContract +npx -y xcodebuildmcp@2.7.0 swift-package test --package-path apps/macos/CodeVetterPackage --filter supervisedRunnerPreservesTheSharedLocalCheckReceiptAndExitSemantics +node apps/desktop/scripts/mcp-benchmark.mjs --smoke --skip-build +``` + +The only known Rust warning remains the pre-existing unused `RawPeriod.totals` +and `RawPeriod.model_breakdowns` fields in `local_usage.rs`. + +## Remaining boundary + +This receipt does not authorize an MCP execution tool, run a live watcher poll, +qualify signing/notarization/updating, establish matched Tauri/native performance, +or approve Tauri retirement. diff --git a/evidence/verification/native-unpack-scan-parity-2026-09-01.md b/evidence/verification/native-unpack-scan-parity-2026-09-01.md new file mode 100644 index 00000000..a6940df6 --- /dev/null +++ b/evidence/verification/native-unpack-scan-parity-2026-09-01.md @@ -0,0 +1,73 @@ +# Native Repo Unpack scan parity receipt + +Date: 2026-09-01 +Scope: deterministic local scan, persistence, and bounded native inspection +Schema: `codevetter.unpack-scan/v1` + +## Result + +The incumbent Tauri command, `codevetter unpack --operation scan`, and the +native Repo Unpack workspace now reuse the same Rust scan and persistence +boundary. The operation invokes no model, writes the canonical snapshot only +to the local SQLite evidence store, and returns a bounded client projection +without the raw full-file list. + +The native workspace can start and cancel the supervised CLI process, refresh +the snapshot ledger, inspect the persisted record, and render Overview, stored +Brief, Activity, Inventory, bounded Graph, and commit-range Delta desks. It can +render deterministic Analysis, observed Rules, and source-qualified Handoff; +the Handoff desk falls back to inventory entrypoints and test leads when no +model-labelled report is attached. It can +also save the Rust-rendered Markdown, offline HTML, graph JSON, agent-context +Markdown, or repository-memory Markdown after the user chooses a destination. +History, topology, and +deterministic health remain labelled as navigation evidence rather than +executable proof. + +## Cross-surface authority + +| Surface | Authority | Qualified behavior | +| --- | --- | --- | +| Rust core | Authoritative execute/persist | Scans, profiles, persists, and emits the canonical receipt | +| Tauri | Supervised projection | Uses the shared core and preserves detailed progress events | +| CLI | Supervised projection | Validates the directory, invokes the shared core, and emits JSON or a human summary | +| Native | Supervised projection | Sends exact scan arguments, validates receipt identity, supports cancellation, and reloads the persisted snapshot | +| MCP | Read-only | May query an explicitly enabled stored graph/history index; it cannot start a scan | + +Comparison is read-only and bounded to 24 commits. Export returns +`codevetter.unpack-export/v1`; Rust renders the content and the native client +writes that exact content to the user-selected local destination. + +## Executable proof + +- Rust shared persistence test: 1/1 passed; the stored inventory remains + complete while the client projection caps the file list. +- CLI parser contract: passed for explicit scan/list/inspect operations and + invalid argument combinations. +- Isolated CLI smoke: passed against a temporary app-data directory and emitted + both `full_scan` and `local_scan_persist` profiles. +- Isolated CLI comparison/export smoke: passed for one exact Git commit range + and a 4,523-byte repository-memory export from a temporary app-data database. +- Swift supervised-runner contract: passed exact arguments, schema, status, + canonical path, and profile validation. +- Swift package gate: 56/56 passed. +- Native Debug compile: passed through XcodeBuildMCP 2.7.0. +- Foreground native UI suite: 9/9 passed. Repo Unpack navigation, explicit + repository selection, disabled-without-input scan authority, export control, + and Rust-owned SQLite boundary are reachable through accessibility IDs. +- Large native projection: passed for 100 snapshots, 700 graph nodes, and 1,000 + source-tree rows; the offscreen dark render is + `artifacts/design/native-unpack-scan-dark.png`. + +The only known Rust warning remains the pre-existing unused +`RawPeriod.totals` and `RawPeriod.model_breakdowns` fields in +`local_usage.rs`. + +## Remaining boundary + +This scan receipt did not qualify interactive graph/history queries. That +later slice is recorded separately in +[Native repository query parity](native-repository-query-2026-09-01.md). Model +synthesis execution, cleanup, richer graph traversal and causal history, +foreground owner acceptance, signing/notarization, release, and Tauri +retirement remain outside this receipt. diff --git a/evidence/verification/native-usage-windows-2026-09-01.md b/evidence/verification/native-usage-windows-2026-09-01.md new file mode 100644 index 00000000..9be4ef6f --- /dev/null +++ b/evidence/verification/native-usage-windows-2026-09-01.md @@ -0,0 +1,57 @@ +# Native Usage time-window qualification + +Date: 2026-09-01 +Surface: native macOS Usage workspace + +## Result + +The native Usage workspace now matches the incumbent 1w, 30d, 90d, and +all-time windows through the Rust-owned `LocalUsageReport` contract. +One selected window consistently scopes the activity chart, generated/cache/ +cost totals, model aggregation, active-day count, and recent sessions. The 1w +window selects daily granularity so a weekly bucket cannot hide the boundary. + +The same receipt now projects indexed Devin sessions, generated/cache tokens, +cost, and model rows for each range. The native Devin desk follows the selected +window while remaining a visibly separate SQLite source, and `codevetter usage` +prints the same bounded window summaries. Devin is never folded into ccusage. + +The range projection remains local and read-only. It does not query provider +quotas or turn spend telemetry into verification evidence. + +## Executable proof + +- Rust Devin projection contract: 1 passed, 0 failed; +- Swift time-boundary and Devin-window projection tests: passed inside the + complete 63-test package suite, with 0 failures and 0 skipped; +- complete Rust library: 1,042 passed, 0 failed, 31 ignored; CLI: 28 passed; + MCP integration: 3 passed; +- native macOS Debug and Release builds: passed through XcodeBuildMCP; +- fresh Release host and local package qualifier: passed at + `artifacts/native-package/qualification-2qmH4t/CodeVetter.app`; +- docs, generated capability registry, strict Swift formatting, and diff + whitespace checks: passed. + +The boundary test fixes the reference time and checks exact 1w/30d/90d/all +totals, overlapping weekly/monthly buckets, session timestamps, missing +timestamps, and agent selection. The large-report render gate uses 365 valid +rolling dates and 100 sessions so the default 30d view cannot pass as an empty +chart. + +## Visual evidence + +![Native Usage with aligned Devin time windows](../design/native-acceptance-2026-09-01/usage.png) + +The current 2560 x 1600 true-black render has SHA-256 +`3422bf1a081a28e389867aa6e1262669b59bd2c01947d6d0586ca2425f46a57d`. +It shows 30 populated daily buckets, coherent 30d ccusage metric details, the +compact window and granularity controls, a separate 30d Devin projection, the +quota boundary, and the Rust adapter-health inspector. + +## Remaining boundary + +Live provider quota telemetry remains separate credential-sensitive parity +work. The previously exported Tauri session scorecard has no mounted caller and +is not a retained visible surface. This receipt does not authorize installation, +production signing, notarization, update cutover, Tauri retirement, or ticket +closure. diff --git a/evidence/verification/native-watcher-live-qualification-2026-09-01.md b/evidence/verification/native-watcher-live-qualification-2026-09-01.md new file mode 100644 index 00000000..97cc5e6b --- /dev/null +++ b/evidence/verification/native-watcher-live-qualification-2026-09-01.md @@ -0,0 +1,63 @@ +# Native PR watcher live qualification + +Date: 2026-09-01 +Repository: `Codevetter/codevetter` +Pull request: `#190` +Head: `170fae78e9a16953f5c622cbdcfd219420959543` + +## Result + +An explicitly approved, one-PR `codevetter watcher` poll completed through the +shared Rust boundary with an isolated qualification database. It fetched the +exact GitHub PR head, created a detached sandbox worktree, installed the pnpm +workspace from its frozen lockfile, ran the strongest zero-argument-safe +repository check it could discover, persisted the canonical watcher receipt, +and posted the `codevetter/t-rex` commit status. + +The final verdict was `NEEDS_REVIEW` at `0.6` confidence. `pnpm run lint` +completed successfully over 870 files, but no unit/e2e tests or browser steps +ran. The watcher therefore did not upgrade static evidence into runtime proof. + +## Qualified boundaries + +- GitHub's 40-character head identity is validated before it reaches Git. +- `refs/pull/190/head` is fetched with `--no-tags --no-write-fetch-head`. +- The user's branch, index, worktree, `FETCH_HEAD`, and durable refs are not + changed by materialization. +- The exact head SHA, rather than a mutable remote branch name, is passed to + `git worktree add --detach`. +- Existing `gh auth` can supply status authority in memory; no newly discovered + token is written to evidence, logs, source, or the qualification database. +- Node installation follows the declared package manager and lockfile instead + of unconditionally creating npm state. +- Only closed, zero-argument-safe script names are auto-selected: `test`, + `test:unit`, `check`, `lint`, or `typecheck`. +- GitHub accepted the final `pending` status and the receipt retained + `status_error: null`. +- A separately confirmed `watcher --operation retry --pr-number 190` reran the + same unchanged head in 17,235 ms, retained a second attempt, and posted the + corrected conservative status without weakening automatic deduplication. + +## Executable proof + +Focused Rust tests cover exact SHA validation, token-source selection, pnpm +command discovery, and a real local bare remote whose PR ref is absent from the +checkout until the watcher fetches it. The focused watcher and sandbox suites +passed with 24 tests and one intentionally ignored sandbox e2e test. + +The clean full all-target Rust rerun passed 1,072 tests with no failures: +1,034 library tests, 27 CLI tests, five app tests, one MCP binary test, three +MCP stdio integration tests, and two additional binary-target tests. Thirty-one +library tests remain intentionally ignored. An earlier run had correctly found +one stale MCP stdio assertion expecting 24 tools after `capability_catalog` and +`resolve_evidence_scope` raised the canonical total to 26; the clean rerun +proves the corrected contract across the full suite. + +## Remaining limits + +- This PR changed a dependency and yielded static lint evidence only; it is not + runtime proof for Tailwind 4 compatibility. +- Automatic polls skip an unchanged head after any persisted receipt. Recovery + requires the explicit CLI/native Retry action and current-open-PR validation. +- The app-lifetime scheduler, owner interaction acceptance, signing, + notarization, updater cutover, and Tauri retirement remain separate gates. diff --git a/evidence/verification/stryker-accounting-oracle-2026-08-31.md b/evidence/verification/stryker-accounting-oracle-2026-08-31.md new file mode 100644 index 00000000..0204cabe --- /dev/null +++ b/evidence/verification/stryker-accounting-oracle-2026-08-31.md @@ -0,0 +1,30 @@ +# Accounting-oracle mutation qualification — 2026-08-31 + +StrykerJS was evaluated against +`scripts/qualify-codex-accounting-oracle.mjs`, a deterministic verification +boundary where a false pass would corrupt CodeVetter's accounting evidence. + +| Measurement | Initial trial | Strengthened suite | +|---|---:|---:| +| StrykerJS | 10.0.0 | 10.0.0 | +| Mutants | 218 | 218 | +| Killed | 88 | 185 | +| Survived | 130 | 33 | +| Mutation score | 40.37% | 84.86% | +| Wall time | 14 seconds | 53 seconds | + +The added tests cover invalid and zero numeric evidence, inverted cost bounds, +provider selection, duplicate and missing daily buckets, CLI success/mismatch/ +malformed-input exits, and the exact CodexBar subprocess arguments and +`CODEX_HOME` handoff. + +The 33 survivors are primarily error-message string changes and equivalent or +low-value implementation mutations. Remaining behavioral cases include exact +epsilon boundaries, deterministic multi-date ordering, and a forced CodexBar +non-zero exit. They remain visible in the ignored JSON report rather than being +excluded from mutation. + +The maintained local command is `pnpm quality:mutation:accounting`. It uses +ephemeral, exactly-versioned StrykerJS and TypeScript packages, writes its report +under ignored `artifacts/tooling/stryker/`, and fails below 80%. It is deliberately +bounded to one high-value oracle rather than applied as a universal score. diff --git a/evidence/verification/tool-collector-foundation-2026-08-31.md b/evidence/verification/tool-collector-foundation-2026-08-31.md new file mode 100644 index 00000000..b6c0ff9d --- /dev/null +++ b/evidence/verification/tool-collector-foundation-2026-08-31.md @@ -0,0 +1,60 @@ +# Tool collector foundation qualification — 2026-08-31 + +## Scope + +This receipt qualifies the unreleased `codevetter.tool-collection/v1` contract, +CLI wiring, and first Gitleaks adapter. It does not claim a sidecar is bundled or +that cargo-audit or Rust coverage executes in the product. + +## Contract + +- Input is one exact clean checked-out Git `base..head` range. +- Collector selection is explicit and duplicate selections are normalized. +- Product resolution accepts only an application-bundle sibling or an explicit + debug/test qualification override; release builds exclude that override and + arbitrary `PATH` discovery. +- Process launch uses no shell, a fixed minimal environment, null stdin, + kill-on-drop, a 120-second timeout, a 256 KiB diagnostic limit, and an 8 MiB + report limit. The JSON report is consumed from a private process pipe rather + than written to a temporary file. +- Tool evidence records exact version, resolution source, and binary SHA-256. +- Gitleaks runs with 100% redaction. Only normalized rule, description, + repository-relative file/line, commit, and fingerprint fields survive; + upstream `Secret` and `Match` fields are not represented by the Rust type. +- Every normalized finding must map to a commit and changed path in the resolved + source receipt or the collector returns an error. +- A finding exits 1, unavailable/error exits 2, and only fully clean collection + exits 0. These are CLI collection outcomes, not the overall CodeVetter + verification verdict. + +## Automated evidence + +Focused Rust tests prove that raw fixture secret and match values cannot enter +serialized receipts, missing product tools stay explicitly unavailable, and +CLI parsing requires a range plus supported explicit collectors. + +The local 8.30.1 Gitleaks binary was then exercised through the compiled CLI on +a disposable clean Git repository: + +| Trial | Observed result | +|---|---| +| Safe one-commit range | `clean`, zero findings, exit 0, 355 ms collector duration | +| Controlled custom-rule finding | `findings`, one normalized finding, exit 1, 383 ms collector duration | +| Missing cargo-audit and cargo-llvm-cov | both `unavailable`, exit 2 | + +The Gitleaks receipt recorded SHA-256 +`f414bc2fb952be6c9072b75cb411e3368614ef4b16d48dbd9ad238034afd2302`. +The controlled finding receipt contained the rule, relative path, line, commit, +and fingerprint, but not the matched fixture value or surrounding match. + +## Remaining gates + +- Pin, qualify, declare, sign, and smoke-test sidecar artifacts inside final app + bundles before claiming shipment. +- Package an offline RustSec advisory database before cargo-audit execution. +- Require the Rust LLVM tools component and implement LCOV changed-executable- + line accounting before cargo-llvm-cov execution. +- Compose collector evidence into CodeVetter's existing overall verdict policy; + do not substitute scanner exit codes for that policy. + +Issue #198 owns these gates.