Adopt external performance, testing, and MCP evidence integrations

[sarthakagrawal927]

Why

CodeVetter already reuses strong external foundations such as Playwright, axe-core, MSW, c8, Biome, Gitleaks, OSV-Scanner, StrykerJS, and rmcp, but its external-tool adoption is concentrated in scanning. Performance review, test-result interoperability, and maintainer MCP connections still rely on custom or disconnected paths.

The owner wants CodeVetter to build thin, evidence-preserving adapters on maintained outside projects even where the underlying operation is simple enough to rebuild. The goal is broader executable evidence and better dogfooding, not more scanners or a transfer of verdict authority.

What

In scope

• Qualify and connect pinned Chrome DevTools MCP, Playwright MCP, and GitHub MCP for maintainer use with bounded privacy and read-only controls.
• Connect the repository-scoped CodeVetter evidence, runtime, and verification-receipt MCP servers where their current authorization contracts permit it.
• Add producer-native ingestion for Playwright JSON, JUnit XML, LCOV, Cobertura XML, Lighthouse JSON, and Chrome trace metadata without executing arbitrary producer commands.
• Reuse upstream parsers/reporters where they are maintained and materially reduce custom parsing risk.
• Replace or complement the handwritten frontend bundle-budget implementation with a qualified Size Limit lane if it preserves CodeVetter's Tauri-specific entry/Home closure contract.
• Add fixtures, normalization, attribution, bounds, redaction, and CLI/MCP parity tests.
• Update canonical tooling, testing, performance, MCP, and current-status documentation with exact wired/trialled boundaries.

Out of scope

• No hosted customer-source upload, production monitoring, authenticated browser journey, arbitrary MCP command execution, deployment, release, or automatic remediation.
• No external tool output directly becomes the CodeVetter verdict.
• No rebuilding Lighthouse, Chrome tracing, Playwright, JUnit XML parsing, coverage formats, or package-size analysis unless an upstream option fails qualification.
• The unresolved owner term sizeform is not treated as an identified dependency; the candidate interpretation is the maintained Size Limit bundle-size project and must be qualified before adoption.

Design

flowchart LR
    A[Upstream tools and reporters] --> B[Bounded CodeVetter adapters]
    A1[Chrome DevTools and Lighthouse] --> B
    A2[Playwright JUnit and coverage] --> B
    A3[Size Limit] --> B
    B --> C[Versioned attributed evidence]
    C --> D[CodeVetter policy and verdict]
    E[Maintainer MCP clients] --> F[Read-only or explicitly executing scoped servers]
    F --> C

Loading

Adapters ingest repository-contained artifacts with exact source hashes, size limits, schema/version checks, path containment, secret rejection, and explicit missing fields. Maintainer MCP configuration stays outside the product runtime and uses pinned commands, isolated browser state, disabled telemetry where supported, and GitHub read-only toolsets. Executing runtime MCP remains visibly distinct from read-only evidence MCP.

There is no deploy impact. New npm packages, if any, are development/tooling dependencies only and require license/advisory/size review before addition.

Specs

Requirement: Reuse maintained evidence producers

CodeVetter SHALL prefer maintained upstream reporters, parsers, and measurement tools for commodity formats while retaining its own normalization and verdict policy.

Scenario: A supported producer artifact is ingested

• WHEN a repository-contained Playwright, JUnit, LCOV, Cobertura, Lighthouse, or Chrome trace artifact passes bounds and identity checks
• THEN CodeVetter emits attributed normalized evidence with the raw SHA-256 and explicit limitations
• AND does not upgrade missing producer evidence.

Requirement: Performance evidence remains reproducible

Performance adapters SHALL separate observed metrics from inferred diagnosis and SHALL require same-scope evidence for regression or improvement claims.

Scenario: Lighthouse or Chrome evidence is supplied

• WHEN the artifact lacks a compatible baseline, exact URL/revision identity, or repeated samples
• THEN the result remains observational or no-confidence
• AND no composite score alone authorizes an optimization claim.

Requirement: Test interoperability is fail closed

Test adapters SHALL reject malformed, oversized, path-escaping, credential-shaped, duplicate, or internally inconsistent reports.

Scenario: A JUnit report contains retries or failures

• WHEN those outcomes are normalized
• THEN attempts, terminal status, retry/flaky evidence, and bounded failure signatures remain separate.

Requirement: MCP connections preserve authority

Maintainer MCP connections SHALL be pinned and scoped. Read-only servers SHALL expose no write tools; executing servers SHALL remain explicit and repository-fixed.

Scenario: GitHub MCP is connected

• WHEN the server lists tools
• THEN only the approved read-only repository, pull-request, Actions, and code-security capabilities are available.

Scenario: CodeVetter MCP is dogfooded

• WHEN a local client connects to the generated repository scope
• THEN the packaged evidence server remains read-only and prepare_review suggestions remain non-executing leads.

Requirement: Existing special-purpose gates are preserved

Upstream replacement SHALL match the meaningful behavior of the existing gate before the handwritten implementation is retired.

Scenario: Size Limit is qualified

• WHEN Size Limit measures the built frontend
• THEN it preserves entry/Home closure, largest-chunk, and full lazy-distribution budgets or runs additively
• AND the existing Tauri-specific gate is not removed merely because total bytes can be measured elsewhere.

Tasks

• 1. Record current MCP, test, performance, bundle-budget, dependency, and receipt-adapter baselines.
• 2. Qualify exact versions, licenses, advisories, telemetry/network behavior, and command schemas for Chrome DevTools MCP, Playwright MCP, GitHub MCP, Size Limit, and any parser dependencies.
• 3. Add bounded producer-native Playwright JSON and JUnit XML adapters with fixtures and failure/retry normalization.
• 4. Add bounded LCOV and Cobertura coverage adapters with aggregate producer attribution and explicit changed-file missing-data limitations.
• 5. Add bounded Lighthouse JSON and Chrome trace metadata adapters with observational versus comparable performance semantics.
• 6. Add or qualify Size Limit additively against the existing Tauri entry/Home and lazy-route bundle budgets; retire custom logic only on parity.
• 7. Configure and verify pinned maintainer Chrome DevTools, isolated Playwright, and read-only GitHub MCP connections without exposing secrets.
• 8. Connect and protocol-smoke the allowed CodeVetter evidence/runtime/receipt MCP servers; record any app-generated scope or restart prerequisite explicitly.
• 9. Run focused adapter tests, MCP protocol tests, lint, typecheck, docs validation, dependency/license/security gates, and relevant builds.
• 10. Update canonical docs and PROJECT_STATUS with only proven wired capabilities; leave deployment/release and unresolved external prerequisites claim-closed.
• 11. Dogfood Playwright JSON, JUnit XML, LCOV, and Cobertura through their built-in producers and the CodeVetter ingestion path.
• 12. Reconcile the active browser-test inventory with retired Work/Board surfaces and fix integration defects exposed by the full producer run.
• 13. Replace the PAT-dependent remote GitHub registration with the checksum-verified GitHub MCP Server 1.11.0 local OAuth path, strict read-only mode, narrow scopes, and bounded toolsets.

Boundaries

• No production dependency without a separate justification.
• No secrets in arguments, logs, repository files, fixtures, or issue updates.
• No commit, push, merge, deploy, migration, or release under this issue without separate authorization.
• MCP configuration changes may require a new Codex session before tools are callable; configuration is not current-session proof.

---

Spec-driven tracking issue. All active design and task state lives here.

[sarthakagrawal927]

Implementation receipts (uncommitted local source): Playwright JSON, JUnit XML, LCOV, Cobertura XML, Lighthouse JSON, and Chrome trace adapters pass 20 focused tests; runtime/MCP suite passes 151 tests. Size Limit 13.0.3 runs additively after the existing Tauri-aware bundle gate (1.26 MB against 1.8 MB; initial + Home 472.5 KiB). Chrome DevTools MCP 1.8.0, Playwright MCP 0.0.79, codevetter-runtime-exec, and codevetter-verification are registered and require a fresh Codex session. GitHub MCP is registered with official read-only toolsets but remains unauthenticated because credentials are operator-owned, so task 7 stays open. The packaged graph/history MCP also remains subject to in-app repository indexing and opaque-scope authorization. Lighthouse CI 0.15.1 passed a three-run local trial but was removed after introducing three high advisories, including unpatched extract-zip traversal; npm high/critical audit is clean after removal. Lint, typecheck, docs, complexity, cycles, Knip, duplication, build, bundle, and current npm advisory gates pass. Offline OSV still reports the tracked repository baseline; none of the newly added parser/Size Limit package names appear in its SARIF.

[sarthakagrawal927]

Dogfood integration receipt (uncommitted local source): pnpm verification:dogfood now runs CodeVetter's active Playwright suite and c8 unit coverage, emits Playwright JSON, JUnit XML, LCOV, and Cobertura via the upstream built-in reporters, and ingests all four artifacts. Both producers exited 0; Playwright passed 62/62 flows in 61.8s, and the JUnit report independently records 62 tests, zero failures/errors/skips. All four artifacts produce a passed bounded correctness projection while correctly retaining overall/inventory/performance/safety as no_confidence. The run exposed stale tests for retired Work/Board surfaces, obsolete Settings provider controls, a missing clipboard permission, a duplicate React sibling key in Testing, JUnit <error> classification, and standard Cobertura DTD handling; those were reconciled and the full rerun passed.

Current gates pass: 20/20 receipt contracts, TypeScript, Biome, docs, changed-file complexity, Knip, import cycles, duplication threshold, npm high/critical audit, full Gitleaks scan, production Vite build, Tauri-aware bundle budgets (472.5 KiB initial + Home), and additive Size Limit (1.26 MB / 1.8 MB). The optional local cargo-deny binary is absent; no Rust or Cargo dependency changed in this slice. Task 7 remains open: the official read-only GitHub MCP registration is correct but GITHUB_PERSONAL_ACCESS_TOKEN is operator-owned and absent, and the packaged graph/history MCP still requires in-app repository authorization. A fresh Codex session is required for newly registered MCPs to become callable.

[sarthakagrawal927]

GitHub MCP hardening update: replaced the unauthenticated PAT-dependent remote registration with the checksum-verified official github-mcp-server v1.11.0 Darwin arm64 binary. Codex now launches it over local stdio with --read-only, the bounded repos,issues,pull_requests,actions,code_security toolsets, and only read:org,security_events OAuth scopes. No token is stored in Codex configuration or the repository; the official server keeps the browser-OAuth token in memory. Binary version/commit and SHA-256 were verified against the v1.11.0 release checksum. Task 7 remains open until the operator approves the first-use GitHub OAuth prompt in a fresh Codex session and we can list/call the resulting read-only tools.

The packaged history MCP is also still correctly blocked: the installed CodeVetter database has no history index or MCP scope for this checkout. Repository docs require Repo Unpack -> Index history -> Settings -> Agent MCP -> Enable; direct database synthesis is intentionally prohibited. Computer-use could inspect the app but its native action pipe closed on every attempted click, so no authorization state was fabricated.

[sarthakagrawal927]

MCP restart verification complete. The pinned GitHub MCP server completed browser OAuth and live read-only calls succeeded for repository contents, open pull requests, the workflow catalog, issue #200, and open CodeQL alerts. Chrome DevTools MCP also listed its isolated about:blank page, and Playwright MCP listed its isolated about:blank tab. The current callable tool registry exposes only the configured GitHub read surfaces; no GitHub mutation tools are present. Task 7 is now complete.

The packaged CodeVetter graph/history MCP remains separately product-authorized. The installed app still has no history index or opaque scope for this checkout, and the macOS computer-control pipe continues to close before every click reaches the Tauri app, so that local enablement was not bypassed with direct database writes.

[sarthakagrawal927]

Packaged CodeVetter graph/history MCP is now complete for the canonical checkout. The product UI selected /Users/sarthak/Desktop/fleet/codevetter, indexed current history, built the canonical tree-sitter structural index, generated an opaque repository scope, and enabled it. The exact app-generated stdio configuration is registered globally as codevetter-history. Direct MCP protocol smoke tests succeeded for both history_search and graph_query, and sequential calls produced bounded local access-audit rows. The structural index is current at HEAD with 1,379 indexed files, 115,884 nodes, and 193,399 edges; history is current with 599 loaded commits and two persisted checkpoints. No database arguments or scope rows were synthesized, and no commit, push, deploy, or release was performed.

[sarthakagrawal927]

Committed and pushed the completed integration slice as b34ce70 (feat: integrate external evidence producers and MCP tooling) on chore/repo-structure-and-quality-gates. Local commit/push hooks passed Biome plus staged and full-history Gitleaks scans; focused receipt tests remain 20/20 and both Tauri-aware and Size Limit bundle gates pass. The push-triggered Docs workflow succeeded: https://github.com/Codevetter/codevetter/actions/runs/33369412710. Full CI is configured for pull requests or main pushes, and this branch's previous PR #185 is closed, so no full CI run or merge is claimed.