Adopt external quality tools without weakening verdict authority

[sarthakagrawal927]

Why

CodeVetter already depends on proven components such as Playwright, Biome, c8, axe-core, MSW, Tree-sitter, and optional ast-grep, but the August 2026 tooling review is mostly an approval ledger rather than an implemented integration ledger. Live repository checks on 2026-08-31 also found CodeQL default setup unconfigured and GitHub secret scanning, push protection, Dependabot alerts, and Dependabot security updates disabled.

The owner wants CodeVetter to gain leverage from external tools instead of rebuilding commodity analysis. The first batch should improve the public repository and produce structured evidence without adding a production dependency, transmitting customer repositories, or allowing an external tool to issue the final CodeVetter verdict.

What

In scope

• Connect the free public DeepWiki MCP to the owner's local Codex client as a maintainer navigation aid, and verify it is listed without reading credential-bearing configuration.
• Enable eligible GitHub-native security features for the public CodeVetter repository: CodeQL default setup, Dependabot alerts/security updates, secret scanning, and push protection.
• Turn the existing Biome installation into a reproducible SARIF-producing check.
• Replace or supplement the handwritten tracked-file secret regex with the already-installed Gitleaks scanner, using redaction and machine-readable output.
• Add a pinned zizmor GitHub Actions audit for workflow security, with least-privilege permissions and inspectable findings.
• Capture before/after receipts and fix only concrete, in-scope findings surfaced by these tools.
• Update the tooling documentation so approved, trialled, and currently wired tools are visibly distinct.

Out of scope

• No DeepWiki or hosted analysis in the CodeVetter product runtime.
• No customer repository upload or hosted verdict dependency.
• No StrykerJS, OSV-Scanner, Schemathesis, Apple Containerization, libkrun, Semgrep registry rules, or Trivy product integration in this first batch; those require separately measured trials.
• No new production dependency, deployment, release, migration, or automatic remediation.
• No claim that enabling tools automatically proves product quality.

Design

External tools remain bounded collectors. Their output may block repository CI or become attributed evidence, but CodeVetter-owned normalization and executable checks remain authoritative.

flowchart LR
    A[Public CodeVetter repository] --> B[GitHub security controls]
    A --> C[Local and CI collectors]
    C --> D[Biome SARIF]
    C --> E[Gitleaks SARIF]
    C --> F[zizmor SARIF]
    B --> G[Reviewable findings]
    D --> G
    E --> G
    F --> G
    G --> H[Concrete fixes and enforced gates]
    H -. does not replace .-> I[CodeVetter executable verdict authority]
    J[DeepWiki MCP] --> K[Maintainer navigation only]

Loading

Affected surfaces are local Codex MCP configuration, GitHub repository security settings, root package scripts, GitHub Actions, Husky secret checks, and canonical tooling documentation. There is no deploy impact; workflow changes run only on repository events.

Specs

Requirement: Separate maintainer tooling from product dependencies

The implementation SHALL permit hosted tools against CodeVetter's public source only where they do not become a runtime dependency or customer-code path.

Scenario: DeepWiki is connected

• WHEN a maintainer uses Codex on a public repository
• THEN DeepWiki may provide navigation and Q&A
• AND its responses are non-authoritative and must not create pass evidence.

Requirement: Enable native public-repository security

The public repository SHALL enable eligible GitHub security controls and record their post-change state.

Scenario: A vulnerable dependency or secret is introduced

• WHEN GitHub detects the supported pattern
• THEN the repository exposes an actionable alert or push block
• AND the evidence remains separate from CodeVetter runtime verdicts.

Requirement: Produce structured local analysis evidence

Biome, Gitleaks, and zizmor checks SHALL emit deterministic or version-attributed machine-readable output with secrets redacted.

Scenario: A scanner finds no issue

• WHEN the exact pinned/configured scanner completes successfully
• THEN the receipt records tool identity, source revision, exit state, and artifact location
• AND does not overstate broader correctness.

Scenario: A scanner finds an issue

• WHEN a scanner reports a concrete finding
• THEN the finding is reviewed and either fixed or explicitly documented
• AND suppressions require a reason rather than silent filtering.

Requirement: Preserve local-first product boundaries

The first batch SHALL add no production dependency and SHALL NOT send customer source, manifests, lockfiles, or evidence to a hosted provider.

Scenario: Product runtime executes

• WHEN CodeVetter verifies a user repository
• THEN none of the new maintainer-only hosted services are invoked.

Tasks

• 1. Record the clean-worktree baseline, current GitHub security states, installed tool identities, and existing CI/Husky coverage.
• 2. Connect and verify the public DeepWiki MCP in the local Codex client without inspecting credential-bearing configuration.
• 3. Enable and verify CodeQL default setup, Dependabot alerts/security updates, secret scanning, and push protection.
• 4. Add a repository-owned Biome SARIF command and verify its output against the current tree.
• 5. Integrate Gitleaks with redaction and structured output, preserving deliberate fixtures through explained configuration rather than broad exclusions.
• 6. Add a commit-SHA-pinned zizmor workflow with least-privilege permissions and run or locally reproduce its first audit.
• 7. Review and fix concrete findings surfaced by the enabled tools; document unresolved or false-positive cases explicitly.
• 8. Update third-party tooling documentation to distinguish approved, trialled, wired, and rejected tools; validate docs.
• 9. Run the smallest focused checks after each task, then the relevant lint/typecheck/tests and diff checks for the completed batch.
• 10. Record final receipts in this issue and leave StrykerJS, OSV-Scanner, Schemathesis, and container isolation as separate evidence-gated follow-ups.

[sarthakagrawal927]

Baseline receipt — 2026-08-31

• Worktree: clean on chore/repo-structure-and-quality-gates at 225480e5b6a018014bef1c190ed8ac3914a472e7.
• GitHub: CodeQL default setup not-configured; secret scanning, push protection, Dependabot alerts, and Dependabot security updates disabled.
• Codex MCP: DeepWiki absent.
• Installed collectors: Gitleaks 8.30.1, ast-grep 0.45.2, Semgrep 1.175.0, Trivy 0.74.0, Biome 2.5.1.
• Not installed: zizmor, OSV-Scanner, cargo-mutants, cargo-llvm-cov, Apple container CLI.
• Read-only cleanup: dependency audit clean (0 advisories), Knip/lint/diff checks pass; 6 conservative and 1 major outdated direct package reported, with no dependency file changes.
• Existing repo coverage: handwritten Husky secret regex, Biome lint, pnpm audit, Knip, complexity/cycle/duplication gates, Playwright failure traces.

No repository file, dependency, or external setting was changed by the baseline.

[sarthakagrawal927]

DeepWiki MCP receipt

• Added global Codex MCP server deepwiki with streamable HTTP URL https://mcp.deepwiki.com/mcp.
• codex mcp get deepwiki --json reports enabled: true, no bearer-token environment variable, and no headers.
• codex mcp list reports the server enabled.
• This running Codex session does not dynamically gain newly configured MCP tools; use is available to a subsequent local session.
• No credential-bearing configuration was read and no repository file changed.

[sarthakagrawal927]

Implementation receipt (local branch, not committed or pushed):

• Biome SARIF: pnpm quality:sarif emits SARIF 2.1.0 at ignored artifacts/tooling/biome.sarif; current result count 0.
• Gitleaks 8.30.1: 1,628 commits / 58.48 MB scanned, 0 findings after four exact historical fingerprints. Staged mode also passes. CI downloads the MIT binary directly and verifies SHA-256; it does not use the relicensed action.
• zizmor 1.29.0: 45 initial findings reduced to four unsuppressed low-confidence cache-poisoning heuristics on trusted release/deploy workflows. Fixed 22 movable action tags, six credential-persisting checkouts, broad CI token permissions, Dependabot cooldown, and 11 shell-template expansions.
• CodeQL: first default-setup run surfaced 14 alerts. Dismissed six intentional vulnerable/test fixtures as used in tests and one explicit maintainer-binary execution as won’t fix with rationale. Local fixes address missing CI permissions, provider-origin matching, two generated-doc origin checks, and two cleartext session/source logs. The genuine localStorage API-key finding remains open and is tracked by Move review-provider API keys from localStorage to OS credential storage #194.
• GitHub controls verified: Dependabot security updates enabled; automated fixes enabled and unpaused; secret scanning enabled; push protection enabled; open Dependabot and secret alerts both 0. Non-provider patterns and validity checks remain unavailable/disabled.
• Validation: root lint, docs validation (83 files), Knip strict, TypeScript typecheck, focused review-service tests (12/12), Rust cargo check, landing + docs production build, changed-file complexity, import cycles, duplication threshold, automation tests (6/6), diff check, Biome SARIF, Gitleaks history/staged, and offline zizmor all pass. Rust retains one pre-existing dead-code warning.

The initial CodeQL run is still waiting only on Swift; the other six language jobs have succeeded. No commit, push, deploy, or release was performed.

[sarthakagrawal927]

CodeQL final receipt: default-setup run https://github.com/Codevetter/codevetter/actions/runs/33330379453 completed successfully. All seven analyzers succeeded: Rust, Go, Swift, JavaScript/TypeScript, Java/Kotlin, Actions, and Python.

[sarthakagrawal927]

Second-wave qualification receipts:

• OSV-Scanner 2.5.1: installed via Homebrew; explicit database download followed by warm offline scan. Four lock/module inputs, 35 affected packages, 51 vulnerabilities (14 high, 18 medium, 2 low, 17 unknown), 34 fixable, 9.16 seconds. Follow-up: Adopt offline OSV lockfile scanning and remediate the qualified baseline #195.
• StrykerJS 10.0.0: ephemeral pinned trial with TypeScript 5.9.3. Initial integration exposed isolated TypeScript resolution and socket/symlink sandbox-copy failures; explicit ignores resolved them. Representative accounting oracle produced 218 mutants, 88 killed, 130 survived, 40.37% in 14 seconds. Follow-up: Qualify changed-file mutation evidence with StrykerJS #196.
• Schemathesis 4.25.2: CLI availability verified, but no OpenAPI/Swagger contract exists anywhere in CodeVetter. Rejected for the current desktop/no-server surface; no feature issue created because that would manufacture an API solely to justify a tool.
• Apple container: host qualifies (arm64, macOS 27), but the CLI/system service is absent and the signed installer requires administrator authorization. Follow-up: Qualify Apple container as the first macOS sandbox backend #197.

Canonical tooling docs now record wired, trialled, approved, and lane-specific rejected status. No production dependency, customer-code upload, commit, push, deploy, or release occurred.

[sarthakagrawal927]

Local follow-up receipt, 2026-08-31:

• Added checksum-pinned actionlint 1.7.12 plus ShellCheck 0.11.0 to the repository-security workflow and a local quality:workflows command.
• actionlint exposed and drove fixes for shell quoting and repeated GitHub output redirects.
• Pedantic zizmor drove per-job permissions, workflow concurrency, job names, safer expression handling, and cache removal from privileged publish jobs. Final audit: zero unsuppressed security findings; two informational Rust setup preferences remain.
• Full tracked-diff and new-file Gitleaks scans report no leaks.
• Biome lint, docs validation, Knip, TypeScript, workflow lint, complexity, cycles, duplication, docs build, desktop unit tests, and focused Rust tests pass locally.
• Full Rust suite result is 967 passed, 3 failed, 23 ignored; the three failures are unrelated exact benchmark fixture drift and are not masked.

No production dependency was added. Nothing was committed, pushed, deployed, or released.

[sarthakagrawal927]

Additional local receipt, 2026-08-31:

• Wired cargo-deny 0.20.2 as a checksum-pinned repository policy gate with native SARIF for Rust license/source violations and a separate enforced bans lane.
• Added an Apple-Silicon-targeted deny.toml: 15 qualified license families, wildcard requirements denied, unknown registries denied, Git dependencies denied, duplicate versions reported.
• The first run exposed that the desktop crate was publishable but unlicensed; it is now explicitly publish = false.
• Locked offline result: bans ok, licenses ok, sources ok. Fourteen Tauri-driven duplicate-version warnings remain visible and non-blocking.
• Qualified ast-grep 0.45.2 instead of blindly wiring it. Structural locations were correct, but its native SARIF envelope is invalid because root version is 0.45.2 instead of SARIF 2.1.0 and the schema field is absent. It remains trialled, not wired.
• Verified DeepWiki remains enabled in the current Codex MCP configuration.
• Biome, docs validation/build, Knip, actionlint, cargo check, rustfmt, diff checks, and focused Gitleaks checks pass.

No production dependency was added and nothing was committed, pushed, deployed, or released.

[sarthakagrawal927]

Trivy configuration-scan qualification, 2026-08-31:

• Tested installed Trivy 0.74.0 only in config mode with telemetry, version checks, and check updates disabled.
• Debug evidence confirms the notification request was skipped and 563 embedded checks were used.
• The unbounded scan reported against a dependency-owned Dockerfile under docs-site/node_modules.
• After excluding dependencies/build artifacts, Trivy misclassified a warm-verification JSON fixture as CloudFormation and found no supported first-party IaC surface.
• Decision: trialled, not wired. Revisit when a real Dockerfile, Terraform, Kubernetes, Helm, Ansible, ARM, or CloudFormation surface exists.

This avoids adding a nominal scanner that currently protects nothing and creates dependency-owned noise. Receipt: evidence/security/trivy-config-qualification-2026-08-31.md.