diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index ee79fa4f3..920b1ad25 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,17 +1,42 @@ -## Summary +## Canonical issue -Describe the outcome and the evidence that supports it. +Closes # -## Linked issue +## Outcome -Fixes # +Describe the user or operational result this PR produces. + +## Scope + +- Included: +- Explicitly excluded: + +## Risk + +- Risk level: low / medium / high +- Failure mode: +- Rollback: ## Verification +List exact automated and manual checks, tied to the current head SHA. + - [ ] Focused tests - [ ] Required CI - [ ] Review threads resolved +## Production evidence + +Provide the Vercel preview, production deployment, runtime evidence, or state why production evidence is not applicable. + +## Agent handoff + +- [ ] One canonical issue is linked +- [ ] No competing PR implements the same issue +- [ ] Acceptance criteria are satisfied +- [ ] Required checks pass on the current head +- [ ] Human decision is requested only for product, security, irreversible infrastructure, or production approval + ## Agent provenance Human-authored pull requests may delete this section. Agent-authored pull requests must replace agent-lock-example with agent-lock-manifest and fill the values. Scope and test paths remain authoritative in the linked issue. diff --git a/.github/workflows/AUDIT.md b/.github/workflows/AUDIT.md index 63fa27412..e6af32577 100644 --- a/.github/workflows/AUDIT.md +++ b/.github/workflows/AUDIT.md @@ -27,7 +27,9 @@ concrete reason, verified against the actual repository tree. | `mcp-optimization.yml` | **DELETE** | Entire workflow targets `mcp-servers/mcp-profiling/` (requirements.txt, investigator_client.py, profiling_server.py) which does not exist — every run fails. | | `phase-goal-tracker.yml` | KEEP | Tracks markdown checklists on phase issues, keeps a single status comment updated, and auto-closes the issue when all checklist goals are complete. | | `pr-checks.yml` | KEEP | Validates PR title/description; fork-safe comment handling. | +| `pr-governance.yml` | **ADD** | Enforces exactly one canonical closing issue, required delivery-evidence sections, and fails when another open PR closes the same issue. Draft PRs are intentionally non-blocking. | | `real-processing.yml` | KEEP | Manual single-video processing; well-formed. | +| `repository-reconciliation.yml` | **ADD** | Scheduled/manual non-destructive reconciliation that updates one `[automation] Repository drift report` issue and proposes stale detached branches without deleting them. | | `secret-scan.yml` | KEEP | gitleaks on the working tree; action pinned to SHA, checksum-verified install. | | `security.yml` | KEEP | npm audit, safety, bandit, trivy; uploads SARIF. | | `verify-litert-mcp.yml` | **DELETE** | Path-filtered smoke test of `mcp-servers/litert-mcp/server.py`; the `mcp-servers/` tree was removed in the dead-code cleanup, so the target no longer exists and every run fails. | @@ -65,4 +67,4 @@ valid. Referenced paths were checked against the working tree: | `agent-completion-enforcement.yml` | **ADD** | Protected-default-branch verifier that creates the independent **Agent completion enforcement** Check directly against the PR head SHA. It accepts only an exact-head machine-readable report from the configured dedicated GitHub App; missing/stale/mutable evidence, untrusted label provenance, and custom roles all fail closed. The existing `agent-completion/truth-gate` status stays advisory and must not be made required. | -The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists and therefore blocks until a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations. \ No newline at end of file +The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists and therefore blocks until a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations. diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 0e5c52aca..8ec53ca08 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -23,6 +23,8 @@ workflow; this README is the index. | Bulk Issue Processor | `bulk-issue-processor.yml` | manual | Bulk label / summarize / close-stale across many issues | | Close stale issues | `stale.yml` | daily (00:00 UTC) | Mark and close stale issues and PRs | | Branch Cleanup | `branch-cleanup.yml` | manual; push sentinel on `claude/branch-cleanup-*` | Gated archive-then-delete of branches (dry-run by default); push `[restore-branch:]` sentinel to restore a deleted branch from its archive tag | +| PR Governance | `pr-governance.yml` | PR opened/edited/synchronize/ready | Enforce one canonical closing issue, required governance sections, and competing-PR detection | +| Repository Reconciliation | `repository-reconciliation.yml` | daily; manual | Non-destructive drift report of PR/issue linkage and stale unattached branches | | E2E Tests | `e2e-tests.yml` | push / PR to `main` | Run Vitest E2E pipeline tests against production or the PR's Vercel preview deployment and report results on the PR | | Autonomous Video Processing | `autonomous-video-processing.yml` | manual | Batch-process YouTube videos by category (matrix) | | Real Video Processing (Cloud) | `real-processing.yml` | manual | Process a single video: transcript and/or AI analysis | @@ -111,6 +113,10 @@ A full audit of this directory was performed (see - **Fixed** `auto-assign.yml` — replaced `gh issue edit` with the REST assignees endpoint after run logs showed GitHub App installation tokens cannot use the CLI's GraphQL assignable mutation for this assignment. +- **Added** `pr-governance.yml` — validates exactly one canonical closing issue, + required delivery evidence sections, and blocks competing open implementation PRs. +- **Added** `repository-reconciliation.yml` — publishes/updates a single + `[automation] Repository drift report` issue without deleting branches. ## Resources @@ -126,4 +132,4 @@ A full audit of this directory was performed (see `pr-checks.yml` retains the advisory `agent-completion/truth-gate/pr-` status; it is never required. `agent-completion-enforcement.yml` runs protected default-branch code, does not execute PR code, and creates the separate **Agent completion enforcement** Check directly on the PR head SHA. It accepts only an exact-head, machine-readable report published by the configured dedicated GitHub App. Missing, stale, edited/deleted, ambiguous, or untrusted evidence fails closed. -Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Empty lists intentionally block. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`. \ No newline at end of file +Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Empty lists intentionally block. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`. diff --git a/.github/workflows/pr-governance.yml b/.github/workflows/pr-governance.yml new file mode 100644 index 000000000..2eab41258 --- /dev/null +++ b/.github/workflows/pr-governance.yml @@ -0,0 +1,80 @@ +name: PR Governance + +on: + pull_request_target: + types: [opened, edited, reopened, synchronize, ready_for_review] + +permissions: + contents: read + issues: read + pull-requests: read + +concurrency: + group: pr-governance-${{ github.event.pull_request.number }} + cancel-in-progress: true + +jobs: + policy: + name: Canonical issue and evidence + runs-on: ubuntu-latest + steps: + - name: Validate delivery contract + uses: actions/github-script@v8 + with: + script: | + const pr = context.payload.pull_request; + if (pr.draft) { + core.notice("Draft PR: governance enforcement begins when marked ready."); + return; + } + + const body = pr.body || ""; + const requiredSections = [ + "## Canonical issue", + "## Outcome", + "## Risk", + "## Verification", + "## Production evidence" + ]; + const missing = requiredSections.filter(section => !body.includes(section)); + + const closingPattern = /(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/gi; + const issueNumbers = [...body.matchAll(closingPattern)].map(match => Number(match[1])); + const canonicalIssues = [...new Set(issueNumbers)]; + + if (canonicalIssues.length !== 1) { + missing.push("exactly one closing reference: Closes #"); + } + + if (canonicalIssues.length === 1) { + const canonical = canonicalIssues[0]; + const pulls = await github.paginate(github.rest.pulls.list, { + owner: context.repo.owner, + repo: context.repo.repo, + state: "open", + per_page: 100 + }); + + const competing = pulls.filter(candidate => { + if (candidate.number === pr.number) return false; + const matches = [...(candidate.body || "").matchAll(closingPattern)] + .map(match => Number(match[1])); + return matches.includes(canonical); + }); + + if (competing.length) { + const links = competing.map(candidate => `#${candidate.number}`).join(", "); + core.setFailed( + `Issue #${canonical} already has another open implementation PR: ${links}. ` + + "Supersede, close, or explicitly consolidate it before this PR becomes canonical." + ); + return; + } + } + + if (missing.length) { + core.setFailed(`PR delivery contract is incomplete: ${missing.join("; ")}`); + return; + } + + core.notice("PR has one canonical issue and the required delivery evidence sections."); diff --git a/.github/workflows/repository-reconciliation.yml b/.github/workflows/repository-reconciliation.yml new file mode 100644 index 000000000..0536a7770 --- /dev/null +++ b/.github/workflows/repository-reconciliation.yml @@ -0,0 +1,115 @@ +name: Repository Reconciliation + +on: + schedule: + - cron: "17 13 * * *" + workflow_dispatch: + +permissions: + contents: read + issues: write + pull-requests: read + +concurrency: + group: repository-reconciliation + cancel-in-progress: true + +jobs: + report: + runs-on: ubuntu-latest + steps: + - name: Reconcile canonical delivery state + uses: actions/github-script@v8 + with: + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const now = Date.now(); + const staleAfterMs = 14 * 24 * 60 * 60 * 1000; + const closingPattern = /(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/gi; + + const pulls = await github.paginate(github.rest.pulls.list, { + owner, repo, state: "open", per_page: 100 + }); + const branches = await github.paginate(github.rest.repos.listBranches, { + owner, repo, protected: false, per_page: 100 + }); + const activeHeads = new Set(pulls.map(pr => pr.head.ref)); + + const untracked = []; + const issueToPulls = new Map(); + for (const pr of pulls) { + const issues = [...(pr.body || "").matchAll(closingPattern)] + .map(match => Number(match[1])); + const unique = [...new Set(issues)]; + if (unique.length !== 1) untracked.push(pr); + for (const issue of unique) { + const existing = issueToPulls.get(issue) || []; + existing.push(pr.number); + issueToPulls.set(issue, existing); + } + } + + const duplicates = [...issueToPulls.entries()] + .filter(([, numbers]) => numbers.length > 1); + + const staleBranches = []; + for (const branch of branches) { + if (branch.name === "main" || activeHeads.has(branch.name)) continue; + const commit = await github.rest.repos.getCommit({ + owner, repo, ref: branch.commit.sha + }); + const date = commit.data.commit.committer?.date || commit.data.commit.author?.date; + if (date && now - new Date(date).getTime() > staleAfterMs) { + staleBranches.push({ name: branch.name, date, sha: branch.commit.sha.slice(0, 8) }); + } + } + + const lines = [ + "## Canonical delivery-state reconciliation", + "", + `Generated: ${new Date().toISOString()}`, + "", + `- Open PRs: **${pulls.length}**`, + `- Remote branches: **${branches.length}**`, + `- Ready PRs without exactly one canonical issue: **${untracked.length}**`, + `- Issues with competing implementation PRs: **${duplicates.length}**`, + `- Unattached branches older than 14 days: **${staleBranches.length}**`, + "", + "### PRs requiring canonical issue", + untracked.length + ? untracked.map(pr => `- #${pr.number} — ${pr.title}`).join("\n") + : "- None", + "", + "### Competing PRs", + duplicates.length + ? duplicates.map(([issue, numbers]) => `- Issue #${issue}: ${numbers.map(n => `#${n}`).join(", ")}`).join("\n") + : "- None", + "", + "### Stale unattached branches", + staleBranches.length + ? staleBranches.slice(0, 100).map(branch => + `- \`${branch.name}\` — ${branch.sha}, last commit ${branch.date}` + ).join("\n") + : "- None", + "", + "> This report is intentionally non-destructive. Branch deletion requires a merged PR or an explicit retention decision.", + "", + "Canonical governance: #898" + ]; + + const title = "[automation] Repository drift report"; + const query = `repo:${owner}/${repo} is:issue is:open in:title "${title}"`; + const existing = await github.rest.search.issuesAndPullRequests({ + q: query, per_page: 10 + }); + const report = existing.data.items.find(item => item.title === title); + const body = lines.join("\n"); + + if (report) { + await github.rest.issues.update({ + owner, repo, issue_number: report.number, body + }); + } else { + await github.rest.issues.create({ owner, repo, title, body }); + } diff --git a/apps/web/vercel.json b/apps/web/vercel.json index e31fb5a41..23ee00c38 100644 --- a/apps/web/vercel.json +++ b/apps/web/vercel.json @@ -1,6 +1,7 @@ { "framework": "nextjs", "installCommand": "cd ../.. && npm install --legacy-peer-deps", + "ignoreCommand": "bash ../../scripts/deployment/vercel-ignore-command.sh", "buildCommand": "npm run build", "outputDirectory": ".next" } diff --git a/scripts/deployment/vercel-ignore-command.sh b/scripts/deployment/vercel-ignore-command.sh new file mode 100755 index 000000000..a3ae96d4d --- /dev/null +++ b/scripts/deployment/vercel-ignore-command.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Vercel ignored-build command semantics: +# - exit 0 => skip build +# - exit 1 => continue build +# +# We skip only when the diff is strictly docs/workflow metadata. + +head_sha="${VERCEL_GIT_COMMIT_SHA:-}" +base_sha="${VERCEL_GIT_PREVIOUS_SHA:-}" + +if [[ -z "${head_sha}" || -z "${base_sha}" ]]; then + echo "preview-ignore: missing commit context; running build" + exit 1 +fi + +if ! git cat-file -e "${head_sha}^{commit}" 2>/dev/null; then + echo "preview-ignore: head commit not available locally; running build" + exit 1 +fi + +if ! git cat-file -e "${base_sha}^{commit}" 2>/dev/null; then + echo "preview-ignore: base commit not available locally; running build" + exit 1 +fi + +mapfile -t changed < <(git diff --name-only "${base_sha}" "${head_sha}") +if [[ ${#changed[@]} -eq 0 ]]; then + echo "preview-ignore: no changed files detected; running build" + exit 1 +fi + +for path in "${changed[@]}"; do + if [[ "${path}" == docs/* ]]; then + continue + fi + if [[ "${path}" == .github/workflows/* ]]; then + continue + fi + if [[ "${path}" == .github/ISSUE_TEMPLATE/* ]]; then + continue + fi + if [[ "${path}" == .github/pull_request_template.md ]]; then + continue + fi + if [[ "${path}" == *.md ]]; then + continue + fi + + echo "preview-ignore: app-impacting change detected (${path}); running build" + exit 1 +done + +echo "preview-ignore: docs/workflow-only change; skipping preview build" +exit 0 diff --git a/tests/unit/test_repository_governance_workflows.py b/tests/unit/test_repository_governance_workflows.py new file mode 100644 index 000000000..2d68cbbcc --- /dev/null +++ b/tests/unit/test_repository_governance_workflows.py @@ -0,0 +1,59 @@ +from __future__ import annotations + +from pathlib import Path + +import yaml + + +REPO_ROOT = Path(__file__).resolve().parents[2] +PR_GOVERNANCE = REPO_ROOT / ".github/workflows/pr-governance.yml" +RECONCILIATION = REPO_ROOT / ".github/workflows/repository-reconciliation.yml" + + +def _load_yaml(path: Path) -> dict: + assert path.exists(), f"Expected workflow to exist: {path}" + return yaml.safe_load(path.read_text(encoding="utf-8")) + +def _workflow_on(workflow: dict) -> dict: + # PyYAML follows YAML 1.1 booleans, where `on` may parse as True. + return workflow.get("on") or workflow.get(True) or {} + + +def test_pr_governance_workflow_enforces_single_issue_contract() -> None: + workflow = _load_yaml(PR_GOVERNANCE) + workflow_on = _workflow_on(workflow) + + assert workflow["name"] == "PR Governance" + assert workflow_on["pull_request_target"]["types"] == [ + "opened", + "edited", + "reopened", + "synchronize", + "ready_for_review", + ] + assert workflow["permissions"] == { + "contents": "read", + "issues": "read", + "pull-requests": "read", + } + script = workflow["jobs"]["policy"]["steps"][0]["with"]["script"] + assert "exactly one closing reference" in script + assert "already has another open implementation PR" in script + + +def test_repository_reconciliation_is_non_destructive_report_only() -> None: + workflow = _load_yaml(RECONCILIATION) + workflow_on = _workflow_on(workflow) + + assert workflow["name"] == "Repository Reconciliation" + assert "workflow_dispatch" in workflow_on + assert workflow["permissions"] == { + "contents": "read", + "issues": "write", + "pull-requests": "read", + } + script = workflow["jobs"]["report"]["steps"][0]["with"]["script"] + assert "[automation] Repository drift report" in script + assert "intentionally non-destructive" in script + assert "issues.create" in script + assert "issues.update" in script diff --git a/tests/unit/test_vercel_preview_ignore_command.py b/tests/unit/test_vercel_preview_ignore_command.py new file mode 100644 index 000000000..57bb02358 --- /dev/null +++ b/tests/unit/test_vercel_preview_ignore_command.py @@ -0,0 +1,94 @@ +from __future__ import annotations + +import os +import subprocess +from pathlib import Path +from tempfile import TemporaryDirectory + +import json + + +REPO_ROOT = Path(__file__).resolve().parents[2] +APPS_WEB_VERCEL = REPO_ROOT / "apps/web/vercel.json" +IGNORE_SCRIPT = REPO_ROOT / "scripts/deployment/vercel-ignore-command.sh" + + +def _git(repo: Path, *args: str) -> None: + subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True) + + +def _run_ignore(repo: Path) -> int: + env = os.environ.copy() + env["VERCEL_GIT_COMMIT_SHA"] = ( + subprocess.run( + ["git", "rev-parse", "HEAD"], cwd=repo, check=True, capture_output=True, text=True + ) + .stdout.strip() + ) + env["VERCEL_GIT_PREVIOUS_SHA"] = ( + subprocess.run( + ["git", "rev-parse", "HEAD~1"], cwd=repo, check=True, capture_output=True, text=True + ) + .stdout.strip() + ) + result = subprocess.run( + ["bash", str(IGNORE_SCRIPT)], + cwd=repo, + env=env, + check=False, + capture_output=True, + text=True, + ) + return result.returncode + + +def test_apps_web_vercel_uses_ignore_command() -> None: + cfg = json.loads(APPS_WEB_VERCEL.read_text(encoding="utf-8")) + assert cfg.get("ignoreCommand"), "apps/web/vercel.json should define ignoreCommand" + assert "vercel-ignore-command.sh" in cfg["ignoreCommand"] + + +def test_ignore_script_skips_docs_only_changes() -> None: + assert IGNORE_SCRIPT.exists(), "vercel ignore command script should exist" + + with TemporaryDirectory() as td: + repo = Path(td) + _git(repo, "init") + _git(repo, "config", "user.email", "test@example.com") + _git(repo, "config", "user.name", "test") + _git(repo, "branch", "-m", "main") + + (repo / "docs").mkdir(parents=True) + (repo / ".github/workflows").mkdir(parents=True) + (repo / "apps/web/src").mkdir(parents=True) + + (repo / "docs/intro.md").write_text("base\n", encoding="utf-8") + (repo / ".github/workflows/ci.yml").write_text("name: ci\n", encoding="utf-8") + (repo / "apps/web/src/page.tsx").write_text("export default function Page(){}\n", encoding="utf-8") + _git(repo, "add", ".") + _git(repo, "commit", "-m", "base") + + (repo / "docs/intro.md").write_text("changed\n", encoding="utf-8") + (repo / ".github/workflows/ci.yml").write_text("name: ci\non: pull_request\n", encoding="utf-8") + _git(repo, "add", ".") + _git(repo, "commit", "-m", "docs and workflow only") + assert _run_ignore(repo) == 0 + + +def test_ignore_script_keeps_preview_for_web_changes() -> None: + with TemporaryDirectory() as td: + repo = Path(td) + _git(repo, "init") + _git(repo, "config", "user.email", "test@example.com") + _git(repo, "config", "user.name", "test") + _git(repo, "branch", "-m", "main") + + (repo / "apps/web/src").mkdir(parents=True) + (repo / "apps/web/src/page.tsx").write_text("export default function Page(){}\n", encoding="utf-8") + _git(repo, "add", ".") + _git(repo, "commit", "-m", "base") + + (repo / "apps/web/src/page.tsx").write_text("export default function Page(){return null}\n", encoding="utf-8") + _git(repo, "add", ".") + _git(repo, "commit", "-m", "web change") + assert _run_ignore(repo) == 1