Skip to content

[Agent Lock] Block completion unless intent, diff, checks, and output agree #870

Description

@groupthinking

Objective

Add a deterministic Agent Completion Truth Gate that evaluates repository evidence instead of trusting agent completion prose.

Agent login

groupthinking

Agent run ID

codex-myx-82-20260717

Incident fixture

PR #813 / issue #802 exposed the required failure modes:

  • the source issue had no description or acceptance criteria;
  • Jules emitted both "Ready for review" and "unexpected error / wasn't able to complete";
  • the PR claimed two source files but changed five files across three commits;
  • unrelated dependency and test changes entered the branch;
  • generic CI was green while an unresolved review identified a behavior-breaking regex regression;
  • no focused regression test demonstrated the intended behavior.

Required behavior

The gate must return a machine-readable verdict with explicit reasons and fail closed.

Pre-dispatch

  • Reject an issue with a blank description.
  • Reject an issue without explicit acceptance criteria.
  • Require a declared file scope or an explicit unrestricted-scope marker.

Post-execution

  • Compare declared file scope with actual changed paths.
  • Treat any undeclared path as scope drift unless explicitly allowlisted.
  • Treat contradictory completion and error events as failed.
  • Treat unresolved blocking review findings as failed even when generic CI is green.
  • Require focused test evidence for behavior-changing source edits.
  • Only allow a completion verdict after the PR is merged and required post-merge checks succeed.

Deliverables

Declared file scope

  • scripts/ci/agent_completion_gate.py
  • tests/unit/test_agent_completion_gate.py
  • tests/fixtures/agent_completion/pr_813.json
  • .github/workflows/pr-checks.yml
  • .github/ISSUE_TEMPLATE/agent-task.yml
  • .github/pull_request_template.md
  • docs/agent-completion-truth-gate.md

Focused test paths

  • tests/unit/test_agent_completion_gate.py

Acceptance tests

  • Blank issue description returns blocked.
  • Missing acceptance criteria returns blocked.
  • Declared two files but actual five returns blocked: scope_drift.
  • Completion plus later error returns blocked: contradictory_terminal_events.
  • Green CI plus unresolved blocking review returns blocked: unresolved_review.
  • Source behavior change without focused test evidence returns blocked: missing_test_evidence.
  • Unmerged PR cannot return completed.
  • A fully scoped, reviewed, tested, merged fixture returns completed.

Pre-dispatch confirmation

  • Objective, acceptance criteria, scope, and focused-test expectations are complete before delegation.

Non-goals

  • No autonomous merging.
  • No LLM judgment inside the enforcement path.
  • No Slack or notification layer.
  • No modification of application runtime behavior.

Related: #813, #802.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions