Skip to content

gates: implement review.py as hard stop before submit packages a PR #14

Description

@Mathews-Tom

Context

src/apprentice/gates/review.py is currently a ~3-line stub. The apprentice submit command invokes agents/packaging.py, which calls open_pr()gh pr create, with no architectural gate between review and packaging.

The original design review (2026-04-07) flagged this: "human-review gate must be architecturally enforced (e.g., opens a PR and stops)". The gate is not bypassable because it does not exist.

Current state

  • src/apprentice/gates/review.py — stub, no implementation
  • agents/review.py:84-126 — runs consistency/schema validators only, marks review_verdict: "passed" without human input
  • agents/packaging.py:39 wires open_pr as a tool the LLM invokes during submit
  • No approval token, no state flag, no prompt gate between build completion and submit start

Risk

Any operator running apprentice submit after build fires a PR against no-magic-ai/no-magic (and no-magic-viz) under their GitHub identity with zero human review. Confirmed during 2026-04-19 smoke test: gh is authed with repo scope and open_pr has no dry-run or bypass flag.

Proposed fix

Implement gates/review.py as a required gate between build and submit:

  1. After agents/review.py completes, write a review_approval_required: true flag to session state.
  2. Gate blocks submit unless an operator has written review_approval: {approved_by: <gh_login>, approved_at: <ts>, artifact_hashes: {...}} to the session.
  3. Approval must hash the artifacts; any subsequent artifact change invalidates the approval.
  4. A apprentice approve <run_id> CLI command writes the approval.

Acceptance

  • Running submit without approval exits non-zero with an explicit message naming the apprentice approve command.
  • Running approve then submit proceeds to packaging.
  • Modifying any artifact after approval and re-running submit fails hash check and requires re-approval.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions