Skip to content

docs: say what the six required security scans actually gate - #1414

Closed
groupthinking wants to merge 1 commit into
claude/clever-heisenberg-0zuq9kfrom
claude/clever-heisenberg-klf4fu
Closed

docs: say what the six required security scans actually gate#1414
groupthinking wants to merge 1 commit into
claude/clever-heisenberg-0zuq9kfrom
claude/clever-heisenberg-klf4fu

Conversation

@groupthinking

Copy link
Copy Markdown
Owner

Canonical issue

Closes #1413

Stacked on #1410 (base is its branch, not main), so the diff here is one commit, MERGE_POLICY.md +17/−0. This is a follow-up to #1410, not a competing implementation of #1412#1410 decides which checks are required; this records what passing them means. Neither claim moves the other's list.

Outcome

Gate 2 requires six security contexts, and every one of them reports success with findings present. A reader configuring branch protection from that list would reasonably conclude a HIGH/CRITICAL finding cannot merge. It can. This records the gap and what requiring them does buy.

Verified against the workflow definitions, not inferred from observed check-run status:

Check Fails the job on Why findings don't fail it
npm-audit npm install breaking continue-on-error: true on the audit step
python-safety pip install safety breaking || true and continue-on-error: true
bandit pip install bandit breaking || true on bandit -r src
trivy the docker build breaking exit-code: '0' passed to trivy-action
Security Scan - python, - javascript CodeQL itself erroring alerts route to the Security tab; blocking is code-scanning merge protection, not a required-check setting

CodeRabbit reached the same conclusion independently on #1410 ("requiring these check names ensures the jobs report, but does not necessarily make vulnerability findings fail the merge"). This is the documentation half of the two options its review offered.

Scope

Risk

  • Risk level: low
  • Failure mode: documentation only, no executable surface. The realistic failure is the one this fixes: someone reads gate 2 as a vulnerability gate and ships branch protection that does not block vulnerable merges.
  • Rollback: git revert. Nothing depends on the file.

Verification

Head 1853285.

  • All four security.yml suppressions confirmed at sourcecontinue-on-error: true (lines 31, 53), || true (lines 52, 69), exit-code: '0' (line 94). Each job's only unguarded steps are toolchain/build steps, so each fails on a broken scanner and never on a finding.
  • The six do all run on every PR to main — re-checked rather than assumed, because the claim would be wrong if any were absent. security.yml and codeql-analysis.yml both trigger on pull_request: branches: [main] with no paths: filter; neither has a job-level if:; CodeQL's matrix is ['javascript', 'python'] with fail-fast: false. docs: require the security scans gate 2 lists but never required #1410's own premise holds.
  • verification.yml's "Gate 3: Security Scan" is not one of the six — different check name, and it triggers only on PRs targeting refactor/hybrid-infra-v2. Checked because a name collision there would have made the required list strand every PR to main.
  • Independent corroboration — CodeRabbit's full review of docs: require the security scans gate 2 lists but never required #1410 flags the same enforcement caveat from the same evidence.
  • Required CI — suppressed by stacking (base is a feature branch); agent-completion/truth-gate is red for the arming defect fix(ci): arm the truth gate only on a real dispatch contract #1409 repairs, which pull_request_target runs from the base so no commit here can affect.
  • Review threads resolved — none yet.

Production evidence

Not applicable — one markdown file, no runtime or build surface. Under the policy this file defines, gate 4 scopes previews to apps/web/**.

Agent handoff

Agent provenance

Agent-authored. No agent-lock-manifest is filled in: the manifest declares a run_id and agent_login the truth gate treats as evidence and expects corroborated by append-only result comments. There is no dispatch record behind this change, and fabricating those values would inject false evidence into the mechanism #1409 is repairing.


Generated by Claude Code

Gate 2 now requires the six security contexts, but every one of them
reports success with findings present, so the list as written reads as a
vulnerability gate it is not.

  npm-audit      continue-on-error: true on the audit step
  python-safety  || true and continue-on-error: true
  bandit         || true on bandit -r src
  trivy          exit-code: '0' passed to trivy-action
  CodeQL x2      alerts route to the Security tab; blocking them is
                 code-scanning merge protection, not a required check

Records what requiring them does buy — detection of a scanner that broke
or stopped running — and states that making findings block is a change to
security.yml and to merge protection, not a branch-protection edit.

Same defect class as the list this stacks on: a list that does not say
what it means.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019VsKKnTAKoWJr49zQP95kK
@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
v0-uvai Building Building Preview, v0 Aug 7, 2026 5:17pm

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • [‘architecture-gap’, ‘bug’, ‘ci-cd’, ‘ci/cd’, ‘copilot-rabbit’, ‘documentation’, ‘duplicate’, ‘enhancement’, ‘frontend’, ‘github_actions’, ‘good first issue’, ‘help wanted’, ‘high-priority’, ‘invalid’, ‘javascript’, ‘ml-model’, ‘needs-triage’, ‘pipeline-critical’, ‘placeholder-code’, ‘priority:high’, ‘python’, ‘python:uv’, ‘question’, ‘styling’, ‘tests’, ‘v0’]

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f2a6f725-7009-478b-ba82-8c12a2c64cde

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Agent Completion Truth Gate: BLOCKED

Reasons: invalid_payload

Machine-readable verdict
{
  "details": {
    "collection_errors": [
      "incomplete_linked_issue_contract",
      "missing_closing_issue_reference",
      "linked_issue_not_agent_task",
      "missing_intent_snapshot",
      "missing_agent_run_id",
      "missing_agent_login"
    ],
    "invalid_fields": [
      "policy.agent_login",
      "policy.run_id"
    ]
  },
  "reasons": [
    "invalid_payload"
  ],
  "verdict": "blocked"
}

Workflow evidence

Copy link
Copy Markdown
Owner Author

Closing as superseded — lost a race, and the version that won is better.

#1410 pushed 81a0785 ("docs: say what requiring the security checks actually buys") at 17:16 UTC, about two minutes after CodeRabbit posted the enforcement finding and while this branch was being prepared. It lands the same fix in the paragraph it belongs to, and goes further than this PR did: it also records the removal of each continue-on-error / || true / exit-code: '0' escape under What is deliberately not here, so making findings block becomes a staged next step rather than an unstated gap.

Keeping this open would leave a competing edit to the same paragraph — the exact thing the governance gate exists to catch, and the backlog problem this run should reduce rather than add to.

For the record, the finding was reached independently twice before either fix landed: from the workflow source here, and by CodeRabbit on #1410. Both agree on the mechanism — the four security.yml jobs fail only on toolchain or build breakage, never on findings, and the two CodeQL contexts route alerts to the Security tab, where blocking is code-scanning merge protection rather than a required-check setting.

The agent-completion/truth-gate failure on this head is the known arming defect #1409 repairs, not a signal about this diff. #1413 is closed alongside this as covered by #1412 / #1410.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants