Skip to content

MERGE_POLICY gate 2 reads as a vulnerability gate, but all six required security scans pass with findings present #1413

Description

@groupthinking

Problem

#1410 moves the six security contexts into gate 2's unconditional required list — correct, and #1412 covers that. This issue was the layer underneath it.

Every one of those six reports success when findings exist. Requiring them by name attests that each scan ran; it does not stop a HIGH/CRITICAL finding from merging. As written at the time, gate 2 read as a vulnerability gate it is not.

Verified against the workflow definitions on main, not inferred from check-run status:

Check Fails the job on Why findings don't fail it
npm-audit npm install breaking continue-on-error: true on the npm audit --audit-level=high step
python-safety pip install safety breaking safety scan ... || true and continue-on-error: true
bandit pip install bandit breaking bandit -r src ... || true
trivy the docker build breaking exit-code: '0' passed to trivy-action
Security Scan - python, Security Scan - javascript CodeQL itself erroring alerts route to the Security tab; blocking on them is code-scanning merge protection, which is not a required-check setting

The four security.yml jobs have no path filters and no job-level if:, and CodeQL's matrix is unguarded, so all six do run on every PR to main#1410's own claim holds. The gap was only about what passing means.

Resolution — closed as already fixed

Fixed on #1410 by 81a0785, which states that gate 2 requires these checks to run and complete, records the per-check reason each cannot fail on findings, and adds the escape-removal steps under What is deliberately not here so blocking enforcement is a staged next step rather than an unstated gap. All four acceptance criteria below are met there.

This issue is closed rather than left open against a fix that already exists, and #1414 (opened for it) is closed as superseded.

Acceptance criteria

  • Gate 2 states plainly that the six security contexts gate scan execution, not absence of findings.
  • The per-check reason each one cannot fail on findings is recorded, so the claim is checkable rather than asserted.
  • It is stated that making findings block is a change to security.yml and to code-scanning merge protection — not a branch-protection edit.
  • No change to which checks are required — that stayed MERGE_POLICY.md gate 2 names six security checks and requires none of them #1412's scope.

Still open, deliberately

Actually making the scans fail on findings. Removing the continue-on-error / || true / exit-code: '0' escapes flips CI red across the repo the moment any HIGH/CRITICAL exists, so it is a maintainer's call about tolerance for known vulnerabilities and wants its own PR. #1410 records the exact escapes to remove.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions