Skip to content

ci: isolate benchmark comment permissions#64

Merged
TimBeyer merged 1 commit into
masterfrom
codex/harden-benchmark-comments
Jul 18, 2026
Merged

ci: isolate benchmark comment permissions#64
TimBeyer merged 1 commit into
masterfrom
codex/harden-benchmark-comments

Conversation

@TimBeyer

Copy link
Copy Markdown
Owner

Summary

  • make every pull-request workflow explicitly read-only and prevent checkout from persisting GitHub credentials
  • move benchmark PR comments into a trusted default-branch workflow_run workflow
  • separate untrusted artifact parsing from the write-capable publication job
  • pin third-party actions to immutable commit SHAs and update their current major versions

Security model

The previous benchmark workflow executed pull-request-controlled dependency lifecycle scripts and benchmark code while its checkout retained the repository token. The repository also inherited a write-capable default token, so a compromised benchmark dependency or competitor could recover that credential.

This change keeps benchmark execution untrusted and read-only. A separate default-branch workflow:

  1. downloads exact, allowlisted artifacts using only actions: read
  2. bounds and sanitizes their Markdown (including HTML, links, mentions, control characters, and bidirectional controls)
  3. passes only base64-encoded sanitized text to a distinct job with issues: write
  4. re-resolves and validates the open PR, repository, base branch, and exact head SHA before publishing

The publishing job never checks out or executes pull-request code and never receives the artifact filesystem.

The repository setting was also changed out of band to default workflow permissions = read and “Allow GitHub Actions to create and approve pull requests” = disabled, closing the exposed token path immediately.

Validation

  • npm run format
  • npm run lint
  • npm test — 70 passing
  • npm run build
  • Prettier and YAML parsing for all touched workflows
  • embedded github-script compilation
  • immutable action-pin, explicit-permission, and non-persistent-checkout audit
  • malicious-artifact integration test covering HTML/comment truncation, mentions, links, and bidirectional controls
  • git diff --check

A normal npm ci on macOS ARM64/Node 25 reaches the existing v8-profiler-next native build failure; npm ci --ignore-scripts succeeded for local validation.

Bootstrap note

This PR itself cannot receive the new benchmark comment: GitHub only runs the privileged workflow_run definition after it exists on the default branch. Benchmark comments begin with subsequent PR runs after merge.

Run pull-request workflows with read-only tokens and discard checkout credentials. Publish sanitized benchmark artifacts from trusted default-branch code.
@TimBeyer
TimBeyer marked this pull request as ready for review July 18, 2026 15:03
@TimBeyer
TimBeyer merged commit e74f1e1 into master Jul 18, 2026
11 checks passed
@TimBeyer
TimBeyer deleted the codex/harden-benchmark-comments branch July 18, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant