Skip to content

Add an AI review workflow for pull requests opened from forks - #407

Merged
haksungjang merged 2 commits into
mainfrom
feat/external-pr-review-workflow
Sep 6, 2026
Merged

Add an AI review workflow for pull requests opened from forks#407
haksungjang merged 2 commits into
mainfrom
feat/external-pr-review-workflow

Conversation

@haksungjang

Copy link
Copy Markdown
Contributor

Summary

`ai-review.yml` deliberately does not run on fork PRs (its own header explains why: GitHub withholds secrets from `pull_request` events on forks, and `pull_request_target` would run with our secrets against a branch the contributor controls). This adds that coverage back the way that stays safe.

How it avoids what ai-review.yml's warning is about

  • Never checks out the fork's head. No `ref:` override, so on `pull_request_target` the checkout resolves to the trusted base branch.
  • The contributor's diff and PR title/body are fetched as plain text through the API and only read by the model. Nothing here executes, imports, or evaluates any of it.
  • The only side effect is a rendered PR comment. `permissions:` grants nothing else.
  • Diff and description are fenced as untrusted data, same defence `ai-review.yml` uses for scanner output. Mentions in the model's reply are defused before posting.

Cost controls

`pull_request_target` itself runs with no approval click, unlike `ci.yml`'s plain `pull_request` trigger. Two things bound spend:

  • `external_review.py` caps the diff (`MAX_DIFF_CHARS`) and the reply (`MAX_TOKENS`) per call.
  • The workflow counts its own runs in the trailing 24 hours and skips the paid call past `MAX_RUNS_PER_DAY` (20), posting a static notice instead.

Neither substitutes for a hard spend limit set on the API key or workspace in the Anthropic Console before the secret is added - that's the real backstop.

Status

Dormant: no `ANTHROPIC_API_KEY` secret exists in this repository yet, same as `ai-review.yml`. Every step is skipped and the job succeeds at no cost until one is added.

Verification

  • `tools/ai-review/external_review_selftest.py`: all checks pass (offline, no key needed).
  • `actionlint`, `ruff`, em-dash lint, SPDX header check: all clean.
  • Network hop itself is unverified, same caveat `ai-review.yml` carries.

ai-review.yml's own header explains why it does not run on fork PRs:
GitHub withholds secrets from pull_request events triggered by forks,
and switching to pull_request_target to work around that would run with
our secrets against a branch the contributor controls.

This adds that coverage back using pull_request_target the way that
stays safe: the job never checks out the fork's head (no ref: override,
so it resolves to the trusted base branch), the contributor's diff and
PR metadata are fetched as plain text through the API and read only by
the model, and the only side effect is a rendered PR comment. Findings
are fenced as untrusted data with the same defence ai-review.yml already
uses for scanner output.

Two cost controls, since pull_request_target itself runs with no
approval click the way ci.yml's plain pull_request trigger does:
external_review.py caps the diff and the reply per call, and the
workflow counts its own runs in the trailing 24 hours and skips the
paid call past a fixed ceiling. Neither substitutes for a hard spend
limit on the API key itself, set before the secret is added.

Dormant with no ANTHROPIC_API_KEY secret present, same as ai-review.yml.
… runner

test_static_checks_match_ci.py caught this: the new ci.yml step had no
entry in tools/static-checks/run.py's CHECKS, so the local runner would
report success without having run it.
@haksungjang
haksungjang merged commit 08a96c7 into main Sep 6, 2026
24 checks passed
@haksungjang
haksungjang deleted the feat/external-pr-review-workflow branch September 6, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant