feat(workflow-sdk): add pr_url verification check#852
Conversation
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR adds support for a ChangesPR URL verification feature
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
f3be8c8 to
042d3b2
Compare
Lets workflow authors gate step completion on the worker actually opening a pull request. Without this, evidence-based completion accepts "tests pass + clean process exit + files modified" as sufficient — which is how multiple Track A-F implementation steps of the proactive-runtime M3 swarm returned OWNER_DECISION: COMPLETE while leaving every PR unopened. - workflow-types: add 'pr_url' to the VerificationCheck type union with a doc-string explaining the `value` semantics - sdk: scan worker output for https://github.com/<owner>/<repo>/pull/<N> URLs, optionally filtered by an `<owner>/<repo>` qualifier; reuses the existing stripInjectedTaskEcho path so PR URLs echoed inside the injected task brief don't satisfy the gate - tests: 8 new vitest cases (4 against runVerification + 4 against the exported findPrUrl helper) covering happy path, missing URL, wrong repo, qualifier case-insensitivity, and injected-task echo guard Workflow author guidance for using this check (and pairing it with `createGitHubStep({ action: 'createPR' })` so the URL flows through the github-primitive instead of raw `gh pr create`) lives in the canonical skills repo: AgentWorkforce/skills @ feat/pr-url-verification-guidance.
042d3b2 to
99c2356
Compare
Summary
Adds a new `'pr_url'` value to the `VerificationCheck` type union. Workflow authors can now gate step completion on the worker leaving behind a GitHub PR URL — closing the loophole that let multiple Track A–F workers of the proactive-runtime M3 swarm post `OWNER_DECISION: COMPLETE` with green tests and a clean process exit but no PR.
Motivation
`resolveOwnerCompletionDecision` already runs `step.verification` before honoring `OWNER_DECISION: COMPLETE` (a failing check throws `failed_verification`). The gap was that no built-in check expressed "this step must publish a PR." Workflow authors could compose it with `type: 'custom'` and a shell command, but the friction is high enough that nobody did — including the M3 workflow, which relied on a free-text brief instructing workers to `gh pr create`. None of the workers did, and evidence-based completion (positive-conclusion text + process-exit=0) waved them through.
API
```ts
// any GitHub PR URL in step output counts
verification: { type: 'pr_url', value: '' }
// require the PR belong to a specific repo
verification: { type: 'pr_url', value: 'AgentWorkforce/relaycast' }
```
Repo qualifier is case-insensitive. PR URLs echoed inside the injected task brief are stripped before scanning (via the existing `stripInjectedTaskEcho` path), so reference URLs in the brief don't satisfy the gate.
Test plan
Follow-ups
🤖 Generated with Claude Code