feat(hooks): add optional quality gate before task can enter done - #666
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in “quality gate” to the enter-done verification flow so projects can block the done transition when configured test and/or lint commands fail, reducing CI failures after PR creation.
Changes:
- Introduces
05-verify-quality.ps1verify hook that (optionally) runs configuredquality_gate.test_command/quality_gate.lint_command. - Extends the
enter-donetransition timeout to allow longer-running suites, and registers/documents the new verify hook + settings. - Adds Layer 1 tests covering disabled/default behavior and pass/fail scenarios for test and lint commands.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Test-VerifyQuality.ps1 | Adds coverage for the new optional quality gate verify hook behavior. |
| tests/Test-Structure.ps1 | Asserts the new verify script is registered as non-core and the default setting is disabled. |
| tests/Run-Tests.ps1 | Wires the new Layer 1 test file into the test runner. |
| src/runtime/Plugins/Hooks/Transitions/enter-done/metadata.json | Increases transition max duration to accommodate running tests/lint. |
| src/hooks/verify/README.md | Documents how to enable/configure the new quality gate. |
| src/hooks/verify/config.json | Registers the new verify script as optional (core: false, required: false). |
| src/hooks/verify/05-verify-quality.ps1 | Implements the optional quality gate execution and JSON reporting. |
| content/settings/settings.default.json | Adds quality_gate defaults (disabled, commands null). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…dresharpe#656) The enter-done verify chain checked git hygiene (clean tree, pushed branch, markdown refs, framework integrity) but never ran tests or a linter, so a task could reach 'done' and open a PR with failing tests. Add 05-verify-quality.ps1: an opt-in verify hook that runs configured test_command/lint_command and blocks the done transition on failure. Off by default (quality_gate.enabled=false in settings.default.json) — a project opts in via its own settings override. Bump enter-done's max_duration from 120s to 600s so real test suites have room to run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a section for 05-verify-quality.ps1 covering the quality_gate settings shape and default no-op behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Resolve BotRoot via Get-DotbotProjectBotPath (walks up to find .bot/) instead of assuming cwd is exactly the project root, so the hook still reads quality_gate settings correctly if invoked from a subdirectory. - Redirect the check command's stderr at the external pwsh call site instead of inside the -Command string, so it also catches the child pwsh's own parser/startup errors and doesn't need to double-escape quotes/redirections already present in the configured command. - Name the failing check(s) in the top-level JSON message (not just in 'failures') — enter-done only surfaces 'message' when reporting a verify failure, so a generic "Quality gate failed" was undiagnosable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
71a7358 to
0b6c74e
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (1)
src/hooks/verify/05-verify-quality.ps1:62
- Resolve-DotbotModulePath builds its initial candidates from
$frameworkRoot = Join-Path $PSScriptRoot ".." "..", which resolves tosrc/hooks/for this script location. Paths likesrc/hooks/src/runtime/...(andsrc/hooks/runtime/...) don’t exist in this repo, so the function effectively depends ongit rev-parse --show-toplevelto find modules. Ifgitis unavailable or$PSScriptRootisn’t inside a git worktree, the quality gate will silently skip (and pass) even when enabled. Prefer resolving module paths from$env:DOTBOT_HOME(when set) and/or a deterministic path relative to$PSScriptRoot, withgitonly as a fallback.
function Resolve-DotbotModulePath {
param([Parameter(Mandatory)][string]$ModuleName)
$frameworkRoot = Join-Path $PSScriptRoot ".." ".."
$found = Resolve-FirstExistingPath @(
carlospedreira
left a comment
There was a problem hiding this comment.
Rebased onto current main with the shared canonical-path fix. Verified the worktree-aware enter-done path and quality gate locally; full Layers 1-3 passed, and all required cross-platform CI plus Playwright checks are green. Approved for squash merge.
Linked issue
Closes #656
Summary of changes
enter-doneverify chain checks git hygiene (clean tree, pushed branch, markdown refs, framework integrity) but never runs tests or a linter — a task can reachdone(and open a PR) with failing tests, only to fail CI later.src/hooks/verify/05-verify-quality.ps1: an opt-in verify hook that runs a project's configuredtest_command/lint_commandand blocks thedonetransition when either fails. Off by default (quality_gate.enabled: falseincontent/settings/settings.default.json) — a project opts in via its own settings override (e.g..bot/.control/settings.json).src/hooks/verify/config.json(core: false,required: false— must stay optional since it's off by default).enter-done'smax_durationfrom 120s to 600s so a real test suite has room to run once a project enables the gate.src/hooks/verify/README.md.Check commands run in a child
pwshprocess (notInvoke-Expressionin-process) — same trust level as a CI yaml step, without touching the hook's own runspace.Screenshots / recordings
N/A — backend-only change, no UI.
Testing notes
tests/Test-VerifyQuality.ps1(19 assertions): disabled by default, enabled-but-unconfigured (no-op), passing test+lint, failing test only, failing lint only, test-only configuration.tests/Test-VerifyQuality.ps1— 19 passed, 0 failedtests/Test-Structure.ps1— 360 passed, 0 failed, 2 skippedtests/Test-Hooks.ps1— 51 passed, 0 failedtests/Test-Components.ps1— 851 passed, 0 failed, 1 skipped (pre-existing, unrelated skip)tests/Test-ProcessDispatch.ps1— 44 passed, 0 failedChecklist
Follow-up hardening