Skip to content

feat(hooks): add optional quality gate before task can enter done - #666

Merged
carlospedreira merged 3 commits into
andresharpe:mainfrom
elmaljevo:feature/656-verify-quality-gate
Aug 3, 2026
Merged

feat(hooks): add optional quality gate before task can enter done#666
carlospedreira merged 3 commits into
andresharpe:mainfrom
elmaljevo:feature/656-verify-quality-gate

Conversation

@elmaljevo

@elmaljevo elmaljevo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Linked issue

Closes #656

Summary of changes

  • The enter-done verify chain checks git hygiene (clean tree, pushed branch, markdown refs, framework integrity) but never runs tests or a linter — a task can reach done (and open a PR) with failing tests, only to fail CI later.
  • Add src/hooks/verify/05-verify-quality.ps1: an opt-in verify hook that runs a project's configured test_command/lint_command and blocks the done transition when either fails. Off by default (quality_gate.enabled: false in content/settings/settings.default.json) — a project opts in via its own settings override (e.g. .bot/.control/settings.json).
  • Register the new script in src/hooks/verify/config.json (core: false, required: false — must stay optional since it's off by default).
  • Bump enter-done's max_duration from 120s to 600s so a real test suite has room to run once a project enables the gate.
  • Document the new setting in src/hooks/verify/README.md.

Check commands run in a child pwsh process (not Invoke-Expression in-process) — same trust level as a CI yaml step, without touching the hook's own runspace.

Screenshots / recordings

N/A — backend-only change, no UI.

Testing notes

  • Added tests/Test-VerifyQuality.ps1 (19 assertions): disabled by default, enabled-but-unconfigured (no-op), passing test+lint, failing test only, failing lint only, test-only configuration.
  • tests/Test-VerifyQuality.ps1 — 19 passed, 0 failed
  • tests/Test-Structure.ps1 — 360 passed, 0 failed, 2 skipped
  • tests/Test-Hooks.ps1 — 51 passed, 0 failed
  • tests/Test-Components.ps1 — 851 passed, 0 failed, 1 skipped (pre-existing, unrelated skip)
  • tests/Test-ProcessDispatch.ps1 — 44 passed, 0 failed

Checklist

  • Tests added or updated
  • Docs updated (if behaviour changed)
  • Linked issue exists
  • Follows the contribution guide

Follow-up hardening

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in “quality gate” to the enter-done verification flow so projects can block the done transition when configured test and/or lint commands fail, reducing CI failures after PR creation.

Changes:

  • Introduces 05-verify-quality.ps1 verify hook that (optionally) runs configured quality_gate.test_command / quality_gate.lint_command.
  • Extends the enter-done transition timeout to allow longer-running suites, and registers/documents the new verify hook + settings.
  • Adds Layer 1 tests covering disabled/default behavior and pass/fail scenarios for test and lint commands.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/Test-VerifyQuality.ps1 Adds coverage for the new optional quality gate verify hook behavior.
tests/Test-Structure.ps1 Asserts the new verify script is registered as non-core and the default setting is disabled.
tests/Run-Tests.ps1 Wires the new Layer 1 test file into the test runner.
src/runtime/Plugins/Hooks/Transitions/enter-done/metadata.json Increases transition max duration to accommodate running tests/lint.
src/hooks/verify/README.md Documents how to enable/configure the new quality gate.
src/hooks/verify/config.json Registers the new verify script as optional (core: false, required: false).
src/hooks/verify/05-verify-quality.ps1 Implements the optional quality gate execution and JSON reporting.
content/settings/settings.default.json Adds quality_gate defaults (disabled, commands null).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/hooks/verify/05-verify-quality.ps1 Outdated
Comment thread src/hooks/verify/05-verify-quality.ps1
Comment thread src/hooks/verify/05-verify-quality.ps1
Copilot AI review requested due to automatic review settings July 24, 2026 12:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Elma Ljevo and others added 3 commits August 3, 2026 09:40
…dresharpe#656)

The enter-done verify chain checked git hygiene (clean tree, pushed
branch, markdown refs, framework integrity) but never ran tests or a
linter, so a task could reach 'done' and open a PR with failing tests.

Add 05-verify-quality.ps1: an opt-in verify hook that runs configured
test_command/lint_command and blocks the done transition on failure.
Off by default (quality_gate.enabled=false in settings.default.json) —
a project opts in via its own settings override. Bump enter-done's
max_duration from 120s to 600s so real test suites have room to run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Add a section for 05-verify-quality.ps1 covering the quality_gate
settings shape and default no-op behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Resolve BotRoot via Get-DotbotProjectBotPath (walks up to find .bot/)
  instead of assuming cwd is exactly the project root, so the hook still
  reads quality_gate settings correctly if invoked from a subdirectory.
- Redirect the check command's stderr at the external pwsh call site
  instead of inside the -Command string, so it also catches the child
  pwsh's own parser/startup errors and doesn't need to double-escape
  quotes/redirections already present in the configured command.
- Name the failing check(s) in the top-level JSON message (not just in
  'failures') — enter-done only surfaces 'message' when reporting a
  verify failure, so a generic "Quality gate failed" was undiagnosable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@carlospedreira
carlospedreira force-pushed the feature/656-verify-quality-gate branch from 71a7358 to 0b6c74e Compare August 3, 2026 08:55
Copilot AI review requested due to automatic review settings August 3, 2026 08:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/hooks/verify/05-verify-quality.ps1:62

  • Resolve-DotbotModulePath builds its initial candidates from $frameworkRoot = Join-Path $PSScriptRoot ".." "..", which resolves to src/hooks/ for this script location. Paths like src/hooks/src/runtime/... (and src/hooks/runtime/...) don’t exist in this repo, so the function effectively depends on git rev-parse --show-toplevel to find modules. If git is unavailable or $PSScriptRoot isn’t inside a git worktree, the quality gate will silently skip (and pass) even when enabled. Prefer resolving module paths from $env:DOTBOT_HOME (when set) and/or a deterministic path relative to $PSScriptRoot, with git only as a fallback.
function Resolve-DotbotModulePath {
    param([Parameter(Mandatory)][string]$ModuleName)

    $frameworkRoot = Join-Path $PSScriptRoot ".." ".."
    $found = Resolve-FirstExistingPath @(

@carlospedreira carlospedreira left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto current main with the shared canonical-path fix. Verified the worktree-aware enter-done path and quality gate locally; full Layers 1-3 passed, and all required cross-platform CI plus Playwright checks are green. Approved for squash merge.

@carlospedreira
carlospedreira merged commit 5b0a3e9 into andresharpe:main Aug 3, 2026
6 checks passed
@github-project-automation github-project-automation Bot moved this from Inbox to Done in Dotbot Product Backlog Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

No test or lint gate before task is marked done and PR is created

3 participants