v0.32.0 — CLIPipe tool mode + Evaluator predicate fix#22
Merged
Conversation
…SS (0.31.1)
The old `!!(await predicate(result))` mapped any truthy return to
{status:'satisfied', pass:true}. A caller predicate that returned a
test-runner RESULT rather than a boolean — {exitCode:1,failures:3}, an
ENOENT crash object, a summary string '3 failing, 0 passing', a failure
count — laundered a FAILING check into a PASS. The optimistic-rounding
class of BA-4/5/6/7/13.
A non-boolean now throws a ValidationError naming the TYPE only (never
the value — audit-log safety, F16/BA-1). Thrown, it routes through
BA-15's runArbiter at recurse's verify slot: a caller wiring opts.evaluate
around an Evaluator predicate surfaces {incomplete, blocker:'broken-verifier'}
instead of a converged-shaped fake green. Standalone callers get a loud
error. Genuine true/false (sync or async) unchanged; null/undefined now
throws too (a non-answer is named, not silently rounded).
Reverses the BA-15 park of this same issue, whose justification rested on
the same wrong premise the feedback carried (that garbage coerced to
pass:false). RLM_PRD record corrected.
Proven by poc/rlmplans-predicate-coercion.mjs (pre-fix documented the bug,
now runs as a post-fix confirmation). +6 regression tests; suite 898 tests
(896 pass / 0 fail / 2 skipped); typecheck clean. Docs synced (CHANGELOG,
eval-assist PRD, RLM PRD, context).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on (0.32.0)
A subscription CLI (`claude -p`, …) was a plain turn-provider: text in, text
out, tools silently ignored. This adds Option C — schema-validated tool
EMULATION — so a caller's tools work over the CLI, letting a Claude sub drive
a full agentic Loop (tools + turns) with NO metered-API spend, while
bareagent's own Loop keeps the cycle (governance, round accounting, spin
guards all still apply — unlike an MCP callback that hands turns to the CLI).
Opt in with { toolProtocol: 'claude' }; tools passed to generate() then
auto-use emulation (system-prompt manifest + JSON envelope, parsed back into
normalized toolCalls). New src/provider-clipipe-tools.js isolates the
claude-specific flags/schema/parse/probe so a second CLI slots in behind the
same seam (claude-only for now).
Design (validated by 5 POCs before building):
- CLI reduced to a bare brain: --tools '' --strict-mcp-config + --setting-sources ''
(a MEASURED ~18x cost drop, 37,423 -> 2,026 tokens/turn — the CLI else
auto-loads the cwd CLAUDE.md/memory every turn).
- Weak models FAIL LOUD + UPFRONT (probeCapability, default on): one probe asks
the model to obtain unknowable info via a tool; prose instead of a tool_call
throws a ProviderError naming the model, before real work. Behaviour-based,
never a model-list (BA-10). The probe MUST mirror real-task shape — a trivial
"call this tool" instruction is a false positive (haiku 4/4 yet fails real
tasks 0/5); the question shape sorts cleanly (sonnet 4/4, haiku 0/4).
Documented model floor: sonnet-class+ for tools, haiku fine for plain text.
- Malformed envelope -> loud ProviderError, never prose (BA-6 closed at parse).
- Tools with no toolProtocol -> plain-text (backward compat: a non-tool-calling
CLI legitimately sits in a Loop with mounted tools) + one-time warn.
Proven on the SHIPPED code end-to-end through a real Loop, multi-round, sonnet
(poc/clipipe-tools-05-shipped.mjs). +18 offline deterministic tests (stubbed
_spawn). Suite 916 tests (914 pass / 0 fail / 2 skipped); typecheck clean.
Also folds in the 0.31.x predicate-boolean fix under this 0.32.0 batch release.
Docs synced: CHANGELOG, CLAUDE.md, context, README, PRDs, worklist.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, redundancy correction) Grounding the full committed state revealed items 3 (persona) and 5 (scrub wording) were ALREADY resolved in a5af8de (the branch base) by better-designed audit2-* probes — my rlmplans-persona/scrub re-runs were redundant (deleted). - Item 4 (hung sensor): back the recurse.js caveat with poc/rlmplans-hung-sensor.mjs. - Item 6 (receipts): token-free audit — receipts already report as-executed (partition floor+width, scan window/passes, fanout steps.length). No change. - Worklist: close all items, record the grounding failure honestly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…review cleanup) Close the two low-severity notes from the 0.32.0 medium code-review: - Extract mapClaudeMeta() so the claude --output-format json usage/model/cost contract lives in ONE place, shared by tool-mode parseResult and the plain _parseClaudeJson preset (was duplicated across the two files). - Document that the one-per-instance capability probe's usage is unmetered (never flows to onLlmResult) — negligible for the subscription use case. Behavior-preserving: 914 pass / 0 fail / 2 skipped, typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Releases v0.32.0. Two deliverables, both validated end-to-end.
1. CLIPipe TOOL MODE (
84f6853)A subscription CLI (
claude -p) can now drive an agentic Loop with the caller's tools — run bareagent against a Claude subscription instead of metered API credits, without giving up Loop governance.--json-schema), envelope parsed back into normalizedtoolCalls. The Loop keeps ownership of the cycle (round accounting,maxConsecutiveDenials,maxIdenticalToolErrors, stop-reason classification all still apply).ProviderError, never laundered prose.--setting-sources ''(37,423 → 2,026 tokens/turn, measured).2. Evaluator predicate false-green fix (
b2e0b48)!!(await predicate(result))coerced any truthy non-boolean (a test-runner result object, a summary string, a failure count) intopass:true— a failing check laundered into a PASS (BA-4/5/6/7/13 optimistic-rounding family). Now a non-boolean throwsValidationError(type named only, never the value — audit-log safety), routing tobroken-verifierat recurse's verify slot.Review cleanup (
9dc6f16,e63bb97)/code-review mediumfindings closed: sharedmapClaudeMeta(kill duplicated claude-json parsing), documented the unmetered probe caveat, corrected the CHANGELOG suite count.Validation
tsc --noEmitclean.poc/clipipe-tools-05-shipped.mjs)./ship·/security·/diff-reviewgates all clean (argv spawn, no shell; library scope).🤖 Generated with Claude Code