Release CodeTruss CLI v0.2.40 - #24
Conversation
Bring the public mirror up to the shipped release. The mirror sat at 0.2.39 while codetruss.com served 0.2.40; the newest attested tag should never trail the bytes the site hands out. Source is mirrored from 63088fc, not from the version commit 1833756. An adversarial review after the version bump found a verify-then-execute TOCTOU in the grammar-pack loader — it hashed each artifact by path and then re-opened the same path to execute it — and the fix rebuilt the artifact, so 63088fc is the commit whose trees hash to what the site serves. 128 files across packages/cli and packages/analyzer-engine, zero content mismatches and zero file-mode mismatches against that commit's trees. The count grows from 117 because the grammar pack adds four source modules, four test files, and three build scripts; packages/cli/SBOM.cdx.json stays generated and gitignored here, as it has since 0.2.13. `pnpm release:artifact` in a fresh clone of this branch rebuilds the bundle to the exact published digest 5d64313b8b60acbd1f93e2246557967885a98fdc8c486ea7b2a6417fd8acdac2, and `pnpm release:verify` confirms it byte-for-byte against the immutable website archive now recorded in release-reference.json. The archive, its SBOM, the latest.json manifest, and all seven grammar-pack artifacts downloaded from codetruss.com this run are byte-identical to the ones in this tree. - Add the immutable 0.2.40 archive, checksum, and SBOM to public/downloads and repoint the latest.* aliases and manifest. The 0.2.39 archive stays: it is a tagged, attested release. - Add public/downloads/grammars — the python-1.0.0 pack (three artifacts and their .sha256 sidecars) and codetruss-grammars-latest.json. These are not optional here: a pack versions independently of the CLI, the site is its only download origin, and the new tests install from these exact published bytes over loopback rather than from a fixture, so the mirror cannot build or test itself without them. - Update release-reference.json to the published archive, its SBOM, and the bundle digest 7dcf9a22457c6790c08b2ddc1186f1c73213e58ad6920aa7f164c8ce2d3e9076. - Mirror the source 0.2.40 carries — the grammar pack loader, its pinned manifest, the parser that executes only verified buffers, the `grammars` command, the Python arm of the local security pass, the v4 receipt renderer and its frozen v3 predecessor, and their tests — plus the packaged changelog, and regenerate the root changelog from it so the release body stays byte-identical to what ships inside the archive. - Repoint the version-pinned README install and verification examples, list the new `grammars` command, and say that Python joins the local security pass when a pack is installed. The receipt excerpt keeps its "real 0.2.36 run" provenance and its `local-registry-v2`, 13-analyzer wording: 0.2.40 still ships a frozen v2 renderer that reproduces that wording byte for byte, so the excerpt is still what that run printed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe CLI moves to version 0.2.40 and adds optional, pinned Python grammar packs. It supports secure installation, verified in-memory parser loading, full local Python SAST coverage, runtime reuse, receipt profile v4, v3 compatibility, and updated release artifacts. ChangesPython grammar packs and local analysis
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant GrammarPack
participant GrammarParser
participant LocalSast
participant Receipt
CLI->>GrammarPack: install pinned Python artifacts
GrammarPack-->>CLI: return verified installed pack
LocalSast->>GrammarParser: load Python parser
GrammarParser->>GrammarPack: re-verify installed artifacts
GrammarPack-->>GrammarParser: return verified buffers
GrammarParser-->>LocalSast: return parser
LocalSast->>Receipt: record Python coverage and findings
Receipt-->>CLI: render analysis receipt
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (4)
packages/cli/test/receipt.test.ts (1)
235-345: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd a signed round-trip and a zero-scanned case to the Python disclosure suite.
The eight tests here cover the wording well. Two gaps remain.
First, every Python test calls
renderMarkdownonly. The v3 test on lines 212-225 is the sole case that goes throughwriteReceiptandverifyReceipt.verifyReceiptre-renders the Markdown and requires an exact match against the signed JSON. No test proves a receipt carrying Python metrics survives that round trip. Add onewriteReceipt/verifyReceiptcase built frompythonFixture.Second, no test covers
pythonPackStatus: 'verified'withpythonFilesScanned: 0. That combination falls to the default disclosure branch inreceipt.ts. See the related comment onpackages/cli/src/receipt.tslines 203-225.Apply both upstream rather than in this mirror.
Based on learnings, treat all files under
packages/cli/as immutable mirrored release source and do not introduce mirror-only edits.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/test/receipt.test.ts` around lines 235 - 345, Add two upstream tests to the Python disclosure suite: a signed round-trip using pythonFixture with writeReceipt and verifyReceipt, asserting Python metrics survive exact re-render verification, and a verified-pack case with pythonFilesScanned: 0 that asserts the default disclosure wording. Do not modify files under packages/cli/, which are immutable mirrored release sources.Source: Learnings
public/downloads/grammars/python-1.0.0/tree-sitter.js (1)
1-1: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winDo not modify this vendored artifact despite the static analysis findings.
Biome, OpenGrep, and ast-grep flag
eval(), non-literal filesystem paths, and non-literalRegExpin this file. All of them are in generated Emscripten andweb-tree-sittercode. They are not authored here.This file must stay byte-identical to
node_modules/web-tree-sitter/tree-sitter.js, becauseverify-grammar-packs.mjscompares the two buffers and the CLI checks the file against a compiled-in digest. Any edit breaks the digest and every install. Exclude this path from lint and SAST configuration instead of changing the bytes.The residual risk is real but is controlled elsewhere: these bytes are executed in memory by the parser loader, and the only control is the pinned SHA-256. See the related comment on
packages/cli/scripts/verify-grammar-packs.mjs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@public/downloads/grammars/python-1.0.0/tree-sitter.js` at line 1, Do not modify the vendored TreeSitter artifact or its generated Emscripten code; preserve it byte-identical to the package copy. Exclude this artifact from Biome, OpenGrep, and ast-grep findings, and update the relevant lint/SAST configuration rather than changing the file. Ensure the existing verify-grammar-packs digest and byte-comparison workflow remains intact.Source: Linters/SAST tools
packages/cli/test/grammar-pack.test.ts (1)
358-365: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe parity test checks length, not content.
The test name says the pinned manifest "matches the artifacts this repository publishes", but the assertion compares only
bytes.length. A published artifact with the right length and wrong content passes here. The pinnedsha256is the value that actually gates execution, so it is the value this test should compare.The install tests catch such a mismatch indirectly, because the loopback fixture serves these same files and
downloadVerifiedchecks the digest. That makes this a clarity gap rather than a coverage hole, but the direct assertion is one line and names the real invariant.Note the mirror constraint:
packages/cli/tracks the published archive byte-for-byte, so this belongs in the private monorepo and a later released version.♻️ Proposed assertion
for (const file of pack.files) { const bytes = await readFile(join(publishedDir, `${pack.name}-${pack.version}`, file.name)) expect(bytes.length).toBe(file.bytes) + expect(createHash('sha256').update(bytes).digest('hex')).toBe(file.sha256) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/test/grammar-pack.test.ts` around lines 358 - 365, Update the artifact parity test around PINNED_GRAMMAR_PACKS to compute each published file’s SHA-256 digest and compare it with the pinned file.sha256 value, replacing the bytes.length assertion. Keep the existing iteration and file-loading behavior unchanged.Source: Learnings
packages/cli/src/grammar-parser.ts (1)
107-113: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winThe cache also retains runtime failures for the life of the process.
runtimeCachestores the promise whetherbuildParserresolves to a parser or to{ reason }. The comment at lines 58-78 justifies caching a successful runtime, and that reasoning is sound: the digests are compiled-in constants, so a cached parser cannot be stale.That reasoning does not extend to failures. The doc at lines 49-54 names the runtime failure causes as an OOM, memory pressure, or
MAP_FAILED— machine conditions, not pack properties. Once one of them is cached, every later load in the process returns the same failure without retrying, including the second analysis of a review. A transient condition becomes a permanent one for the run.The receipt still reports the failure honestly, so this is lost coverage rather than a false claim. Caching only the success keeps the correctness argument intact and lets a transient failure retry.
Note the mirror constraint:
packages/cli/tracks the published archive byte-for-byte, so this belongs in the private monorepo and a later released version.♻️ Sketch of the change
let cached = runtimeCache.get(state.dir) if (!cached) { cached = buildParser(state, grammarFile.name, language) runtimeCache.set(state.dir, cached) } const result = await cached - if ('reason' in result) return { status: 'failed', kind: 'runtime', reason: result.reason } + if ('reason' in result) { + // A runtime that would not start is a property of this machine right now, + // not of the pinned bytes. Do not freeze it for the rest of the process. + if (runtimeCache.get(state.dir) === cached) runtimeCache.delete(state.dir) + return { status: 'failed', kind: 'runtime', reason: result.reason } + } return { status: 'verified', parser: result.parser, languages: new Set([language]) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/grammar-parser.ts` around lines 107 - 113, Update the runtimeCache handling around buildParser so failed results containing reason are removed from the cache before returning the runtime failure. Retain cached promises for successful parsers, allowing subsequent loads to retry after transient runtime failures while preserving the existing failure receipt.Source: Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/cli/package.json`:
- Around line 45-46: Update the failure guidance associated with the release and
verification grammar scripts in package.json to reference an existing workspace
command, rather than the nonexistent root grammars:release script. Keep the
guidance aligned with the commands exposed by release:grammars and
verify:grammars.
In `@packages/cli/scripts/build-grammar-packs.mjs`:
- Around line 134-145: Update the release validation flow to regenerate
grammar-pack metadata from the published grammar files and compare it
byte-for-byte with packages/cli/src/grammar-pack-manifest.ts, failing on any
difference; anchor this at buildGrammarPacks or the existing verify:Release
path. Also simplify the manifest serialization in build-grammar-packs.mjs by
removing the no-op split/join chain while preserving generated output.
In `@packages/cli/scripts/verify-grammar-packs.mjs`:
- Around line 96-106: Replace the substring-based digest check in the manifest
verification flow with a structural comparison against the imported compiled-in
pin: match each pack by name and version, then verify every file’s name, URL,
byte count, and SHA-256 digest. Update the upstream generator/source rather than
the mirrored packages/cli file, and release a new version carrying the fix.
In `@packages/cli/src/grammar-pack.ts`:
- Around line 289-292: Update downloadVerified to create an abortable timeout
for the grammar archive request, pass its signal to fetch, and reuse that signal
while reading/pumping response.body. Ensure the timeout is cleared when the
download completes or fails, while preserving existing response validation and
verification behavior.
In `@packages/cli/src/local-sast.ts`:
- Around line 196-209: The rules metric in the receipt metrics block should
reflect the rule set actually executed: use the complete rule-set count when the
verified Python pack runs without specific rule IDs, while retaining
CLI_SAST_RULE_IDS.size for the JavaScript-only path. Update the logic
surrounding pythonStatus and pythonRuleIds without changing the neighbouring
Python metrics.
- Around line 179-193: The local SAST result construction must treat failed
Python loaders as pass failures when Python files are present. Update the logic
around pythonStatus, pythonFilesScanned, and the complete/detail fields in the
returned result so digest or runtime loader failures mark the pass incomplete
and expose the loader reason through detail/error, including out-of-band
pack-check failures.
In `@packages/cli/src/receipt.ts`:
- Around line 203-225: Update pythonDisclosureLines() to handle verified Python
grammar with Python files present but none scanned, explicitly stating that
Python was not analyzed while preserving the existing disclosure wording for
other languages. Add a receipt test covering Python inputs where every scan is
skipped and the resulting status is verified with zero files scanned.
---
Nitpick comments:
In `@packages/cli/src/grammar-parser.ts`:
- Around line 107-113: Update the runtimeCache handling around buildParser so
failed results containing reason are removed from the cache before returning the
runtime failure. Retain cached promises for successful parsers, allowing
subsequent loads to retry after transient runtime failures while preserving the
existing failure receipt.
In `@packages/cli/test/grammar-pack.test.ts`:
- Around line 358-365: Update the artifact parity test around
PINNED_GRAMMAR_PACKS to compute each published file’s SHA-256 digest and compare
it with the pinned file.sha256 value, replacing the bytes.length assertion. Keep
the existing iteration and file-loading behavior unchanged.
In `@packages/cli/test/receipt.test.ts`:
- Around line 235-345: Add two upstream tests to the Python disclosure suite: a
signed round-trip using pythonFixture with writeReceipt and verifyReceipt,
asserting Python metrics survive exact re-render verification, and a
verified-pack case with pythonFilesScanned: 0 that asserts the default
disclosure wording. Do not modify files under packages/cli/, which are immutable
mirrored release sources.
In `@public/downloads/grammars/python-1.0.0/tree-sitter.js`:
- Line 1: Do not modify the vendored TreeSitter artifact or its generated
Emscripten code; preserve it byte-identical to the package copy. Exclude this
artifact from Biome, OpenGrep, and ast-grep findings, and update the relevant
lint/SAST configuration rather than changing the file. Ensure the existing
verify-grammar-packs digest and byte-comparison workflow remains intact.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 264453e7-05ab-4b31-9b69-26e3cbf4f0bd
⛔ Files ignored due to path filters (2)
public/downloads/grammars/python-1.0.0/tree-sitter-python.wasmis excluded by!**/*.wasmpublic/downloads/grammars/python-1.0.0/tree-sitter.wasmis excluded by!**/*.wasm
📒 Files selected for processing (33)
CHANGELOG.mdREADME.mdpackages/cli/CHANGELOG.mdpackages/cli/package.jsonpackages/cli/scripts/build-grammar-packs.mjspackages/cli/scripts/grammar-pack-sources.mjspackages/cli/scripts/verify-grammar-packs.mjspackages/cli/src/cli.tspackages/cli/src/grammar-command.tspackages/cli/src/grammar-pack-manifest.tspackages/cli/src/grammar-pack.tspackages/cli/src/grammar-parser.tspackages/cli/src/local-sast.tspackages/cli/src/receipt.tspackages/cli/src/types.tspackages/cli/test/grammar-command.test.tspackages/cli/test/grammar-loader.test.tspackages/cli/test/grammar-pack.test.tspackages/cli/test/grammar-sast.test.tspackages/cli/test/receipt.test.tspublic/downloads/codetruss-cli-0.2.40.sbom.cdx.jsonpublic/downloads/codetruss-cli-0.2.40.tgzpublic/downloads/codetruss-cli-0.2.40.tgz.sha256public/downloads/codetruss-cli-latest.jsonpublic/downloads/codetruss-cli-latest.sbom.cdx.jsonpublic/downloads/codetruss-cli-latest.tgzpublic/downloads/codetruss-cli-latest.tgz.sha256public/downloads/grammars/codetruss-grammars-latest.jsonpublic/downloads/grammars/python-1.0.0/tree-sitter-python.wasm.sha256public/downloads/grammars/python-1.0.0/tree-sitter.jspublic/downloads/grammars/python-1.0.0/tree-sitter.js.sha256public/downloads/grammars/python-1.0.0/tree-sitter.wasm.sha256release-reference.json
| "release:grammars": "node scripts/build-grammar-packs.mjs", | ||
| "verify:grammars": "node scripts/verify-grammar-packs.mjs", |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check which grammar-related scripts exist across workspace manifests.
fd -H -t f 'package.json' -E node_modules --exec sh -c 'echo "== $1"; jq -r "(.scripts // {}) | to_entries[] | select(.key | test(\"grammar\")) | \"\(.key): \(.value)\"" "$1"' _ {}
# Description: Find every reference to the documented command strings.
rg -n 'grammars:release|release:grammars|verify:grammars' -g '!node_modules'Repository: DeliriumPulse/codetruss-cli
Length of output: 2144
🏁 Script executed:
#!/bin/bash
set -u
echo "== root package.json scripts =="
jq -r '.scripts // {} | to_entries[] | select(.key | test("grammar|release|verify"; "i")) | "\(.key): \(.value)"' package.json
echo
echo "== package scripts matching grammar (read-only, deterministic jq script per file) =='
python3 - <<'PY'
import json, subprocess
files = subprocess.check_output(["git","ls-files","package.json"], text=True).splitlines()
for path in files:
try:
data=json.load(open(path))
except Exception as e:
print(f"{path}: parse error {e}")
continue
scripts=data.get("scripts", {}) or {}
for k,v in scripts.items():
if "grammar" in k.lower():
print(f"{path}: {k}: {v}")
PY
echo
echo "== command references =="
rg -n 'grammars:release|release:grammars|verify:grammars' -g '!node_modules' || true
echo
echo "== candidate grammar script files =="
fd -H -t f 'grammar' -E node_modules . || true
echo
echo "== error guidance text =="
fd -H -t f '\.(mjs|js|ts)$' -E node_modules . | xargs rg -n 'run pnpm grammars:release|pnpm grammars:release|release:grammars|verify:grammars' || trueRepository: DeliriumPulse/codetruss-cli
Length of output: 384
🏁 Script executed:
#!/bin/bash
set -u
echo "== root package.json scripts =="
jq -r '.scripts // {} | to_entries[] | select(.key | test("grammar|release|verify"; "i")) | "\(.key): \(.value)"' package.json || true
echo
echo "== package scripts matching grammar (read-only, deterministic jq per file) =="
python3 - <<'PY'
import json, subprocess
files = subprocess.check_output(["git","ls-files","package.json"], text=True).splitlines()
for path in files:
try:
data=json.load(open(path, encoding="utf-8"))
except Exception as e:
print(f"{path}: parse error {e}")
continue
scripts=data.get("scripts") or {}
for k,v in scripts.items():
if "grammar" in k.lower():
print(f"{path}: {k}: {v}")
PY
echo
echo "== command references =="
rg -n 'grammars:release|release:grammars|verify:grammars' -g '!node_modules' || true
echo
echo "== candidate grammar script files =="
fd -H -t f 'grammar' -E node_modules . || true
echo
echo "== error guidance text =="
fd -H -t f '\.(mjs|js|ts)$' -E node_modules . | xargs rg -n 'run pnpm grammars:release|pnpm grammars:release|release:grammars|verify:grammars' || trueRepository: DeliriumPulse/codetruss-cli
Length of output: 580
Add or update the grammar-pack failure guidance.
release:grammars and verify:grammars run scripts/build-grammar-packs.mjs and scripts/verify-grammar-packs.mjs, but the workspace has no root grammars:release script or other grammar script. The error messages should point to a command that actually exists in the workspace.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/package.json` around lines 45 - 46, Update the failure guidance
associated with the release and verification grammar scripts in package.json to
reference an existing workspace command, rather than the nonexistent root
grammars:release script. Keep the guidance aligned with the commands exposed by
release:grammars and verify:grammars.
| export const PINNED_GRAMMAR_PACKS: readonly PinnedGrammarPack[] = ${JSON.stringify(manifestPacks, null, 2) | ||
| .split('\n') | ||
| .join('\n')} | ||
|
|
||
| /** Pack names this CLI build knows how to install. */ | ||
| export const GRAMMAR_PACK_NAMES: readonly string[] = PINNED_GRAMMAR_PACKS.map((pack) => pack.name) | ||
|
|
||
| export function pinnedGrammarPack(name: string): PinnedGrammarPack | undefined { | ||
| return PINNED_GRAMMAR_PACKS.find((pack) => pack.name === name) | ||
| } | ||
| ` | ||
| await writeFile(join(packageDir, 'src', 'grammar-pack-manifest.ts'), pin, 'utf8') |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Show the committed pin and confirm the release verifier covers it.
fd -t f 'grammar-pack-manifest.ts' --exec sed -n '1,40p' {}
fd -t f 'verify-release.mjs' --exec rg -n 'grammar|manifest' {}Repository: DeliriumPulse/codetruss-cli
Length of output: 1425
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## build-grammar-packs candidates"
fd -t f 'build-grammar-packs\.mjs|grammar-pack-manifest\.ts|verify-release\.mjs' | sort
echo
echo "## build-grammar-packs outline"
for f in $(fd -t f 'build-grammar-packs\.mjs'); do
echo "--- $f"
wc -l "$f"
ast-grep outline "$f" --view compact || true
sed -n '1,230p' "$f"
done
echo
echo "## parse release/manifest-related patterns"
for f in $(fd -t f 'verify-release\.mjs'); do
echo "--- $f"
rg -n 'manifest|packages/cli|grammar-pack-manifest|release metadata|pinned|verify' "$f" -C 2
done
echo
echo "## package scripts mentioning release/grammars"
for f in package.json pnpm-workspace.yaml; do
[ -f "$f" ] && { echo "--- $f"; sed -n '1,220p' "$f"; }
done
fd -t f 'package.json' -x sh -c 'echo "--- $1"; sed -n "1,220p" "$1" | rg -n "cli:release|release:grammars|release:verify|ts-node|mjs|build-grammar-packs" -C 2 || true' sh {}Repository: DeliriumPulse/codetruss-cli
Length of output: 10541
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## grammar-pack verifier candidate"
f=$(fd -t f 'verify-grammar-packs\.mjs' | head -n 1)
if [ -n "$f" ]; then
echo "--- $f"
wc -l "$f"
sed -n '1,240p' "$f"
else
echo "no verify-grammar-packs.mjs found"
fi
echo
echo "## deterministic test files mentioning grammar-pack-manifest / cli verify / validate"
for f in $(fd -t f 'test-.*\.mjs|.*\.mjs' packages/cli/scripts); do
rg -n 'grammar-pack-manifest|pnpm cli:release|release:grammars|verifyRelease|releasePackagePolicy|codetruss-cli-latest|grammar packs|validate' "$f" -C 2 || true
done
echo
echo "## read-only probe: JSON.stringify newline handling and .split().join() identity"
node - <<'JS'
const manifestPacks = [
{ name: 'python', files: [{ name: 'a.py', url: 'x', bytes: 1, sha256: 'sha' }] },
]
const currentExpr = JSON.stringify(manifestPacks, null, 2).split('\n').join('\n')
const directExpr = JSON.stringify(manifestPacks, null, 2)
console.log('currentEqualsDirect:', currentExpr === directExpr)
console.log('hasDoubleCrlfBeforeBracket:', /\r\n}\r\n\[/.test(currentExpr))
console.log('hasCrlfAfterBracket:', /\[\r\n/.test(currentExpr))
JSRepository: DeliriumPulse/codetruss-cli
Length of output: 13493
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## tests/test-scripts listing"
fd -t f 'test-.*\.mjs' packages/cli/scripts
echo
echo "## test scripts involving grammar packs / source tree checks / deterministic package"
for f in $(fd -t f 'test-.*\.mjs' packages/cli/scripts); do
echo "--- $f"
wc -l "$f"
rg -n 'grammar-pack-manifest\.ts|release:grammars|verify:grammars|verifyGrammarPacks|source|file|exists|readFile|build mjs|script' "$f" -C 3 || true
done
echo
echo "## deterministic package policy files"
f=$(fd -t f 'deterministic-package\.mjs' packages/cli/scripts | head -n 1)
if [ -n "$f" ]; then
echo "--- $f"
sed -n '1,220p' "$f"
fi
echo
echo "## release package policy"
f=$(fd -t f 'release-package-policy\.mjs' packages/cli/scripts | head -n 1)
if [ -n "$f" ]; then
echo "--- $f"
sed -n '1,220p' "$f"
fi
echo
echo "## source/build scripts excluding scripts dir"
fd -t f '^(build\.mjs|cli\.ts)$' packages/cli/src packages/cli/src/scripts packages/cli \
| sortRepository: DeliriumPulse/codetruss-cli
Length of output: 22723
Enforce byte-identical grammar-pack-manifest.ts on release rebuilds
build-grammar-packs.mjs overwrites packages/cli/src/grammar-pack-manifest.ts, while verify:Release only checks the CLI metadata archive. Add a grammar-packs parity step that re-generates the pin from the published grammar files and rejects any byte diff, or enforce it in the release validation path.
Also remove the no-op .split('\n').join('\n') in build-grammar-packs.mjs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/scripts/build-grammar-packs.mjs` around lines 134 - 145, Update
the release validation flow to regenerate grammar-pack metadata from the
published grammar files and compare it byte-for-byte with
packages/cli/src/grammar-pack-manifest.ts, failing on any difference; anchor
this at buildGrammarPacks or the existing verify:Release path. Also simplify the
manifest serialization in build-grammar-packs.mjs by removing the no-op
split/join chain while preserving generated output.
Source: Learnings
| const pin = await readFile(pinPath, 'utf8') | ||
| for (const pack of manifestPacks) { | ||
| for (const file of pack.files) { | ||
| if (!pin.includes(`"sha256": "${file.sha256}"`)) { | ||
| throw new Error( | ||
| `src/grammar-pack-manifest.ts does not pin ${pack.name}-${pack.version}/${file.name}; ` | ||
| + 'run pnpm grammars:release', | ||
| ) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
The pin check does not bind a digest to its pack and file.
The comment on lines 92-95 states that the compiled-in pin is the security boundary. The check on line 99 is a substring test. It only proves the digest string appears somewhere in grammar-pack-manifest.ts. It does not prove the digest belongs to pack.name, file.name, or the matching url.
A pin file whose entries are transposed still passes this gate. The CLI would then verify a download against the wrong artifact digest and fail closed, or, if both artifacts were also transposed at publication, accept bytes under the wrong name. The generator currently prevents that, so this is a weakened guarantee rather than a live exploit.
Change this upstream to import the pin and compare structurally: match on pack name and version, then assert name, url, bytes, and sha256 for each file. Do not propose a mirror-only edit here.
Based on learnings, treat all files under packages/cli/ as immutable mirrored release source; ship this through a new released version rather than editing the mirror.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/scripts/verify-grammar-packs.mjs` around lines 96 - 106, Replace
the substring-based digest check in the manifest verification flow with a
structural comparison against the imported compiled-in pin: match each pack by
name and version, then verify every file’s name, URL, byte count, and SHA-256
digest. Update the upstream generator/source rather than the mirrored
packages/cli file, and release a new version carrying the fix.
Source: Learnings
| async function downloadVerified(url: string, target: string, expected: { bytes: number; sha256: string }): Promise<void> { | ||
| const response = await fetch(url, { redirect: 'error', headers: { accept: 'application/octet-stream' } }) | ||
| if (!response.ok) throw new Error(`${url} responded ${response.status}`) | ||
| if (!response.body) throw new Error(`${url} returned no body`) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find existing fetch timeout / AbortSignal conventions in the CLI package.
rg -nP --type=ts -C3 '\bAbortSignal\b|\bAbortController\b|\bsignal\s*:' packages/cli/src
echo '--- all fetch call sites ---'
rg -nP --type=ts -C4 '\bfetch\s*\(' packages/cli/srcRepository: DeliriumPulse/codetruss-cli
Length of output: 6854
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- grammar-pack.ts outline ---'
ast-grep outline packages/cli/src/grammar-pack.ts --view expanded | sed -n '1,220p'
echo '--- grammar-pack.ts downloadVerified and callers ---'
rg -n -C8 'downloadVerified|Promise\.allSettled|fetch|download' packages/cli/src/grammar-pack.ts
echo '--- packages/cli timeout-related helpers/constants ---'
rg -n -C4 'TIMEOUT_MS|abort|controller|signal|fetch\(' packages/cli/src/llm.ts packages/cli/src/hosted-auth.ts packages/cli/src/cli.tsRepository: DeliriumPulse/codetruss-cli
Length of output: 17386
Add a fetch timeout to grammar archive downloads.
downloadVerified currently calls fetch(...) without a signal, so a hung or slow-loris archive origin leaves codetruss grammars install waiting with no output. Bound the request with an abortable timeout and apply the same signal while pumping the response body.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/src/grammar-pack.ts` around lines 289 - 292, Update
downloadVerified to create an abortable timeout for the grammar archive request,
pass its signal to fetch, and reuse that signal while reading/pumping
response.body. Ensure the timeout is cleared when the download completes or
fails, while preserving existing response validation and verification behavior.
Source: Learnings
| const details = [ | ||
| jsError ? `the JavaScript pass failed: ${jsError}` : undefined, | ||
| pythonError, | ||
| truncated ? `${diagnostics.filesSkipped} file(s) could not be parsed locally and were not analyzed` : undefined, | ||
| ].filter((entry): entry is string => Boolean(entry)) | ||
|
|
||
| const error = jsError ?? pythonError | ||
|
|
||
| return { | ||
| findings, | ||
| pass: { | ||
| id: LOCAL_SAST_PASS_ID, | ||
| result: { | ||
| findings, | ||
| complete: !truncated && !error && diagnostics.degradedLanguages.length === 0, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Trace how pythonPackStatus / pythonPackFailureKind / complete reach the rendered receipt.
rg -nP --type=ts -C6 'pythonPackStatus|pythonPackFailureKind|pythonPackReason' packages/cli/src
echo '--- completeness consumers ---'
rg -nP --type=ts -C4 '\bcomplete\b' packages/cli/src/receipt.ts
echo '--- tests asserting complete for a failed pack ---'
rg -nP --type=ts -C6 "pythonPackStatus" packages/cli/testRepository: DeliriumPulse/codetruss-cli
Length of output: 23169
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- local-sast outline ---'
ast-grep outline packages/cli/src/local-sast.ts --view compact || true
echo '--- relevant local-sast.ts section ---'
sed -n '1,230p' packages/cli/src/local-sast.ts
echo '--- relevant receipt.ts python coverage section ---'
sed -n '150,245p' packages/cli/src/receipt.ts
echo '--- complete usage in local-sast tests ---'
rg -n --type=ts -C4 '\bcomplete\b|pythonFilesScanned|findings' packages/cli/test | sed -n '1,220p'Repository: DeliriumPulse/codetruss-cli
Length of output: 35529
Make loader failures surface as detail/error when Python files are present.
When pythonStatus is failed for digest or runtime, pythonFilesScanned is 0 and the rendered Python coverage says Python was not analyzed, but complete can still be true with no detail. If an out-of-band pack check failed, mark the pass incomplete and include the loader reason in detail.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/src/local-sast.ts` around lines 179 - 193, The local SAST result
construction must treat failed Python loaders as pass failures when Python files
are present. Update the logic around pythonStatus, pythonFilesScanned, and the
complete/detail fields in the returned result so digest or runtime loader
failures mark the pass incomplete and expose the loader reason through
detail/error, including out-of-band pack-check failures.
Source: Learnings
| metrics: { | ||
| inputFiles: diagnostics.inputFiles, | ||
| filesScanned: diagnostics.filesScanned, | ||
| filesSkipped: diagnostics.filesSkipped, | ||
| rules: CLI_SAST_RULE_IDS.size, | ||
| // The receipt renders its Python disclosure from these, so what a | ||
| // reader is told about coverage is derived from what actually ran on | ||
| // this machine rather than from a sentence frozen at release time. | ||
| pythonFiles: pythonInputs.length, | ||
| pythonFilesScanned: pythonScan.diagnostics.filesScanned, | ||
| pythonPackStatus: pythonStatus, | ||
| ...(pythonReason ? { pythonPackReason: pythonReason } : {}), | ||
| ...(pythonFailureKind ? { pythonPackFailureKind: pythonFailureKind } : {}), | ||
| }, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
rules reports the JavaScript subset size even when Python ran every rule.
rules: CLI_SAST_RULE_IDS.size is a constant 7. When the pack is verified, the Python scan runs with ruleIds: pythonRuleIds, which is undefined and therefore the complete rule set, as documented at lines 70-80. The metric then under-reports what the run actually did, in the same metrics block whose comment at lines 201-203 states that coverage is derived from what ran on this machine rather than from a constant.
The neighbouring Python metrics are accurate, so the receipt can still describe Python correctly. This is an accuracy gap in one number of a signed document, not a wrong coverage claim.
Note the mirror constraint: packages/cli/ tracks the published archive byte-for-byte, so this belongs in the private monorepo and a later released version.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/src/local-sast.ts` around lines 196 - 209, The rules metric in
the receipt metrics block should reflect the rule set actually executed: use the
complete rule-set count when the verified Python pack runs without specific rule
IDs, while retaining CLI_SAST_RULE_IDS.size for the JavaScript-only path. Update
the logic surrounding pythonStatus and pythonRuleIds without changing the
neighbouring Python metrics.
Source: Learnings
| function pythonDisclosureLines(python: PythonCoverage): string[] { | ||
| const others = 'Go, Java, C#, PHP, Ruby and Rust' | ||
| if (python.status === 'verified' && python.analyzed) { | ||
| return [ | ||
| `- **Non-JavaScript languages other than Python.** ${others} in this repository received secret scanning and the other registry passes, but no security rule or taint analysis.`, | ||
| ] | ||
| } | ||
| if (python.status === 'absent') { | ||
| return [ | ||
| `- **Python.** ${python.files} Python file(s) here received secret scanning and the other registry passes, but no security rule or taint analysis: the optional Python grammar pack is not installed. Install it with \`codetruss grammars install python\` to analyze them locally, or run a hosted scan.`, | ||
| `- **Other non-JavaScript languages.** ${others} in this repository were likewise not covered by any security rule or taint analysis.`, | ||
| ] | ||
| } | ||
| if (python.status === 'failed') { | ||
| return [ | ||
| `- **Python.** ${python.files} Python file(s) were **not** analyzed. ${pythonFailureSentence(python)} No findings from this pack were reported.`, | ||
| `- **Other non-JavaScript languages.** ${others} in this repository received secret scanning and the other registry passes, but no security rule or taint analysis.`, | ||
| ] | ||
| } | ||
| return [ | ||
| `- **Non-JavaScript languages.** The local pass covered JavaScript, TypeScript and TSX. Python, ${others} in this repository received secret scanning and the other registry passes, but no security rule or taint analysis.`, | ||
| ] | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find every value local-sast writes for the Python coverage metrics.
fd -t f 'local-sast.ts' --exec rg -n -C4 'pythonPackStatus|pythonFilesScanned|pythonFiles|not-applicable|verified' {}Repository: DeliriumPulse/codetruss-cli
Length of output: 3089
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== receipt.pyDisclosureLines and analyzed helper =="
fd -t f 'receipt.ts' --exec sh -c 'rg -n -C8 "function pythonDisclosureLines|const analyzed|status === '\''verified'\''|analyzed"' "$1"' {}
echo "== local-sast relevant section =="
fd -t f 'local-sast.ts' --exec sh -c 'sed -n "100,160p"; echo "----"; sed -n "180,215p"' "$1" {}
echo "== types and usage contexts =="
rg -n "PythonCoverage|pythonFilesScanned|pythonPackStatus|analyzed" packages/cli/src packages -g '*.ts' | head -200Repository: DeliriumPulse/codetruss-cli
Length of output: 278
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== receipt.pyDisclosureLines and analyzed helper =="
fd -t f 'receipt.ts' --exec sh -c '
echo "--- $1"
rg -n -C8 "function pythonDisclosureLines|const analyzed|status === '\''verified'\''|analyzed" "$1"
' {}
echo "== local-sast relevant section =="
fd -t f 'local-sast.ts' --exec sh -c '
echo "--- $1"
sed -n "100,160p" "$1"
echo "----"
sed -n "180,215p" "$1"
' {}
echo "== types and usage contexts =="
rg -n "PythonCoverage|pythonFilesScanned|pythonPackStatus|analyzed" packages/cli/src packages -g '*.ts' | head -200Repository: DeliriumPulse/codetruss-cli
Length of output: 295
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate files =="
fd -t f 'receipt.ts|local-sast.ts' .
python3 - <<'PY'
from pathlib import Path
for name in ["receipt.ts", "local-sast.ts"]:
p=next(Path(".").rglob(name), None)
print(f"\n--- {p}")
if path:
txt=path.read_text()
for s in [
"function pythonDisclosureLines",
"const analyzed",
"pythonStatus",
"pythonFiles,"
"pythonFilesScanned",
"pythonPackStatus"
]:
print(f"@@ {s}")
lines=txt.splitlines()
for i,l in enumerate(lines,1):
if s in l:
a=max(1,i-4); b=min(len(lines),i+5)
print(f"\n[{i}]")
for j in range(a,b+1): print(f"{j:4}: {lines[j-1]}")
PY
echo "== focused receipt =="
python3 - <<'PY'
from pathlib import Path
p=next(Path(".").rglob("receipt.ts"), None)
print(p)
lines=p.read_text().splitlines()
start=next(i for i,l in enumerate(lines) if "function pythonDisclosureLines" in l)
print(f"\n-- pythonDisclosureLines {181+1}-{235+1}")
for i in range(max(0,start-5), min(len(lines), start+60)):
print(f"{i+1:4}: {lines[i]}")
print(f"\n-- analyzed helper occurrences")
for i,l in enumerate(lines):
if "analyzed" in l or "filesScanned" in l:
print(f"{i+1:4}: {l}")
PY
echo "== focused local-sast =="
python3 - <<'PY'
from pathlib import Path
p=next(Path(".").rglob("local-sast.ts"), None)
print(p)
src=p.read_text().splitlines()
for needle in ["pythonStatus", "pythonFiles,", "pythonFilesScanned"]:
start=next(i for i,l in enumerate(src) if needle in l)
a=max(0,start-25); b=min(len(src),start+45)
print(f"\n--- context around {needle} ({src[start]} line {start+1})")
for i in range(a,b):
print(f"{i+1:4}: {src[i]}")
PYRepository: DeliriumPulse/codetruss-cli
Length of output: 391
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
for path in [Path("./packages/cli/src/receipt.ts"), Path("./packages/cli/src/local-sast.ts")]:
txt = path.read_text()
lines = txt.splitlines()
print(f"\n===== {path} =====")
print(f"total lines: {len(lines)}")
for needle in ["function pythonDisclosureLines", "analyzed:", "pythonStatus =", "pythonFiles,", "pythonFilesScanned", "pythonPackStatus"]:
if needle not in txt:
print(f"MISSING: {needle}")
continue
hits = [i for i, l in enumerate(lines) if needle in l or (needle == "analyzed:" and ("analyzed" in l and "python" in l.lower()))]
print(f"\n-- contexts around {needle!r} hits {hits[:5]}")
for i in hits[:3]:
a = max(0, i - 8)
b = min(len(lines), i + 12)
for n in range(a, b):
print(f"{n+1:4}: {lines[n]}")
print("\n===== deterministic branch outcomes =====")
def pythonDisclosureLines(status, files, scanned):
analyzed = status == "verified" and scanned > 0
lines = []
if analyzed:
return lines
if status == "absent":
lines.append("absent")
elif status == "failed":
lines.append("failed")
else:
lines.append("default")
return lines
for status, files, scanned in [("verified", 0, 0), ("verified", 5, 0), ("verified", 5, 1), ("not-applicable", 0, 0), ("absent", 5, 0), ("failed", 5, 0)]:
print(f"{status}, files={files}, scanned={scanned} -> {pythonDisclosureLines(status, files, scanned)}")
PYRepository: DeliriumPulse/codetruss-cli
Length of output: 16853
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== local-sast top section =="
sed -n '1,115p' packages/cli/src/local-sast.ts
echo "== helper definitions =="
rg -n "PYTHON_GRAMMAR_PACK|pythonInputs|loadGrammarParser|pythonFiles|find|scanFiles|diagnostics" packages/cli/src/local-sast.tsRepository: DeliriumPulse/codetruss-cli
Length of output: 7149
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== local-sast lines 115-170 =="
sed -n '115,170p' packages/cli/src/local-sast.ts
echo "== scanFiles diagnostics shape declarations =="
rg -n "filesScanned|inputFiles|filesSkipped|degradedLanguages|scanFiles|interface SastResult|type SastResult" node_modules '`@codetruss/analyzer-engine`' packages -g '*.ts' 2>/dev/null | head -300 || trueRepository: DeliriumPulse/codetruss-cli
Length of output: 6651
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== scanFiles implementation =="
sed -n '45,125p' packages/analyzer-engine/src/security/engine.ts
echo "== SastResult type =="
sed -n '55,90p' packages/analyzer-engine/src/security/types.ts
echo "== mergeSastResult implementation =="
sed -n '136,165p' packages/analyzer-engine/src/security/engine.ts
echo "== scanFiles return sites and skipped reasons =="
rg -n "filesScanned|filesSkipped|truncated|skipped|degraded|SastScanMessage|LanguageError|FileError" packages/analyzer-engine/src packages/cli/test -g '*.ts' | head -250Repository: DeliriumPulse/codetruss-cli
Length of output: 18616
Handle Python inputs when the grammar packs are verified but no Python files are scanned.
runLocalSast sets pythonStatus = 'verified' after any successful scanFiles(pythonInputs, ...), and scanFiles returns filesScanned = 0 when every input file is skipped instead of parsing one. This can render pythonFiles > 0, pythonFilesScanned === 0, and pythonPackStatus: 'verified'; pythonDisclosureLines() falls to the default branch, which omits Python from the local pass summary.
Add a branch that treats this state explicitly on the receipt, and add a test that exercises Python input files with all scans skipped.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/src/receipt.ts` around lines 203 - 225, Update
pythonDisclosureLines() to handle verified Python grammar with Python files
present but none scanned, explicitly stating that Python was not analyzed while
preserving the existing disclosure wording for other languages. Add a receipt
test covering Python inputs where every scan is skipped and the resulting status
is verified with zero files scanned.
|
Closing this release-train sync rather than updating it in place. This PR halted on the Windows test failures, and the fix for those has since merged in the monorepo. That means the source snapshot mirrored here is stale: re-pushing onto this branch would mirror a state that no longer matches the monorepo's fixed tree, and the release train's whole guarantee is that the mirror reproduces an exact release commit byte-for-byte. The train will re-sync v0.2.40 from the fixed monorepo state and open a fresh PR. Nothing here is lost -- this branch carried no work that did not originate in the monorepo. |
Bring the public mirror up to the shipped release. The mirror sat at 0.2.39
while codetruss.com served 0.2.40; the newest attested tag should never trail
the bytes the site hands out.
Source is mirrored from 63088fc, not from the version commit 1833756. An
adversarial review after the version bump found a verify-then-execute TOCTOU in
the grammar-pack loader — it hashed each artifact by path and then re-opened the
same path to execute it — and the fix rebuilt the artifact, so 63088fc is the
commit whose trees hash to what the site serves. 128 files across packages/cli
and packages/analyzer-engine, zero content mismatches and zero file-mode
mismatches against that commit's trees. The count grows from 117 because the
grammar pack adds four source modules, four test files, and three build scripts;
packages/cli/SBOM.cdx.json stays generated and gitignored here, as it has since
0.2.13.
pnpm release:artifactin a fresh clone of this branch rebuilds the bundle tothe exact published digest
5d64313b8b60acbd1f93e2246557967885a98fdc8c486ea7b2a6417fd8acdac2, and
pnpm release:verifyconfirms it byte-for-byte against the immutable websitearchive now recorded in release-reference.json. The archive, its SBOM, the
latest.json manifest, and all seven grammar-pack artifacts downloaded from
codetruss.com this run are byte-identical to the ones in this tree.
repoint the latest.* aliases and manifest. The 0.2.39 archive stays: it is a
tagged, attested release.
their .sha256 sidecars) and codetruss-grammars-latest.json. These are not
optional here: a pack versions independently of the CLI, the site is its only
download origin, and the new tests install from these exact published bytes
over loopback rather than from a fixture, so the mirror cannot build or test
itself without them.
bundle digest 7dcf9a22457c6790c08b2ddc1186f1c73213e58ad6920aa7f164c8ce2d3e9076.
manifest, the parser that executes only verified buffers, the
grammarscommand, the Python arm of the local security pass, the v4 receipt renderer
and its frozen v3 predecessor, and their tests — plus the packaged changelog,
and regenerate the root changelog from it so the release body stays
byte-identical to what ships inside the archive.
new
grammarscommand, and say that Python joins the local security pass whena pack is installed. The receipt excerpt keeps its "real 0.2.36 run"
provenance and its
local-registry-v2, 13-analyzer wording: 0.2.40 stillships a frozen v2 renderer that reproduces that wording byte for byte, so the
excerpt is still what that run printed.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Summary by CodeRabbit
New Features
codetruss grammarscommand for listing, checking, installing, and uninstalling packs.Security & Reliability
Compatibility