diff --git a/.github/workflows/publish-npm.yml b/.github/workflows/publish-npm.yml index c8c75e6..3925f32 100644 --- a/.github/workflows/publish-npm.yml +++ b/.github/workflows/publish-npm.yml @@ -88,7 +88,6 @@ jobs: `codetruss-cli-${version}.sbom.cdx.json`, `codetruss-cli-${version}.tgz`, `codetruss-cli-${version}.tgz.sha256`, - 'release-manifest.json', ].sort() if (JSON.stringify(actual) !== JSON.stringify(expected)) { throw new Error(`release assets differ from the exact publish set: ${actual.join(', ')}`) diff --git a/CHANGELOG.md b/CHANGELOG.md index b882ccf..5cf4ae2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ CodeTruss CLI follows semantic versioning. Release artifacts and their SHA-256 checksums are published at . -The current public release is [v0.2.30 on GitHub](https://github.com/DeliriumPulse/codetruss-cli/releases/tag/v0.2.30), +The current public release is [v0.2.36 on GitHub](https://github.com/DeliriumPulse/codetruss-cli/releases/tag/v0.2.36), distributed from . The npm `latest` tag is still [`@codetruss/cli@0.2.24`](https://www.npmjs.com/package/@codetruss/cli/v/0.2.24): @@ -16,6 +16,175 @@ were superseded before distribution. No unreleased changes. +## 0.2.36 — 2026-08-07 + +- **Indexed file paths are now the same bytes on every platform.** The + repository walk emitted whatever separator the host used, so a Windows run + produced `src\users.ts` while every other path surface in the CLI — receipts, + git snapshots, policy globs, scope inference — normalized to `src/users.ts`. + Three consequences, all fixed by normalizing at the source: a signed receipt's + findings table and its changed-files table named the same file two different + ways; receipt bytes differed by platform for an identical tree, so a + cross-platform reproduction could not match; and vendored-directory exclusion + (`.claude/`, `vendor/`, …) silently stopped matching on Windows, pulling + tooling payloads back into analysis. +- `changedFindings()` now compares paths separator-agnostically as well. The + source fix already makes both sides POSIX, so this is defense in depth against + any future caller that hands in a raw platform path. + +- **Security analysis now runs locally.** The rule pack and taint solver that + previously existed only in hosted scans execute on your machine, offline, over + the JavaScript, TypeScript and TSX in your repository — the same engine, not a + reimplementation. Seven rules ship: SQL injection tracked from request source + to query execution, mass assignment, open-record write payloads, un-awaited + database writes, swallowed errors, coercion-prone `==` comparisons, and N+1 + queries in loops. +- The blocker was packaging, not the rules: the tree-sitter WASM grammars the + engine parses with are six times the CLI's entire 1 MB release budget. So the + engine moved behind an injected parser interface and the CLI got a + zero-dependency JavaScript/TypeScript/JSX parser that emits the same syntax + vocabulary. One rule pack, two front-ends, nothing duplicated to drift. +- That parser is strict where a normal one recovers: anything it cannot + represent exactly is skipped and reported as lost coverage, so unsupported + syntax costs a finding CodeTruss never makes rather than one it makes wrongly. + Both parsers were run over 1,314 real files with the same rules and produced + identical findings — nothing found only locally, nothing lost. +- **Local security findings are REVIEW_REQUIRED, never FAILED.** They do not + fail a verdict or halt an agent turn on their own. Blocking is a promotion + precision has to earn on real repositories; severity alone does not grant it. +- `unawaited-persistence` now also catches a raw driver write whose promise is + dropped — `pool.query("INSERT …")` with no `await` — which the ORM-shaped + checks could not see. Reads, callback-style calls, and handled promises are + untouched. +- **Receipts state the new boundary.** The analysis profile is now + `local-registry-v2`, and the receipt names both what the local pass checked and + what it still did not: command injection, path traversal, SSRF, XSS, + deserialization, and every non-JavaScript language. Receipts signed by earlier + versions keep verifying byte-for-byte against the wording they were signed + with. +- Cost: about 0.3 s over 2.5 MB of source, roughly a quarter of one percent of + the agent hook's budget, and faster than the hosted parser it stands in for. + +## 0.2.34 — 2026-08-07 + +- Findings can now carry a suggested fix: a description, a unified diff or + snippet, and a required safety note, rendered in the receipt as a **Suggested + fixes** section and available as `fix` on the JSON finding. A suggestion is + never applied, written, or run, and never framed as required — a change derived + from one matched line cannot see the rest of the codebase. +- An analyzer attaches one only where the finding's own evidence determines a + single correct change. Where the right fix is ambiguous the prose suggestion + stays the whole answer and no `fix` is attached, because a wrong autofix is a + false positive with extra damage. +- **Committed secrets** get a move-to-env diff: the literal becomes + `process.env.X` (or `os.environ`, `os.Getenv`, `ENV.fetch`, `getenv` by + language), with the matching `.env.example` line appended at the correct + offset. The removed line is shown with the credential **masked** — CodeTruss + never echoes a credential, not even into its own suggestion — so the diff + cannot apply cleanly by design, and the note says so and leads with rotation. + A tracked `.env` gets the untracking commands instead. A key inside a call, a + private-key block, or an unsupported language gets prose only. +- A credential found inside a **generated** file is still reported in full, but + gets no diff: the next generation would overwrite the edit, so the fix belongs + in the generator's input, not in that line. +- **No lockfile committed** gets the refresh command for the package manager the + repository itself declares. With no such evidence it lists every option rather + than guessing one — a lockfile from the wrong manager is worse than none. +- **Missing README** and **No CI pipeline** get minimal starter blocks. The + workflow is built only from scripts `package.json` actually defines, and is + withheld entirely where the setup steps would have to be guessed. +- Under an agent hook, the highest-severity suggestion is appended to the Stop + summary in its own field, so the five-reason display cap can never drop it and + the agent can correct the change before a person opens the receipt. +- Suggested fixes quote real source lines, so they are stripped from the hosted + sync copy. Receipts whose findings carry no fix render byte for byte as before, + so earlier signatures keep verifying. + +## 0.2.33 — 2026-08-07 + +- A generated-file banner can no longer hide a committed credential. A four-line + file whose first line read `// AUTO-GENERATED FILE - DO NOT EDIT` produced a + signed PASS with a live Stripe key on line four; the identical file without + that comment FAILED. Generated classification exists to stop machine-written + output producing spurious "oversized file" findings, but it was excluding those + files from every analyzer, secret scanning included, so one comment bypassed + the whole scanner. The excluded text is now retained for the secrets pass only: + LOC totals, the architecture graph, and the quality analyzers keep skipping + generated files exactly as before. +- Name every excluded file, at any size. The exclusion note used to appear only + above 500 LOC or 50KB and named just the first file as "e.g." — so a small + generated stub was dropped from analysis with nothing on the receipt to say so. + It now lists every excluded path whenever anything is excluded, and reports + small volumes in bytes rather than rounding them to "0 KB". +- `setup` stops reporting "No repository verification commands were detected" + when it detected them. An unattended run deliberately withholds commands it has + no permission to execute; it now prints the list it found, why it withheld it, + and the exact step that enables it. When detection genuinely fails because no + lockfile is committed, it says the lockfile is the reason instead of implying + the repository has no tests. +- npm and yarn repositories get the same `[lint, test]` collection pnpm already + had, instead of losing their lint command over which lockfile they commit. +- `setup`'s own footprint — `.codetruss.yml`, `.claude/**`, `.codex/**`, + `.githooks/**` — is in scope by default, so installing CodeTruss no longer + reports CodeTruss as scope drift on a user's first commit. `.codetruss.yml` + remains a sensitive policy surface, and setup now says to commit it so the + policy stays reviewable. +- The installer no longer prints "Ready" while an older binary shadows the one it + just installed. `install.sh` compares what `command -v codetruss` resolves to + against the completed install and prints the PATH fix when they differ; + `hooks doctor` warns on the same version skew, reading the shadowing install's + manifest rather than executing whatever is first on PATH. +- `codetruss verify-policy trust-key` is listed in `--help`. Blocked-commit + errors already told people to run it. +- The verification-command trust store honors `XDG_CONFIG_HOME`, matching where + the saved login already lives. An approval left at the legacy `~/.config` path + keeps being read, so setting that variable never orphans a trust store. + +## 0.2.32 — 2026-08-06 + +- Infer this turn's scope so a first session reads as signal, not noise. Scope + drift used to fire the moment an agent touched anything outside the + directories `setup` happened to find on disk, which made the one detection + nobody else ships debut as a false alarm. A path with no approved allow root + can now be classified `inferred` on this turn's own evidence: the task naming + the path or the feature directory, a cohesive working set under one shared + parent, or a test file mirroring a source file already in scope. +- Disclose every inferred allowance on the receipt. An "Inferred scope" section + names each root, what it was read from, and the approved roots it sits beside, + and the changed-file row reads `allowed (inferred)` rather than `allowed`. The + PASS reason no longer claims those files were within approved scope. Receipts + that inferred nothing render byte for byte as before, so receipts signed by + earlier versions keep verifying. +- Never infer past a hard line. Deny rules win outright, secrets, config and + dependency surfaces are not inferable, the repository root is never a root, + and no inferred root may climb above an allow root the repository deliberately + narrowed. With no allow roots configured at all, the turn is its own scope and + the receipt says exactly that. +- Keep the mid-turn PostToolUse check quiet about a path inference already + covers. It sees one tool call, so it infers strictly less than the Stop-time + receipt and never more. + +## 0.2.31 — 2026-08-06 + +- Capture a baseline for turns that carry no prompt. Harness machine events — + background-task notifications, hook feedback continuations, and resumed agents + — reach `UserPromptSubmit` with no prompt at all, and capture assumed there + always was one. A promptless turn now snapshots tree state like any other turn, + is labelled honestly, and earns a receipt; previously those turns reached Stop + with no baseline and went entirely unreviewed. +- `UserPromptSubmit` never blocks. Blocking there erased a person's prompt + because CodeTruss could not take a snapshot — failing closed against the user + rather than the agent. Capture failures now emit a note and let the prompt + through. Stop remains the enforcement point and still fails closed on a turn + with no provable baseline, so an agent cannot finish unreviewed. +- Exact capture retries a working tree that changes mid-snapshot, bounded at + three attempts, and the Stop hook timeout moves to 360s. It had been 300s — + exactly the internal review timeout — so the harness killed the hook at the + moment the graceful timeout receipt would have been written. +- Shipped alongside, outside the CLI: production deploys now apply database + migrations before serving new code, which had been running against an old + schema. + ## 0.2.30 — 2026-08-06 - Publish the 0.2.25 through 0.2.29 release history, which shipped without diff --git a/README.md b/README.md index 92ebbcb..8937f66 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,8 @@ The deterministic first-pass verification gate for AI-written code. An agent finishes a change. Something has to look at it before a human does. CodeTruss Boundary is that first pass: it captures an exact before/after Git evidence pair, checks the change against the task contract you declared, runs 13 -deterministic analyzers and your own project checks, then signs a `PASS`, -`REVIEW_REQUIRED`, or `FAILED` receipt you can re-verify later. +deterministic analyzers, a local security pass, and your own project checks, then +signs a `PASS`, `REVIEW_REQUIRED`, or `FAILED` receipt you can re-verify later. Routine changes clear the checks and pass. Material changes escalate for human sign-off. Every verdict leaves a receipt that records the reasons, and the @@ -44,7 +44,7 @@ To pin an exact version, install the immutable archive directly: ```bash npm install --global --ignore-scripts --no-audit --no-fund \ - https://codetruss.com/downloads/codetruss-cli-0.2.30.tgz + https://codetruss.com/downloads/codetruss-cli-0.2.36.tgz ``` The `@codetruss/cli` package on the npm registry is published as a separate, @@ -59,8 +59,10 @@ codetruss verify latest ``` That first receipt needs no account and no configuration. Without an allow -policy every changed file is deliberately unexpected, so the review exits `1` -with `REVIEW_REQUIRED` and still writes valid signed evidence. +policy CodeTruss infers the scope of the turn, marks each file that inference +covers `allowed (inferred)`, and discloses on the receipt that it did so, which +keeps a first run readable as signal instead of blanket scope drift. Anything +the inference does not cover is still unexpected. Run `codetruss setup` once at the Git root to make it automatic. It proposes conventional source roots rather than repository-wide access, shows any detected @@ -126,8 +128,6 @@ detected verification commands are trusted to run. `verify-policy trust-key` is separate: it appends your local signing key to `signing.publicKeys` so a teammate can sign receipts as themselves instead of sharing a private key. Commit `.codetruss.yml` afterward so the rest of the team inherits the change. -It is accepted by the CLI but omitted from the built-in `--help` banner in -0.2.30. ## Fail-closed policy @@ -175,14 +175,14 @@ The verdict is not a confidence score. Receipts are written as Markdown and JSON next to hashed patch evidence, and can be rechecked later with `codetruss verify latest`. Every receipt states the detection gaps in its own body, so a `PASS` is never mistaken for a security -clearance. Abridged from a real 0.2.30 run: +clearance. Abridged from a real 0.2.36 run: ```markdown # CodeTruss receipt — REVIEW_REQUIRED - **Task:** Fix auth callback validation -- **Evidence trees:** `f8c28a26…` → `52fdebbe…` -- **Policy SHA-256:** `368f88df…` +- **Evidence trees:** `a2303191…` → `0f481c3c…` +- **Policy SHA-256:** `82db19fe…` ## Verdict: REVIEW_REQUIRED @@ -191,31 +191,63 @@ clearance. Abridged from a real 0.2.30 run: ## Analysis profile -Profile: `local-registry-v1`. +Profile: `local-registry-v2`. -The 13 deterministic registry analyzers ran locally on this machine. +The 13 deterministic registry analyzers ran locally on this machine, plus a +local security pass: the shared SAST engine — the same rules and the same +source-to-sink taint tracking as the hosted audit — over the JavaScript, +TypeScript and TSX in this repository. + +### What the local security pass checked + +- **SQL injection (CWE-89).** Untrusted input tracked from request sources + through string building into query execution. +- **Mass assignment (CWE-915).** A raw request body spread into a database + write, and write helpers whose payload type accepts arbitrary keys. +- **Un-awaited database writes, swallowed errors, coercion-prone `==` + comparisons, and N+1 queries in loops** — the defect classes coding agents + most often introduce. ### What did not run -- **Security static analysis (SAST).** No injection or taint analysis was - performed. SQL injection, command injection, code injection, path traversal, - SSRF, open redirect, XSS and insecure deserialization were never checked. +- **The rest of the security rule pack.** Command injection, code injection, + path traversal, SSRF, open redirect, XSS and insecure deserialization were + **not** checked here. +- **Non-JavaScript languages.** The local pass covers JavaScript, TypeScript + and TSX only. - **Hosted symbol graph.** No cross-file call or data-flow graph was built. +- **Optional LLM review.** No model read this diff. - **Hosted Health scores.** Not calculated, reported as **N/A**. +Local security findings are reported for review and do not fail the verdict on +their own. + A PASS verdict means the passes listed above never ran and the passes that did run found nothing new. It is not a statement that this change is secure. ``` -SAST and the symbol graph are hosted-only. A local run never performs injection -or taint analysis. +Since 0.2.35 the security rule pack and its taint solver run locally and +offline over JavaScript, TypeScript and TSX — the same engine as the hosted +audit, not a reimplementation. The rest of the rule pack, every other language, +and the symbol graph remain hosted-only, and the receipt names them rather than +leaving their absence to be inferred. Local security findings are +`REVIEW_REQUIRED` at most; they never fail a verdict on their own. + +Where a finding's own evidence determines a single correct change, the receipt +also carries a **Suggested fixes** section with a diff and a required safety +note. Nothing is ever applied, written, or run. A committed credential is shown +with its value masked, so that diff cannot apply cleanly by design and the note +leads with rotation. ## Measured accuracy On a nine-case adversarial corpus of AI-agent bug classes, the analyzers caught -five at the exact file and line, with zero false positives across eight -repositories. The four misses are named, each with the reason it needs dataflow -analysis the local passes do not perform. +six at the exact file and line, with zero false positives across 177,703 lines +in eight repositories. The three misses are named, each with the reason a rule +that caught it would fire on legitimate code more often than on the bug. + +The read-modify-write race was published as a miss and now sits in the caught +half: it moved because the rule shipped, not because the bar moved. Method, per-case reasoning, and the misses are published at [codetruss.com/benchmark](https://codetruss.com/benchmark). @@ -281,8 +313,8 @@ clean global install. Verify a downloaded release yourself: ```bash -gh attestation verify codetruss-cli-0.2.30.tgz --repo DeliriumPulse/codetruss-cli -shasum -a 256 -c codetruss-cli-0.2.30.tgz.sha256 +gh attestation verify codetruss-cli-0.2.36.tgz --repo DeliriumPulse/codetruss-cli +shasum -a 256 -c codetruss-cli-0.2.36.tgz.sha256 ``` Maintainers should follow [docs/RELEASE.md](docs/RELEASE.md). Tag-driven GitHub diff --git a/packages/analyzer-engine/src/coverage.ts b/packages/analyzer-engine/src/coverage.ts index 4dcb99f..08f5d1a 100644 --- a/packages/analyzer-engine/src/coverage.ts +++ b/packages/analyzer-engine/src/coverage.ts @@ -257,6 +257,36 @@ export const coverageAnalyzer: Analyzer = { ] } + // The pass ran, but with a reduced rule set. Naming only what it covered + // would let a reader infer the rest was checked and clean — the same + // inference the block above exists to prevent, one level down. + const unchecked = context.sastUncheckedClasses + if (context.sast && unchecked && unchecked.length > 0 && coverage.securityLoc >= MIN_ANALYZABLE_LOC) { + return [ + { + category: 'SECURITY_HYGIENE', + severity: 'INFO', + title: 'Security static analysis ran with a reduced rule set', + description: + `The security pass ran on the ${coverage.securityLoc} lines it covers here, including ` + + `taint tracking from untrusted source to SQL execution. It did NOT check ` + + `${unchecked.join(', ')}. Those rules need analysis this profile does not perform, so ` + + `the absence of a finding in those classes means "not checked", not "clean".`, + suggestion: + `A hosted SECURITY or FULL_AUDIT scan runs the complete rule pack — every class listed ` + + `above — over the same code.`, + impactScore: 20, + effort: 'low', + metadata: { + sastPassRan: true, + sastUncheckedClasses: [...unchecked], + securityLoc: coverage.securityLoc, + totalLoc: coverage.totalLoc, + }, + }, + ] + } + if (coverage.securityLimited) { const lang = coverage.primaryLanguage ?? coverage.structureOnlyLanguages[0] ?? 'this language' diff --git a/packages/analyzer-engine/src/dependencies.ts b/packages/analyzer-engine/src/dependencies.ts index 56fe794..b076f8f 100644 --- a/packages/analyzer-engine/src/dependencies.ts +++ b/packages/analyzer-engine/src/dependencies.ts @@ -1,5 +1,7 @@ import { readFile } from 'fs/promises' import { join } from 'path' +import { nodePackageManager } from './detect' +import { lockfileRefreshFix } from './fixes' import type { Analyzer, AnalyzerFinding } from './types' /** @@ -57,6 +59,7 @@ export const dependencyAnalyzer: Analyzer = { title: 'No lockfile committed', description: 'package.json exists but no lockfile is committed. Builds are not reproducible and supply-chain drift is invisible.', suggestion: 'Commit the lockfile for your package manager and enforce frozen-lockfile installs in CI.', + fix: lockfileRefreshFix(nodePackageManager(index.files)), impactScore: isLibrary ? 55 : 75, effort: 'low', }) diff --git a/packages/analyzer-engine/src/detect.ts b/packages/analyzer-engine/src/detect.ts index 8387a3c..54a3e9a 100644 --- a/packages/analyzer-engine/src/detect.ts +++ b/packages/analyzer-engine/src/detect.ts @@ -133,6 +133,43 @@ export interface DetectableFile { content: string | null } +/** + * The one Node package manager this repository actually uses, or undefined when + * the evidence does not name exactly one. + * + * Deliberately stricter than detectPackageManagers(): that function reports + * everything it sees for display, while a fix suggestion may only name a + * manager the repository itself declares. Order matters — an explicit + * `packageManager` field outranks a lockfile, and two competing lockfiles + * resolve to nothing rather than to whichever was checked first. + */ +export function nodePackageManager(files: DetectableFile[]): 'pnpm' | 'yarn' | 'npm' | 'bun' | undefined { + const paths = files.map((f) => f.path) + const has = (name: string) => paths.includes(name) + const packageJson = files.find((f) => f.path === 'package.json') + if (!packageJson) return undefined + + const declared = packageJson.content?.match(/"packageManager"\s*:\s*"([a-z]+)@/)?.[1] + if (declared === 'pnpm' || declared === 'yarn' || declared === 'npm' || declared === 'bun') return declared + + const fromLockfiles = ([ + ['pnpm', 'pnpm-lock.yaml'], + ['yarn', 'yarn.lock'], + ['npm', 'package-lock.json'], + ['bun', 'bun.lock'], + ['bun', 'bun.lockb'], + ] as const).filter(([, lockfile]) => has(lockfile)).map(([manager]) => manager) + const uniqueLockfileManagers = [...new Set(fromLockfiles)] + if (uniqueLockfileManagers.length === 1) return uniqueLockfileManagers[0] + if (uniqueLockfileManagers.length > 1) return undefined + + // No lockfile at all: only a manager-specific config file is evidence. + if (has('pnpm-workspace.yaml')) return 'pnpm' + if (has('.yarnrc.yml') || has('.yarnrc')) return 'yarn' + if (has('bunfig.toml')) return 'bun' + return undefined +} + export function detectPackageManagers(files: DetectableFile[]): string[] { const paths = files.map((f) => f.path) const managers: string[] = [] diff --git a/packages/analyzer-engine/src/fixes.ts b/packages/analyzer-engine/src/fixes.ts new file mode 100644 index 0000000..1ed6fe7 --- /dev/null +++ b/packages/analyzer-engine/src/fixes.ts @@ -0,0 +1,329 @@ +import type { FindingFix } from './types' + +/** + * Builders for `AnalyzerFinding.fix`. + * + * Every builder here is total and honest: it returns `undefined` the moment the + * evidence stops determining a single correct change, so the caller falls back + * to prose guidance instead of shipping a plausible-looking wrong edit. Nothing + * in this module touches the filesystem — a fix is text for a human or an agent + * to read, never an action CodeTruss performs. + */ + +/** How each language reads an environment variable at runtime. */ +const ENV_ACCESSOR: Record string> = { + ts: (name) => `process.env.${name}`, + tsx: (name) => `process.env.${name}`, + mts: (name) => `process.env.${name}`, + cts: (name) => `process.env.${name}`, + js: (name) => `process.env.${name}`, + jsx: (name) => `process.env.${name}`, + mjs: (name) => `process.env.${name}`, + cjs: (name) => `process.env.${name}`, + py: (name) => `os.environ["${name}"]`, + go: (name) => `os.Getenv("${name}")`, + rb: (name) => `ENV.fetch("${name}")`, + php: (name) => `getenv('${name}')`, +} + +/** Languages whose accessor needs an import the file may not already have. */ +const ACCESSOR_NEEDS_IMPORT = new Set(['py', 'go']) + +function extensionOf(path: string): string { + const base = path.split('/').pop() ?? path + const dot = base.lastIndexOf('.') + return dot <= 0 ? '' : base.slice(dot + 1).toLowerCase() +} + +/** + * `awsSecretKey` → `AWS_SECRET_KEY`. Returns undefined for identifiers that + * cannot produce a legal environment-variable name, rather than guessing one. + */ +export function envVarNameFrom(identifier: string): string | undefined { + const screaming = identifier + .replace(/([a-z0-9])([A-Z])/g, '$1_$2') + .replace(/[^A-Za-z0-9]+/g, '_') + .replace(/_+/g, '_') + .replace(/^_+|_+$/g, '') + .toUpperCase() + return /^[A-Z][A-Z0-9_]*$/.test(screaming) ? screaming : undefined +} + +interface ParsedAssignment { + /** Everything on the line before the opening quote. */ + head: string + /** Everything after the closing quote. */ + tail: string + quote: string + identifier: string + value: string +} + +/** `const awsKey: string = "…"` / `let x = '…'` / `AWS_KEY = "…"`. */ +const ASSIGNMENT_RE = + /^(\s*(?:(?:export|public|private|protected|static|final|readonly|const|let|var|val)\s+)*([A-Za-z_$][A-Za-z0-9_$]*)\s*(?::[^=]*?)?\s*=\s*)(['"])([^'"]*)\3(\s*[;,]?\s*)$/ + +/** `password: "…"` inside an object literal or a Python dict. */ +const KEY_VALUE_RE = + /^(\s*['"]?([A-Za-z_$][A-Za-z0-9_$-]*)['"]?\s*:\s*)(['"])([^'"]*)\3(\s*,?\s*)$/ + +/** + * Parse a single-literal assignment. Lines with more than one string literal, + * concatenation, or a call around the value are rejected: the replacement would + * be a guess about which fragment is the credential. + */ +export function parseSingleLiteralAssignment(lineText: string): ParsedAssignment | undefined { + const match = ASSIGNMENT_RE.exec(lineText) ?? KEY_VALUE_RE.exec(lineText) + if (!match) return undefined + return { head: match[1], identifier: match[2], quote: match[3], value: match[4], tail: match[5] } +} + +/** Unified-diff hunk header for replacing exactly one line in place. */ +function replaceLineHunk(path: string, line: number, before: string, after: string): string[] { + return [`--- a/${path}`, `+++ b/${path}`, `@@ -${line} +${line} @@`, `-${before}`, `+${after}`] +} + +/** Unified-diff hunk that appends one line to an existing or absent file. */ +function appendLineHunk(path: string, existingLines: number | undefined, added: string): string[] { + if (existingLines === undefined) { + return [`--- /dev/null`, `+++ b/${path}`, `@@ -0,0 +1 @@`, `+${added}`] + } + return [`--- a/${path}`, `+++ b/${path}`, `@@ -${existingLines},0 +${existingLines + 1} @@`, `+${added}`] +} + +export interface SecretFixEvidence { + filePath: string + line: number + /** The exact source line the secret pattern matched. */ + lineText: string + /** Human name of the credential type, used only in the redaction marker. */ + credentialType: string + /** Line count of the repository's `.env.example`, or undefined when absent. */ + envExampleLines?: number +} + +/** + * Move-to-env refactor for a credential committed in source. + * + * The removed line is shown with the VALUE MASKED — CodeTruss never echoes a + * credential, not even into its own suggestion — which is also why the diff is + * deliberately not directly appliable. The safety note says so, and leads with + * rotation: replacing the line does not remove the value from Git history. + */ +export function moveSecretToEnvFix(evidence: SecretFixEvidence): FindingFix | undefined { + const extension = extensionOf(evidence.filePath) + const accessor = ENV_ACCESSOR[extension] + if (!accessor) return undefined + const parsed = parseSingleLiteralAssignment(evidence.lineText) + if (!parsed) return undefined + // The parsed literal must be the one the scanner matched, or the replacement + // would rewrite an unrelated string on a line that also carries a secret. + if (!evidence.lineText.includes(parsed.value) || parsed.value.length === 0) return undefined + const variable = envVarNameFrom(parsed.identifier) + if (!variable) return undefined + + const masked = `${parsed.head}${parsed.quote}<${evidence.credentialType} value — never printed by CodeTruss>${parsed.quote}${parsed.tail}` + const replaced = `${parsed.head}${accessor(variable)}${parsed.tail.replace(/^\s*/, '')}` + const content = [ + ...replaceLineHunk(evidence.filePath, evidence.line, masked, replaced), + ...appendLineHunk('.env.example', evidence.envExampleLines, `${variable}=`), + '', + ].join('\n') + + const importNote = ACCESSOR_NEEDS_IMPORT.has(extension) + ? ` Add the import the accessor needs (\`os\`) if this file does not already have it.` + : '' + return { + description: `Read the credential from \`${variable}\` at runtime and document it in .env.example.`, + kind: 'diff', + language: 'diff', + content, + safetyNote: + `Rotate this credential first — it is already in Git history, and editing the line does not remove it from earlier commits. ` + + `The removed line is shown with the value masked, so this diff will NOT apply cleanly by design; make the edit by hand.${importNote}`, + } +} + +/** + * A committed `.env`: the fix is to stop tracking the file, not to rewrite a + * line. Concrete because the path is known; history rewriting is named as a + * separate decision rather than scripted. + */ +export function untrackEnvFileFix(filePath: string): FindingFix { + const content = [ + '# 1. Rotate every credential in this file at its provider first.', + `# 2. Stop tracking the file (this does NOT remove it from earlier commits):`, + `git rm --cached ${filePath}`, + `printf '%s\\n' '${filePath}' >> .gitignore`, + '# 3. Commit a value-free .env.example in its place.', + '', + ].join('\n') + return { + description: `Stop tracking ${filePath} and keep its values out of the repository.`, + kind: 'snippet', + language: 'sh', + content, + safetyNote: + 'Run this only after rotating the credentials. Untracking leaves every past commit intact, so treat the values as ' + + 'exposed until they are rotated; purging history is a separate, coordinated decision for a shared repository.', + } +} + +/** Package managers whose lockfile-refresh command is unambiguous. */ +export type LockfileManager = 'pnpm' | 'yarn' | 'npm' | 'bun' + +const LOCKFILE_REFRESH: Record = { + pnpm: { command: 'pnpm install --lockfile-only', lockfile: 'pnpm-lock.yaml' }, + yarn: { command: 'yarn install', lockfile: 'yarn.lock' }, + npm: { command: 'npm install --package-lock-only', lockfile: 'package-lock.json' }, + bun: { command: 'bun install', lockfile: 'bun.lock' }, +} + +/** + * Lockfile refresh for the detected package manager. With no manager evidence + * the snippet lists every command instead of picking one — a lockfile written + * by the wrong manager is worse than no lockfile. + */ +export function lockfileRefreshFix(manager: LockfileManager | undefined): FindingFix { + if (manager) { + const { command, lockfile } = LOCKFILE_REFRESH[manager] + return { + description: `Generate and commit ${lockfile} with ${manager}.`, + kind: 'snippet', + language: 'sh', + content: `${command}\ngit add ${lockfile}\n`, + safetyNote: + `Detected from this repository's own ${manager} configuration. Review the generated ${lockfile} before committing — ` + + 'it pins every transitive version resolved on the machine that ran the command.', + } + } + const lines = (Object.keys(LOCKFILE_REFRESH) as LockfileManager[]).flatMap((name) => [ + `# ${name}`, + `${LOCKFILE_REFRESH[name].command} && git add ${LOCKFILE_REFRESH[name].lockfile}`, + ]) + return { + description: 'Generate and commit a lockfile with the package manager your team uses.', + kind: 'snippet', + language: 'sh', + content: `${lines.join('\n')}\n`, + safetyNote: + 'No package-manager evidence was found in this repository, so every option is listed rather than one guessed. ' + + 'Run only the line for the manager your team uses — a lockfile from the wrong manager is worse than none.', + } +} + +export interface ReadmeStarterEvidence { + projectName: string + /** Install command for the detected package manager, when there is one. */ + installCommand?: string +} + +/** Minimal README skeleton: the four sections the docs analyzer looks for. */ +export function readmeStarterFix(evidence: ReadmeStarterEvidence): FindingFix { + const quickStart = evidence.installCommand + ? ['```sh', evidence.installCommand, '```'] + : ['Document the install and run commands for this project.'] + const content = [ + `# ${evidence.projectName}`, + '', + 'One paragraph: what this project does and who it is for.', + '', + '## Quick start', + '', + ...quickStart, + '', + '## Environment variables', + '', + 'Every variable this project reads, with a one-line purpose. Real values belong in a secret manager, never here.', + '', + '## Deployment', + '', + 'How a change reaches production.', + '', + ].join('\n') + return { + description: 'Add a root README covering purpose, quick start, environment variables, and deployment.', + kind: 'snippet', + language: 'markdown', + content, + safetyNote: + 'A skeleton, not a description of this project — the prose is placeholder text that has to be replaced before it ' + + 'is worth committing.', + } +} + +export interface CiStarterEvidence { + /** Package manager the workflow should install with. */ + manager: LockfileManager + /** True when a lockfile is committed, which decides frozen vs plain install. */ + hasLockfile: boolean + /** `packageManager` field in package.json, required by corepack for pnpm/yarn. */ + hasPackageManagerField: boolean + /** package.json scripts that actually exist, in the order they should run. */ + scripts: string[] +} + +const CI_SETUP_STEPS: Record string[] | undefined> = { + npm: () => [' - uses: actions/setup-node@v4', ' with:', ' node-version: lts/*'], + pnpm: (evidence) => (evidence.hasPackageManagerField + ? [ + ' - uses: actions/setup-node@v4', + ' with:', + ' node-version: lts/*', + ' - run: corepack enable', + ] + : undefined), + yarn: (evidence) => (evidence.hasPackageManagerField + ? [ + ' - uses: actions/setup-node@v4', + ' with:', + ' node-version: lts/*', + ' - run: corepack enable', + ] + : undefined), + bun: () => [' - uses: oven-sh/setup-bun@v2'], +} + +const CI_INSTALL: Record string> = { + npm: (frozen) => (frozen ? 'npm ci' : 'npm install'), + pnpm: (frozen) => (frozen ? 'pnpm install --frozen-lockfile' : 'pnpm install'), + yarn: (frozen) => (frozen ? 'yarn install --immutable' : 'yarn install'), + bun: (frozen) => (frozen ? 'bun install --frozen-lockfile' : 'bun install'), +} + +/** + * Minimal GitHub Actions workflow. Emitted only for Node repositories whose + * package.json names the scripts to run, so the workflow never invokes a script + * that does not exist. Returns undefined when the setup steps would have to be + * guessed (pnpm/yarn without a `packageManager` field for corepack). + */ +export function ciWorkflowFix(evidence: CiStarterEvidence): FindingFix | undefined { + const setup = CI_SETUP_STEPS[evidence.manager](evidence) + if (!setup || evidence.scripts.length === 0) return undefined + const runner = evidence.manager === 'npm' ? 'npm run' : `${evidence.manager} run` + const content = [ + 'name: CI', + 'on:', + ' push:', + ' branches: [main]', + ' pull_request:', + 'jobs:', + ' verify:', + ' runs-on: ubuntu-latest', + ' steps:', + ' - uses: actions/checkout@v4', + ...setup, + ` - run: ${CI_INSTALL[evidence.manager](evidence.hasLockfile)}`, + ...evidence.scripts.map((script) => ` - run: ${runner} ${script}`), + '', + ].join('\n') + return { + description: `Add .github/workflows/ci.yml running ${evidence.scripts.map((script) => `\`${script}\``).join(' and ')} on every push and pull request.`, + kind: 'snippet', + language: 'yaml', + content, + safetyNote: + `The script names come from this repository's package.json; the action versions and \`lts/*\` Node version are ` + + 'defaults to pin to whatever your organization allows before merging.', + } +} diff --git a/packages/analyzer-engine/src/indexer.ts b/packages/analyzer-engine/src/indexer.ts index 23f8756..88dee8c 100644 --- a/packages/analyzer-engine/src/indexer.ts +++ b/packages/analyzer-engine/src/indexer.ts @@ -100,7 +100,12 @@ async function walk( if (IGNORED_DIRS.has(entry.name)) continue await walk(full, root, state) } else if (entry.isFile()) { - state.paths.push(relative(root, full)) + // POSIX separators always: `relative` yields `src\users.ts` on Windows, + // and every other path surface (receipts, git snapshots, policy globs, + // scope inference) normalizes to `/`. Emitting the raw platform form here + // would make the same file read differently on different platforms and + // break every path comparison against those surfaces. + state.paths.push(relative(root, full).replaceAll('\\', '/')) } } } @@ -209,13 +214,21 @@ export async function indexWorkingTree( // knowledge graph too), and record it for one consolidated finding. Only // code kinds are eligible — a doc/config with a "do not edit" banner is not // machine-written source. + // + // The text is retained in `excludedContent` because this exclusion exists to + // suppress FALSE findings about machine-written code, not to stop looking + // for credentials. A `// AUTO-GENERATED` banner above a live Stripe key must + // never buy that key a pass — see secretsAnalyzer. if ( content && (kind === 'source' || kind === 'component' || kind === 'route' || kind === 'test') && generatedLabel(content) ) { generatedFiles[path] = loc - files.push({ path, language, kind: 'generated', sizeBytes: size, loc, sha, content: null }) + files.push({ + path, language, kind: 'generated', sizeBytes: size, loc, sha, + content: null, excludedContent: content, + }) continue } diff --git a/packages/analyzer-engine/src/secrets.ts b/packages/analyzer-engine/src/secrets.ts index 9dd47fd..cc17e38 100644 --- a/packages/analyzer-engine/src/secrets.ts +++ b/packages/analyzer-engine/src/secrets.ts @@ -1,3 +1,4 @@ +import { moveSecretToEnvFix, untrackEnvFileFix } from './fixes' import { incompleteAnalyzerOutput, type Analyzer, type AnalyzerFinding } from './types' /** @@ -73,12 +74,26 @@ export const secretsAnalyzer: Analyzer = { async run(index) { const findings: AnalyzerFinding[] = [] const findingLimit = 50 + // Where a move-to-env fix appends its variable. Undefined means the file + // does not exist, which the diff renders as a new-file hunk. + const envExample = index.files.find((file) => file.path === '.env.example')?.content + const envExampleLines = envExample === undefined || envExample === null + ? undefined + : envExample.length === 0 ? 0 : envExample.replace(/\n$/, '').split('\n').length for (const file of index.files) { - if (!file.content || SKIP_FILES.test(file.path)) continue + // Generated/minified files are excluded from every OTHER analyzer to stop + // machine-written output producing spurious quality findings. That + // exclusion must never extend to credentials: a leaked key is a leak no + // matter which tool emitted the line, and a `DO NOT EDIT` banner would + // otherwise be a one-comment bypass of the whole secret scanner. + const content = file.content ?? file.excludedContent + if (!content || SKIP_FILES.test(file.path)) continue + /** Machine-written text: read for credentials, but never hand-edited. */ + const isGeneratedFile = !file.content const isTestContext = TEST_PATH_RE.test(file.path) const isSeedScript = SEED_PATH_RE.test(file.path) - const lines = file.content.split('\n') + const lines = content.split('\n') for (let i = 0; i < lines.length && findings.length < findingLimit; i++) { const line = lines[i] if (PLACEHOLDER.test(line)) continue @@ -157,6 +172,27 @@ export const secretsAnalyzer: Analyzer = { metadata: { credentialType: name, testContext: isTestContext, messageString }, }) } else { + // A concrete fix only where the evidence determines one: a tracked + // .env is untracked wholesale, and a source assignment becomes an + // environment read. Anything else (a key inside a call, a private + // key block, an unsupported language) keeps the prose suggestion. + // + // A generated file is read here (`content` came from + // excludedContent) precisely so its credentials are not exempt — + // but its LINE is not the place to fix them. The next generation + // overwrites any edit; the credential has to leave the generator's + // input. So the leak is reported and the diff is withheld. + const fix = isEnvFile + ? untrackEnvFileFix(file.path) + : isGeneratedFile + ? undefined + : moveSecretToEnvFix({ + filePath: file.path, + line: i + 1, + lineText: line, + credentialType: name, + envExampleLines, + }) findings.push({ category: 'SECURITY_HYGIENE', severity: isEnvFile ? 'CRITICAL' : 'HIGH', @@ -165,6 +201,7 @@ export const secretsAnalyzer: Analyzer = { filePath: file.path, line: i + 1, suggestion: 'Rotate this credential immediately, move it to environment configuration, and add the file to .gitignore. Consider a pre-commit secret scanner.', + ...(fix ? { fix } : {}), impactScore: 95, effort: 'low', metadata: { credentialType: name }, diff --git a/packages/analyzer-engine/src/security/analysis-scope.ts b/packages/analyzer-engine/src/security/analysis-scope.ts new file mode 100644 index 0000000..254545b --- /dev/null +++ b/packages/analyzer-engine/src/security/analysis-scope.ts @@ -0,0 +1,79 @@ +import { CWE, type CweInfo } from './cwe' +import { PATTERN_RULES, TAINT_SINKS } from './rules' + +/** + * What the SAST engine's security verdict does — and does not — cover. + * + * A security score computed over the classes the engine can see must never be + * read as a verdict over the classes it cannot. This module is the single + * source of truth for both lists: the analyzed classes are DERIVED from the + * live rule pack (they cannot drift from reality), and the not-analyzed list + * is curated with a sync-guard test (tests/scoring.test.ts) that forces an + * entry out the moment a rule for its class ships. + */ + +/** Vulnerability classes with rule-backed analysis, derived from the rule pack. */ +export function analyzedVulnerabilityClasses(): CweInfo[] { + const keys = new Set() + for (const rule of TAINT_SINKS) keys.add(rule.cweKey) + for (const rule of PATTERN_RULES) keys.add(rule.cweKey) + return [...keys].map((key) => CWE[key]).sort((a, b) => a.title.localeCompare(b.title)) +} + +export interface UnanalyzedClass { + name: string + /** CWE id when one exists — feeds the no-overlap sync guard. */ + cwe?: string + /** One sentence on why this needs eyes, phrased for a report reader. */ + detail: string +} + +/** + * Classes a syntactic rule/taint engine structurally cannot judge. These need + * a human review (or a future analysis the engine does not have yet) — the + * report says so explicitly instead of letting a green score imply them. + */ +export const UNANALYZED_VULNERABILITY_CLASSES: UnanalyzedClass[] = [ + { + name: 'Authorization & tenant isolation (IDOR)', + cwe: 'CWE-639', + detail: + 'whether each endpoint verifies the caller may access the specific resource it names — this requires understanding your permission model, not just the code', + }, + { + name: 'Authentication & session management logic', + cwe: 'CWE-287', + detail: 'correctness of login flows, token lifetimes, session fixation and revocation', + }, + { + name: 'Cross-site scripting (XSS) — partially analyzed', + cwe: 'CWE-79', + detail: + 'tainted dangerouslySetInnerHTML / innerHTML / outerHTML assignments ARE analyzed in JavaScript, TypeScript and TSX; server-side template engines, hand-built HTML strings and non-JS languages are not', + }, + { + name: 'Cross-site request forgery (CSRF)', + cwe: 'CWE-352', + detail: 'presence and correctness of anti-CSRF protections on state-changing routes', + }, + { + name: 'Webhook & callback signature verification', + cwe: 'CWE-347', + detail: 'whether inbound webhooks verify their signatures before acting on the payload', + }, + { + name: 'Business-logic flaws', + cwe: 'CWE-840', + detail: 'price manipulation, workflow bypass, quantity and limit abuse', + }, + { + name: 'Race conditions — partially analyzed', + cwe: 'CWE-367', + detail: + 'unguarded read-modify-write pairs — a row read, recomputed arithmetically, and written back outside a transaction — ARE analyzed in JavaScript, TypeScript and TSX; check-then-act windows (validate then insert, existence check before a write, multi-request state machines) are not', + }, + { + name: 'Infrastructure & deployment configuration', + detail: 'cloud IAM, network policy and runtime hardening living outside this repository', + }, +] diff --git a/packages/analyzer-engine/src/security/cwe.ts b/packages/analyzer-engine/src/security/cwe.ts new file mode 100644 index 0000000..6ad3c1b --- /dev/null +++ b/packages/analyzer-engine/src/security/cwe.ts @@ -0,0 +1,42 @@ +/** + * CWE → OWASP Top-10 (2021) mapping and canonical titles for the classes this + * engine detects. Keeping this in one table means every rule cites the same, + * accurate identifiers — credibility depends on getting these exactly right. + */ + +export interface CweInfo { + cwe: string + owasp: string + title: string +} + +export const CWE: Record = { + SQLI: { cwe: 'CWE-89', owasp: 'A03:2021 Injection', title: 'SQL Injection' }, + CMDI: { cwe: 'CWE-78', owasp: 'A03:2021 Injection', title: 'OS Command Injection' }, + CODEI: { cwe: 'CWE-95', owasp: 'A03:2021 Injection', title: 'Code Injection (eval)' }, + PATH: { cwe: 'CWE-22', owasp: 'A01:2021 Broken Access Control', title: 'Path Traversal' }, + SSRF: { cwe: 'CWE-918', owasp: 'A10:2021 Server-Side Request Forgery', title: 'Server-Side Request Forgery' }, + DESER: { cwe: 'CWE-502', owasp: 'A08:2021 Software and Data Integrity Failures', title: 'Insecure Deserialization' }, + ALLOC: { cwe: 'CWE-789', owasp: 'A08:2021 Software and Data Integrity Failures', title: 'Uncontrolled Memory Allocation' }, + DECOMP: { cwe: 'CWE-409', owasp: 'A08:2021 Software and Data Integrity Failures', title: 'Decompression Bomb (Data Amplification)' }, + WEAKHASH: { cwe: 'CWE-328', owasp: 'A02:2021 Cryptographic Failures', title: 'Weak Hash' }, + WEAKCIPHER: { cwe: 'CWE-327', owasp: 'A02:2021 Cryptographic Failures', title: 'Broken/Weak Cryptographic Algorithm' }, + WEAKRNG: { cwe: 'CWE-338', owasp: 'A02:2021 Cryptographic Failures', title: 'Cryptographically Weak PRNG' }, + TLS: { cwe: 'CWE-295', owasp: 'A07:2021 Identification and Authentication Failures', title: 'Improper Certificate Validation' }, + HARDCRED: { cwe: 'CWE-798', owasp: 'A07:2021 Identification and Authentication Failures', title: 'Hard-coded Credentials' }, + CLIENTSECRET: { cwe: 'CWE-200', owasp: 'A01:2021 Broken Access Control', title: 'Exposure of Sensitive Information' }, + XSS: { cwe: 'CWE-79', owasp: 'A03:2021 Injection', title: 'Cross-Site Scripting' }, + XXE: { cwe: 'CWE-611', owasp: 'A05:2021 Security Misconfiguration', title: 'XML External Entity (XXE)' }, + OPENREDIR: { cwe: 'CWE-601', owasp: 'A01:2021 Broken Access Control', title: 'Open Redirect' }, + COOKIE: { cwe: 'CWE-1004', owasp: 'A05:2021 Security Misconfiguration', title: 'Insecure Cookie' }, + REDOS: { cwe: 'CWE-1333', owasp: 'A05:2021 Security Misconfiguration', title: 'Regular Expression Denial of Service' }, + TIMING: { cwe: 'CWE-208', owasp: 'A02:2021 Cryptographic Failures', title: 'Observable Timing Discrepancy' }, + // CWE-252/362/697/1050 have no official OWASP Top-10 (2021) mapping — say so + // rather than borrow an injection bucket the finding does not belong to. + UNAWAITED: { cwe: 'CWE-252', owasp: 'Not mapped — reliability', title: 'Unchecked Return Value (floating promise)' }, + SWALLOW: { cwe: 'CWE-1069', owasp: 'A09:2021 Security Logging and Monitoring Failures', title: 'Empty Exception Block' }, + LOOSEEQ: { cwe: 'CWE-697', owasp: 'Not mapped — correctness', title: 'Incorrect Comparison' }, + NPLUSONE: { cwe: 'CWE-1050', owasp: 'Not mapped — performance', title: 'Excessive Platform Resource Consumption within a Loop' }, + MASSASSIGN: { cwe: 'CWE-915', owasp: 'A08:2021 Software and Data Integrity Failures', title: 'Mass Assignment' }, + RMWRACE: { cwe: 'CWE-362', owasp: 'Not mapped — correctness', title: 'Race Condition (lost update)' }, +} diff --git a/packages/analyzer-engine/src/security/engine.ts b/packages/analyzer-engine/src/security/engine.ts new file mode 100644 index 0000000..8250047 --- /dev/null +++ b/packages/analyzer-engine/src/security/engine.ts @@ -0,0 +1,586 @@ +import { + sastLanguageForPath, + MAX_NODES, + type SastLanguage, + type SastParser, + type SyntaxNode, +} from './lang' +import { asCall, asFunction, urlHeadOf, walk, type NCall, type NFunc } from './normalize' +import { + analyzeFunction, + bindsToLocalFn, + firstSource, + hasRealSource, + paramIndexes, + taintBudgetExhausted, + taintOf, + type FunctionTaint, +} from './taint' +import { PATTERN_RULES, TAINT_ASSIGN_SINKS, TAINT_SINKS, asNamedValue, ruleAppliesTo, type TaintAssignSink, type TaintSink } from './rules' +import { CWE } from './cwe' +import type { CodeLocation, Flow, SastFinding, SastResult } from './types' + +/** + * The SAST engine: parse → taint → match rules → findings. + * + * Per file we run one taint solve per function, evaluate every call against the + * taint sinks (direct findings + interprocedural param summaries), then a single + * pattern-rule pass over the whole tree. Everything is bounded per file and the + * whole thing is fail-soft: a parse error, a grammar that won't load, or a + * runaway file degrades to fewer/zero findings for that file — never a thrown + * scan. Absence of a finding is therefore never a proof of safety, which the + * diagnostics make explicit. + * + * The parser is injected ({@link SastParser}) so the same rule pack runs behind + * the hosted WASM grammars and behind the CLI's zero-dependency JS parser. A + * language the injected parser does not cover is reported as degraded — the one + * honest answer, and the reason a leaner parser can never turn into a wrong + * finding. + */ + +export interface ScanInput { + filePath: string + content: string +} + +/** Rules this scan is allowed to report. Absent = the whole pack. */ +export interface ScanOptions { + /** Rule ids to keep. A rule outside the set never runs against a node. */ + ruleIds?: ReadonlySet +} + +const MAX_FINDINGS_PER_FILE = 100 +const MAX_FINDINGS_PER_SCAN = 100_000 + +/** Scan a set of source files and return findings + honest diagnostics. */ +export async function scanFiles( + files: ScanInput[], + parser: SastParser, + options: ScanOptions = {}, +): Promise { + const findings: SastFinding[] = [] + let filesScanned = 0 + let filesSkipped = 0 + let truncatedFiles = 0 + let findingsTruncated = false + const degraded = new Set() + + // web-tree-sitter grows its WASM heap while parsing and does not return that + // high-water allocation to the host process between files. Continuing with + // even a pattern-only parse after crossing the function budget can therefore + // OOM the worker. Stop parsing new files once RSS crosses the limit, report + // them as skipped, and let the scan authority layer withhold scores. This is + // fail-closed: partial findings survive, but absence is never called clean. + const configuredRssLimit = Number(process.env.SAST_TAINT_RSS_LIMIT_MB) + const defaultRssLimit = process.env.NODE_ENV === 'test' ? Number.POSITIVE_INFINITY : 850 + const taintRssLimit = + (Number.isFinite(configuredRssLimit) && configuredRssLimit > 0 + ? configuredRssLimit + : defaultRssLimit) * 1048576 + let memoryLimitReached = false + + for (const file of files) { + const lang = sastLanguageForPath(file.filePath) + if (!lang) { + filesSkipped++ + continue + } + if (memoryLimitReached || process.memoryUsage().rss > taintRssLimit) { + memoryLimitReached = true + filesSkipped++ + continue + } + try { + const fileFindings = await scanOne(file.filePath, file.content, lang, degraded, parser, options) + if (fileFindings === null) { + filesSkipped++ + continue + } + filesScanned++ + if (fileFindings.truncated) truncatedFiles++ + for (const f of fileFindings.findings) { + if (findings.length >= MAX_FINDINGS_PER_SCAN) { + findingsTruncated = true + break + } + findings.push(f) + } + if (process.memoryUsage().rss > taintRssLimit) memoryLimitReached = true + } catch { + // never let one file crash the scan + filesSkipped++ + } + } + + dedupe(findings) + sortFindings(findings) + + return { + findings, + diagnostics: { + inputFiles: files.length, + filesScanned, + filesSkipped, + degradedLanguages: [...degraded].sort(), + truncatedFiles, + findingsTruncated, + resourceLimitReached: memoryLimitReached, + }, + } +} + +/** Merge isolated batch results back into the same deterministic contract as a + * single in-process scan. Each source file must belong to exactly one retained + * batch result; callers discard a memory-truncated parent batch before retrying + * its smaller children. */ +export function mergeSastResults(results: SastResult[], inputFiles: number): SastResult { + const findings = results.flatMap((result) => result.findings) + dedupe(findings) + sortFindings(findings) + const findingsTruncated = + results.some((result) => result.diagnostics.findingsTruncated) || + findings.length > MAX_FINDINGS_PER_SCAN + if (findings.length > MAX_FINDINGS_PER_SCAN) findings.length = MAX_FINDINGS_PER_SCAN + + const degradedLanguages = new Set() + let filesScanned = 0 + let filesSkipped = 0 + let truncatedFiles = 0 + for (const result of results) { + filesScanned += result.diagnostics.filesScanned + filesSkipped += result.diagnostics.filesSkipped + truncatedFiles += result.diagnostics.truncatedFiles + for (const language of result.diagnostics.degradedLanguages) degradedLanguages.add(language) + } + + return { + findings, + diagnostics: { + inputFiles, + filesScanned, + filesSkipped, + degradedLanguages: [...degradedLanguages].sort(), + truncatedFiles, + findingsTruncated, + resourceLimitReached: results.some((result) => result.diagnostics.resourceLimitReached), + budgetExceeded: results.some((result) => result.diagnostics.budgetExceeded), + failureReason: results.find((result) => result.diagnostics.failureReason)?.diagnostics.failureReason, + }, + } +} + +interface FileScan { + findings: SastFinding[] + truncated: boolean +} + +/** Which param indexes of a local function reach which sink. */ +interface ParamSinkRecord { + sink: TaintSink + node: SyntaxNode + line: number +} +interface FnRecord { + fn: NFunc + ft: FunctionTaint + /** param index → the sink it reaches (first wins). */ + sinkParams: Map +} + +async function scanOne( + filePath: string, + content: string, + lang: SastLanguage, + degraded: Set, + parser: SastParser, + options: ScanOptions, +): Promise { + const parsed = await parser.parse(lang, content) + if (!parsed) { + degraded.add(lang) + return null + } + + const lines = content.split('\n') + const loc = (node: SyntaxNode, label: string): CodeLocation & { label: string } => ({ + filePath, + line: node.startPosition.row + 1, + column: node.startPosition.column + 1, + snippet: snippetAt(lines, node.startPosition.row), + label, + }) + + const findings: SastFinding[] = [] + const enabled = (id: string) => !options.ruleIds || options.ruleIds.has(id) + const applicableSinks = TAINT_SINKS.filter((s) => ruleAppliesTo(s, lang) && enabled(s.id)) + const applicablePatterns = PATTERN_RULES.filter((p) => ruleAppliesTo(p, lang) && enabled(p.id)) + const applicableAssignSinks = TAINT_ASSIGN_SINKS.filter((s) => ruleAppliesTo(s, lang) && enabled(s.id)) + + let nodeCount = 0 + let truncated = false + + // ---- gather functions & solve taint per function ---- + const fnRecords: FnRecord[] = [] + walk(parsed.rootNode, (node) => { + if (++nodeCount > MAX_NODES) { + truncated = true + return + } + const fn = asFunction(node, lang) + if (fn && fn.body) { + const ft = analyzeFunction(fn, lang) + fnRecords.push({ fn, ft, sinkParams: new Map() }) + } + }) + + // ---- direct sink findings + build interprocedural summaries ---- + for (const rec of fnRecords) { + if (!rec.fn.body) continue + walk(rec.fn.body, (node) => { + if (findings.length >= MAX_FINDINGS_PER_FILE) return + if (isNestedFnBoundary(node, rec.fn, lang)) return + // Assignment-shaped sinks (XSS): `__html:` is a JSX pair and + // `el.innerHTML =` an assignment, so neither reaches a call-based rule. + for (const sink of applicableAssignSinks) { + const nv = asNamedValue(node, lang) + if (!nv || !sink.matchName(nv.name)) continue + if (sink.safeValue?.(nv.value, lang)) continue + const origins = taintOf(nv.value, rec.ft) + if (!hasRealSource(origins)) continue + const src = firstSource(origins)! + findings.push(makeAssignFinding(sink, lang, filePath, nv.node, src.node, src.sourceKind, loc, lines)) + } + const call = asCall(node, lang) + if (!call) return + for (const sink of applicableSinks) { + const idxs = sink.match(call, lang) + if (!idxs) continue + for (const i of idxs) { + const arg = call.args[i] + if (!arg) continue + const origins = taintOf(arg, rec.ft) + if (origins.length === 0) continue + // Head-position sinks (SSRF): taint confined to path/query segments + // of a constant-authority URL cannot steer the request target. + if (sink.taintPosition === 'head' && headTaintSuppressed(arg, rec.ft, lang)) continue + if (hasRealSource(origins)) { + const src = firstSource(origins)! + findings.push(makeTaintFinding(sink, lang, filePath, call, src.node, src.sourceKind, false, loc, lines)) + } + for (const pi of paramIndexes(origins)) { + if (!rec.sinkParams.has(pi)) rec.sinkParams.set(pi, { sink, node: call.node, line: call.line }) + } + } + break // one sink per call site + } + }) + } + + // ---- one-hop interprocedural: tainted arg → param that reaches a sink ---- + const summaries = new Map() + for (const rec of fnRecords) { + if (rec.sinkParams.size > 0 && !summaries.has(rec.fn.name)) summaries.set(rec.fn.name, rec) + } + if (summaries.size > 0) { + for (const rec of fnRecords) { + if (!rec.fn.body) continue + walk(rec.fn.body, (node) => { + if (findings.length >= MAX_FINDINGS_PER_FILE) return + const call = asCall(node, lang) + if (!call || call.isConstruct) return + if (!bindsToLocalFn(call)) return // don't bind a member call to a same-name local fn + const target = summaries.get(call.method) + if (!target || target.fn === rec.fn) return + for (const [pi, record] of target.sinkParams) { + const arg = call.args[pi] + if (!arg) continue + const origins = taintOf(arg, rec.ft) + const src = firstSource(origins) + if (!src) continue + findings.push( + makeInterprocFinding(record.sink, lang, filePath, call, target.fn, record, src.node, src.sourceKind, loc, lines), + ) + } + }) + } + } + + // ---- pattern rules (single pass over the whole tree) ---- + runPatternRules(parsed.rootNode, applicablePatterns, lang, filePath, findings, lines) + + parsed.release() + // A drained taint budget means some solve or sink query degraded to + // no-taint — real coverage loss, same as the node budget. A tree with error + // nodes (parsed.hasError) is still fully walked best-effort, and the + // per-file findings cap only bounds OUTPUT — neither degrades coverage. + if (fnRecords.some((rec) => taintBudgetExhausted(rec.ft))) truncated = true + return { findings, truncated } +} + +/** Pattern-rule pass over a tree. Cheap (no taint), so it always runs even when + * the taint solve is skipped under memory pressure. */ +function runPatternRules( + root: SyntaxNode, + patterns: typeof PATTERN_RULES, + lang: SastLanguage, + filePath: string, + findings: SastFinding[], + lines: string[], +): void { + // Per-rule path exclusion (seed/migration/ops directories where the pattern + // is expected and harmless) — resolved once per file, not per node. + const activePatterns = patterns.filter((rule) => !rule.excludePath || !rule.excludePath.test(filePath)) + walk(root, (node) => { + if (findings.length >= MAX_FINDINGS_PER_FILE) return + for (const rule of activePatterns) { + for (const hit of rule.test(node, lang)) { + const info = CWE[rule.cweKey] + findings.push({ + ruleId: rule.id, + kind: 'pattern', + cwe: info.cwe, + owasp: info.owasp, + severity: rule.severity, + title: rule.title, + message: rule.message, + language: lang, + filePath, + line: hit.line, + column: hit.node.startPosition.column + 1, + remediation: rule.remediation, + metadata: sortMeta({ detail: hit.detail, snippet: snippetAt(lines, hit.line - 1) }), + }) + } + } + }) +} + +/** A finding for a nested function is handled by that function's own record. */ +function isNestedFnBoundary(node: SyntaxNode, fn: NFunc, lang: SastLanguage): boolean { + return node !== fn.node && node !== fn.body && asFunction(node, lang) !== null +} + +/** Constant URL head that pins scheme + authority (`scheme://host/…`) — taint + * after it can only land in the path/query/fragment. */ +const AUTHORITY_PINNED_PREFIX = /^[a-z][a-z0-9+.-]*:\/\/[^\/?#]+[\/?#]/i +/** Single-slash relative path with a constant character after the slash. A + * lone '/' does NOT qualify: taint abutting it becomes `//host`, which + * fetch/axios treat as protocol-relative and follow off-origin. */ +const SINGLE_SLASH_PATH = /^\/[^\/]/ +/** Constant fragment after the head expression that ends the authority: a + * single '/' starts the path, and '?' or '#' start the query/fragment — past + * any of them a tainted part cannot reach the host. A fragment that is empty + * or starts with ':' or '//' leaves the following part in scheme/authority + * position (`${scheme}://${host}`) and must NOT anchor. */ +const PATH_ANCHOR = /^(\/(?!\/)|[?#])/ +/** A scheme/protocol-relative opener (`http://`, `http://a`, `//`) — when the + * authority-pinning match above failed, the host is still attacker-extendable. */ +const AUTHORITY_OPENER = /^([a-z][a-z0-9+.-]*:)?\/\//i + +/** + * Position-aware suppression for head-position sinks (SSRF). A URL built as a + * template/concat is only attacker-steerable when taint can reach its scheme/ + * authority: suppress when a constant head pins the authority (or is a + * single-slash relative path), or when the head expression is untainted (e.g. + * a `${BASE}` config var) and every tainted part lands after a path-anchoring + * constant fragment. Deliberately conservative — an empty, protocol-relative + * ('//') or incomplete-host ('http://', 'http://a') prefix, or a tainted head, + * still flags, and a non-template/concat argument is never suppressed. + */ +function headTaintSuppressed(arg: SyntaxNode, ft: FunctionTaint, lang: SastLanguage): boolean { + const { constantPrefix, headExpr, parts } = urlHeadOf(arg, lang) + if (!parts) return false // opaque expression — the whole argument is the URL + if (constantPrefix !== null && (AUTHORITY_PINNED_PREFIX.test(constantPrefix) || SINGLE_SLASH_PATH.test(constantPrefix))) { + return true + } + // An unclosed authority in the constant head ('http://a…') means whatever + // follows extends the HOST, not the path — never suppress. + if (constantPrefix && AUTHORITY_OPENER.test(constantPrefix)) return false + if (headExpr && taintOf(headExpr, ft).length > 0) return false // taint steers the authority + // A tainted part is harmless only once a constant fragment AFTER the head + // expression path-anchors it. A connecting fragment that could still place + // the tainted part in scheme/authority position — empty, ':', '//', '://' — + // must not suppress (`${scheme}://${req.query.host}/x` steers the HOST). + let pastHead = false + let pathAnchored = false + for (const part of parts) { + if (part.kind === 'const') { + if (pastHead && PATH_ANCHOR.test(part.text)) pathAnchored = true + } else if (part.node === headExpr) { + pastHead = true + } else if (!pathAnchored && taintOf(part.node, ft).length > 0) { + return false + } + } + return true +} + +function makeTaintFinding( + sink: TaintSink, + lang: SastLanguage, + filePath: string, + call: NCall, + sourceNode: SyntaxNode, + sourceKind: string, + interprocedural: boolean, + loc: (n: SyntaxNode, label: string) => CodeLocation & { label: string }, + lines: string[], +): SastFinding { + const info = CWE[sink.cweKey] + const source = loc(sourceNode, `untrusted input (${sourceKind})`) + const sinkLoc = loc(call.node, `${call.fullName}()`) + const flow: Flow = { + source, + sink: sinkLoc, + steps: [source, sinkLoc], + summary: `${sourceKind} → ${call.fullName}()`, + interprocedural, + } + // A same-line source→sink is one expression, not a two-hop journey. + const sameLine = source.filePath === sinkLoc.filePath && source.line === sinkLoc.line + return { + ruleId: sink.id, + kind: 'taint', + cwe: info.cwe, + owasp: info.owasp, + severity: sink.severity, + title: sink.title, + message: sameLine + ? `${sink.message} Untrusted data from ${sourceKind} reaches ${call.fullName}() in the same expression (line ${sinkLoc.line}).` + : `${sink.message} Untrusted data from ${sourceKind} (line ${source.line}) reaches ${call.fullName}() at line ${sinkLoc.line}.`, + language: lang, + filePath, + line: call.line, + column: call.node.startPosition.column + 1, + flow, + remediation: sink.remediation, + metadata: sortMeta({ sourceKind, sink: call.fullName, snippet: snippetAt(lines, call.line - 1) }), + } +} + +/** Finding for an assignment-shaped sink, where the location is the binding. */ +function makeAssignFinding( + sink: TaintAssignSink, + lang: SastLanguage, + filePath: string, + sinkNode: SyntaxNode, + sourceNode: SyntaxNode, + sourceKind: string, + loc: (n: SyntaxNode, label: string) => CodeLocation & { label: string }, + lines: string[], +): SastFinding { + const info = CWE[sink.cweKey] + const source = loc(sourceNode, `untrusted input (${sourceKind})`) + const sinkLoc = loc(sinkNode, sink.title) + const flow: Flow = { + source, + sink: sinkLoc, + steps: [source, sinkLoc], + summary: `${sourceKind} -> raw HTML`, + interprocedural: false, + } + const sameLine = source.filePath === sinkLoc.filePath && source.line === sinkLoc.line + return { + ruleId: sink.id, + kind: 'taint', + cwe: info.cwe, + owasp: info.owasp, + severity: sink.severity, + title: sink.title, + message: sameLine + ? `${sink.message} Untrusted data from ${sourceKind} is assigned to a raw-HTML binding in the same expression (line ${sinkLoc.line}).` + : `${sink.message} Untrusted data from ${sourceKind} (line ${source.line}) is assigned to a raw-HTML binding at line ${sinkLoc.line}.`, + language: lang, + filePath, + line: sinkLoc.line, + column: sinkNode.startPosition.column + 1, + flow, + remediation: sink.remediation, + metadata: sortMeta({ sourceKind, sink: 'raw HTML', snippet: snippetAt(lines, sinkLoc.line - 1) }), + } +} + +function makeInterprocFinding( + sink: TaintSink, + lang: SastLanguage, + filePath: string, + call: NCall, + callee: NFunc, + record: ParamSinkRecord, + sourceNode: SyntaxNode, + sourceKind: string, + loc: (n: SyntaxNode, label: string) => CodeLocation & { label: string }, + lines: string[], +): SastFinding { + const info = CWE[sink.cweKey] + const source = loc(sourceNode, `untrusted input (${sourceKind})`) + const callSite = loc(call.node, `${call.method}(…) → ${callee.name}()`) + const sinkLoc = loc(record.node, `${sink.title} in ${callee.name}()`) + const flow: Flow = { + source, + sink: sinkLoc, + steps: [source, callSite, sinkLoc], + summary: `${sourceKind} → ${callee.name}(…) → sink in ${callee.name}() (line ${sinkLoc.line})`, + interprocedural: true, + } + return { + ruleId: sink.id, + kind: 'taint', + cwe: info.cwe, + owasp: info.owasp, + severity: sink.severity, + title: sink.title, + message: `${sink.message} Untrusted data from ${sourceKind} (line ${source.line}) is passed to ${callee.name}() and reaches ${sink.title.toLowerCase()} at line ${sinkLoc.line}.`, + language: lang, + filePath, + line: call.line, + column: call.node.startPosition.column + 1, + flow, + remediation: sink.remediation, + metadata: sortMeta({ sourceKind, callee: callee.name, sinkLine: sinkLoc.line, snippet: snippetAt(lines, call.line - 1) }), + } +} + +function snippetAt(lines: string[], row: number): string | undefined { + const raw = lines[row] + if (raw === undefined) return undefined + const trimmed = raw.trim() + return trimmed.length > 200 ? trimmed.slice(0, 200) + '…' : trimmed +} + +/** Deterministic metadata ordering so findings serialize identically each run. */ +function sortMeta(meta: Record): Record { + const out: Record = {} + for (const k of Object.keys(meta).sort()) { + if (meta[k] !== undefined) out[k] = meta[k] + } + return out +} + +const SEV_RANK: Record = { CRITICAL: 4, HIGH: 3, MEDIUM: 2, LOW: 1 } + +/** Stable, severity-then-location ordering — deterministic across runs. */ +function sortFindings(findings: SastFinding[]): void { + findings.sort( + (a, b) => + SEV_RANK[b.severity] - SEV_RANK[a.severity] || + a.filePath.localeCompare(b.filePath) || + a.line - b.line || + a.ruleId.localeCompare(b.ruleId) || + (a.column ?? 0) - (b.column ?? 0), + ) +} + +/** Collapse identical (rule, file, line) findings that different passes emit. */ +function dedupe(findings: SastFinding[]): void { + const seen = new Set() + let w = 0 + for (let r = 0; r < findings.length; r++) { + const f = findings[r] + const key = `${f.ruleId}|${f.filePath}|${f.line}|${f.column ?? ''}` + if (seen.has(key)) continue + seen.add(key) + findings[w++] = f + } + findings.length = w +} diff --git a/packages/analyzer-engine/src/security/finding-map.ts b/packages/analyzer-engine/src/security/finding-map.ts new file mode 100644 index 0000000..5805af7 --- /dev/null +++ b/packages/analyzer-engine/src/security/finding-map.ts @@ -0,0 +1,68 @@ +import type { AnalyzerFinding, FindingCategory } from '../types' +import type { SastFinding, Severity } from './types' + +/** + * Engine finding → analyzer finding, shared by both front-ends. + * + * The hosted pipeline and the CLI must agree on severity weight and category or + * the same defect would rank differently depending on where it was found, and + * the receipt would stop matching the audit. This module is the single place + * that decision lives. + */ + +/** Severity → impactScore (drives fix-plan ranking), aligned with other analyzers. */ +export const SAST_SEVERITY_IMPACT: Record = { + CRITICAL: 95, + HIGH: 80, + MEDIUM: 55, + LOW: 30, + INFO: 15, +} + +/** + * CWE-1050 (db-call-in-loop / N+1) is a performance defect, not a security one: + * PERFORMANCE participates in no score axis, so the rule can never deduct the + * security score while still ranking in fix plans via impactScore and persisting + * through the finding lifecycle. + */ +export function sastFindingCategory(cwe: string): FindingCategory { + return cwe === 'CWE-1050' ? 'PERFORMANCE' : 'SECURITY_HYGIENE' +} + +/** Pure map: one engine finding → one AnalyzerFinding. */ +export function mapSastFinding(finding: SastFinding): AnalyzerFinding { + const locate = (location: { label: string; filePath: string; line: number }) => + `${location.label} (${location.filePath}:${location.line})` + const flow = finding.flow + ? { + source: locate(finding.flow.source), + sink: locate(finding.flow.sink), + path: finding.flow.steps.map(locate), + } + : { + // Pattern findings have no dataflow — the sink site IS the finding. + source: `${finding.title} (${finding.filePath}:${finding.line})`, + sink: `${finding.title} (${finding.filePath}:${finding.line})`, + path: [`${finding.filePath}:${finding.line}`], + } + return { + category: sastFindingCategory(finding.cwe), + severity: finding.severity, + title: finding.title.slice(0, 140), + description: finding.message, + filePath: finding.filePath, + line: finding.line, + suggestion: finding.remediation, + impactScore: SAST_SEVERITY_IMPACT[finding.severity] ?? 50, + effort: 'medium', + metadata: { + // `sast: true` distinguishes real taint findings from regex secret hits + // in the same category — the report Security section keys off it. + sast: true, + cwe: finding.cwe, + owasp: finding.owasp, + ruleId: finding.ruleId, + flow, + }, + } +} diff --git a/packages/analyzer-engine/src/security/js-parse/index.ts b/packages/analyzer-engine/src/security/js-parse/index.ts new file mode 100644 index 0000000..513999b --- /dev/null +++ b/packages/analyzer-engine/src/security/js-parse/index.ts @@ -0,0 +1,37 @@ +import { MAX_SOURCE_BYTES, type ParsedTree, type SastLanguage, type SastParser } from '../lang' +import { parseJs, type JsDialect } from './parser' + +export { ParseError } from './lexer' +export { parseJs } from './parser' +export type { JsDialect } from './parser' + +const DIALECTS: Record = { + javascript: 'javascript', + typescript: 'typescript', + tsx: 'tsx', +} + +const JS_LANGUAGES: ReadonlySet = new Set(['javascript', 'typescript', 'tsx']) + +/** + * The CLI's SAST parser: JavaScript, TypeScript and TSX, no dependencies. + * + * Returns null — degrading that file to "not analyzed" — for any source it + * cannot parse exactly, including every non-JS language. That is the honest + * answer and the safe one: the CLI's receipt reports degraded languages, and a + * file we could not read never produces a finding. + */ +export const zeroDependencyJsParser: SastParser = { + languages: JS_LANGUAGES, + async parse(lang: SastLanguage, content: string): Promise { + const dialect = DIALECTS[lang] + if (!dialect) return null + if (content.length > MAX_SOURCE_BYTES) return null + try { + const rootNode = parseJs(content, dialect) + return { rootNode, hasError: false, release: () => {} } + } catch { + return null + } + }, +} diff --git a/packages/analyzer-engine/src/security/js-parse/lexer.ts b/packages/analyzer-engine/src/security/js-parse/lexer.ts new file mode 100644 index 0000000..95a79bb --- /dev/null +++ b/packages/analyzer-engine/src/security/js-parse/lexer.ts @@ -0,0 +1,318 @@ +/** + * A strict JavaScript/TypeScript tokenizer. + * + * "Strict" is the whole point: anything this lexer cannot read with certainty + * throws {@link ParseError}, the file is skipped, and the scan reports that as + * missing coverage. A tokenizer that guesses would hand the rule pack a wrong + * tree, and a wrong tree is how a security tool produces a false positive. + * + * Regex-vs-division is resolved by the parser, not by heuristics here: the + * parser knows whether it is in operand or operator position and passes + * `regexAllowed`. Template literals are likewise driven by the parser, because + * `${...}` nests arbitrary expressions. + */ + +export class ParseError extends Error {} + +export type TokenKind = + | 'identifier' + | 'private' + | 'number' + | 'string' + | 'regex' + | 'punct' + | 'template_start' + | 'eof' + +export interface Token { + kind: TokenKind + start: number + end: number + /** Punctuator/identifier text; empty for literals (read the span instead). */ + value: string + /** A line terminator appeared between the previous token and this one. */ + newlineBefore: boolean +} + +const PUNCTUATORS: string[] = [ + '>>>=', + '...', '===', '!==', '**=', '<<=', '>>=', '>>>', '&&=', '||=', '??=', + '=>', '==', '!=', '<=', '>=', '&&', '||', '??', '?.', '++', '--', + '+=', '-=', '*=', '/=', '%=', '&=', '|=', '^=', '**', '<<', '>>', + '{', '}', '(', ')', '[', ']', ';', ',', '<', '>', '+', '-', '*', '/', '%', + '&', '|', '^', '!', '~', '?', ':', '=', '.', '@', +] + +function isIdStart(code: number): boolean { + return ( + (code >= 97 && code <= 122) || // a-z + (code >= 65 && code <= 90) || // A-Z + code === 36 || // $ + code === 95 || // _ + code >= 0x80 + ) +} + +function isIdPart(code: number): boolean { + return isIdStart(code) || (code >= 48 && code <= 57) +} + +function isLineTerminator(code: number): boolean { + return code === 10 || code === 13 || code === 0x2028 || code === 0x2029 +} + +function isSpace(code: number): boolean { + return ( + code === 32 || + code === 9 || + code === 11 || + code === 12 || + code === 0xa0 || + code === 0xfeff || + (code >= 0x1680 && /\s/.test(String.fromCharCode(code))) + ) +} + +export interface CommentSpan { + start: number + end: number +} + +export class Lexer { + readonly text: string + pos = 0 + /** Comments seen anywhere during lexing, keyed by start offset (dedupes + * re-lexing during the parser's speculative scans). */ + readonly comments = new Map() + /** Discovery order of {@link comments}. Speculative scans can look ahead and + * record a later comment before an earlier one, so the parser sorts when + * {@link commentsUnsorted} is set rather than on every flush. */ + readonly commentList: CommentSpan[] = [] + commentsUnsorted = false + + private recordComment(start: number, end: number): void { + if (this.comments.has(start)) return + const span = { start, end } + this.comments.set(start, span) + const last = this.commentList[this.commentList.length - 1] + if (last && last.start > start) this.commentsUnsorted = true + this.commentList.push(span) + } + + constructor(text: string) { + this.text = text + // A hashbang is only legal on line 1 and is not a comment node. + if (text.startsWith('#!')) { + while (this.pos < text.length && !isLineTerminator(text.charCodeAt(this.pos))) this.pos++ + } + } + + /** Skip whitespace and comments; returns true when a line break was crossed. */ + private skipTrivia(): boolean { + let newline = false + const text = this.text + while (this.pos < text.length) { + const code = text.charCodeAt(this.pos) + if (isLineTerminator(code)) { + newline = true + this.pos++ + continue + } + if (isSpace(code)) { + this.pos++ + continue + } + if (code === 47 /* / */) { + const next = text.charCodeAt(this.pos + 1) + if (next === 47 /* / */) { + const start = this.pos + this.pos += 2 + while (this.pos < text.length && !isLineTerminator(text.charCodeAt(this.pos))) this.pos++ + this.recordComment(start, this.pos) + continue + } + if (next === 42 /* * */) { + const start = this.pos + this.pos += 2 + for (;;) { + if (this.pos >= text.length) throw new ParseError('unterminated block comment') + if (text.charCodeAt(this.pos) === 42 && text.charCodeAt(this.pos + 1) === 47) { + this.pos += 2 + break + } + if (isLineTerminator(text.charCodeAt(this.pos))) newline = true + this.pos++ + } + this.recordComment(start, this.pos) + continue + } + } + break + } + return newline + } + + /** + * Advance past whitespace and comments WITHOUT producing a token, and return + * the resulting offset. JSX attribute values must be classified from the raw + * character (a multi-line `class="…"` is markup, not a JS string literal), so + * the caller has to look before it lexes. + */ + peekPosition(): number { + this.skipTrivia() + return this.pos + } + + /** Read the next token. `regexAllowed` decides `/` — regex or division. */ + next(regexAllowed: boolean): Token { + const newlineBefore = this.skipTrivia() + const text = this.text + const start = this.pos + if (start >= text.length) { + return { kind: 'eof', start, end: start, value: '', newlineBefore } + } + const code = text.charCodeAt(start) + + if (isIdStart(code)) { + if (code === 92 /* \ */) throw new ParseError(`unicode escape in identifier at ${start}`) + this.pos++ + while (this.pos < text.length && isIdPart(text.charCodeAt(this.pos))) this.pos++ + return { kind: 'identifier', start, end: this.pos, value: text.slice(start, this.pos), newlineBefore } + } + if (code === 92 /* \ */) throw new ParseError(`unicode escape in identifier at ${start}`) + + if (code === 35 /* # */) { + this.pos++ + if (!isIdStart(text.charCodeAt(this.pos))) throw new ParseError(`bad private name at ${start}`) + while (this.pos < text.length && isIdPart(text.charCodeAt(this.pos))) this.pos++ + return { kind: 'private', start, end: this.pos, value: text.slice(start, this.pos), newlineBefore } + } + + if ((code >= 48 && code <= 57) || (code === 46 && text.charCodeAt(start + 1) >= 48 && text.charCodeAt(start + 1) <= 57)) { + return { kind: 'number', start, end: this.readNumber(), value: '', newlineBefore } + } + + if (code === 34 || code === 39) { + return { kind: 'string', start, end: this.readString(code), value: '', newlineBefore } + } + + if (code === 96 /* ` */) { + this.pos++ + return { kind: 'template_start', start, end: this.pos, value: '`', newlineBefore } + } + + if (code === 47 /* / */ && regexAllowed) { + return { kind: 'regex', start, end: this.readRegex(), value: '', newlineBefore } + } + + for (const punct of PUNCTUATORS) { + if (text.startsWith(punct, start)) { + // `?.3` is a conditional followed by a number, not optional chaining. + if (punct === '?.' && text.charCodeAt(start + 2) >= 48 && text.charCodeAt(start + 2) <= 57) continue + this.pos = start + punct.length + return { kind: 'punct', start, end: this.pos, value: punct, newlineBefore } + } + } + + throw new ParseError(`unexpected character ${JSON.stringify(text[start])} at ${start}`) + } + + private readNumber(): number { + const text = this.text + let i = this.pos + if (text.charCodeAt(i) === 48 /* 0 */ && /[xXoObB]/.test(text[i + 1] ?? '')) { + // One digit class covers hex/octal/binary: the grammar of the radix is not + // this scanner's problem, only where the literal ends. + i += 2 + while (i < text.length && /[0-9a-fA-F_]/.test(text[i]!)) i++ + } else { + while (i < text.length && /[0-9_]/.test(text[i]!)) i++ + if (text[i] === '.') { + i++ + while (i < text.length && /[0-9_]/.test(text[i]!)) i++ + } + if (/[eE]/.test(text[i] ?? '')) { + i++ + if (/[+-]/.test(text[i] ?? '')) i++ + if (!/[0-9]/.test(text[i] ?? '')) throw new ParseError('bad exponent') + while (i < text.length && /[0-9_]/.test(text[i]!)) i++ + } + } + if (text[i] === 'n') i++ // bigint + if (i === this.pos) throw new ParseError(`bad numeric literal at ${i}`) + if (isIdStart(text.charCodeAt(i))) throw new ParseError(`numeric literal followed by identifier at ${i}`) + this.pos = i + return i + } + + private readString(quote: number): number { + const text = this.text + let i = this.pos + 1 + for (;;) { + if (i >= text.length) throw new ParseError(`unterminated string at ${i}`) + const code = text.charCodeAt(i) + if (code === 92 /* \ */) { + // A backslash-newline is a line continuation; anything else is one escape. + i += 2 + if (text.charCodeAt(i - 1) === 13 && text.charCodeAt(i) === 10) i++ + continue + } + if (code === quote) { + i++ + break + } + if (isLineTerminator(code)) throw new ParseError(`newline in string literal at ${i}`) + i++ + } + this.pos = i + return i + } + + private readRegex(): number { + const text = this.text + let i = this.pos + 1 + let inClass = false + for (;;) { + if (i >= text.length) throw new ParseError('unterminated regex') + const code = text.charCodeAt(i) + if (isLineTerminator(code)) throw new ParseError('newline in regex') + if (code === 92 /* \ */) { + i += 2 + continue + } + if (code === 91 /* [ */) inClass = true + else if (code === 93 /* ] */) inClass = false + else if (code === 47 /* / */ && !inClass) { + i++ + break + } + i++ + } + while (i < text.length && isIdPart(text.charCodeAt(i))) i++ + this.pos = i + return i + } + + /** + * Scan one raw chunk of a template literal starting at `from` (just past a + * backtick or a substitution's `}`). Returns where the literal text ends and + * whether a `${` substitution or the closing backtick terminated it. + */ + templateChunk(from: number): { fragmentEnd: number; kind: 'substitution' | 'end' } { + const text = this.text + let i = from + for (;;) { + if (i >= text.length) throw new ParseError('unterminated template literal') + const code = text.charCodeAt(i) + if (code === 92 /* \ */) { + i += 2 + continue + } + if (code === 96 /* ` */) return { fragmentEnd: i, kind: 'end' } + if (code === 36 /* $ */ && text.charCodeAt(i + 1) === 123 /* { */) { + return { fragmentEnd: i, kind: 'substitution' } + } + i++ + } + } +} diff --git a/packages/analyzer-engine/src/security/js-parse/node.ts b/packages/analyzer-engine/src/security/js-parse/node.ts new file mode 100644 index 0000000..3f2aec3 --- /dev/null +++ b/packages/analyzer-engine/src/security/js-parse/node.ts @@ -0,0 +1,138 @@ +import type { SyntaxNode } from '../lang' + +/** + * Tree-sitter-shaped nodes over a plain source string. + * + * Spans are offsets, not substrings: `text` slices on demand and row/column come + * from a line table by binary search. A 400 KB file therefore costs one string + * plus one Int32Array, not one substring per node. + */ + +export class Source { + readonly text: string + /** Offset of the first character of each line. */ + private readonly lineStarts: Int32Array + + constructor(text: string) { + this.text = text + const starts: number[] = [0] + for (let i = 0; i < text.length; i++) { + const c = text.charCodeAt(i) + // \n, \r (bare or CRLF), U+2028, U+2029 all start a new line. + if (c === 10) starts.push(i + 1) + else if (c === 13) { + if (text.charCodeAt(i + 1) === 10) i++ + starts.push(i + 1) + } else if (c === 0x2028 || c === 0x2029) starts.push(i + 1) + } + this.lineStarts = Int32Array.from(starts) + } + + /** 0-based row/column for an offset. */ + pointAt(offset: number): { row: number; column: number } { + const starts = this.lineStarts + let lo = 0 + let hi = starts.length - 1 + while (lo < hi) { + const mid = (lo + hi + 1) >> 1 + if (starts[mid] <= offset) lo = mid + else hi = mid - 1 + } + return { row: lo, column: offset - starts[lo] } + } + + /** 0-based row for an offset — the hot path (every finding location). */ + rowAt(offset: number): number { + return this.pointAt(offset).row + } +} + +export class JsNode implements SyntaxNode { + readonly type: string + readonly isNamed: boolean + readonly id: number + startIndex: number + endIndex: number + parent: JsNode | null = null + readonly children: JsNode[] = [] + private fieldMap: Map | null = null + private namedCache: JsNode[] | null = null + private readonly source: Source + + constructor(source: Source, type: string, isNamed: boolean, startIndex: number, endIndex: number, id: number) { + this.source = source + this.type = type + this.isNamed = isNamed + this.startIndex = startIndex + this.endIndex = endIndex + this.id = id + } + + get text(): string { + return this.source.text.slice(this.startIndex, this.endIndex) + } + + get startPosition(): { row: number; column: number } { + return this.source.pointAt(this.startIndex) + } + + get endPosition(): { row: number; column: number } { + return this.source.pointAt(this.endIndex) + } + + get childCount(): number { + return this.children.length + } + + get namedChildren(): JsNode[] { + if (!this.namedCache) this.namedCache = this.children.filter((child) => child.isNamed) + return this.namedCache + } + + get namedChildCount(): number { + return this.namedChildren.length + } + + get previousNamedSibling(): JsNode | null { + if (!this.parent) return null + const siblings = this.parent.namedChildren + const index = siblings.indexOf(this) + return index > 0 ? siblings[index - 1] : null + } + + get nextNamedSibling(): JsNode | null { + if (!this.parent) return null + const siblings = this.parent.namedChildren + const index = siblings.indexOf(this) + return index >= 0 && index + 1 < siblings.length ? siblings[index + 1] : null + } + + child(index: number): JsNode | null { + return this.children[index] ?? null + } + + childForFieldName(fieldName: string): JsNode | null { + return this.fieldMap?.get(fieldName) ?? null + } + + // ---- construction (parser-internal) ---- + + add(child: JsNode, field?: string): JsNode { + child.parent = this + this.children.push(child) + this.namedCache = null + if (field) { + if (!this.fieldMap) this.fieldMap = new Map() + // Tree-sitter keeps the FIRST child bound to a field when a rule repeats + // one (e.g. sequence_expression left/right); match that. + if (!this.fieldMap.has(field)) this.fieldMap.set(field, child) + } + return child + } + + /** Re-key an existing child under a field name (used when a node is reshaped). */ + setField(field: string, child: JsNode): void { + if (!this.fieldMap) this.fieldMap = new Map() + this.fieldMap.set(field, child) + } +} diff --git a/packages/analyzer-engine/src/security/js-parse/parser.ts b/packages/analyzer-engine/src/security/js-parse/parser.ts new file mode 100644 index 0000000..3e1bd24 --- /dev/null +++ b/packages/analyzer-engine/src/security/js-parse/parser.ts @@ -0,0 +1,2354 @@ +import { JsNode, Source } from './node' +import { Lexer, ParseError, type Token } from './lexer' + +/** + * A recursive-descent JavaScript/TypeScript/JSX parser that emits + * tree-sitter-grammar node names. + * + * WHY: the SAST rule pack is written against tree-sitter's JS/TS vocabulary + * (`expression_statement`, `catch_clause`, `for_in_statement`, `spread_element`, + * …). The WASM grammars that produce it are 6 MB — six times the CLI's entire + * release budget. Producing the *same vocabulary* from a ~40 KB parser lets the + * identical rules, taint solver and tests run locally, instead of a second, + * separately-drifting implementation of every rule. + * + * SOUNDNESS: this parser is strict where tree-sitter is error-tolerant. Any + * construct it cannot represent faithfully throws, the file is skipped, and the + * engine reports the language as degraded. Unsupported syntax therefore costs a + * finding we never make — never a finding we make wrongly. That asymmetry is the + * reason a hand-written parser can be trusted behind a zero-false-positive bar. + * + * Divergences from tree-sitter that are deliberate and rule-neutral: + * - `if_statement`'s `alternative` points straight at the else statement rather + * than through an `else_clause` wrapper. + * - TypeScript type syntax is captured as an opaque `type_annotation` / + * `type_arguments` span rather than a parsed type tree. The rules only ever + * read `type_annotation.text`. + */ + +const KEYWORD_LITERALS = new Set(['true', 'false', 'null']) +/** + * Words that can never be a binding name. Deliberately NOT the statement-starter + * list: `type`, `async`, `interface` and friends start declarations but are also + * ordinary identifiers, and treating them as reserved loses every + * `list.map(type => …)`. + */ +const RESERVED_WORDS = new Set([ + 'break', 'case', 'catch', 'class', 'const', 'continue', 'debugger', 'default', + 'delete', 'do', 'else', 'enum', 'export', 'extends', 'false', 'finally', 'for', + 'function', 'if', 'import', 'in', 'instanceof', 'new', 'null', 'return', + 'super', 'switch', 'this', 'throw', 'true', 'try', 'typeof', 'var', 'void', + 'while', 'with', +]) +/** Binary operators by precedence (higher binds tighter). */ +const BINARY_PRECEDENCE: Record = { + '??': 1, + '||': 2, + '&&': 3, + '|': 4, + '^': 5, + '&': 6, + '==': 7, '!=': 7, '===': 7, '!==': 7, + '<': 8, '>': 8, '<=': 8, '>=': 8, instanceof: 8, in: 8, + '<<': 9, '>>': 9, '>>>': 9, + '+': 10, '-': 10, + '*': 11, '/': 11, '%': 11, + '**': 12, +} + +const ASSIGN_OPS = new Set([ + '+=', '-=', '*=', '/=', '%=', '**=', '<<=', '>>=', '>>>=', '&=', '|=', '^=', '&&=', '||=', '??=', +]) + +const MODIFIER_KEYWORDS = new Set([ + 'public', 'private', 'protected', 'readonly', 'static', 'abstract', 'override', 'declare', 'accessor', +]) + +/** After one of these, more type syntax must follow. */ +const TYPE_OPERATOR_TOKENS = new Set([ + '|', '&', '=>', '<', ',', '?', ':', '(', '[', '{', '.', '-', '+', '...', + 'extends', 'keyof', 'typeof', 'readonly', 'infer', 'is', 'in', 'new', 'asserts', 'out', 'const', 'import', +]) +/** + * Words that may legally follow a COMPLETE type and continue it. + * + * `as` is deliberately absent: at depth 0 it is the expression-level operator in + * `x as unknown as Y`, and swallowing it would make the second cast disappear. + * Mapped-type `as` clauses only occur inside `[...]`, where depth > 0. + */ +const TYPE_CONTINUATION_WORDS = new Set(['extends', 'is', 'in']) + +export type JsDialect = 'javascript' | 'typescript' | 'tsx' + +interface SavedState { + pos: number + token: Token + prevEnd: number + /** + * Speculative parses (arrow-function lookahead, generic-vs-less-than) can + * descend into real sub-parses that attach comments. Rolling the comment + * watermark back with the lexer is what keeps a discarded attempt from + * consuming a comment the committed parse still needs — a swallowed comment + * turns `catch { /* intentional *\/ }` back into an empty catch, which is a + * false positive. + */ + commentFlushedTo: number +} + +export function parseJs(text: string, dialect: JsDialect): JsNode { + return new Parser(text, dialect).parseProgram() +} + +class Parser { + private readonly source: Source + private readonly lexer: Lexer + private readonly dialect: JsDialect + private readonly jsx: boolean + private token: Token + private nextId = 1 + private commentCursor = 0 + private commentFlushedTo = 0 + private commentCursorDirty = false + private depth = 0 + + constructor(text: string, dialect: JsDialect) { + this.source = new Source(text) + this.lexer = new Lexer(text) + this.dialect = dialect + this.jsx = dialect !== 'typescript' + this.token = this.lexer.next(false) + } + + // ---- node plumbing ------------------------------------------------------- + + private make(type: string, start: number, end: number, isNamed = true): JsNode { + return new JsNode(this.source, type, isNamed, start, end, this.nextId++) + } + + private open(type: string, start = this.token.start): JsNode { + return this.make(type, start, start) + } + + private close(node: JsNode, end = this.prevEnd): JsNode { + node.endIndex = end + return node + } + + private prevEnd = 0 + + // ---- token plumbing ------------------------------------------------------ + + private advance(): void { + this.prevEnd = this.token.end + this.token = this.lexer.next(false) + } + + private save(): SavedState { + return { + pos: this.lexer.pos, + token: this.token, + prevEnd: this.prevEnd, + commentFlushedTo: this.commentFlushedTo, + } + } + + private restore(state: SavedState): void { + this.lexer.pos = state.pos + this.token = state.token + this.prevEnd = state.prevEnd + if (state.commentFlushedTo !== this.commentFlushedTo) { + this.commentFlushedTo = state.commentFlushedTo + this.commentCursorDirty = true + } + } + + private is(value: string): boolean { + return this.token.kind === 'punct' && this.token.value === value + } + + private isWord(value: string): boolean { + return this.token.kind === 'identifier' && this.token.value === value + } + + /** Method form so control-flow narrowing of `this.token` cannot go stale + * across the many calls that advance it. */ + private atEof(): boolean { + return this.token.kind === 'eof' + } + + private atTemplate(): boolean { + return this.token.kind === 'template_start' + } + + /** Attach the current token to `parent` as an anonymous child and advance. */ + private take(parent: JsNode, field?: string): JsNode { + const node = this.make(this.token.value || this.token.kind, this.token.start, this.token.end, false) + parent.add(node, field) + this.advance() + return node + } + + private eat(value: string, parent: JsNode, field?: string): boolean { + if (!this.is(value)) return false + this.take(parent, field) + return true + } + + private eatWord(value: string, parent: JsNode, field?: string): boolean { + if (!this.isWord(value)) return false + this.take(parent, field) + return true + } + + private expect(value: string, parent: JsNode, field?: string): void { + if (!this.is(value)) throw new ParseError(`expected "${value}" at ${this.token.start}`) + this.take(parent, field) + } + + private expectWord(value: string, parent: JsNode, field?: string): void { + if (!this.isWord(value)) throw new ParseError(`expected "${value}" at ${this.token.start}`) + this.take(parent, field) + } + + /** Re-read the current token allowing a regex literal (operand position). */ + private relexAsRegex(): void { + this.lexer.pos = this.token.start + this.token = this.lexer.next(true) + } + + private enter(): void { + if (++this.depth > 400) throw new ParseError('expression nesting too deep') + } + + private exit(): void { + this.depth-- + } + + // ---- comments ------------------------------------------------------------ + + /** + * Attach every comment discovered before `until` as a named `comment` child. + * + * Comment placement is load-bearing: `catch { /* ignore *\/ }` must not read as + * an empty catch, and a comment above a `try` documents intent. Attaching a + * comment can only ever SUPPRESS a finding, so an imprecise placement costs + * recall, not precision. + */ + private flushComments(parent: JsNode, until = this.token.start): void { + const list = this.lexer.commentList + if (this.lexer.commentsUnsorted) { + list.sort((a, b) => a.start - b.start) + this.lexer.commentsUnsorted = false + this.commentCursorDirty = true + } + if (this.commentCursorDirty) { + this.commentCursor = 0 + while (this.commentCursor < list.length && list[this.commentCursor].start < this.commentFlushedTo) { + this.commentCursor++ + } + this.commentCursorDirty = false + } + while (this.commentCursor < list.length && list[this.commentCursor].start < until) { + const span = list[this.commentCursor++] + if (span.start >= this.commentFlushedTo) parent.add(this.make('comment', span.start, span.end)) + } + if (until > this.commentFlushedTo) this.commentFlushedTo = until + } + + // ---- program ------------------------------------------------------------- + + parseProgram(): JsNode { + const program = this.open('program', 0) + while (!this.atEof()) { + this.flushComments(program) + if (this.atEof()) break + program.add(this.parseStatement()) + } + this.flushComments(program, this.source.text.length) + return this.close(program, this.source.text.length) + } + + private semicolon(parent: JsNode): void { + if (this.eat(';', parent)) return + // Automatic semicolon insertion, restricted to the three legal cases. Any + // other continuation is a parse we do not understand — fail closed. + if (this.token.kind === 'eof' || this.is('}') || this.token.newlineBefore) return + throw new ParseError(`expected ";" at ${this.token.start}`) + } + + // ---- statements ---------------------------------------------------------- + + private parseStatement(): JsNode { + this.enter() + try { + return this.parseStatementInner() + } finally { + this.exit() + } + } + + private parseStatementInner(): JsNode { + if (this.is('{')) return this.parseBlock() + if (this.is(';')) { + const node = this.open('empty_statement') + this.take(node) + return this.close(node) + } + if (this.is('@')) return this.parseDecorated() + + if (this.token.kind === 'identifier') { + switch (this.token.value) { + case 'var': + case 'let': + case 'const': + // `let` is only a declaration when a binding follows. + if (this.token.value !== 'let' || this.letStartsDeclaration()) return this.parseVariableDeclaration() + break + case 'function': + return this.parseFunctionDeclaration(this.open('function_declaration')) + case 'class': + return this.parseClass('class_declaration') + case 'if': + return this.parseIf() + case 'for': + return this.parseFor() + case 'while': + return this.parseWhile() + case 'do': + return this.parseDoWhile() + case 'try': + return this.parseTry() + case 'switch': + return this.parseSwitch() + case 'return': + case 'throw': + return this.parseReturnLike(this.token.value === 'return' ? 'return_statement' : 'throw_statement') + case 'break': + case 'continue': + return this.parseBreakLike(this.token.value === 'break' ? 'break_statement' : 'continue_statement') + case 'debugger': { + const node = this.open('debugger_statement') + this.take(node) + this.semicolon(node) + return this.close(node) + } + case 'import': + if (!this.importIsExpression()) return this.parseImport() + break + case 'export': + return this.parseExport() + case 'async': + if (this.asyncStartsFunction()) { + const node = this.open('function_declaration') + this.take(node) + return this.parseFunctionDeclaration(node) + } + break + default: + break + } + const typeDecl = this.tryParseTypeDeclaration() + if (typeDecl) return typeDecl + if (this.isLabel()) return this.parseLabeled() + } + + const node = this.open('expression_statement') + node.add(this.parseExpression()) + this.semicolon(node) + return this.close(node) + } + + private parseDecorated(): JsNode { + // Decorators bind to the following class or member; keep them as siblings of + // the declaration so statement shape (and therefore rule anchors) is stable. + const node = this.open('decorator') + this.take(node) + node.add(this.parseLeftHandSide(this.parsePrimary())) + this.close(node) + const statement = this.parseStatement() + const wrapper = this.open('decorated_statement', node.startIndex) + wrapper.add(node) + wrapper.add(statement) + return this.close(wrapper) + } + + private letStartsDeclaration(): boolean { + const state = this.save() + this.advance() + const ok = + this.token.kind === 'identifier' || + this.is('[') || + this.is('{') + this.restore(state) + return ok + } + + private importIsExpression(): boolean { + const state = this.save() + this.advance() + const ok = this.is('(') || this.is('.') + this.restore(state) + return ok + } + + private asyncStartsFunction(): boolean { + const state = this.save() + this.advance() + const ok = this.isWord('function') && !this.token.newlineBefore + this.restore(state) + return ok + } + + private isLabel(): boolean { + if (RESERVED_WORDS.has(this.token.value)) return false + const state = this.save() + this.advance() + const ok = this.is(':') + this.restore(state) + return ok + } + + private parseLabeled(): JsNode { + const node = this.open('labeled_statement') + const label = this.open('statement_identifier') + this.advance() + node.add(this.close(label, this.prevEnd), 'label') + this.expect(':', node) + node.add(this.parseStatement(), 'body') + return this.close(node) + } + + private parseBlock(): JsNode { + const node = this.open('statement_block') + this.expect('{', node) + while (!this.is('}')) { + this.flushComments(node) + if (this.is('}')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated block') + node.add(this.parseStatement()) + } + this.flushComments(node) + this.expect('}', node) + return this.close(node) + } + + private parseVariableDeclaration(): JsNode { + const kind = this.token.value + const node = this.open(kind === 'var' ? 'variable_declaration' : 'lexical_declaration') + this.take(node) + for (;;) { + node.add(this.parseVariableDeclarator()) + if (!this.eat(',', node)) break + } + this.semicolon(node) + return this.close(node) + } + + private parseVariableDeclarator(): JsNode { + const node = this.open('variable_declarator') + node.add(this.parseBindingTarget(), 'name') + this.eat('!', node) + this.parseOptionalTypeAnnotation(node) + if (this.eat('=', node)) node.add(this.parseAssignment(), 'value') + return this.close(node) + } + + private parseIf(): JsNode { + const node = this.open('if_statement') + this.take(node) + node.add(this.parseParenthesized(), 'condition') + node.add(this.parseStatement(), 'consequence') + if (this.isWord('else')) { + this.take(node) + node.add(this.parseStatement(), 'alternative') + } + return this.close(node) + } + + private parseParenthesized(): JsNode { + const node = this.open('parenthesized_expression') + this.expect('(', node) + node.add(this.parseExpression()) + this.expect(')', node) + return this.close(node) + } + + private parseWhile(): JsNode { + const node = this.open('while_statement') + this.take(node) + node.add(this.parseParenthesized(), 'condition') + node.add(this.parseStatement(), 'body') + return this.close(node) + } + + private parseDoWhile(): JsNode { + const node = this.open('do_statement') + this.take(node) + node.add(this.parseStatement(), 'body') + this.expectWord('while', node) + node.add(this.parseParenthesized(), 'condition') + this.eat(';', node) + return this.close(node) + } + + private parseFor(): JsNode { + const start = this.token.start + const head = this.open('for_statement', start) + this.take(head) // for + const isAwait = this.eatWord('await', head) + this.expect('(', head) + + // Distinguish `for (x of y)` / `for (x in y)` from the three-clause form by + // scanning the head; `in` inside the initializer of a C-style for is only + // reachable through parentheses, which the scan tracks. + const kind = this.scanForKind() + if (kind === 'in' || kind === 'of') { + const node = this.open('for_in_statement', start) + for (const child of head.children) node.add(child) + if (this.isWord('var') || this.isWord('const') || (this.isWord('let') && this.letStartsDeclaration())) { + this.take(node) + } + node.add(this.parseBindingTarget(), 'left') + if (!this.isWord('in') && !this.isWord('of')) throw new ParseError(`expected in/of at ${this.token.start}`) + this.take(node, 'operator') + node.add(kind === 'of' ? this.parseAssignment() : this.parseExpression(), 'right') + this.expect(')', node) + node.add(this.parseStatement(), 'body') + return this.close(node) + } + if (isAwait) throw new ParseError('for await requires of') + + if (!this.is(';')) { + if (this.isWord('var') || this.isWord('const') || (this.isWord('let') && this.letStartsDeclaration())) { + const decl = this.open(this.token.value === 'var' ? 'variable_declaration' : 'lexical_declaration') + this.take(decl) + for (;;) { + decl.add(this.parseVariableDeclarator()) + if (!this.eat(',', decl)) break + } + head.add(this.close(decl), 'initializer') + } else { + head.add(this.parseExpression(), 'initializer') + } + } + this.expect(';', head) + if (!this.is(';')) head.add(this.parseExpression(), 'condition') + this.expect(';', head) + if (!this.is(')')) head.add(this.parseExpression(), 'increment') + this.expect(')', head) + head.add(this.parseStatement(), 'body') + return this.close(head) + } + + /** Look ahead over a `for` head to classify it, without consuming anything. */ + private scanForKind(): 'in' | 'of' | 'classic' { + const state = this.save() + let depth = 0 + try { + for (let i = 0; i < 5000; i++) { + if (this.token.kind === 'eof') break + if (this.is('(') || this.is('[') || this.is('{')) depth++ + else if (this.is(')') || this.is(']') || this.is('}')) { + if (depth === 0) break + depth-- + } else if (depth === 0) { + if (this.is(';')) return 'classic' + if (this.isWord('of')) return 'of' + if (this.isWord('in')) return 'in' + } + this.advance() + } + return 'classic' + } finally { + this.restore(state) + } + } + + private parseTry(): JsNode { + const node = this.open('try_statement') + this.take(node) + node.add(this.parseBlock(), 'body') + if (this.isWord('catch')) { + const clause = this.open('catch_clause') + this.take(clause) + if (this.is('(')) { + this.expect('(', clause) + clause.add(this.parseBindingTarget(), 'parameter') + this.parseOptionalTypeAnnotation(clause) + this.expect(')', clause) + } + clause.add(this.parseBlock(), 'body') + node.add(this.close(clause), 'handler') + } + if (this.isWord('finally')) { + const clause = this.open('finally_clause') + this.take(clause) + clause.add(this.parseBlock(), 'body') + node.add(this.close(clause), 'finalizer') + } + return this.close(node) + } + + private parseSwitch(): JsNode { + const node = this.open('switch_statement') + this.take(node) + node.add(this.parseParenthesized(), 'value') + const body = this.open('switch_body') + this.expect('{', body) + while (!this.is('}')) { + this.flushComments(body) + if (this.is('}')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated switch') + if (this.isWord('case')) { + const clause = this.open('switch_case') + this.take(clause) + clause.add(this.parseExpression(), 'value') + this.expect(':', clause) + this.parseCaseBody(clause) + body.add(this.close(clause)) + } else if (this.isWord('default')) { + const clause = this.open('switch_default') + this.take(clause) + this.expect(':', clause) + this.parseCaseBody(clause) + body.add(this.close(clause)) + } else { + throw new ParseError(`expected case/default at ${this.token.start}`) + } + } + this.flushComments(body) + this.expect('}', body) + node.add(this.close(body), 'body') + return this.close(node) + } + + private parseCaseBody(clause: JsNode): void { + while (!this.is('}') && !this.isWord('case') && !this.isWord('default')) { + this.flushComments(clause) + if (this.is('}') || this.isWord('case') || this.isWord('default')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated switch case') + clause.add(this.parseStatement(), 'body') + } + } + + private parseReturnLike(type: string): JsNode { + const node = this.open(type) + this.take(node) + if (!this.is(';') && !this.is('}') && this.token.kind !== 'eof' && !this.token.newlineBefore) { + node.add(this.parseExpression()) + } + this.semicolon(node) + return this.close(node) + } + + private parseBreakLike(type: string): JsNode { + const node = this.open(type) + this.take(node) + if (this.token.kind === 'identifier' && !this.token.newlineBefore && !this.is(';')) { + const label = this.open('statement_identifier') + this.advance() + node.add(this.close(label, this.prevEnd), 'label') + } + this.semicolon(node) + return this.close(node) + } + + // ---- modules ------------------------------------------------------------- + + private parseImport(): JsNode { + const node = this.open('import_statement') + this.take(node) + this.skipBalancedUntilStatementEnd(node) + return this.close(node) + } + + private parseExport(): JsNode { + const node = this.open('export_statement') + this.take(node) + // `export = X` — the CommonJS-interop form in TypeScript declarations. + if (this.is('=')) { + this.skipBalancedUntilStatementEnd(node) + return this.close(node) + } + if (this.eatWord('default', node)) { + if (this.isWord('function') || this.isWord('class') || this.asyncStartsFunction()) { + node.add(this.parseStatement(), 'declaration') + } else { + node.add(this.parseAssignment(), 'value') + this.semicolon(node) + } + return this.close(node) + } + if (this.is('{') || this.is('*')) { + this.skipBalancedUntilStatementEnd(node) + return this.close(node) + } + // `export type { A } from '…'` is a re-export clause, not a type alias. + if (this.isWord('type') && this.typeReexportAhead()) { + this.take(node) + this.skipBalancedUntilStatementEnd(node) + return this.close(node) + } + if (this.isWord('type') || this.isWord('interface') || this.isWord('enum') || this.isWord('declare')) { + const decl = this.tryParseTypeDeclaration() + if (decl) { + node.add(decl, 'declaration') + return this.close(node) + } + } + node.add(this.parseStatement(), 'declaration') + return this.close(node) + } + + private typeReexportAhead(): boolean { + const state = this.save() + this.advance() + const ahead = this.is('{') || this.is('*') + this.restore(state) + return ahead + } + + /** Consume the remainder of an import/export clause (never contains code). */ + private skipBalancedUntilStatementEnd(node: JsNode): void { + let depth = 0 + for (let i = 0; i < 20_000; i++) { + if (this.token.kind === 'eof') return + if (depth === 0 && this.is(';')) { + this.take(node) + return + } + if (depth === 0 && this.token.newlineBefore && i > 0) return + if (this.is('{') || this.is('(') || this.is('[')) depth++ + else if (this.is('}') || this.is(')') || this.is(']')) { + if (depth === 0) return + depth-- + } + this.take(node) + } + throw new ParseError('module clause too long') + } + + // ---- TypeScript declarations -------------------------------------------- + + /** `interface` / `type` / `enum` / `namespace` / `declare` / `abstract class`. */ + private tryParseTypeDeclaration(): JsNode | null { + if (this.dialect === 'javascript') return null + const word = this.token.value + if (word === 'abstract') { + const state = this.save() + this.advance() + if (this.isWord('class')) { + this.restore(state) + const node = this.open('abstract_class_declaration') + this.take(node) + return this.parseClassInto(node) + } + this.restore(state) + return null + } + if (word === 'declare') { + const state = this.save() + this.advance() + const followsDeclaration = this.token.kind === 'identifier' && !this.token.newlineBefore + this.restore(state) + if (!followsDeclaration) return null + const node = this.open('ambient_declaration') + this.take(node) + this.skipDeclarationBody(node) + return this.close(node) + } + if (word === 'interface' || word === 'enum') { + const state = this.save() + this.advance() + if (this.token.kind !== 'identifier') { + this.restore(state) + return null + } + this.restore(state) + const node = this.open(word === 'interface' ? 'interface_declaration' : 'enum_declaration') + this.take(node) + this.skipDeclarationBody(node, true) + return this.close(node) + } + if (word === 'type') { + const state = this.save() + this.advance() + if (this.token.kind !== 'identifier' || this.token.newlineBefore) { + this.restore(state) + return null + } + this.advance() + const isAlias = this.is('=') || this.is('<') + this.restore(state) + if (!isAlias) return null + const node = this.open('type_alias_declaration') + this.take(node) // type + node.add(this.parseIdentifier('type_identifier'), 'name') + this.parseOptionalTypeParameters(node) + this.expect('=', node) + this.skipTypeExpression(node, false) + this.eat(';', node) + return this.close(node) + } + if (word === 'namespace' || word === 'module') { + const state = this.save() + this.advance() + const named = this.token.kind === 'identifier' || this.token.kind === 'string' + this.restore(state) + if (!named) return null + const node = this.open('internal_module') + this.take(node) + while (!this.is('{') && this.token.kind !== 'eof') this.take(node) + node.add(this.parseBlock(), 'body') + return this.close(node) + } + return null + } + + /** + * Consume a declaration whose body carries no executable code + * (`interface`, `enum`, `declare …`). + * + * The header before the brace can itself nest brackets — `interface A extends + * B<{ x: 1 }>` — so brace counting alone stops in the wrong place. This finds + * the block by scanning the header with full bracket tracking, then consumes + * exactly one balanced `{ … }`. A declaration with no block (`declare const x: + * T`) ends at its semicolon or line break. + */ + private skipDeclarationBody(node: JsNode, requireBlock = false): void { + let depth = 0 + let angle = 0 + let sawBlock = false + for (let i = 0; i < 200_000; i++) { + if (this.atEof()) return + if (this.is('{')) { + depth++ + // A brace inside `<…>` belongs to a type argument + // (`ApiFromModules<{ … }>`), not to the declaration's own block. + if (angle === 0 && depth === 1) sawBlock = true + } else if (this.is('(') || this.is('[')) depth++ + else if (this.is('}') || this.is(')') || this.is(']')) { + depth-- + if (depth < 0) return + this.take(node) + if (depth === 0 && angle === 0 && sawBlock) return + continue + } else if (this.is('<')) angle++ + else if (angle > 0 && /^>+=?$/.test(this.token.value)) { + angle -= (this.token.value.match(/>/g) ?? []).length + if (angle < 0) angle = 0 + } else if (depth === 0 && angle === 0 && !sawBlock) { + if (this.is(';')) { + this.take(node) + return + } + // `interface`/`enum` always end in a block, so a line break inside the + // header (`interface X\n extends Y`) is not the end of the declaration. + if (!requireBlock && this.token.newlineBefore && i > 0) return + } + if (this.atTemplate()) { + this.skipTemplateRaw(node) + continue + } + this.take(node) + } + throw new ParseError('declaration too long') + } + + // ---- functions & classes ------------------------------------------------- + + private parseFunctionDeclaration(node: JsNode): JsNode { + this.expectWord('function', node) + if (this.eat('*', node)) { + // generator_function_declaration is a distinct grammar node. + const generator = this.make('generator_function_declaration', node.startIndex, node.endIndex) + for (const child of node.children) generator.add(child) + node = generator + } + if (this.token.kind === 'identifier') node.add(this.parseIdentifier('identifier'), 'name') + this.parseOptionalTypeParameters(node) + node.add(this.parseFormalParameters(), 'parameters') + this.parseOptionalTypeAnnotation(node) + if (this.is('{')) node.add(this.parseBlock(), 'body') + else this.semicolon(node) // TypeScript overload signature + return this.close(node) + } + + private parseClass(type: string): JsNode { + return this.parseClassInto(this.open(type)) + } + + private parseClassInto(node: JsNode): JsNode { + this.expectWord('class', node) + if (this.token.kind === 'identifier' && !this.isWord('extends') && !this.isWord('implements')) { + node.add(this.parseIdentifier('type_identifier'), 'name') + } + this.parseOptionalTypeParameters(node) + while (this.isWord('extends') || this.isWord('implements')) { + const heritage = this.open('class_heritage') + this.take(heritage) + for (;;) { + heritage.add(this.parseLeftHandSide(this.parsePrimary())) + // `implements Promise` — type arguments here are never call generics, + // so they are consumed unconditionally. + if (this.is('<') && this.dialect !== 'javascript') { + const typeArguments = this.open('type_arguments') + this.skipAngleBracketed(typeArguments) + heritage.add(this.close(typeArguments)) + } + if (!this.eat(',', heritage)) break + } + node.add(this.close(heritage)) + } + node.add(this.parseClassBody(), 'body') + return this.close(node) + } + + private parseClassBody(): JsNode { + const body = this.open('class_body') + this.expect('{', body) + while (!this.is('}')) { + this.flushComments(body) + if (this.is('}')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated class body') + if (this.eat(';', body)) continue + body.add(this.parseClassMember()) + } + this.flushComments(body) + this.expect('}', body) + return this.close(body) + } + + private parseClassMember(): JsNode { + const start = this.token.start + const scratch = this.open('class_member', start) + while (this.is('@')) { + const decorator = this.open('decorator') + this.take(decorator) + decorator.add(this.parseLeftHandSide(this.parsePrimary())) + scratch.add(this.close(decorator)) + } + // Modifiers are only modifiers when another member token follows; `static` + // and friends are also legal member names. + for (let i = 0; i < 8; i++) { + if (this.token.kind !== 'identifier' || !MODIFIER_KEYWORDS.has(this.token.value)) break + const state = this.save() + this.advance() + const isModifier = !this.is('(') && !this.is('=') && !this.is(';') && !this.is(':') && !this.is('?') && !this.is('<') && !this.is('}') + this.restore(state) + if (!isModifier) break + this.take(scratch) + } + // `static { … }` — a class static initialization block. + if (this.is('{')) { + const block = this.open('class_static_block', start) + for (const child of scratch.children) block.add(child) + block.add(this.parseBlock(), 'body') + return this.close(block) + } + if (this.is('[') && this.indexSignatureAhead()) { + const signature = this.open('index_signature', start) + for (const child of scratch.children) signature.add(child) + this.skipDeclarationBody(signature) + return this.close(signature) + } + + let isAsync = false + let isGenerator = false + let accessor: 'get' | 'set' | null = null + if (this.isWord('async')) { + const state = this.save() + this.advance() + if (!this.is('(') && !this.is('=') && !this.is(';') && !this.is(':') && !this.token.newlineBefore) { + this.restore(state) + this.take(scratch) + isAsync = true + } else this.restore(state) + } + if (this.is('*')) { + this.take(scratch) + isGenerator = true + } + if (this.isWord('get') || this.isWord('set')) { + const word = this.token.value as 'get' | 'set' + const state = this.save() + this.advance() + if (!this.is('(') && !this.is('=') && !this.is(';') && !this.is(':') && !this.is('}')) { + this.restore(state) + this.take(scratch) + accessor = word + } else this.restore(state) + } + void isAsync + void isGenerator + void accessor + + const name = this.parsePropertyName() + const optional = this.is('?') || this.is('!') + if (optional) this.take(scratch) + + if (this.is('(') || this.is('<')) { + const method = this.open('method_definition', start) + for (const child of scratch.children) method.add(child) + method.add(name, 'name') + this.parseOptionalTypeParameters(method) + method.add(this.parseFormalParameters(), 'parameters') + this.parseOptionalTypeAnnotation(method) + if (this.is('{')) method.add(this.parseBlock(), 'body') + else this.semicolon(method) + return this.close(method) + } + + const field = this.open('field_definition', start) + for (const child of scratch.children) field.add(child) + field.add(name, 'property') + this.parseOptionalTypeAnnotation(field) + if (this.eat('=', field)) field.add(this.parseAssignment(), 'value') + this.semicolon(field) + return this.close(field) + } + + private indexSignatureAhead(): boolean { + if (this.dialect === 'javascript') return false + const state = this.save() + try { + this.advance() + if (this.token.kind !== 'identifier') return false + this.advance() + return this.is(':') + } finally { + this.restore(state) + } + } + + private parsePropertyName(): JsNode { + if (this.is('[')) { + const node = this.open('computed_property_name') + this.take(node) + node.add(this.parseAssignment()) + this.expect(']', node) + return this.close(node) + } + if (this.token.kind === 'string') return this.parseStringLiteral() + if (this.token.kind === 'number') { + const node = this.open('number') + this.advance() + return this.close(node, this.prevEnd) + } + if (this.token.kind === 'private') { + const node = this.open('private_property_identifier') + this.advance() + return this.close(node, this.prevEnd) + } + if (this.token.kind !== 'identifier') throw new ParseError(`expected property name at ${this.token.start}`) + return this.parseIdentifier('property_identifier') + } + + private parseIdentifier(type: string): JsNode { + if (this.token.kind !== 'identifier') throw new ParseError(`expected identifier at ${this.token.start}`) + const node = this.open(type) + this.advance() + return this.close(node, this.prevEnd) + } + + private parseFormalParameters(): JsNode { + const node = this.open('formal_parameters') + this.expect('(', node) + while (!this.is(')')) { + this.flushComments(node) + if (this.is(')')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated parameter list') + node.add(this.parseParameter()) + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect(')', node) + return this.close(node) + } + + /** + * One parameter. TypeScript files wrap the binding in + * `required_parameter`/`optional_parameter` with `pattern` and `type` fields — + * exactly what `open-record-write` reads to see an `any`/`Record` annotation. Plain JavaScript has no annotations, so the binding + * pattern stands alone, matching tree-sitter-javascript. + */ + private parseParameter(): JsNode { + const start = this.token.start + const scratch = this.open('parameter', start) + while (this.is('@')) { + const decorator = this.open('decorator') + this.take(decorator) + decorator.add(this.parseLeftHandSide(this.parsePrimary())) + scratch.add(this.close(decorator)) + } + let hasModifier = false + for (let i = 0; i < 4; i++) { + if (this.token.kind !== 'identifier' || !MODIFIER_KEYWORDS.has(this.token.value)) break + const state = this.save() + this.advance() + const isModifier = this.token.kind === 'identifier' || this.is('{') || this.is('[') || this.is('...') + this.restore(state) + if (!isModifier) break + this.take(scratch) + hasModifier = true + } + + const rest = this.is('...') + const pattern = this.parseBindingTarget() + const optional = this.is('?') + const typed = optional || this.is(':') + if (optional) this.take(scratch) + + if (this.dialect === 'javascript' || (!typed && !hasModifier && scratch.children.length === 0)) { + // Plain binding: default values become assignment_pattern, matching + // tree-sitter-javascript's `formal_parameters` children. + if (this.is('=')) { + const node = this.open('assignment_pattern', start) + node.add(pattern, 'left') + this.expect('=', node) + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + return pattern + } + + const node = this.open(optional ? 'optional_parameter' : 'required_parameter', start) + for (const child of scratch.children) node.add(child) + node.add(pattern, 'pattern') + void rest + this.parseOptionalTypeAnnotation(node) + if (this.eat('=', node)) node.add(this.parseAssignment(), 'value') + return this.close(node) + } + + /** A binding target: identifier, object/array pattern, or rest. */ + private parseBindingTarget(): JsNode { + if (this.is('...')) { + const node = this.open('rest_pattern') + this.take(node) + node.add(this.parseBindingTarget()) + return this.close(node) + } + if (this.is('{')) return this.parseObjectPattern() + if (this.is('[')) return this.parseArrayPattern() + if (this.isWord('this')) { + const node = this.open('this') + this.advance() + return this.parseBindingSuffixes(this.close(node, this.prevEnd)) + } + return this.parseBindingSuffixes(this.parseIdentifier('identifier')) + } + + /** + * Member and index suffixes on a binding target. + * + * Destructuring assignment targets are not restricted to plain names — + * `[node.leadingComments, last] = f()` is ordinary code — so a target may be a + * member chain. Declarations never produce one (`.`/`[` cannot follow a + * declared name), so this costs nothing where it does not apply. + */ + private parseBindingSuffixes(target: JsNode): JsNode { + let expression = target + for (let i = 0; i < 64; i++) { + if (this.is('.')) { + const node = this.open('member_expression', expression.startIndex) + node.add(expression, 'object') + this.take(node) + node.add(this.parseIdentifier('property_identifier'), 'property') + expression = this.close(node) + continue + } + if (this.is('[')) { + const node = this.open('subscript_expression', expression.startIndex) + node.add(expression, 'object') + this.take(node) + node.add(this.parseExpression(), 'index') + this.expect(']', node) + expression = this.close(node) + continue + } + break + } + return expression + } + + private parseObjectPattern(): JsNode { + const node = this.open('object_pattern') + this.expect('{', node) + while (!this.is('}')) { + this.flushComments(node) + if (this.is('}')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated object pattern') + node.add(this.parseObjectPatternProperty()) + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect('}', node) + return this.close(node) + } + + private parseObjectPatternProperty(): JsNode { + if (this.is('...')) { + const node = this.open('rest_pattern') + this.take(node) + node.add(this.parseBindingTarget()) + return this.close(node) + } + const start = this.token.start + const computed = this.is('[') + const key = this.parsePropertyName() + if (this.is(':')) { + const node = this.open('pair_pattern', start) + node.add(key, 'key') + this.expect(':', node) + node.add(this.parseBindingTargetWithDefault(), 'value') + return this.close(node) + } + if (computed) throw new ParseError('computed key requires a binding') + // `{ a }` / `{ a = 1 }` — the shorthand forms. + const shorthand = this.make('shorthand_property_identifier_pattern', key.startIndex, key.endIndex) + if (this.is('=')) { + const node = this.open('object_assignment_pattern', start) + node.add(shorthand, 'left') + this.expect('=', node) + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + return shorthand + } + + private parseBindingTargetWithDefault(): JsNode { + const start = this.token.start + const target = this.parseBindingTarget() + if (!this.is('=')) return target + const node = this.open('assignment_pattern', start) + node.add(target, 'left') + this.expect('=', node) + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + + private parseArrayPattern(): JsNode { + const node = this.open('array_pattern') + this.expect('[', node) + while (!this.is(']')) { + this.flushComments(node) + if (this.is(']')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated array pattern') + if (this.is(',')) { + this.take(node) // elision + continue + } + node.add(this.parseBindingTargetWithDefault()) + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect(']', node) + return this.close(node) + } + + // ---- TypeScript type syntax --------------------------------------------- + + /** `: T` after a binding, parameter, or signature. Opaque by design. */ + private parseOptionalTypeAnnotation(parent: JsNode, stopAtArrow = false): void { + if (this.dialect === 'javascript' || !this.is(':')) return + const node = this.open('type_annotation') + this.take(node) + this.skipTypeExpression(node, stopAtArrow) + parent.add(this.close(node), 'type') + } + + private parseOptionalTypeParameters(parent: JsNode): void { + if (this.dialect === 'javascript' || !this.is('<')) return + const node = this.open('type_parameters') + this.skipAngleBracketed(node) + parent.add(this.close(node), 'type_parameters') + } + + /** + * Consume a type expression as an opaque span. + * + * The rules never inspect a parsed type — `open-record-write` reads + * `type_annotation.text` and nothing else — so parsing TypeScript's full type + * grammar would be cost without a consumer. What must be exact is where the + * type ENDS, which is what this tracks: bracket depth, `?:` conditional-type + * pairing, and (for arrow return types) the `=>` that starts the body. + */ + private skipTypeExpression(node: JsNode, stopAtArrow: boolean): void { + let depth = 0 + let angle = 0 + let conditional = 0 + // A type ends where an operand would start a *new* construct. Tracking + // "does the syntax so far still demand more type?" is what separates + // `function f(): Foo {` (body) from `function f(): { a: X } {` (object type), + // and `let x: Foo` + newline + `bar()` from a continuation. Getting this + // wrong swallows executable code into a type span, so it is deliberately + // conservative: when in doubt, stop. + let expectingType = true + for (let i = 0; i < 200_000; i++) { + if (this.token.kind === 'eof') return + const value = this.token.value + if (depth === 0 && angle === 0) { + if (this.is(')') || this.is(']') || this.is(',') || this.is(';')) return + if (this.is('=') || ASSIGN_OPS.has(value)) return + if (stopAtArrow && this.is('=>')) return + if (!expectingType) { + if (this.is('{') || this.is('(')) return + if (this.token.kind === 'number' || this.token.kind === 'string' || this.token.kind === 'template_start') return + if (this.token.kind === 'identifier' && !TYPE_CONTINUATION_WORDS.has(value)) return + } + if (this.is('}')) return + if (this.is('?')) conditional++ + else if (this.is(':')) { + if (conditional === 0) return + conditional-- + } + } + if (this.is('(') || this.is('[') || this.is('{')) depth++ + else if (this.is(')') || this.is(']') || this.is('}')) depth-- + else if (this.is('<')) angle++ + else if (angle > 0 && /^>+=?$/.test(value)) { + angle -= (value.match(/>/g) ?? []).length + if (angle < 0) angle = 0 + } + if (this.atTemplate()) { + this.skipTemplateRaw(node) + expectingType = false + continue + } + expectingType = TYPE_OPERATOR_TOKENS.has(value) + this.take(node) + } + throw new ParseError('type expression too long') + } + + /** + * Consume a whole template literal at the character level. + * + * Used wherever the surrounding syntax is a TYPE: a template literal *type* + * (`` `${infer A} ${infer B}` ``) holds type syntax in its substitutions, so + * parsing them as expressions fails on perfectly valid code. Types have no + * consumer in the rule pack, so the literal only needs to be spanned, not + * understood. + */ + private skipTemplateRaw(node: JsNode): void { + const text = this.lexer.text + const start = this.token.start + type Frame = { kind: 'literal' } | { kind: 'substitution'; depth: number } + const stack: Frame[] = [{ kind: 'literal' }] + let i = start + 1 + while (stack.length) { + if (i >= text.length) throw new ParseError('unterminated template literal') + const top = stack[stack.length - 1] + const char = text[i] + if (char === '\\') { + i += 2 + continue + } + if (top.kind === 'literal') { + if (char === '`') { + stack.pop() + i++ + } else if (char === '$' && text[i + 1] === '{') { + stack.push({ kind: 'substitution', depth: 0 }) + i += 2 + } else i++ + continue + } + if (char === '`') { + stack.push({ kind: 'literal' }) + i++ + } else if (char === '"' || char === "'") { + const closing = text.indexOf(char, i + 1) + if (closing === -1) throw new ParseError('unterminated string in template type') + i = closing + 1 + } else if (char === '{') { + top.depth++ + i++ + } else if (char === '}') { + if (top.depth === 0) stack.pop() + else top.depth-- + i++ + } else i++ + } + node.add(this.make('template_string', start, i)) + this.lexer.pos = i + this.prevEnd = i + this.token = this.lexer.next(false) + } + + /** Consume a balanced `<...>` run, tolerating `>>` closing two levels. */ + private skipAngleBracketed(node: JsNode): void { + let angle = 0 + for (let i = 0; i < 20_000; i++) { + if (this.token.kind === 'eof') throw new ParseError('unterminated type arguments') + if (this.is('<')) angle++ + else if (this.is('>') || this.is('>>') || this.is('>>>')) { + angle -= this.token.value.length + } else if (this.is('>=') || this.is('>>=') || this.is('>>>=')) { + angle -= this.token.value.length - 1 + } + if (this.atTemplate()) { + this.skipTemplateRaw(node) + continue + } + this.take(node) + if (angle <= 0) return + } + throw new ParseError('type arguments too long') + } + + // ---- expressions --------------------------------------------------------- + + private parseExpression(): JsNode { + let expression = this.parseAssignment() + while (this.is(',')) { + const node = this.open('sequence_expression', expression.startIndex) + node.add(expression, 'left') + this.take(node) + node.add(this.parseAssignment(), 'right') + expression = this.close(node) + } + return expression + } + + private parseAssignment(): JsNode { + this.enter() + try { + return this.parseAssignmentInner() + } finally { + this.exit() + } + } + + private parseAssignmentInner(): JsNode { + if (this.isWord('yield')) return this.parseYield() + const arrow = this.tryParseArrowFunction() + if (arrow) return arrow + + const start = this.token.start + + // A destructuring assignment must be recognized BEFORE its target is + // parsed: `({ bg = "#fff" } = style)` is not a valid object literal, and + // reading it as one both fails and would leave `bg` unbound for taint. + if ((this.is('{') || this.is('[')) && this.destructuringAssignmentAhead()) { + const node = this.open('assignment_expression', start) + node.add(this.parseBindingTarget(), 'left') + this.expect('=', node, 'operator') + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + + const left = this.parseConditional() + + if (this.is('=')) { + const node = this.open('assignment_expression', start) + node.add(left, 'left') + this.take(node, 'operator') + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + if (this.token.kind === 'punct' && ASSIGN_OPS.has(this.token.value)) { + const node = this.open('augmented_assignment_expression', start) + node.add(left, 'left') + this.take(node, 'operator') + node.add(this.parseAssignment(), 'right') + return this.close(node) + } + return left + } + + /** Does a balanced `{…}`/`[…]` starting here close and get assigned to? */ + private destructuringAssignmentAhead(): boolean { + const state = this.save() + try { + let depth = 0 + for (let i = 0; i < 20_000; i++) { + if (this.atEof()) return false + if (this.is('{') || this.is('[') || this.is('(')) depth++ + else if (this.is('}') || this.is(']') || this.is(')')) { + depth-- + if (depth === 0) { + this.advance() + return this.is('=') + } + if (depth < 0) return false + } else if (this.atTemplate()) { + this.skipTemplateRaw(this.open('scratch')) + continue + } + this.advance() + } + return false + } catch { + return false + } finally { + this.restore(state) + } + } + + private parseYield(): JsNode { + const node = this.open('yield_expression') + this.take(node) + this.eat('*', node) + if (!this.is(')') && !this.is(']') && !this.is('}') && !this.is(',') && !this.is(';') && this.token.kind !== 'eof' && !this.token.newlineBefore) { + node.add(this.parseAssignment()) + } + return this.close(node) + } + + private parseConditional(): JsNode { + const start = this.token.start + const test = this.parseBinary(0) + if (!this.is('?')) return test + const node = this.open('ternary_expression', start) + node.add(test, 'condition') + this.take(node) + node.add(this.parseAssignment(), 'consequence') + this.expect(':', node) + node.add(this.parseAssignment(), 'alternative') + return this.close(node) + } + + private parseBinary(minPrecedence: number): JsNode { + let left = this.parseUnary() + for (;;) { + // `as` / `satisfies` bind like a postfix operator on the left operand. + if (this.dialect !== 'javascript' && (this.isWord('as') || this.isWord('satisfies')) && !this.token.newlineBefore) { + const node = this.open(this.token.value === 'as' ? 'as_expression' : 'satisfies_expression', left.startIndex) + node.add(left) + this.take(node) + // Not stopAtArrow: `x as (c: T) => U` is a function TYPE, and cutting it + // at the `=>` would leave the return type as loose expression tokens. + if (this.isWord('const')) this.take(node) + else this.skipTypeExpression(node, false) + left = this.close(node) + continue + } + const operator = this.token.kind === 'identifier' ? this.token.value : this.token.kind === 'punct' ? this.token.value : null + if (!operator) break + if (operator === 'in' || operator === 'instanceof') { + if (this.token.kind !== 'identifier') break + } else if (this.token.kind !== 'punct') break + const precedence = BINARY_PRECEDENCE[operator] + if (precedence === undefined || precedence <= minPrecedence) break + const node = this.open('binary_expression', left.startIndex) + node.add(left, 'left') + this.take(node, 'operator') + // `**` is right-associative; everything else is left-associative. + node.add(this.parseBinary(operator === '**' ? precedence - 1 : precedence), 'right') + left = this.close(node) + } + return left + } + + private parseUnary(): JsNode { + const start = this.token.start + if (this.token.kind === 'punct' && (this.token.value === '!' || this.token.value === '~' || this.token.value === '+' || this.token.value === '-')) { + const node = this.open('unary_expression', start) + this.take(node, 'operator') + node.add(this.parseUnary(), 'argument') + return this.close(node) + } + if (this.token.kind === 'identifier' && (this.token.value === 'typeof' || this.token.value === 'void' || this.token.value === 'delete')) { + const node = this.open('unary_expression', start) + this.take(node, 'operator') + node.add(this.parseUnary(), 'argument') + return this.close(node) + } + if (this.isWord('await')) { + const state = this.save() + this.advance() + // `await` is a plain identifier outside async code; only treat it as an + // operator when an operand actually follows. + if (this.startsExpression()) { + const node = this.open('await_expression', start) + this.restore(state) + this.take(node) + node.add(this.parseUnary()) + return this.close(node) + } + this.restore(state) + } + if (this.is('++') || this.is('--')) { + const node = this.open('update_expression', start) + this.take(node, 'operator') + node.add(this.parseUnary(), 'argument') + return this.close(node) + } + if (this.dialect === 'typescript' && this.is('<')) { + const node = this.open('type_assertion', start) + this.skipAngleBracketed(node) + node.add(this.parseUnary()) + return this.close(node) + } + let expression = this.parseLeftHandSide(this.parsePrimary()) + if ((this.is('++') || this.is('--')) && !this.token.newlineBefore) { + const node = this.open('update_expression', start) + node.add(expression, 'argument') + this.take(node, 'operator') + expression = this.close(node) + } + return expression + } + + private startsExpression(): boolean { + switch (this.token.kind) { + case 'identifier': + return !['in', 'instanceof', 'as', 'satisfies', 'of'].includes(this.token.value) + case 'number': + case 'string': + case 'regex': + case 'template_start': + case 'private': + return true + case 'punct': + return ['(', '[', '{', '!', '~', '+', '-', '++', '--', '...', '<'].includes(this.token.value) + default: + return false + } + } + + /** Member/subscript/call chains, including optional chaining and templates. */ + private parseLeftHandSide(base: JsNode): JsNode { + let expression = base + for (let i = 0; i < 10_000; i++) { + if (this.is('.') || this.is('?.')) { + const optional = this.is('?.') + const node = this.open('member_expression', expression.startIndex) + node.add(expression, 'object') + this.take(node) + if (optional && this.is('(')) { + // `a?.()` — an optional CALL, not a member access. + const call = this.open('call_expression', expression.startIndex) + for (const child of node.children) call.add(child) + call.setField('function', expression) + call.add(this.parseArguments(), 'arguments') + expression = this.close(call) + continue + } + if (optional && this.is('[')) { + const subscript = this.open('subscript_expression', expression.startIndex) + for (const child of node.children) subscript.add(child) + subscript.setField('object', expression) + this.expect('[', subscript) + subscript.add(this.parseExpression(), 'index') + this.expect(']', subscript) + expression = this.close(subscript) + continue + } + node.add( + this.token.kind === 'private' ? this.parsePrivateName() : this.parseIdentifier('property_identifier'), + 'property', + ) + expression = this.close(node) + continue + } + if (this.is('[')) { + const node = this.open('subscript_expression', expression.startIndex) + node.add(expression, 'object') + this.take(node) + node.add(this.parseExpression(), 'index') + this.expect(']', node) + expression = this.close(node) + continue + } + if (this.is('(')) { + const node = this.open('call_expression', expression.startIndex) + node.add(expression, 'function') + node.add(this.parseArguments(), 'arguments') + expression = this.close(node) + continue + } + if (this.atTemplate()) { + // Tagged template: tree-sitter models it as a call whose `arguments` + // field IS the template_string. + const node = this.open('call_expression', expression.startIndex) + node.add(expression, 'function') + node.add(this.parseTemplateString(), 'arguments') + expression = this.close(node) + continue + } + if (this.is('!') && !this.token.newlineBefore && this.dialect !== 'javascript') { + const node = this.open('non_null_expression', expression.startIndex) + node.add(expression) + this.take(node) + expression = this.close(node) + continue + } + if (this.is('<') && this.dialect !== 'javascript') { + const typeArguments = this.tryParseTypeArguments() + if (!typeArguments) break + if (this.is('(')) { + const node = this.open('call_expression', expression.startIndex) + node.add(expression, 'function') + node.add(typeArguments, 'type_arguments') + node.add(this.parseArguments(), 'arguments') + expression = this.close(node) + continue + } + if (this.atTemplate()) { + const node = this.open('call_expression', expression.startIndex) + node.add(expression, 'function') + node.add(typeArguments, 'type_arguments') + node.add(this.parseTemplateString(), 'arguments') + expression = this.close(node) + continue + } + break + } + break + } + return expression + } + + private parsePrivateName(): JsNode { + const node = this.open('private_property_identifier') + this.advance() + return this.close(node, this.prevEnd) + } + + /** + * `f(x)` vs `f < T > (x)`. Only accept type arguments when the balanced + * `<...>` is immediately followed by a call or tagged template — the same + * disambiguation TypeScript itself applies. + */ + private tryParseTypeArguments(): JsNode | null { + const state = this.save() + try { + const node = this.open('type_arguments') + this.skipAngleBracketed(node) + this.close(node) + if (this.is('(') || this.atTemplate()) return node + this.restore(state) + return null + } catch { + this.restore(state) + return null + } + } + + private parseArguments(): JsNode { + const node = this.open('arguments') + this.expect('(', node) + while (!this.is(')')) { + this.flushComments(node) + if (this.is(')')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated argument list') + if (this.is('...')) { + const spread = this.open('spread_element') + this.take(spread) + spread.add(this.parseAssignment()) + node.add(this.close(spread)) + } else { + node.add(this.parseAssignment()) + } + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect(')', node) + return this.close(node) + } + + // ---- arrow functions ----------------------------------------------------- + + /** + * Arrow functions require unbounded lookahead (`(a, b): T => …` vs a + * parenthesized expression), so this speculatively scans to the matching `)` + * and commits only when `=>` follows. A failed attempt restores exactly. + */ + private tryParseArrowFunction(): JsNode | null { + const start = this.token.start + const state = this.save() + + let isAsync = false + if (this.isWord('async')) { + const probe = this.save() + this.advance() + if (this.token.newlineBefore || (!this.is('(') && this.token.kind !== 'identifier' && !this.is('<'))) { + this.restore(probe) + return null + } + isAsync = true + } + + if (this.token.kind === 'identifier' && !RESERVED_WORDS.has(this.token.value)) { + const probe = this.save() + this.advance() + if (this.is('=>')) { + this.restore(probe) + const node = this.open('arrow_function', start) + if (isAsync) { + this.restore(state) + this.take(node) + } + node.add(this.parseIdentifier('identifier'), 'parameter') + return this.finishArrow(node) + } + this.restore(state) + return null + } + + if (!this.is('(') && !(this.is('<') && this.dialect !== 'javascript')) { + this.restore(state) + return null + } + if (!this.arrowFollows()) { + this.restore(state) + return null + } + + const node = this.open('arrow_function', start) + this.restore(state) + if (isAsync) this.take(node) + if (this.is('<')) this.parseOptionalTypeParameters(node) + node.add(this.parseFormalParameters(), 'parameters') + this.parseOptionalTypeAnnotation(node, true) + return this.finishArrow(node) + } + + /** Scan past a balanced parameter list (and optional return type) for `=>`. */ + private arrowFollows(): boolean { + const state = this.save() + try { + if (this.is('<')) { + const scratch = this.open('type_parameters') + this.skipAngleBracketed(scratch) + if (!this.is('(')) return false + } + let depth = 0 + for (let i = 0; i < 20_000; i++) { + if (this.token.kind === 'eof') return false + if (this.is('(') || this.is('[') || this.is('{')) depth++ + else if (this.is(')') || this.is(']') || this.is('}')) { + depth-- + if (depth === 0) { + this.advance() + break + } + if (depth < 0) return false + } else if (this.atTemplate()) { + this.parseTemplateString() + continue + } + this.advance() + } + if (this.is('=>')) return true + if (this.is(':') && this.dialect !== 'javascript') { + const scratch = this.open('type_annotation') + this.take(scratch) + this.skipTypeExpression(scratch, true) + return this.is('=>') + } + return false + } catch { + return false + } finally { + this.restore(state) + } + } + + private finishArrow(node: JsNode): JsNode { + this.expect('=>', node) + if (this.is('{')) node.add(this.parseBlock(), 'body') + else node.add(this.parseAssignment(), 'body') + return this.close(node) + } + + // ---- primary expressions ------------------------------------------------- + + private parsePrimary(): JsNode { + const start = this.token.start + if (this.token.kind === 'punct' && (this.token.value === '/' || this.token.value === '/=')) { + this.relexAsRegex() + } + + switch (this.token.kind) { + case 'number': { + const node = this.open('number') + this.advance() + return this.close(node, this.prevEnd) + } + case 'string': + return this.parseStringLiteral() + case 'regex': { + const node = this.open('regex') + this.advance() + return this.close(node, this.prevEnd) + } + case 'template_start': + return this.parseTemplateString() + case 'private': { + // `#x in obj` — a private-name brand check. + return this.parsePrivateName() + } + case 'identifier': + return this.parsePrimaryIdentifier() + case 'punct': + break + default: + throw new ParseError(`unexpected token at ${start}`) + } + + if (this.is('(')) { + const node = this.open('parenthesized_expression') + this.take(node) + node.add(this.parseExpression()) + this.expect(')', node) + return this.close(node) + } + if (this.is('[')) return this.parseArrayLiteral() + if (this.is('{')) return this.parseObjectLiteral() + if (this.is('<') && this.jsx) return this.parseJsx() + throw new ParseError(`unexpected token ${JSON.stringify(this.token.value)} at ${start}`) + } + + private parsePrimaryIdentifier(): JsNode { + const word = this.token.value + if (KEYWORD_LITERALS.has(word)) { + const node = this.open(word) + this.advance() + return this.close(node, this.prevEnd) + } + if (word === 'this' || word === 'super') { + const node = this.open(word) + this.advance() + return this.close(node, this.prevEnd) + } + if (word === 'function') return this.parseFunctionExpression(this.open('function_expression')) + if (word === 'class') return this.parseClass('class') + if (word === 'new') return this.parseNew() + if (word === 'import') { + const node = this.open('import') + this.advance() + return this.close(node, this.prevEnd) + } + if (word === 'async') { + const state = this.save() + this.advance() + if (this.isWord('function') && !this.token.newlineBefore) { + this.restore(state) + const node = this.open('function_expression') + this.take(node) + return this.parseFunctionExpression(node) + } + this.restore(state) + } + return this.parseIdentifier('identifier') + } + + private parseFunctionExpression(node: JsNode): JsNode { + this.expectWord('function', node) + if (this.eat('*', node)) { + const generator = this.make('generator_function', node.startIndex, node.endIndex) + for (const child of node.children) generator.add(child) + node = generator + } + if (this.token.kind === 'identifier' && !this.is('(')) node.add(this.parseIdentifier('identifier'), 'name') + this.parseOptionalTypeParameters(node) + node.add(this.parseFormalParameters(), 'parameters') + this.parseOptionalTypeAnnotation(node) + node.add(this.parseBlock(), 'body') + return this.close(node) + } + + private parseNew(): JsNode { + const node = this.open('new_expression') + this.take(node) + if (this.is('.')) { + // new.target + this.take(node) + this.parseIdentifier('property_identifier') + return this.close(node, this.prevEnd) + } + let constructor = this.parsePrimary() + // Member access binds tighter than `new`, but a call terminates it. + for (let i = 0; i < 1000; i++) { + if (this.is('.') || this.is('[')) { + if (this.is('.')) { + const member = this.open('member_expression', constructor.startIndex) + member.add(constructor, 'object') + this.take(member) + member.add(this.parseIdentifier('property_identifier'), 'property') + constructor = this.close(member) + } else { + const subscript = this.open('subscript_expression', constructor.startIndex) + subscript.add(constructor, 'object') + this.take(subscript) + subscript.add(this.parseExpression(), 'index') + this.expect(']', subscript) + constructor = this.close(subscript) + } + continue + } + break + } + node.add(constructor, 'constructor') + if (this.is('<') && this.dialect !== 'javascript') { + const typeArguments = this.tryParseTypeArguments() + if (typeArguments) node.add(typeArguments, 'type_arguments') + } + if (this.is('(')) node.add(this.parseArguments(), 'arguments') + return this.close(node) + } + + private parseStringLiteral(): JsNode { + const node = this.open('string') + const start = this.token.start + const end = this.token.end + this.advance() + // tree-sitter exposes the unquoted body as a `string_fragment` child. + if (end - start > 2) node.add(this.make('string_fragment', start + 1, end - 1)) + return this.close(node, end) + } + + private parseTemplateString(): JsNode { + const node = this.open('template_string') + if (this.token.kind !== 'template_start') throw new ParseError('expected template literal') + // The opening backtick is consumed WITHOUT advancing: what follows is raw + // literal text, and tokenizing it would read `2px` as a bad number. + node.add(this.make('`', this.token.start, this.token.end, false)) + let cursor = this.token.end + this.prevEnd = cursor + for (let i = 0; i < 20_000; i++) { + const chunk = this.lexer.templateChunk(cursor) + if (chunk.fragmentEnd > cursor) node.add(this.make('string_fragment', cursor, chunk.fragmentEnd)) + if (chunk.kind === 'end') { + node.add(this.make('`', chunk.fragmentEnd, chunk.fragmentEnd + 1, false)) + this.lexer.pos = chunk.fragmentEnd + 1 + this.prevEnd = this.lexer.pos + this.token = this.lexer.next(false) + return this.close(node, chunk.fragmentEnd + 1) + } + const substitution = this.make('template_substitution', chunk.fragmentEnd, chunk.fragmentEnd) + substitution.add(this.make('${', chunk.fragmentEnd, chunk.fragmentEnd + 2, false)) + this.lexer.pos = chunk.fragmentEnd + 2 + this.token = this.lexer.next(true) + substitution.add(this.parseExpression()) + if (!this.is('}')) throw new ParseError(`unterminated template substitution at ${this.token.start}`) + substitution.add(this.make('}', this.token.start, this.token.start + 1, false)) + this.close(substitution, this.token.start + 1) + node.add(substitution) + cursor = this.token.start + 1 + } + throw new ParseError('template literal too long') + } + + private parseArrayLiteral(): JsNode { + const node = this.open('array') + this.expect('[', node) + while (!this.is(']')) { + this.flushComments(node) + if (this.is(']')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated array literal') + if (this.is(',')) { + this.take(node) // elision + continue + } + if (this.is('...')) { + const spread = this.open('spread_element') + this.take(spread) + spread.add(this.parseAssignment()) + node.add(this.close(spread)) + } else { + node.add(this.parseAssignment()) + } + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect(']', node) + return this.close(node) + } + + private parseObjectLiteral(): JsNode { + const node = this.open('object') + this.expect('{', node) + while (!this.is('}')) { + this.flushComments(node) + if (this.is('}')) break + if (this.token.kind === 'eof') throw new ParseError('unterminated object literal') + node.add(this.parseObjectMember()) + if (!this.eat(',', node)) break + } + this.flushComments(node) + this.expect('}', node) + return this.close(node) + } + + private parseObjectMember(): JsNode { + const start = this.token.start + if (this.is('...')) { + const node = this.open('spread_element') + this.take(node) + node.add(this.parseAssignment()) + return this.close(node) + } + + const scratch = this.open('object_member', start) + let sawModifier = false + if (this.isWord('async')) { + const state = this.save() + this.advance() + if (!this.is(':') && !this.is('(') && !this.is(',') && !this.is('}') && !this.token.newlineBefore) { + this.restore(state) + this.take(scratch) + sawModifier = true + } else this.restore(state) + } + if (this.is('*')) { + this.take(scratch) + sawModifier = true + } + if (this.isWord('get') || this.isWord('set')) { + const state = this.save() + this.advance() + if (!this.is(':') && !this.is('(') && !this.is(',') && !this.is('}') && !this.is('=')) { + this.restore(state) + this.take(scratch) + sawModifier = true + } else this.restore(state) + } + + const key = this.parsePropertyName() + if (this.is('(') || this.is('<')) { + const method = this.open('method_definition', start) + for (const child of scratch.children) method.add(child) + method.add(key, 'name') + this.parseOptionalTypeParameters(method) + method.add(this.parseFormalParameters(), 'parameters') + this.parseOptionalTypeAnnotation(method) + method.add(this.parseBlock(), 'body') + return this.close(method) + } + if (sawModifier) throw new ParseError(`expected method body at ${this.token.start}`) + if (this.is(':')) { + const pair = this.open('pair', start) + pair.add(key, 'key') + this.expect(':', pair) + pair.add(this.parseAssignment(), 'value') + return this.close(pair) + } + if (key.type !== 'property_identifier') throw new ParseError(`expected ":" at ${this.token.start}`) + // `{ a }` and `{ a = 1 }` (the latter only inside a destructuring target, + // which parseBindingTarget handles; here it is a syntax error we fail on). + if (this.is('=')) throw new ParseError(`unexpected "=" at ${this.token.start}`) + return this.make('shorthand_property_identifier', key.startIndex, key.endIndex) + } + + // ---- JSX ----------------------------------------------------------------- + + private parseJsx(): JsNode { + const state = this.save() + try { + return this.parseJsxElement(false) + } catch (error) { + // In .tsx a leading `<` may instead open a generic arrow's type + // parameters (`(x) => x`); retry that reading before giving up. + if (this.dialect !== 'tsx') throw error + this.restore(state) + const scratch = this.open('type_parameters') + this.skipAngleBracketed(scratch) + const arrow = this.tryParseArrowFunction() + if (!arrow) throw error + return arrow + } + } + + /** + * `resumeRaw` says what follows this element: inside another element's + * children the next thing is markup, so the lexer must stay parked instead of + * tokenizing `` as operators. Everywhere else a normal token follows. + */ + private parseJsxElement(resumeRaw: boolean): JsNode { + const start = this.token.start + if (this.lexer.text[start] !== '<') throw new ParseError('expected JSX') + const fragment = this.jsxFragmentAhead() + const opening = this.parseJsxOpening(resumeRaw, fragment) + if (opening.type === 'jsx_self_closing_element') return opening + const element = this.open(fragment ? 'jsx_fragment' : 'jsx_element', start) + element.add(opening, 'open_tag') + for (let i = 0; i < 100_000; i++) { + if (this.parseJsxChild(element, resumeRaw) === 'closed') return this.close(element) + } + throw new ParseError('JSX element too long') + } + + /** + * A JSX tag closes on a single `>` CHARACTER, which the tokenizer's maximal + * munch would happily swallow into `>=` or `>>`. `==` is real + * markup, so every tag boundary is tested against the source text, never + * against a token. + */ + private atJsxGt(): boolean { + return this.lexer.text[this.token.start] === '>' + } + + private jsxFragmentAhead(): boolean { + const state = this.save() + this.advance() + const isFragment = this.atJsxGt() + this.restore(state) + return isFragment + } + + private parseJsxOpening(elementResumeRaw: boolean, fragment: boolean): JsNode { + // Attributes tokenize normally; only the text between tags needs raw + // scanning, which starts the moment the tag's `>` is consumed. + const node = this.open('jsx_opening_element') + this.expect('<', node) + if (fragment) { + this.consumeJsxTagEnd(node, true) + return this.close(node) + } + node.add(this.parseJsxName(), 'name') + while (!this.atJsxGt() && !this.is('/')) { + if (this.atEof()) throw new ParseError('unterminated JSX tag') + if (this.is('{')) { + const spread = this.open('jsx_expression') + this.take(spread) + this.eat('...', spread) + spread.add(this.parseAssignment()) + this.expect('}', spread) + node.add(this.close(spread)) + continue + } + const attribute = this.open('jsx_attribute') + attribute.add(this.parseJsxName(), 'name') + if (this.is('=')) { + // Consume `=` WITHOUT lexing ahead: the value may be a multi-line + // attribute string, which the JS string tokenizer would reject. + attribute.add(this.make('=', this.token.start, this.token.end, false)) + this.lexer.pos = this.token.end + const valueStart = this.lexer.peekPosition() + const quote = this.lexer.text[valueStart] + if (quote === '"' || quote === "'") { + attribute.add(this.parseJsxAttributeString(valueStart)) + } else { + this.prevEnd = this.lexer.pos + this.token = this.lexer.next(false) + if (this.is('{')) attribute.add(this.parseJsxExpressionContainer(false)) + else if (this.lexer.text[this.token.start] === '<') attribute.add(this.parseJsxElement(false)) + else throw new ParseError(`bad JSX attribute value at ${this.token.start}`) + } + } + node.add(this.close(attribute)) + } + if (this.is('/')) { + const selfClosing = this.make('jsx_self_closing_element', node.startIndex, node.endIndex) + for (const child of node.children) selfClosing.add(child) + const name = node.childForFieldName('name') + if (name) selfClosing.setField('name', name) + this.expect('/', selfClosing) + this.consumeJsxTagEnd(selfClosing, elementResumeRaw) + return this.close(selfClosing) + } + this.consumeJsxTagEnd(node, true) + return this.close(node) + } + + /** Consume a tag's `>`, then either park the lexer for raw child text or + * resume normal tokenization. */ + private consumeJsxTagEnd(node: JsNode, resumeRaw: boolean): void { + if (!this.atJsxGt()) throw new ParseError(`expected ">" at ${this.token.start}`) + const end = this.token.start + 1 + node.add(this.make('>', this.token.start, end, false)) + this.lexer.pos = end + this.prevEnd = end + this.token = resumeRaw + ? { kind: 'punct', start: end, end, value: '', newlineBefore: false } + : this.lexer.next(false) + } + + private parseJsxChild(element: JsNode, elementResumeRaw: boolean): 'closed' | 'child' { + const text = this.lexer.text + let i = this.lexer.pos + const textStart = i + while (i < text.length && text[i] !== '<' && text[i] !== '{') i++ + if (i > textStart && text.slice(textStart, i).trim().length > 0) { + element.add(this.make('jsx_text', textStart, i)) + } + if (i >= text.length) throw new ParseError('unterminated JSX element') + this.lexer.pos = i + if (text[i] === '{') { + this.token = this.lexer.next(false) + element.add(this.parseJsxExpressionContainer(true)) + return 'child' + } + this.token = this.lexer.next(false) + if (text[i + 1] === '/') { + const closing = this.open('jsx_closing_element', i) + this.expect('<', closing) + this.expect('/', closing) + if (!this.atJsxGt()) closing.add(this.parseJsxName(), 'name') + this.consumeJsxTagEnd(closing, elementResumeRaw) + element.add(this.close(closing), 'close_tag') + return 'closed' + } + element.add(this.parseJsxElement(true)) + return 'child' + } + + /** + * A JSX attribute string is markup, not a JS literal: it may span lines and + * processes no escapes, so a multi-line `className="…"` must be read raw + * rather than handed to the string tokenizer. + */ + private parseJsxAttributeString(start: number): JsNode { + const text = this.lexer.text + const quote = text[start] + if (quote !== '"' && quote !== "'") throw new ParseError(`expected JSX attribute string at ${start}`) + const closing = text.indexOf(quote, start + 1) + if (closing === -1) throw new ParseError(`unterminated JSX attribute string at ${start}`) + const node = this.make('string', start, closing + 1) + if (closing > start + 1) node.add(this.make('string_fragment', start + 1, closing)) + this.lexer.pos = closing + 1 + this.prevEnd = this.lexer.pos + this.token = this.lexer.next(false) + return node + } + + private parseJsxExpressionContainer(resumeRaw: boolean): JsNode { + const node = this.open('jsx_expression') + this.expect('{', node) + if (!this.is('}')) { + this.eat('...', node) + node.add(this.parseExpression()) + } + if (!this.is('}')) throw new ParseError(`expected "}" at ${this.token.start}`) + const end = this.token.start + 1 + node.add(this.make('}', this.token.start, end, false)) + this.lexer.pos = end + this.prevEnd = end + this.close(node, end) + this.token = resumeRaw + ? { kind: 'punct', start: end, end, value: '', newlineBefore: false } + : this.lexer.next(false) + return node + } + + private parseJsxName(): JsNode { + const start = this.token.start + if (this.token.kind !== 'identifier') throw new ParseError(`expected JSX name at ${start}`) + let node = this.parseIdentifier('identifier') + for (let i = 0; i < 32; i++) { + if (this.is(':') || this.is('-')) { + this.advance() + if (this.token.kind !== 'identifier') throw new ParseError('bad JSX name') + this.advance() + node = this.close(this.make('jsx_namespace_name', start, this.prevEnd), this.prevEnd) + continue + } + if (this.is('.')) { + const member = this.open('member_expression', start) + member.add(node, 'object') + this.take(member) + member.add(this.parseIdentifier('property_identifier'), 'property') + node = this.close(member) + continue + } + break + } + return node + } +} diff --git a/packages/analyzer-engine/src/security/lang.ts b/packages/analyzer-engine/src/security/lang.ts new file mode 100644 index 0000000..ef1c0f7 --- /dev/null +++ b/packages/analyzer-engine/src/security/lang.ts @@ -0,0 +1,118 @@ +/** + * Language vocabulary and the parser contract the SAST engine analyzes against. + * + * The engine owns rules, taint and findings; it does NOT own a parser. Every + * front-end injects one: + * + * - the hosted app injects `web-tree-sitter` + the prebuilt WASM grammars + * (10 languages, full fidelity), and + * - the CLI injects the zero-dependency JS-family parser in `js-parse/`, + * because the WASM grammars are 6x the CLI's entire 1 MB release budget. + * + * Both produce the SAME node vocabulary — tree-sitter's grammar node names — + * so one rule pack runs unchanged behind either. {@link SyntaxNode} is the + * structural subset of `web-tree-sitter`'s `SyntaxNode` the rules actually + * touch; the real thing satisfies it as-is, with no adapter. + */ + +/** Languages with a grammar somewhere in the product and at least one rule. */ +export type SastLanguage = + | 'javascript' + | 'typescript' + | 'tsx' + | 'python' + | 'java' + | 'csharp' + | 'go' + | 'php' + | 'ruby' + | 'rust' + +const EXT_LANG: Record = { + '.js': 'javascript', + '.jsx': 'javascript', + '.mjs': 'javascript', + '.cjs': 'javascript', + '.ts': 'typescript', + '.mts': 'typescript', + '.cts': 'typescript', + '.tsx': 'tsx', + '.py': 'python', + '.pyi': 'python', + '.java': 'java', + '.cs': 'csharp', + '.go': 'go', + '.php': 'php', + '.rb': 'ruby', + '.rs': 'rust', +} + +/** Language for a path, or null when the extension isn't SAST-covered. */ +export function sastLanguageForPath(path: string): SastLanguage | null { + const dot = path.lastIndexOf('.') + if (dot === -1) return null + return EXT_LANG[path.slice(dot).toLowerCase()] ?? null +} + +/** JS-family languages share one adapter. */ +export function isJsFamily(lang: SastLanguage): boolean { + return lang === 'javascript' || lang === 'typescript' || lang === 'tsx' +} + +// ---- bounds --------------------------------------------------------------- +export const MAX_SOURCE_BYTES = 400_000 +export const MAX_NODES = 400_000 + +/** + * The subset of a tree-sitter `SyntaxNode` the rule pack and taint engine read. + * + * Deliberately structural and minimal: anything an injected parser must + * implement is listed here, and nothing else may be relied on. `web-tree-sitter` + * satisfies this interface without a wrapper, which is what keeps the hosted + * path byte-for-byte unchanged by the extraction. + */ +export interface SyntaxNode { + readonly type: string + readonly text: string + readonly isNamed: boolean + /** Identity token. Node wrappers are not reference-stable, so rules compare ids. */ + readonly id: number + readonly startIndex: number + readonly endIndex: number + readonly startPosition: { row: number; column: number } + readonly endPosition: { row: number; column: number } + readonly parent: SyntaxNode | null + readonly children: SyntaxNode[] + readonly namedChildren: SyntaxNode[] + readonly childCount: number + readonly namedChildCount: number + readonly previousNamedSibling: SyntaxNode | null + readonly nextNamedSibling: SyntaxNode | null + child(index: number): SyntaxNode | null + childForFieldName(fieldName: string): SyntaxNode | null +} + +/** A parsed file: the root node plus whether the parse recovered from errors. */ +export interface ParsedTree { + rootNode: SyntaxNode + /** True when the parse produced error nodes (best-effort results still usable). */ + hasError: boolean + /** Release parser-owned memory. No-op for parsers with none. */ + release(): void +} + +/** + * The injected parser. + * + * Contract: `parse` returns null — never throws, never guesses — when the + * source is too large, the grammar is unavailable, or the input is outside the + * syntax this parser handles soundly. A null is a *coverage* answer, and the + * engine reports that language as degraded. That is what lets the zero-dep + * parser be strict: anything it cannot parse with certainty becomes a missing + * finding, never a wrong one. + */ +export interface SastParser { + /** Languages this parser can produce trees for. */ + readonly languages: ReadonlySet + parse(lang: SastLanguage, content: string): Promise +} diff --git a/packages/analyzer-engine/src/security/local-profile.ts b/packages/analyzer-engine/src/security/local-profile.ts new file mode 100644 index 0000000..8712233 --- /dev/null +++ b/packages/analyzer-engine/src/security/local-profile.ts @@ -0,0 +1,57 @@ +import type { SastLanguage } from './lang' + +/** + * What the CLI's local SAST pass is allowed to report. + * + * The engine's full pack is 22 rules across 10 languages. The CLI runs a + * deliberate SUBSET, and the subset is a precision decision, not a packaging + * one: every rule listed here has been differentially validated — same rule, + * same file, zero disagreement against the hosted tree-sitter parser — across + * the real-repository corpus, and adjudicated to zero false positives. A rule + * earns its way into this list; it is not added because it compiles. + * + * The five AI-agent defect rules are the classes coding agents actually get + * wrong (floating writes, swallowed errors, coercion comparisons, N+1 loops, + * mass assignment). `sql-injection` is the one injection class that reaches a + * sink through plain string building, so it survives without the hosted symbol + * graph. + */ +export const CLI_SAST_RULE_IDS: ReadonlySet = new Set([ + 'unawaited-persistence', + 'swallowed-error', + 'loose-equality', + 'db-call-in-loop', + 'mass-assignment', + 'open-record-write', + 'sql-injection', +]) + +/** Languages the CLI's zero-dependency parser covers. */ +export const CLI_SAST_LANGUAGES: ReadonlySet = new Set([ + 'javascript', + 'typescript', + 'tsx', +]) + +/** Display names of {@link CLI_SAST_LANGUAGES}, matching the indexer's labels. */ +export const CLI_SAST_LANGUAGE_NAMES: ReadonlySet = new Set(['TypeScript', 'JavaScript']) + +/** + * Classes the local pass STILL does not check, named so a receipt can say so. + * + * This is the honest complement of {@link CLI_SAST_RULE_IDS}: the CLI gained + * concat-SQL-injection but not command injection, path traversal, SSRF, XSS, + * open redirect or insecure deserialization, because those need either the + * hosted symbol graph or rules that have not cleared the same precision bar + * locally. A receipt that listed only what ran would repeat the exact mistake + * the coverage analyzer exists to prevent. + */ +export const CLI_SAST_UNCHECKED_CLASSES = [ + 'command injection', + 'code injection', + 'path traversal', + 'SSRF', + 'open redirect', + 'XSS', + 'insecure deserialization', +] as const diff --git a/packages/analyzer-engine/src/security/normalize.ts b/packages/analyzer-engine/src/security/normalize.ts new file mode 100644 index 0000000..5572e11 --- /dev/null +++ b/packages/analyzer-engine/src/security/normalize.ts @@ -0,0 +1,791 @@ +import type { SastLanguage, SyntaxNode } from './lang' +import { isJsFamily } from './lang' + +/** + * Normalized AST adapter. + * + * Tree-sitter grammars disagree on node names (`call_expression` vs `call` vs + * `method_invocation` vs `invocation_expression`), so this layer projects each + * grammar onto ONE small vocabulary the rules and taint engine speak: calls, + * member/property access, string concatenation, string interpolation, + * subscripts, assignments and function definitions. Rules never touch a raw + * grammar node type; they ask questions like "is this a call to X, and is arg N + * tainted?" — the same question in every language. + * + * Everything here is total: unknown shapes return null / empty, never throw. + */ + +const field = (n: SyntaxNode, name: string): SyntaxNode | null => n.childForFieldName(name) + +/** Strip surrounding quotes / string prefixes from a raw literal token. */ +function stripStringToken(raw: string): string { + let s = raw.trim() + // language string prefixes: f"" r"" b"" u"" (python), @"" $"" (c#), L"" etc. + s = s.replace(/^[a-zA-Z@$]{1,2}(?=['"])/, '') + const q = s[0] + if ((q === '"' || q === "'" || q === '`') && s.endsWith(q)) return s.slice(1, -1) + return s +} + +// ---- node-type vocabularies per language ---------------------------------- + +const FUNCTION_TYPES: Record> = { + javascript: new Set(['function_declaration', 'function_expression', 'arrow_function', 'method_definition', 'generator_function_declaration', 'generator_function']), + typescript: new Set(['function_declaration', 'function_expression', 'arrow_function', 'method_definition', 'generator_function_declaration', 'generator_function']), + tsx: new Set(['function_declaration', 'function_expression', 'arrow_function', 'method_definition', 'generator_function_declaration', 'generator_function']), + python: new Set(['function_definition', 'lambda']), + java: new Set(['method_declaration', 'constructor_declaration', 'lambda_expression']), + csharp: new Set(['method_declaration', 'constructor_declaration', 'local_function_statement', 'lambda_expression']), + go: new Set(['function_declaration', 'method_declaration', 'func_literal']), + php: new Set(['function_definition', 'method_declaration', 'anonymous_function_creation_expression', 'arrow_function']), + ruby: new Set(['method', 'singleton_method']), + rust: new Set(['function_item', 'closure_expression']), +} + +export function isFunctionNode(node: SyntaxNode, lang: SastLanguage): boolean { + return FUNCTION_TYPES[lang].has(node.type) +} + +export interface NFunc { + name: string + params: string[] + body: SyntaxNode | null + node: SyntaxNode + line: number +} + +export function asFunction(node: SyntaxNode, lang: SastLanguage): NFunc | null { + if (!isFunctionNode(node, lang)) return null + const name = field(node, 'name')?.text ?? '' + const body = field(node, 'body') ?? null + return { name, params: paramNames(node), body, node, line: node.startPosition.row + 1 } +} + +function paramNames(node: SyntaxNode): string[] { + // The parameter container differs per grammar; find it, then pull leaf names. + const containerTypes = new Set([ + 'formal_parameters', 'parameters', 'parameter_list', 'method_parameters', + 'formal_parameter_list', 'block_parameters', 'closure_parameters', + ]) + let container: SyntaxNode | null = field(node, 'parameters') + if (!container) { + for (let i = 0; i < node.childCount; i++) { + const c = node.child(i) + if (c && containerTypes.has(c.type)) { + container = c + break + } + } + } + // JS arrow with a single bare identifier param: `x => ...` + if (!container) { + const p = field(node, 'parameter') + if (p) return [leafParamName(p) ?? p.text].filter(Boolean) as string[] + return [] + } + const names: string[] = [] + for (const c of container.namedChildren) { + // Go groups several names under one parameter_declaration. + if (c.type === 'parameter_declaration') { + for (const id of c.namedChildren) if (id.type === 'identifier') names.push(id.text) + continue + } + const n = leafParamName(c) + if (n) names.push(n) + } + return names +} + +function leafParamName(p: SyntaxNode): string | null { + switch (p.type) { + case 'identifier': + case 'shorthand_property_identifier_pattern': + return p.text + case 'variable_name': // php + return p.text.replace(/^\$/, '') + default: { + // required_parameter/optional_parameter (TS), typed_parameter (py), + // default_parameter, formal_parameter (java), parameter (c#/rust), + // simple_parameter (php): the name is a nested identifier/variable_name. + const named = field(p, 'name') ?? field(p, 'pattern') + if (named) { + if (named.type === 'identifier') return named.text + if (named.type === 'variable_name') return named.text.replace(/^\$/, '') + } + for (const c of p.namedChildren) { + if (c.type === 'identifier') return c.text + if (c.type === 'variable_name') return c.text.replace(/^\$/, '') + } + return null + } + } +} + +// ---- expression normalization --------------------------------------------- + +/** Strip parentheses / await / casts so callers see through wrappers. */ +export function unwrap(node: SyntaxNode, lang: SastLanguage): SyntaxNode { + let cur = node + for (let i = 0; i < 12; i++) { + const t = cur.type + if (t === 'parenthesized_expression' || t === 'await_expression') { + const inner = t === 'await_expression' ? (field(cur, 'argument') ?? cur.namedChildren[0]) : cur.namedChildren[0] + if (inner) { + cur = inner + continue + } + } + if (isJsFamily(lang) && (t === 'as_expression' || t === 'satisfies_expression' || t === 'non_null_expression')) { + const inner = cur.namedChildren[0] + if (inner) { + cur = inner + continue + } + } + if (lang === 'csharp' && (t === 'cast_expression')) { + const inner = field(cur, 'value') ?? cur.namedChildren[cur.namedChildren.length - 1] + if (inner) { + cur = inner + continue + } + } + break + } + return cur +} + +export interface NCall { + node: SyntaxNode + line: number + callee: SyntaxNode | null + /** Dotted callee name, e.g. "child_process.exec", "cursor.execute", "os.system". */ + fullName: string + /** Last dotted segment, e.g. "exec". */ + method: string + /** Object of a member/method call, e.g. `db` in `db.query()`. */ + receiver: SyntaxNode | null + /** Dotted receiver name, e.g. "child_process", "os". */ + receiverName: string | null + args: SyntaxNode[] + /** True for `new X()` / object-creation expressions. */ + isConstruct: boolean +} + +const CALL_TYPES: Record> = { + javascript: new Set(['call_expression', 'new_expression']), + typescript: new Set(['call_expression', 'new_expression']), + tsx: new Set(['call_expression', 'new_expression']), + python: new Set(['call']), + java: new Set(['method_invocation', 'object_creation_expression', 'explicit_constructor_invocation']), + csharp: new Set(['invocation_expression', 'object_creation_expression']), + go: new Set(['call_expression']), + php: new Set(['function_call_expression', 'member_call_expression', 'scoped_call_expression', 'object_creation_expression']), + ruby: new Set(['call', 'method_call', 'command', 'command_call']), + rust: new Set(['call_expression', 'macro_invocation']), +} + +export function asCall(node: SyntaxNode, lang: SastLanguage): NCall | null { + if (!CALL_TYPES[lang].has(node.type)) return null + const line = node.startPosition.row + 1 + const isConstruct = node.type === 'new_expression' || node.type === 'object_creation_expression' + + if (lang === 'java') { + if (node.type === 'method_invocation') { + const obj = field(node, 'object') + const nameNode = field(node, 'name') + const name = nameNode?.text ?? '' + const receiverName = obj ? dottedName(obj, lang) : null + const fullName = receiverName ? `${receiverName}.${name}` : name + return { + node, + line, + callee: nameNode, + fullName, + method: name, + receiver: obj, + receiverName, + args: argList(field(node, 'arguments')), + isConstruct: false, + } + } + // object_creation_expression: new Type(args) + const type = field(node, 'type') + const name = type?.text ?? '' + return mk(node, line, type, null, name, argList(field(node, 'arguments')), true, lang) + } + + if (lang === 'python') { + const fn = field(node, 'function') + return mk(node, line, fn, receiverOf(fn, lang), lastSegment(fn, lang), argList(field(node, 'arguments')), false, lang) + } + + if (lang === 'ruby') { + const recv = field(node, 'receiver') + const method = field(node, 'method')?.text ?? '' + return mk(node, line, field(node, 'method'), recv, method, argList(field(node, 'arguments')), false, lang) + } + + if (lang === 'php') { + if (node.type === 'object_creation_expression') { + const type = node.namedChildren.find((c) => c.type === 'name' || c.type === 'qualified_name') + return mk(node, line, type ?? null, null, type?.text ?? '', argList(field(node, 'arguments')), true, lang) + } + if (node.type === 'function_call_expression') { + const fn = field(node, 'function') + return mk(node, line, fn, null, fn?.text?.replace(/^\\/, '') ?? '', argList(field(node, 'arguments')), false, lang) + } + // member_call_expression / scoped_call_expression + const obj = field(node, 'object') ?? field(node, 'scope') + const name = field(node, 'name')?.text ?? '' + return mk(node, line, field(node, 'name'), obj, name, argList(field(node, 'arguments')), false, lang) + } + + if (lang === 'rust' && node.type === 'macro_invocation') { + const macro = field(node, 'macro') + return mk(node, line, macro, null, macro?.text ?? '', argList(node.namedChildren.find((c) => c.type === 'token_tree') ?? null), false, lang) + } + + // JS family, C#, Go, Rust call_expression / new / invocation: + const fnField = node.type === 'new_expression' + ? field(node, 'constructor') + : field(node, 'function') ?? field(node, 'type') + const fn = fnField + return mk(node, line, fn, receiverOf(fn, lang), lastSegment(fn, lang), argList(field(node, 'arguments')), isConstruct, lang) +} + +function mk( + node: SyntaxNode, + line: number, + callee: SyntaxNode | null, + receiver: SyntaxNode | null, + method: string, + args: SyntaxNode[], + isConstruct: boolean, + lang: SastLanguage, +): NCall { + const fullName = callee ? dottedName(callee, lang) ?? method : method + return { + node, + line, + callee, + fullName, + method, + receiver, + receiverName: receiver ? dottedName(receiver, lang) : null, + args, + isConstruct, + } +} + +/** Extract argument value nodes from an arguments container, unwrapping wrappers. */ +function argList(container: SyntaxNode | null): SyntaxNode[] { + if (!container) return [] + const out: SyntaxNode[] = [] + for (const c of container.namedChildren) { + // C#/PHP wrap each argument in an `argument` node. + if (c.type === 'argument') { + const inner = c.namedChildren[c.namedChildren.length - 1] + out.push(inner ?? c) + } else { + out.push(c) + } + } + return out +} + +/** Object expression of a member/method-call callee. */ +function receiverOf(callee: SyntaxNode | null, lang: SastLanguage): SyntaxNode | null { + if (!callee) return null + const m = asMember(callee, lang) + return m ? m.object : null +} + +/** Last dotted segment of a callee expression (the method/function name). */ +function lastSegment(callee: SyntaxNode | null, lang: SastLanguage): string { + if (!callee) return '' + const m = asMember(callee, lang) + if (m) return m.property + const d = dottedName(callee, lang) + if (!d) return callee.text + const dot = d.lastIndexOf('.') + return dot === -1 ? d : d.slice(dot + 1) +} + +export interface NMember { + object: SyntaxNode + property: string + node: SyntaxNode +} + +const MEMBER_TYPES: Record = { + member_expression: { object: 'object', property: 'property' }, // js + attribute: { object: 'object', property: 'attribute' }, // python + field_access: { object: 'object', property: 'field' }, // java + member_access_expression: { object: 'expression', property: 'name' }, // c#/php (php uses object) + selector_expression: { object: 'operand', property: 'field' }, // go + field_expression: { object: 'value', property: 'field' }, // rust + scoped_identifier: { object: 'path', property: 'name' }, // rust :: + scoped_property_access_expression: { object: 'scope', property: 'name' }, // php :: +} + +export function asMember(node: SyntaxNode, lang: SastLanguage): NMember | null { + const spec = MEMBER_TYPES[node.type] + if (spec) { + // php member_access_expression uses field 'object', c# uses 'expression' + let obj = field(node, spec.object) + if (!obj && node.type === 'member_access_expression') obj = field(node, 'object') + const prop = field(node, spec.property)?.text + if (obj && prop) return { object: obj, property: prop.replace(/^\$/, ''), node } + } + // Ruby member access is modeled as a call with a receiver and no args. + if (lang === 'ruby' && node.type === 'call') { + const recv = field(node, 'receiver') + const m = field(node, 'method')?.text + if (recv && m) return { object: recv, property: m, node } + } + return null +} + +/** + * Dotted textual name of a callee / reference expression, e.g. `a.b.c`, + * `os.system`, `child_process.exec`. Subscripts collapse to their base + * (`x.y['z']` → `x.y`). Returns null for expressions with no stable name. + */ +export function dottedName(node: SyntaxNode, lang: SastLanguage): string | null { + const n = unwrap(node, lang) + const m = asMember(n, lang) + if (m) { + const base = dottedName(m.object, lang) + return base ? `${base}.${m.property}` : m.property + } + const sub = subscriptBase(n) + if (sub) return dottedName(sub, lang) + const call = asCall(n, lang) + if (call && call.callee) return dottedName(call.callee, lang) + switch (n.type) { + case 'identifier': + case 'property_identifier': + case 'field_identifier': + case 'type_identifier': + case 'name': + case 'qualified_name': + return n.text.replace(/^\\/, '') + case 'variable_name': // php $x + return n.text.replace(/^\$/, '') + case 'this': + case 'self': + case 'super': + return n.type + default: + return null + } +} + +/** True and the referenced name if the node is a bare variable reference. */ +export function identifierName(node: SyntaxNode, lang: SastLanguage): string | null { + const n = unwrap(node, lang) + if (n.type === 'identifier') return n.text + if (n.type === 'variable_name') return n.text.replace(/^\$/, '') // php + if (lang === 'ruby' && n.type === 'identifier') return n.text + return null +} + +/** Base expression of an index/subscript access, e.g. `a[i]` → `a`. */ +export function subscriptBase(node: SyntaxNode): SyntaxNode | null { + switch (node.type) { + case 'subscript_expression': // js + return field(node, 'object') + case 'subscript': // python + return field(node, 'value') ?? field(node, 'object') + case 'index_expression': // go/php + return field(node, 'operand') ?? field(node, 'object') ?? node.namedChildren[0] ?? null + case 'element_access_expression': // c# + return field(node, 'expression') + case 'array_access': // java + return field(node, 'array') + default: + return null + } +} + +/** + * If the node is string concatenation (`a + b`, or PHP `a . b`), return its + * flattened operands; otherwise null. Only `+` (or `.` in PHP) counts — other + * binary operators don't build strings. + */ +export function concatOperands(node: SyntaxNode, lang: SastLanguage): SyntaxNode[] | null { + const n = unwrap(node, lang) + const concatOp = lang === 'php' ? '.' : '+' + const isBinary = + n.type === 'binary_expression' || n.type === 'binary_operator' /* python */ || n.type === 'binary' /* ruby */ + if (!isBinary) return null + const op = field(n, 'operator')?.text ?? operatorToken(n) + if (op !== concatOp) return null + const left = field(n, 'left') + const right = field(n, 'right') + const parts: SyntaxNode[] = [] + const push = (side: SyntaxNode | null) => { + if (!side) return + const nested = concatOperands(side, lang) + if (nested) parts.push(...nested) + else parts.push(side) + } + push(left) + push(right) + return parts.length ? parts : null +} + +function operatorToken(n: SyntaxNode): string | null { + // Fallback for grammars that expose the operator as an anonymous child. + for (let i = 0; i < n.childCount; i++) { + const c = n.child(i) + if (c && !c.isNamed) return c.text + } + return null +} + +/** + * Embedded expressions of a string with interpolation (template literals, + * python f-strings, c# interpolated strings, php/ruby interpolation). + * Returns null for non-interpolated strings. + */ +export function interpolationExprs(node: SyntaxNode, lang: SastLanguage): SyntaxNode[] | null { + const n = unwrap(node, lang) + const interpTypes = new Set([ + 'template_substitution', // js `${...}` + 'interpolation', // python f-string / c# / ruby / php + ]) + const stringContainerTypes = new Set([ + 'template_string', 'string', 'interpolated_string_expression', 'encapsed_string', 'string_literal', + ]) + if (!stringContainerTypes.has(n.type)) return null + const out: SyntaxNode[] = [] + const collect = (x: SyntaxNode) => { + if (interpTypes.has(x.type)) { + // the embedded expression is the named child (skip format specs) + const expr = x.namedChildren.find((c) => c.type !== 'format_specifier' && c.type !== 'type_conversion') + if (expr) out.push(expr) + return + } + // php encapsed strings embed bare variable_name children directly + if (x.type === 'variable_name') { + out.push(x) + return + } + for (const c of x.namedChildren) collect(c) + } + for (const c of n.namedChildren) collect(c) + return out.length ? out : null +} + +/** Content of a plain (non-interpolated) string literal, else null. */ +export function stringLiteralValue(node: SyntaxNode | null | undefined, lang: SastLanguage): string | null { + if (!node) return null + const n = unwrap(node, lang) + const stringTypes = new Set([ + 'string', 'string_literal', 'template_string', 'interpreted_string_literal', + 'raw_string_literal', 'encapsed_string', 'char_literal', + ]) + if (!stringTypes.has(n.type)) return null + if (interpolationExprs(n, lang)) return null // has interpolation → not a constant + return stripStringToken(n.text) +} + +/** Ordered constituent of a string-building expression: constant text or an embedded expression. */ +export type UrlPart = { kind: 'const'; text: string } | { kind: 'expr'; node: SyntaxNode } + +export interface UrlHead { + /** Constant text before the first embedded expression ('' when the string + * starts with one); null when the head is not a known constant. */ + constantPrefix: string | null + /** The first embedded expression contributing to the value, if any. */ + headExpr: SyntaxNode | null + /** All parts in source order; null when the node is not a template/concat shape. */ + parts: UrlPart[] | null +} + +/** Constant string-fragment node types inside interpolated strings, per grammar. */ +const STRING_FRAGMENT_TYPES = new Set([ + 'string_fragment', // js + 'string_content', // python / ruby + 'interpolated_string_text', // c# + 'string_value', // php +]) + +/** + * Decompose a URL-ish string expression into its head (the part that decides + * scheme/authority) and ordered parts. For a template/interpolated string the + * head is the constant text before the first substitution plus that + * substitution's expression; for a `+`/concat chain it is the leading literal + * operand(s) or the leftmost expression. Anything else is opaque: the node + * itself is the head and there is no part decomposition. + */ +export function urlHeadOf(node: SyntaxNode, lang: SastLanguage): UrlHead { + const n = unwrap(node, lang) + + // Template / interpolated string: fragments and substitutions in source order. + if (interpolationExprs(n, lang)) { + const interpTypes = new Set(['template_substitution', 'interpolation']) + const parts: UrlPart[] = [] + const collect = (x: SyntaxNode) => { + if (interpTypes.has(x.type)) { + const expr = x.namedChildren.find((c) => c.type !== 'format_specifier' && c.type !== 'type_conversion') + if (expr) parts.push({ kind: 'expr', node: expr }) + return + } + // php encapsed strings embed bare variable_name children directly + if (x.type === 'variable_name') { + parts.push({ kind: 'expr', node: x }) + return + } + if (STRING_FRAGMENT_TYPES.has(x.type)) { + parts.push({ kind: 'const', text: x.text }) + return + } + for (const c of x.namedChildren) collect(c) + } + for (const c of n.namedChildren) collect(c) + let prefix = '' + let head: SyntaxNode | null = null + for (const p of parts) { + if (p.kind === 'const') prefix += p.text + else { + head = p.node + break + } + } + return { constantPrefix: prefix, headExpr: head, parts } + } + + // Concatenation chain: leading literal operands form the constant prefix. + const operands = concatOperands(n, lang) + if (operands) { + const parts: UrlPart[] = operands.map((o) => { + const lit = stringLiteralValue(o, lang) + return lit !== null ? { kind: 'const', text: lit } : { kind: 'expr', node: o } + }) + let prefix: string | null = null + let head: SyntaxNode | null = null + for (const p of parts) { + if (p.kind === 'const') prefix = (prefix ?? '') + p.text + else { + head = p.node + break + } + } + return { constantPrefix: prefix, headExpr: head, parts } + } + + return { constantPrefix: null, headExpr: n, parts: null } +} + +/** Boolean value of a literal, else null. */ +export function boolLiteralValue(node: SyntaxNode | null | undefined, lang: SastLanguage): boolean | null { + if (!node) return null + const t = unwrap(node, lang).text.trim().toLowerCase() + if (t === 'true') return true + if (t === 'false') return false + return null +} + +/** Numeric value of an integer/float literal, else null. */ +export function numberLiteralValue(node: SyntaxNode | null | undefined, lang: SastLanguage): number | null { + if (!node) return null + const n = unwrap(node, lang) + if (!/literal|number|integer|float/.test(n.type)) return null + const v = Number(n.text.replace(/[_lLfFdD]/g, '')) + return Number.isFinite(v) ? v : null +} + +// ---- assignments ---------------------------------------------------------- + +export interface NAssign { + target: string + value: SyntaxNode +} + +/** + * All simple `variable ← expression` bindings inside a function body, NOT + * descending into nested function scopes. Multi-target assignments + * (`a, b := f()`, `x = y = z`) are expanded per target. Member/index targets + * (`obj.x = ...`) are ignored — the engine tracks locals only. + */ +export function collectAssignments(body: SyntaxNode, lang: SastLanguage): NAssign[] { + const out: NAssign[] = [] + const visit = (node: SyntaxNode) => { + if (node !== body && isFunctionNode(node, lang)) return // separate scope + for (const a of assignmentAt(node, lang)) out.push(a) + for (const c of node.namedChildren) visit(c) + } + visit(body) + return out +} + +function assignmentAt(node: SyntaxNode, lang: SastLanguage): NAssign[] { + const t = node.type + const pairTargets = (target: SyntaxNode | null, value: SyntaxNode | null): NAssign[] => { + if (!target || !value) return [] + const names = targetNames(target, lang) + // list ← list: align by index; otherwise each name gets the whole value. + const values = target.type.includes('list') || target.type === 'tuple' || target.type === 'expression_list' + ? valueList(value) + : null + if (values && values.length === names.length) { + return names.map((n, i) => ({ target: n, value: values[i] })) + } + return names.map((n) => ({ target: n, value })) + } + + switch (t) { + case 'variable_declarator': { // js/java/c# + const nameNode = field(node, 'name') + const value = field(node, 'value') ?? equalsValue(node) + if (!value) return [] + if (nameNode && PATTERN_TYPES.has(nameNode.type)) { + return patternLeafNames(nameNode, lang).map((target) => ({ target, value })) + } + const name = nameNode?.text ?? leafParamName(node) + return name ? [{ target: name.replace(/^\$/, ''), value }] : [] + } + case 'assignment_expression': // js/java/php/c# + case 'augmented_assignment_expression': { + const left = field(node, 'left') + const right = field(node, 'right') + if (!left || !right) return [] + if (PATTERN_TYPES.has(left.type)) { + return patternLeafNames(left, lang).map((target) => ({ target, value: right })) + } + const name = identifierName(left, lang) + return name ? [{ target: name, value: right }] : [] + } + case 'assignment': // python/ruby + case 'augmented_assignment': + case 'operator_assignment': { + const left = field(node, 'left') + const right = field(node, 'right') + return pairTargets(left, right) + } + case 'short_var_declaration': // go := + case 'assignment_statement': { // go = + const left = field(node, 'left') + const right = field(node, 'right') + return pairTargets(left, right) + } + case 'var_spec': { // go var x = ... + const value = field(node, 'value') + const names = node.namedChildren.filter((c) => c.type === 'identifier').map((c) => c.text) + if (!value) return [] + const values = valueList(value) + return names.map((n, i) => ({ target: n, value: values[i] ?? value })) + } + case 'let_declaration': { // rust + const pat = field(node, 'pattern') + const value = field(node, 'value') + const name = pat ? identifierName(pat, lang) ?? (pat.type === 'identifier' ? pat.text : null) : null + return name && value ? [{ target: name, value }] : [] + } + default: + return [] + } +} + +/** C# variable_declarator stores its initializer under an `equals_value_clause`. */ +function equalsValue(node: SyntaxNode): SyntaxNode | null { + const clause = node.namedChildren.find((c) => c.type === 'equals_value_clause') + if (clause) { + const val = clause.namedChildren[clause.namedChildren.length - 1] + if (val) return val + } + const eq = node.children.findIndex((c) => c.type === '=') + if (eq !== -1) { + for (let i = eq + 1; i < node.childCount; i++) { + const c = node.child(i) + if (c && c.isNamed) return c + } + } + return null +} + +/** Destructuring pattern node types (JS/TS). */ +const PATTERN_TYPES = new Set([ + 'object_pattern', + 'array_pattern', + 'object_assignment_pattern', + 'assignment_pattern', + 'rest_pattern', + 'pair_pattern', +]) + +/** + * Identifiers bound by a destructuring pattern: `{ id }`, `{ q: search }`, + * `[first]`, `{ a = 1 }`, `{ ...rest }`. + * + * Each leaf is bound to the WHOLE right-hand side — the same conservative + * over-approximation `pairTargets` already makes when target and value arities + * differ. Without this, `const { id } = req.query` keys the taint map on the + * literal text "{ id }", so every injection rule goes blind on the idiomatic + * Next.js/Express handler shape. + */ +function patternLeafNames(node: SyntaxNode, lang: SastLanguage, depth = 0): string[] { + if (depth > 6) return [] + if (node.type === 'identifier' || node.type === 'shorthand_property_identifier_pattern') { + return [node.text.replace(/^\$/, '')] + } + if (!PATTERN_TYPES.has(node.type)) { + const direct = identifierName(node, lang) + return direct ? [direct] : [] + } + const names: string[] = [] + for (const child of node.namedChildren) { + if (child.type === 'pair_pattern') { + // `{ q: search }` binds the VALUE side, not the key. + const bound = field(child, 'value') ?? child.namedChildren[child.namedChildren.length - 1] + if (bound) names.push(...patternLeafNames(bound, lang, depth + 1)) + continue + } + if (child.type === 'object_assignment_pattern' || child.type === 'assignment_pattern') { + // `{ a = fallback }` binds the LEFT side; the default is not the binding. + const bound = field(child, 'left') ?? child.namedChildren[0] + if (bound) names.push(...patternLeafNames(bound, lang, depth + 1)) + continue + } + names.push(...patternLeafNames(child, lang, depth + 1)) + } + return names +} + +function targetNames(target: SyntaxNode, lang: SastLanguage): string[] { + const direct = identifierName(target, lang) + if (direct) return [direct] + if (target.type === 'identifier') return [target.text] + // tuple/list/expression_list of identifiers + const names: string[] = [] + for (const c of target.namedChildren) { + const n = identifierName(c, lang) ?? (c.type === 'identifier' ? c.text : null) + if (n) names.push(n) + } + return names +} + +function valueList(value: SyntaxNode): SyntaxNode[] { + if (value.type === 'expression_list' || value.type === 'tuple' || value.type.includes('list')) { + return value.namedChildren.slice() + } + return [value] +} + +// ---- generic walk --------------------------------------------------------- + +export function walk(root: SyntaxNode, visit: (n: SyntaxNode) => void, maxNodes = 400_000): void { + let count = 0 + const stack: SyntaxNode[] = [root] + while (stack.length) { + const n = stack.pop()! + if (++count > maxNodes) return + visit(n) + for (let i = n.childCount - 1; i >= 0; i--) { + const c = n.child(i) + if (c) stack.push(c) + } + } +} diff --git a/packages/analyzer-engine/src/security/rmw.ts b/packages/analyzer-engine/src/security/rmw.ts new file mode 100644 index 0000000..70ca942 --- /dev/null +++ b/packages/analyzer-engine/src/security/rmw.ts @@ -0,0 +1,258 @@ +import type { SastLanguage, SyntaxNode } from './lang' +import { asCall, asFunction, asMember, dottedName, isFunctionNode, unwrap, type NCall } from './normalize' + +/** + * Read-modify-write race detection (CWE-362) — the one concurrency shape a + * file-local AST rule can prove. + * + * This lives outside rules.ts because it is the only rule that reasons across + * STATEMENTS rather than over one node: it pairs a read with a later write in + * the same function scope. Keeping the pair search here also keeps the rule + * pack readable — rules.ts only carries the metadata and the receiver gate. + * + * The predicate is deliberately narrow. A measured probe over this repo plus + * five sweep repos found that the loose shape (any read followed by a write to + * the same model) yields 20 pairs of which 17 are benign — CAS updates, + * transactional claims, unrelated writes. Requiring the write's payload to be + * ARITHMETIC on a field of the read result is the entire difference between a + * useless rule and a clean one, so every clause below is load-bearing: + * + * 1. `const V = await ..find*({ where: K })` — awaited, bound + * 2. `..update({ where: K', data: D })` — later, same scope + * 3. K ≡ K' — same row, keyed identically + * 4. D contains `V. …` — the write recomputes what it read + * 5. no transaction / tx receiver / row lock around the pair + * + * Clause 4 is also what keeps the FIX silent: the atomic operator idiom + * (`data: { balance: { increment: amount } }`) is an object, never a binary + * arithmetic expression, so it can never match. Likewise a last-write-wins + * overwrite (`data: { name }`) carries no arithmetic and never fires. + */ + +const lc = (s: string | null | undefined) => (s ?? '').toLowerCase() + +/** Reads that return a single record whose fields can be recomputed. */ +const READ_METHODS = new Set([ + 'findunique', + 'finduniqueorthrow', + 'findfirst', + 'findfirstorthrow', + 'findone', +]) +/** The update family. `create` cannot lose an update; `delete` writes no value. */ +const WRITE_METHODS = new Set(['update', 'updatemany']) +/** Transaction wrappers: inside one, the read and the write are already atomic. */ +const TX_METHODS = new Set(['transaction', 'runintransaction', 'withtransaction', 'intransaction', 'transact']) +/** Transaction-callback receivers (`tx.invoice.update`) — the same knowledge + * db-call-in-loop leans on, stated here because a `tx` receiver also passes + * the rule pack's DB_RECEIVER when it appears as `db.tx` or `this.trx`. */ +const TX_RECEIVER = /(^|[._])(tx|trx)([._]|$)/i +/** Explicit row locking in raw SQL anywhere in the enclosing scope. */ +const ROW_LOCK = /\bfor\s+update\b|advisory_?(xact_)?lock/i +/** Binary operators that recompute a value from what was read. */ +const ARITHMETIC_OPS = new Set(['+', '-', '*', '/', '%']) + +/** Structurally a `PatternHit` — declared locally so this module never has to + * import from rules.ts (that edge would close an import cycle). */ +export interface RmwHit { + node: SyntaxNode + line: number + detail: string +} + +/** + * Report the read of an unguarded read-modify-write pair, anchored on the + * binding — that is where a reviewer starts reading, and the detail names the + * write line. `dbReceiver` is the rule pack's DB_RECEIVER, passed in rather + * than imported for the cycle reason above. + */ +export function readModifyWriteHits( + node: SyntaxNode, + lang: SastLanguage, + dbReceiver: RegExp, +): RmwHit[] { + if (node.type !== 'variable_declarator') return [] + const nameNode = node.childForFieldName('name') + if (nameNode?.type !== 'identifier') return [] + const binding = nameNode.text + // Awaited only: an un-awaited read binds a promise, so nothing downstream can + // read a field off it and write it back. + const value = node.childForFieldName('value') + if (value?.type !== 'await_expression') return [] + const read = asCall(unwrap(value, lang), lang) + if (!read || read.isConstruct) return [] + if (!READ_METHODS.has(lc(read.method))) return [] + // `db.invoice` — the receiver identifies both the client and the model, so + // requiring the write to carry the SAME receiver is the same-model test. + const target = read.receiverName + if (!target || !dbReceiver.test(target) || TX_RECEIVER.test(target)) return [] + const readWhere = whereShape(read, lang) + if (!readWhere) return [] + if (enclosedByTransaction(node, lang)) return [] + + const scope = enclosingScope(node, lang) + if (!scope) return [] + for (const write of laterWrites(scope, read.node, lang)) { + if (write.receiverName !== target) continue + const arg = write.args[0] + if (!arg || unwrap(arg, lang).type !== 'object') continue + const payload = objectValue(unwrap(arg, lang), 'data') + const writeWhere = objectValue(unwrap(arg, lang), 'where') + if (!payload || !writeWhere) continue + // Same row, keyed identically. An extra key in the write's where is a + // compare-and-set guard (`{ id, status: 'PROPOSED' }`), not a blind write. + const shape = keyShape(unwrap(writeWhere, lang)) + if (!shape || !sameShape(readWhere, shape)) continue + const source = arithmeticOnBinding(unwrap(payload, lang), binding, lang) + if (!source) continue + if (ROW_LOCK.test(scope.text)) return [] + return [ + { + node: read.node, + line: read.line, + detail: `${write.fullName} (line ${write.line}) recomputes ${source.field} from ${source.ref}`, + }, + ] + } + return [] +} + +/** `{ where: { … } }` of a call's first argument, normalized to key → value. */ +function whereShape(call: NCall, lang: SastLanguage): Map | null { + const arg = call.args[0] + if (!arg) return null + const obj = unwrap(arg, lang) + if (obj.type !== 'object') return null + const where = objectValue(obj, 'where') + return where ? keyShape(unwrap(where, lang)) : null +} + +/** Value node of an object literal's `key:` property, or null. */ +function objectValue(obj: SyntaxNode, key: string): SyntaxNode | null { + if (obj.type !== 'object') return null + for (const child of obj.namedChildren) { + if (child.type !== 'pair') continue + const k = child.childForFieldName('key')?.text?.replace(/['"]/g, '') + if (k === key) return child.childForFieldName('value') + } + return null +} + +/** + * An object literal's keys mapped to their whitespace-stripped value text, so + * `{ id }` and `{ id: id }` compare equal. Returns null for anything not + * comparable by text (a spread, a computed key, an empty object) — an + * unprovable key match must never count as a match. + */ +function keyShape(obj: SyntaxNode): Map | null { + if (obj.type !== 'object') return null + const out = new Map() + for (const child of obj.namedChildren) { + if (child.type === 'shorthand_property_identifier') { + out.set(child.text, child.text) + continue + } + if (child.type !== 'pair') return null + const k = child.childForFieldName('key')?.text?.replace(/['"]/g, '') + const v = child.childForFieldName('value')?.text + if (!k || v === undefined) return null + out.set(k, v.replace(/\s+/g, '')) + } + return out.size > 0 ? out : null +} + +function sameShape(a: Map, b: Map): boolean { + if (a.size !== b.size) return false + for (const [k, v] of a) if (b.get(k) !== v) return false + return true +} + +/** + * A binary arithmetic expression inside the write payload whose operand IS a + * field access on the read binding (`balance: inv.balance + amount`). The + * operand must be the member access itself: a value derived through a call + * (`inv.createdAt.getTime() + 1000`) is not the lost-update shape. + */ +function arithmeticOnBinding( + data: SyntaxNode, + binding: string, + lang: SastLanguage, +): { field: string; ref: string } | null { + const found: Array<{ field: string; ref: string }> = [] + const visit = (n: SyntaxNode) => { + if (found.length > 0) return + if (n.type === 'binary_expression' && ARITHMETIC_OPS.has(n.childForFieldName('operator')?.text ?? '')) { + const left = n.childForFieldName('left') + const right = n.childForFieldName('right') + const ref = + (left ? bindingField(left, binding, lang) : null) ?? + (right ? bindingField(right, binding, lang) : null) + if (ref) { + found.push({ field: writtenField(n) ?? ref.slice(ref.lastIndexOf('.') + 1), ref }) + return + } + } + for (const c of n.namedChildren) visit(c) + } + visit(data) + return found[0] ?? null +} + +/** Dotted name of a field access rooted at `binding`, or null. */ +function bindingField(operand: SyntaxNode, binding: string, lang: SastLanguage): string | null { + const n = unwrap(operand, lang) + if (!asMember(n, lang)) return null + const name = dottedName(n, lang) + return name && name.startsWith(`${binding}.`) ? name : null +} + +/** The payload key the arithmetic lands in, for the finding's detail line. */ +function writtenField(from: SyntaxNode): string | null { + let cur: SyntaxNode | null = from.parent + for (let i = 0; i < 8 && cur; i++) { + if (cur.type === 'pair') return cur.childForFieldName('key')?.text?.replace(/['"]/g, '') ?? null + cur = cur.parent + } + return null +} + +/** True when any enclosing call is a transaction wrapper. */ +function enclosedByTransaction(from: SyntaxNode, lang: SastLanguage): boolean { + let cur: SyntaxNode | null = from.parent + for (let i = 0; i < 80 && cur; i++) { + const call = asCall(cur, lang) + if (call && TX_METHODS.has(lc(call.method).replace(/^\$/, ''))) return true + cur = cur.parent + } + return false +} + +/** Body of the function the node lives in, or the program root at module scope. */ +function enclosingScope(from: SyntaxNode, lang: SastLanguage): SyntaxNode | null { + let cur: SyntaxNode | null = from.parent + for (let i = 0; i < 80 && cur; i++) { + if (isFunctionNode(cur, lang)) return asFunction(cur, lang)?.body ?? null + if (!cur.parent) return cur + cur = cur.parent + } + return null +} + +/** + * Update-family calls in the same scope that start after the read ends. Nested + * function scopes are not descended into: a write inside a callback runs under + * a different ordering than the statement sequence this rule reasons about. + */ +function laterWrites(scope: SyntaxNode, read: SyntaxNode, lang: SastLanguage): NCall[] { + const out: NCall[] = [] + const visit = (n: SyntaxNode) => { + if (n !== scope && isFunctionNode(n, lang)) return + if (n.startIndex >= read.endIndex) { + const call = asCall(n, lang) + if (call && !call.isConstruct && WRITE_METHODS.has(lc(call.method))) out.push(call) + } + for (const c of n.namedChildren) visit(c) + } + visit(scope) + return out +} diff --git a/packages/analyzer-engine/src/security/rules.ts b/packages/analyzer-engine/src/security/rules.ts new file mode 100644 index 0000000..e7cee39 --- /dev/null +++ b/packages/analyzer-engine/src/security/rules.ts @@ -0,0 +1,1721 @@ +import type { SastLanguage, SyntaxNode } from './lang' +import { CWE } from './cwe' +import type { Severity } from './types' +import { + asCall, + asFunction, + asMember, + boolLiteralValue, + collectAssignments, + concatOperands, + dottedName, + identifierName, + interpolationExprs, + isFunctionNode, + numberLiteralValue, + stringLiteralValue, + unwrap, + walk, + type NCall, +} from './normalize' +import { readModifyWriteHits } from './rmw' + +/** + * The curated, high-precision rule pack. + * + * Two rule shapes share the CWE/OWASP metadata surface: + * - {@link TaintSink}: a dangerous call whose flagged argument MUST be taint- + * influenced. This is what makes injection rules precise — a constant or + * parameterized argument produces no origins, so no finding. + * - {@link PatternRule}: a dangerous API or insecure configuration that is a + * finding by its mere presence (weak hash, disabled TLS, deserialization of + * an inherently unsafe format). No dataflow to show — hence no flow in the + * finding, only the sink site. + */ + +export interface RuleMeta { + id: string + cweKey: keyof typeof CWE + severity: Severity + /** Languages this rule applies to; null = all covered languages. */ + languages: Set | null + title: string + message: string + remediation: string +} + +export interface TaintSink extends RuleMeta { + /** + * If this call is a sink, return the argument indexes whose taint makes it a + * finding (any one tainted → report). Return null if the call is not this + * sink. An empty array means "reached at all is dangerous" (rare). + */ + match(call: NCall, lang: SastLanguage): number[] | null + /** + * 'head' = only taint that can steer the START of the argument (a URL's + * scheme/authority) is dangerous; the engine suppresses taint confined to + * later path/query segments of a constant-authority string. + */ + taintPosition?: 'head' +} + +export interface PatternHit { + node: SyntaxNode + line: number + detail?: string +} + +export interface PatternRule extends RuleMeta { + /** Skip the rule entirely for files whose path matches (seed/migration/ops + * directories where the pattern is expected and harmless). */ + excludePath?: RegExp + /** Inspect one AST node; return hits (usually 0 or 1). */ + test(node: SyntaxNode, lang: SastLanguage): PatternHit[] +} + +const ALL = null + +// --------------------------------------------------------------------------- +// helpers +// --------------------------------------------------------------------------- + +const lc = (s: string | null | undefined) => (s ?? '').toLowerCase() + +/** Receivers that denote a database connection/cursor/query builder. */ +const DB_RECEIVER = /(^|[._])(db|conn|connection|client|pool|knex|sequelize|prisma|pg|mysql|sqlite|cursor|session|stmt|statement|sql|repository|dao|orm)/i +/** Method names that always execute SQL. */ +/** Receivers that denote Node's child_process module (child_process.exec, + * childProcess.exec, cp.exec) rather than a RegExp. */ +const CHILD_PROCESS_RECEIVER = /(^|[._])(child_?process|cp|proc|shell)$/i + +const SQL_ALWAYS = new Set(['query', 'raw', 'unsafe', 'executequery', 'executeupdate', 'executesql', 'rawquery']) +/** Method names that execute SQL only on a DB-ish receiver. */ +const SQL_GATED = new Set(['execute', 'exec']) + +// NOTE: generic receiver names like `client`/`session` are deliberately NOT here. +// They match Redis/DB/gRPC/GraphQL clients (`client.get(id)`), which are not HTTP +// requests — including them fires SSRF on cache/DB reads (false positives). +const HTTP_CLIENTS = new Set([ + 'axios', 'http', 'https', 'requests', 'got', 'superagent', 'fetch', + 'httpclient', 'urllib', 'urllib.request', 'node-fetch', 'undici', 'webclient', +]) + +/** A keyword/option argument `name=value` present among a call's args (py/js). */ +function findOption(call: NCall, lang: SastLanguage, name: string): SyntaxNode | null { + const want = name.toLowerCase() + for (const a of call.args) { + // python keyword_argument + if (a.type === 'keyword_argument') { + const kn = a.childForFieldName('name')?.text + if (lc(kn) === want) return a.childForFieldName('value') + } + // js object option: look one level into object literals for a matching pair + if (a.type === 'object') { + for (const pair of a.namedChildren) { + if (pair.type !== 'pair') continue + const kn = pair.childForFieldName('key')?.text?.replace(/['"]/g, '') + if (lc(kn) === want) return pair.childForFieldName('value') + } + } + } + return null +} + +function optionIsTrue(call: NCall, lang: SastLanguage, name: string): boolean { + const v = findOption(call, lang, name) + return v ? boolLiteralValue(v, lang) === true : false +} + +// --------------------------------------------------------------------------- +// TAINT SINKS +// --------------------------------------------------------------------------- + +export const TAINT_SINKS: TaintSink[] = [ + // ---- SQL injection ---- + { + id: 'sql-injection', + cweKey: 'SQLI', + severity: 'CRITICAL', + languages: ALL, + title: 'SQL injection', + message: 'Untrusted input is concatenated into a SQL query and executed. An attacker can alter the query to read or modify arbitrary data.', + remediation: 'Use parameterized queries / prepared statements and pass user input as bound parameters, never string concatenation.', + match(call) { + const m = lc(call.method) + if (SQL_ALWAYS.has(m)) return [0] + if (SQL_GATED.has(m) && DB_RECEIVER.test(call.receiverName ?? '')) return [0] + // Go database/sql, gated on a DB-ish receiver. Context variants take + // (ctx, query, ...args) so the SQL string is argument 1, not 0. + if (DB_RECEIVER.test(call.receiverName ?? '')) { + if (m === 'queryrow') return [0] + if (m === 'querycontext' || m === 'queryrowcontext' || m === 'execcontext') return [1] + } + // Prisma raw escape hatches. Only the Unsafe variants take a plain SQL + // string — tagged $queryRaw/$executeRaw templates are parameterized by + // Prisma and must NOT flag. (asCall strips the leading $, so match both.) + if (/^\$?(query|execute)rawunsafe$/.test(m)) return [0] + // C#/Java: new SqlCommand(sql) / new Statement(sql) + if (call.isConstruct && /sqlcommand|npgsqlcommand|mysqlcommand|oledbcommand/i.test(call.fullName)) return [0] + return null + }, + }, + + // ---- Command injection ---- + { + id: 'command-injection', + cweKey: 'CMDI', + severity: 'CRITICAL', + languages: ALL, + title: 'OS command injection', + message: 'Untrusted input flows into a shell command. An attacker can inject additional commands and run arbitrary code on the host.', + remediation: 'Avoid the shell: pass an argument array to execFile/spawn/subprocess without shell=True, and validate against an allow-list.', + match(call, lang) { + const m = lc(call.method) + const recv = lc(call.receiverName) + // Node child_process shell APIs. In JS a bare `.exec()` on an arbitrary + // receiver is overwhelmingly RegExp.prototype.exec (`/re/.exec(s)`, + // minified `S.exec(a)`), so require a child_process-shaped receiver or a + // destructured import. Other languages have no regex `.exec` idiom and + // keep the broader gate. + if (m === 'exec' || m === 'execsync') { + if (lang === 'javascript' || lang === 'typescript' || lang === 'tsx') { + // Require positive evidence of child_process. An `exec` reached + // through ANY member access whose object is not child_process-shaped + // (`/re/.exec(s)`, `pattern.exec(s)`, minified `S.exec(a)`) is + // RegExp.prototype.exec. Only a bare callee — the destructured + // `const { exec } = require('child_process')` — flags without a name. + const memberCallee = call.callee ? asMember(call.callee, lang) : null + if (!memberCallee || CHILD_PROCESS_RECEIVER.test(recv)) return [0] + // Decided: a JS `.exec` reached through a non-child_process object is + // RegExp.prototype.exec. Return rather than fall through — the PHP + // branch below keys off a null receiverName, which a regex LITERAL + // receiver also has. + return null + } + if (!DB_RECEIVER.test(recv)) return [0] + } + if ((m === 'spawn' || m === 'spawnsync' || m === 'execfile') && optionIsTrue(call, lang, 'shell')) return [0] + // Python + if (recv === 'os' && (m === 'system' || m === 'popen')) return [0] + if (/subprocess/.test(recv) && (m === 'call' || m === 'run' || m === 'popen' || m === 'check_output' || m === 'check_call')) { + return optionIsTrue(call, lang, 'shell') ? [0] : null + } + // Java Runtime.exec / new ProcessBuilder + if (m === 'exec' && /runtime/.test(recv)) return [0] + if (call.isConstruct && /processbuilder/i.test(call.fullName)) return call.args.map((_, i) => i) + // C# Process.Start + if (m === 'start' && /process/.test(recv)) return [0] + // Go exec.Command + if (m === 'command' && recv === 'exec') return call.args.map((_, i) => i) + // PHP + if (['system', 'exec', 'shell_exec', 'passthru', 'proc_open'].includes(m) && !call.receiverName) return [0] + return null + }, + }, + + // ---- Path traversal ---- + { + id: 'path-traversal', + cweKey: 'PATH', + severity: 'HIGH', + languages: ALL, + title: 'Path traversal', + message: 'Untrusted input is used as a filesystem path without normalization, allowing access to files outside the intended directory (e.g. via "../").', + remediation: 'Normalize and confine the path (path.basename / realpath) and verify it stays within an allowed base directory before opening.', + match(call) { + const m = lc(call.method) + const recv = lc(call.receiverName) + const FS_METHODS = new Set([ + 'readfile', 'readfilesync', 'writefile', 'writefilesync', 'appendfile', 'createreadstream', + 'createwritestream', 'open', 'opensync', 'unlink', 'readdir', 'readalltext', 'readallbytes', + 'openread', 'openwrite', 'readalllines', + ]) + if (FS_METHODS.has(m) && !/^(process|require)$/.test(recv)) return [0] + if (call.isConstruct && /(^|\.)file$|filestream|fileinfo|streamreader/i.test(call.fullName)) return [0] + return null + }, + }, + + // ---- SSRF ---- + { + id: 'ssrf', + cweKey: 'SSRF', + severity: 'HIGH', + languages: ALL, + title: 'Server-side request forgery (SSRF)', + message: 'A user-controlled URL is passed to an HTTP client, letting an attacker make the server issue requests to internal or arbitrary hosts.', + remediation: 'Validate the URL against a strict allow-list of hosts/schemes and resolve/deny internal addresses before making the request.', + // SSRF requires control of the request TARGET — taint in a path segment of + // a constant-authority URL cannot redirect the request to another host. + taintPosition: 'head', + match(call) { + const m = lc(call.method) + const recv = lc(call.receiverName) + if (m === 'fetch' && !recv) return [0] + if (m === 'urlopen') return [0] + if ((m === 'get' || m === 'post' || m === 'request' || m === 'put' || m === 'delete') && HTTP_CLIENTS.has(recv)) return [0] + if (m === 'openconnection') return [0] + return null + }, + }, + + // ---- Code injection (eval) & reflection ---- + { + id: 'code-injection', + cweKey: 'CODEI', + severity: 'CRITICAL', + languages: ALL, + title: 'Code injection via eval', + message: 'Untrusted input is passed to a dynamic code-evaluation / reflection primitive, allowing arbitrary code execution.', + remediation: 'Never evaluate untrusted input. Replace eval/exec/Function with an explicit parser or a safe lookup table.', + match(call) { + const m = lc(call.method) + const recv = lc(call.receiverName) + if (!recv && ['eval', 'exec', 'compile', 'execscript', 'assert', 'create_function'].includes(m)) return [0] + if (call.isConstruct && /^function$/i.test(call.fullName)) return [0] // new Function(str) + if (m === 'forname' && /class/.test(recv)) return [0] // Java reflection + if (['instance_eval', 'class_eval', 'module_eval'].includes(m)) return [0] + return null + }, + }, + + // ---- Open redirect ---- + { + id: 'open-redirect', + cweKey: 'OPENREDIR', + severity: 'MEDIUM', + languages: ALL, + title: 'Open redirect', + message: 'A user-controlled value is used as a redirect target, enabling phishing by redirecting victims to attacker-chosen sites.', + remediation: 'Redirect only to a fixed set of allowed paths, or validate the target against an allow-list of hosts.', + // A redirect is attacker-steerable only when taint can reach the scheme or + // authority. Taint confined to a path segment of an origin-relative target + // (`/users/`) cannot send the victim off-origin. + taintPosition: 'head', + match(call) { + const m = lc(call.method) + if (m === 'redirect' || m === 'sendredirect') return [0] + return null + }, + }, + + // ---- ReDoS (user-controlled regex) ---- + { + id: 'redos', + cweKey: 'REDOS', + severity: 'MEDIUM', + languages: ALL, + title: 'User-controlled regular expression (ReDoS)', + message: 'A regular expression is built from untrusted input; a crafted pattern can cause catastrophic backtracking and hang the process.', + remediation: 'Do not build regexes from user input. If unavoidable, use a linear-time regex engine (RE2) or strictly validate the pattern.', + match(call) { + const m = lc(call.method) + const recv = lc(call.receiverName) + if (call.isConstruct && /^regexp$/i.test(call.fullName)) return [0] + if (m === 'compile' && /^re$/.test(recv)) return [0] // python re.compile + return null + }, + }, + + // ---- PHP unserialize (insecure deserialization, tainted) ---- + { + id: 'php-object-injection', + cweKey: 'DESER', + severity: 'HIGH', + languages: new Set(['php']), + title: 'PHP object injection via unserialize()', + message: 'unserialize() on untrusted input can instantiate arbitrary objects and trigger magic methods, leading to code execution.', + remediation: 'Use json_decode() for untrusted data, or unserialize() with a strict allowed_classes list.', + match(call) { + if (lc(call.method) === 'unserialize' && !call.receiverName) return [0] + return null + }, + }, + + // ---- Unbounded allocation from an untrusted length (CardShopCoop Msg.cs) ---- + { + id: 'unbounded-read', + cweKey: 'ALLOC', + severity: 'MEDIUM', + languages: new Set(['csharp', 'java']), + title: 'Unbounded allocation from untrusted length', + message: 'A length read off an untrusted stream is used to allocate/read a buffer with no bound. A hostile peer can send a huge length to exhaust memory (DoS), the core of untrusted binary-protocol deserialization bugs.', + remediation: 'Validate the declared length against a sane maximum before allocating or calling ReadBytes; reject oversized frames.', + match(call) { + const m = lc(call.method) + // Include the idiomatic `br`/`rdr` reader abbreviations, not just receivers + // literally named reader/stream (CardShopCoop `var br = Msg.Reader(...)`). + if (m === 'readbytes' && /reader|stream|buffer|^(br|rdr)$/.test(lc(call.receiverName))) return [0] + return null + }, + }, + + // ---- Decompression of untrusted compressed data (zip/gzip bomb) ---- + { + id: 'decompression-bomb', + cweKey: 'DECOMP', + severity: 'MEDIUM', + languages: new Set(['csharp', 'java', 'javascript', 'typescript', 'tsx']), + title: 'Decompression of untrusted compressed data', + message: 'Attacker-influenced compressed data is decompressed with no size cap. A crafted highly-compressible payload (zip/gzip bomb) expands to exhaust memory or disk (DoS).', + remediation: 'Cap the decompressed size (bounded copy / max output length) and reject inputs whose expansion ratio or absolute size exceeds a sane limit before decompressing.', + match(call) { + const m = lc(call.method) + if (['gunzip', 'ungzip', 'inflate', 'decompress', 'unzip', 'inflatesync', 'gunzipsync'].includes(m)) return [0] + // C#/Java decompression streams: new GZipStream(src, Decompress) / new GZIPInputStream(src) + if (call.isConstruct && /gzipstream|deflatestream|brotlistream|gzipinputstream|inflaterinputstream|zipinputstream/i.test(call.fullName)) { + return [0] + } + return null + }, + }, +] + +// --------------------------------------------------------------------------- +// PATTERN RULES +// --------------------------------------------------------------------------- + +/** Extract a `name = value` / `name: value` / `name=value` binding from a node. */ +export function asNamedValue( + node: SyntaxNode, + lang: SastLanguage, +): { name: string; value: SyntaxNode; node: SyntaxNode } | null { + const t = node.type + const clean = (s: string | undefined | null) => lc(s).replace(/^[$'"]+|['"]+$/g, '') + const f = (name: string) => node.childForFieldName(name) + + if (t === 'pair') { + const key = f('key') + const value = f('value') + if (key && value) return { name: clean(key.text), value, node } + } + if (t === 'keyword_argument') { + const value = f('value') + if (value) return { name: clean(f('name')?.text), value, node } + } + if (t === 'keyed_element') { + // go composite literal field + const key = f('key') ?? node.namedChildren[0] + const value = f('value') ?? node.namedChildren[node.namedChildren.length - 1] + if (key && value && key !== value) return { name: clean(key.text), value, node } + } + if (t === 'variable_declarator') { + const nameNode = f('name') + const value = f('value') + if (nameNode && value) return { name: clean(nameNode.text), value, node } + } + if (t === 'assignment_expression' || t === 'assignment') { + const left = f('left') + const right = f('right') + if (left && right) { + const name = identifierName(left, lang) ?? dottedName(left, lang) + const last = name ? name.slice(name.lastIndexOf('.') + 1) : null + if (last) return { name: clean(last), value: right, node } + } + } + return null +} + +/** Climb parents looking for a binding whose name matches `re`. */ +function nameContext(node: SyntaxNode, lang: SastLanguage, re: RegExp): string | null { + let cur: SyntaxNode | null = node + for (let i = 0; i < 5 && cur; i++) { + const nv = asNamedValue(cur, lang) + if (nv && re.test(nv.name)) return nv.name + cur = cur.parent + } + return null +} + +/** Climb parents looking for an enclosing call whose method matches `re`. */ +function callContext(node: SyntaxNode, lang: SastLanguage, re: RegExp): boolean { + let cur: SyntaxNode | null = node + for (let i = 0; i < 6 && cur; i++) { + const call = asCall(cur, lang) + if (call && re.test(call.method)) return true + cur = cur.parent + } + return false +} + +// ---- weak-hash purpose context -------------------------------------------- +/** Non-security purposes for an MD5/SHA-1 hash: content-addressing, dedup, ETag. */ +const NONSEC_HASH_PURPOSE = /fingerprint|dedup|duplicat|etag|cache|checksum|identity|shard|bucket|content.?hash|node.?id|filename/i +/** A variable literally named for a generic content hash (not a security digest). */ +const GENERIC_HASH_NAME = /^(sha|sha1|hash|digest|etag|checksum|fingerprint|cachekey|contenthash|nodeid)$/i +/** Words that mark a hash as security-relevant — never suppress these. */ +const SECURITY_HASH_WORD = /password|passwd|secret|token|sign|hmac|cert|credential|\bauth\b|private|integrity/i + +/** + * True when an MD5/SHA-1 hash is used for a non-security purpose (fingerprinting, + * dedup, ETag, content identity), inferred from the enclosing variable/function + * names. A single overriding security word (password, signature, hmac, …) keeps + * the finding. This kills the clean-repo weak-hash false positives without + * requiring a positive security signal on the true-positive cases. + */ +function isNonSecurityHash(node: SyntaxNode, lang: SastLanguage): boolean { + let cur: SyntaxNode | null = node + let purpose = false + for (let i = 0; i < 12 && cur; i++) { + const nv = asNamedValue(cur, lang) + if (nv) { + if (SECURITY_HASH_WORD.test(nv.name)) return false + if (NONSEC_HASH_PURPOSE.test(nv.name) || GENERIC_HASH_NAME.test(nv.name)) purpose = true + } + const fn = asFunction(cur, lang) + if (fn) { + if (SECURITY_HASH_WORD.test(fn.name)) return false + if (NONSEC_HASH_PURPOSE.test(fn.name)) purpose = true + } + cur = cur.parent + } + return purpose +} + +const SECRET_NAME = /(api[_-]?key|access[_-]?key|secret[_-]?key|client[_-]?secret|auth[_-]?token|access[_-]?token|private[_-]?key|bearer[_-]?token|encryption[_-]?key)/i +const CRED_PLACEHOLDER = /(example|placeholder|your[_-]?|xxx|changeme|dummy|test|fake|sample|<[^>]+>|\$\{|process\.env|os\.getenv|getenv)/i +/** + * Values that are a NAME/label, not a secret: an HTTP header name (`x-api-key`), + * an OAuth field/param name (`client_secret`), or a bare key prefix constant + * (`ct_live_`). Real credentials carry entropy (digits / mixed case); these are + * lowercase dictionary tokens or `x-…` header names. Excluding them kills the + * `API_KEY_PREFIX = 'ct_live_'` / `authToken = 'x-auth-token'` false positives. + */ +// Kebab/dotted segments are label shapes too (`sb-access-token`, +// `next-auth.session-token`). Digits stay out of those segments so a real key +// like `sk_live_abc1234567890abcdef` is still reported. +const CRED_LABEL_VALUE = /^(x-[a-z0-9-]+|[a-z][a-z_]*|[a-z]+([-._][a-z]+)*)$/i + +/** + * Documentation/API endpoint constants often have credential-shaped names + * (`*_ACCESS_TOKENS`, `*_PRIVATE_KEYS`) but their values are ordinary public + * URLs. Suppress only URLs with no embedded credentials, query, fragment, or + * token-looking path segment. Signed URLs, webhook secrets, and other + * high-entropy URL credentials therefore remain findings. + */ +/** + * A relative path is a destination, not a credential: dashboards are full of + * route and asset maps keyed by an entity enum (`API_KEY: '/settings/api-keys'`). + * Requiring a path separator keeps a bare token like `sk_live_…` reportable. + */ +const RELATIVE_PATH_VALUE = /^\.{0,2}\/[A-Za-z0-9._~-]+(\/[A-Za-z0-9._~-]+)*\/?$/ + +function isNonCredentialUrlLiteral(value: string): boolean { + if (RELATIVE_PATH_VALUE.test(value)) return true + if (!/^https?:\/\//i.test(value)) return false + try { + const url = new URL(value) + if (url.username || url.password || url.search || url.hash) return false + const tokenLikePath = url.pathname + .split('/') + .filter(Boolean) + .some((segment) => { + let decoded: string + try { + decoded = decodeURIComponent(segment) + } catch { + return true + } + if (decoded.length < 20) return false + const jwtParts = decoded.split('.') + if ( + decoded.length >= 40 && + jwtParts.length === 3 && + jwtParts.every((part) => part.length >= 8 && /^[A-Za-z0-9_-]+$/.test(part)) + ) { + return true + } + if (!/^[A-Za-z0-9_=-]+$/.test(decoded)) return false + return ( + (/[A-Za-z]/.test(decoded) && /\d/.test(decoded)) || + (/[a-z]/.test(decoded) && /[A-Z]/.test(decoded)) || + /^[A-Fa-f0-9]{20,}$/.test(decoded) || + /^[A-Z0-9_=-]{20,}$/.test(decoded) + ) + }) + return !tokenLikePath + } catch { + return false + } +} +// Only names that unambiguously denote a SECRET. Bare `key`/`session`/`iv`/`seed` +// are excluded: they overwhelmingly name non-secret values (React list keys, map +// keys, cache keys, session objects) and `iv`/`seed` even match as substrings of +// unrelated words (`private`, `seeded`). Compound secret names are kept. +const RNG_NAME = /(token|secret|nonce|salt|otp|password|passwd|apikey|privatekey|secretkey|encryptionkey|signingkey|sessionid|accesskey|credential)/i + +// ---- timing-unsafe secret comparison --------------------------------------- +const EQ_OPS = new Set(['==', '===', '!=', '!==']) +/** + * Names that unambiguously denote a secret VALUE being verified. Bare `token` + * is excluded: pagination/cancellation/CSRF-in-state tokens are compared with + * `===` legitimately all the time — compound *_token names are kept. + */ +const SECRET_COMPARE_NAME = + /(secret|password|passwd|signature|hmac|api[_-]?key|private[_-]?key|access[_-]?token|auth[_-]?token|session[_-]?token)/i + +/** Identifier continues past the secret word with an attribute segment + * (APIKeyAuthorityAdmin, apiKeyStatus): it names a property or category OF + * the secret, not the secret bytes. Plain continuations like SecretKey or + * passwordHash still denote the value itself and stay eligible. */ +const SECRET_ATTRIBUTE_REST = + /(authority|type|kind|status|state|name|prefix|preview|scope|mode|source|version|count|limit|role|level|owner|label|header|error|policy|expir)/i + +/** Leaf property names that dereference the actual secret bytes when only the + * RECEIVER is secret-named (secret.value, apiKey.hash). */ +const SECRET_VALUE_LEAF = /^(value|val|text|str|string|raw|data|bytes|content|plaintext|hash|digest)$/i + +/** The password-confirmation idiom: `password != confirm`, + * `form.password !== form.confirmPassword`. Both sides are values the same + * user just supplied, so timing reveals nothing they do not already know. */ +const CONFIRMATION_NAME = /(confirm|retype|repeat|again|re[_-]?enter)/i + +const PASSWORD_NAME = /pass(word|wd|phrase)?/i +/** Yup's sibling-field accessor (`context.parent.password`). Comparing a field + * to a sibling of the SAME form is client-side match validation, not secret + * verification — both values were typed by the same person. */ +const FORM_SIBLING_FIELD = /(^|\.)parent\.[a-z_$]/i +/** Password-change UX: `currentPassword === newPassword` compares two values + * from the same user's form, not a stored secret against input. */ +const SAME_USER_PASSWORD_PREFIX = /(^|[._-])(new|old|current|previous|prev|proposed)[_-]?pass/i + +/** A secret-named value referenced by this comparison operand (directly, or + * inside a template/concat like `Bearer ${secret}`). `secret.length`-style + * size accesses compare a NUMBER, not the secret bytes — never a hit. */ +function secretCompareSide(node: SyntaxNode, lang: SastLanguage, depth = 0): string | null { + if (depth > 4) return null + const n = unwrap(node, lang) + // A call RESULT is a transform (errorSignature(a) === errorSignature(b), + // hash(x) === hash(y)) — its name describes a function, not a stored secret + // being verified byte-by-byte. + if (asCall(n, lang)) return null + const name = dottedName(n, lang) ?? identifierName(n, lang) + if (name) { + const last = name.slice(name.lastIndexOf('.') + 1) + if (/^(length|size|count)$/i.test(last)) return null + // *Id / *_id names reference an identifier of the secret, not its bytes — + // `row.apiKeyId === req.query.apiKeyId` is an identity check. + if (/id$/i.test(last)) return null + // The compared VALUE is the final segment: `apiKey.Authority` compares an + // enum on a secret-named receiver, not the key itself. + const match = SECRET_COMPARE_NAME.exec(last) + if (match) { + const rest = last.slice(match.index + match[0].length) + return SECRET_ATTRIBUTE_REST.test(rest) ? null : last + } + if (SECRET_COMPARE_NAME.test(name) && SECRET_VALUE_LEAF.test(last)) return last + } + const parts = interpolationExprs(n, lang) ?? concatOperands(n, lang) + if (parts) { + for (const p of parts) { + const hit = secretCompareSide(p, lang, depth + 1) + if (hit) return hit + } + } + return null +} + +/** Presence/shape checks — comparing a secret to null/undefined/a literal/typeof + * is not a byte-by-byte verification, so timing leaks nothing useful. */ +function isTrivialCompareOperand(node: SyntaxNode, lang: SastLanguage): boolean { + const n = unwrap(node, lang) + if (stringLiteralValue(n, lang) !== null) return true + if (numberLiteralValue(n, lang) !== null) return true + if (boolLiteralValue(n, lang) !== null) return true + if (/^(null|undefined|none|nil)$/i.test(n.text)) return true + if (/^typeof\b/.test(n.text)) return true + return false +} + +/** + * Sinks reached by ASSIGNING to a named binding rather than by calling a + * function — `dangerouslySetInnerHTML={{__html: x}}` is a JSX pair and + * `el.innerHTML = x` an assignment, so neither is visible to a `match(call)` + * rule no matter how it is written. + */ +export interface TaintAssignSink extends RuleMeta { + /** True when this binding writes to the dangerous surface. */ + matchName(name: string): boolean + /** Sink-local safety: the assigned expression is already safe by construction. */ + safeValue?(value: SyntaxNode, lang: SastLanguage): boolean +} + +/** + * `JSON.stringify(x).replace(/ a.text).join(''))) return false + node = unwrap(call.receiver, lang) + call = asCall(node, lang) + } + return Boolean(call && /^json\.stringify$/i.test(call.fullName)) +} + +export const TAINT_ASSIGN_SINKS: TaintAssignSink[] = [ + { + id: 'xss', + cweKey: 'XSS', + severity: 'HIGH', + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Cross-site scripting (XSS)', + message: 'Untrusted input is written to a raw-HTML surface. A crafted value can inject script that runs with the victim\'s session.', + remediation: 'Render the value as text instead of HTML, or sanitize it (DOMPurify) before assigning. For JSON embedded in a script tag, serialize with JSON.stringify and escape "<".', + matchName: (name) => name === '__html' || name === 'innerhtml' || name === 'outerhtml', + safeValue: isEscapedJsonLd, + }, +] + +/** Build-tool prefixes that inline an env var into the CLIENT bundle. */ +const CLIENT_ENV_PREFIX = /^(NEXT_PUBLIC_|VITE_|REACT_APP_|EXPO_PUBLIC_|GATSBY_|PUBLIC_)/ +/** + * Server-only credential names, deliberately NARROW. Reusing the general + * SECRET_NAME regex here would fire on `NEXT_PUBLIC_GOOGLE_MAPS_API_KEY`, + * `NEXT_PUBLIC_MAPBOX_ACCESS_TOKEN` and `NEXT_PUBLIC_SUPABASE_ANON_KEY` — + * all of which are DESIGNED to reach the browser. Only names that have no + * legitimate client-side use belong here. + */ +const SERVER_ONLY_ENV_NAME = + /(SERVICE_ROLE|SECRET_KEY|CLIENT_SECRET|PRIVATE_KEY|WEBHOOK_SECRET|SESSION_SECRET|JWT_SECRET|NEXTAUTH_SECRET|ENCRYPTION_KEY|DATABASE_URL|CONNECTION_STRING|_PASSWORD|STRIPE_SECRET|ANTHROPIC_API_KEY|OPENAI_API_KEY|AWS_SECRET)/ + +// ---- un-awaited database write --------------------------------------------- +/** + * Receiver ROOT segments for the floating-write rule. Deliberately NOT + * DB_RECEIVER: its client/session/pool alternates are far too generic for a + * rule keyed on verbs as common as delete/update/create. + */ +const FLOATING_DB_ROOTS = new Set(['db', 'prisma', 'tx', 'trx', 'database']) +const ORM_WRITE_METHODS = new Set([ + 'create', 'createmany', 'createmanyandreturn', 'update', 'updatemany', 'upsert', + 'delete', 'deletemany', 'insertone', 'insertmany', 'updateone', 'replaceone', + 'deleteone', 'bulkwrite', 'findoneandupdate', 'findoneanddelete', 'findoneandreplace', +]) +/** Verbs that also exist on Map/Set/cookies — these need the object-literal + * argument guard (Prisma/Mongo delete filters are always object literals, + * `sessions.delete(key)` takes a bare key). */ +const DELETEISH_METHODS = new Set(['delete', 'deleteone', 'deletemany', 'findoneanddelete']) +const CONVEX_WRITE_METHODS = new Set(['insert', 'patch', 'replace', 'delete']) +/** A .then/.catch/.finally terminal means the promise IS handled. */ +const PROMISE_TERMINALS = new Set(['then', 'catch', 'finally']) +/** Raw SQL driver entry points (pg, mysql2 promise API, generic pools). */ +const RAW_QUERY_METHODS = new Set(['query', 'execute']) +/** First keyword of a statement that CHANGES data. */ +const SQL_WRITE_STATEMENT = /^\s*(insert|update|delete|upsert|merge|replace)\b/i + +/** + * True when the argument is a SQL string literal (plain or templated) whose + * first keyword writes. Only the constant HEAD is inspected, so an interpolated + * query still classifies without evaluating it. + */ +function isSqlWriteArgument(node: SyntaxNode | undefined, lang: SastLanguage): boolean { + if (!node) return false + const n = unwrap(node, lang) + if (n.type !== 'string' && n.type !== 'template_string') return false + const literal = stringLiteralValue(n, lang) + return SQL_WRITE_STATEMENT.test(literal ?? n.text.replace(/^[`'"]/, '')) +} + +/** Strip AT MOST ONE leading context segment (NestJS `this.prisma.…`, GraphQL + * `ctx.prisma.…`). */ +const stripContextSegment = (s: string) => s.replace(/^(this|self|ctx|context)\./, '') + +/** + * True when no level of the receiver chain is itself a call. A call inside the + * chain means a factory/lookup idiom — IndexedDB `tx.objectStore('s').delete(k)`, + * raw Mongo `db.collection('c').insertOne(…)` — which must not fire. + */ +function receiverChainCallFree(receiver: SyntaxNode | null, lang: SastLanguage): boolean { + let cur = receiver + for (let i = 0; i < 12 && cur; i++) { + const n = unwrap(cur, lang) + if (asCall(n, lang)) return false + const m = asMember(n, lang) + if (!m) return true + cur = m.object + } + return true +} + +// ---- swallowed error -------------------------------------------------------- +/** Best-effort cleanup calls whose failure is legitimately ignorable. */ +const CLEANUP_METHODS = new Set([ + 'unlink', 'unlinksync', 'rm', 'rmsync', 'rmdir', 'rmdirsync', 'rimraf', 'remove', 'rmtree', + 'close', 'closesync', 'destroy', 'abort', 'disconnect', 'end', 'kill', 'terminate', + 'dispose', 'release', 'releaselock', 'stop', 'unsubscribe', 'removelistener', 'removealllisteners', +]) +/** Receivers whose failures are cosmetic (copy buttons, storage feature checks). */ +const CLEANUP_RECEIVER = /(^|\.)clipboard$|^(localstorage|sessionstorage)$/ +const BROAD_EXCEPT_TYPES = new Set(['Exception', 'BaseException']) + +/** First call in a try body (descending through expression statements and + * assignments) — the best-effort-cleanup signal for swallowed-error. */ +function firstTryBodyCall(tryNode: SyntaxNode, lang: SastLanguage): NCall | null { + const body = tryNode.childForFieldName('body') + const first = body?.namedChildren.find((c) => c.type !== 'comment') + if (!first) return null + let expr: SyntaxNode = first + if (expr.type === 'expression_statement') expr = expr.namedChildren[0] ?? expr + if (expr.type === 'assignment_expression' || expr.type === 'assignment') { + expr = expr.childForFieldName('right') ?? expr + } + return asCall(unwrap(expr, lang), lang) +} + +/** Documented intent adjacent to the try: a comment directly above it, or a + * trailing comment on the catch/except clause's closing line. */ +function tryHasAdjacentComment(tryNode: SyntaxNode, clauseEndRow: number): boolean { + const prev = tryNode.previousNamedSibling + if (prev?.type === 'comment' && prev.endPosition.row >= tryNode.startPosition.row - 1) return true + let sib = tryNode.nextNamedSibling + for (let i = 0; i < 3 && sib; i++) { + if (sib.type === 'comment' && sib.startPosition.row === clauseEndRow) return true + if (sib.startPosition.row > clauseEndRow) break + sib = sib.nextNamedSibling + } + return false +} + +/** try { await unlink(tmp) } catch {} — temp-file/socket cleanup on an error + * path, and clipboard/storage best-effort writes. Deliberately ignorable. */ +function isBestEffortCleanup(tryNode: SyntaxNode, lang: SastLanguage): boolean { + const call = firstTryBodyCall(tryNode, lang) + if (!call) return false + if (CLEANUP_METHODS.has(lc(call.method))) return true + return CLEANUP_RECEIVER.test(lc(call.receiverName ?? '')) +} + +/** Bare `except:` / Exception / BaseException. A SPECIFIC exception type is a + * deliberate-suppression signal and never fires. */ +function isBroadExceptType(typeNode: SyntaxNode): boolean { + if (typeNode.type === 'identifier') return BROAD_EXCEPT_TYPES.has(typeNode.text) + if (typeNode.type === 'as_pattern') { + const first = typeNode.namedChildren[0] + return first ? BROAD_EXCEPT_TYPES.has(first.text) : false + } + if (typeNode.type === 'tuple') return typeNode.namedChildren.some((c) => BROAD_EXCEPT_TYPES.has(c.text)) + return false +} + +// ---- loose equality --------------------------------------------------------- +const LOOSE_EQ_OPS = new Set(['==', '!=']) +const NUMERIC_STRING_VALUE = /^[+-]?(\d+\.?\d*|\.\d+)$/ +const NULLISH_TEXT = /^(null|undefined)$/ +/** .length/.size/.count are guaranteed numbers — `items.length == 0` behaves + * identically to `===`. */ +const GUARANTEED_NUMBER_LEAF = /^(length|size|count)$/i + +/** + * A literal whose loose comparison invokes coercion: numbers, empty/whitespace + * strings, numeric strings, booleans. 'POST'/'per_unit'-style enum strings only + * loosely equal themselves and are NOT hazardous. `-1` parses as a + * unary_expression, which numberLiteralValue rejects — silently excluding the + * legit `indexOf(x) == -1` idiom. + */ +function isCoercionHazardousLiteral(node: SyntaxNode, lang: SastLanguage): boolean { + if (numberLiteralValue(node, lang) !== null) return true + const s = stringLiteralValue(node, lang) + if (s !== null) return /^\s*$/.test(s) || NUMERIC_STRING_VALUE.test(s.trim()) + return boolLiteralValue(node, lang) !== null +} + +function isAnyLiteral(node: SyntaxNode, lang: SastLanguage): boolean { + return ( + numberLiteralValue(node, lang) !== null || + stringLiteralValue(node, lang) !== null || + boolLiteralValue(node, lang) !== null + ) +} + +// ---- database query inside a loop (N+1) ------------------------------------ +/** ORM/query-builder READ methods. Writes are deliberately absent — that alone + * excludes every seed/migration/backfill loop swept. */ +const DB_READ_METHODS = new Set([ + 'findunique', 'finduniqueorthrow', 'findfirst', 'findfirstorthrow', 'findmany', + 'findone', 'findall', 'findbyid', 'findbypk', 'count', 'aggregate', 'groupby', +]) +const LOOP_TYPES = new Set(['for_statement', 'for_in_statement', 'while_statement', 'do_statement']) +/** Loop variables that name deliberate batching (`for (const chunk of chunks)`). */ +const BATCH_LOOP_VAR = /^(chunk|batch|page|group|slice|part|partition|window|bucket)(e?s)?$/i + +/** identifier / destructuring-leaf names under a node (loop-variable patterns). */ +function identifierLeaves(node: SyntaxNode): string[] { + const out: string[] = [] + walk(node, (n) => { + if (n.type === 'identifier' || n.type === 'shorthand_property_identifier_pattern') out.push(n.text) + }) + return out +} + +/** True when the subtree references any of the given names. */ +function referencesAny(node: SyntaxNode, names: Set): boolean { + let found = false + walk(node, (n) => { + if (!found && (n.type === 'identifier' || n.type === 'shorthand_property_identifier') && names.has(n.text)) { + found = true + } + }) + return found +} + +function containsNumberLiteral(node: SyntaxNode, lang: SastLanguage): boolean { + let found = false + walk(node, (n) => { + if (!found && numberLiteralValue(n, lang) !== null) found = true + }) + return found +} + +/** + * Per-iteration dependence — the actual N+1 signature. D = loop vars plus names + * bound BEFORE the call by a declarator/assignment whose value references a name + * already in D (`const user = teamMembers[i]`). Single forward pass, skipping + * nested-function subtrees. + */ +function perIterationNames( + body: SyntaxNode, + loopVars: string[], + callNode: SyntaxNode, + lang: SastLanguage, +): Set { + const derived = new Set(loopVars) + const visit = (n: SyntaxNode) => { + if (n !== body && isFunctionNode(n, lang)) return + if (n.startIndex < callNode.startIndex) { + if (n.type === 'variable_declarator') { + const nameNode = n.childForFieldName('name') + const value = n.childForFieldName('value') + if (nameNode?.type === 'identifier' && value && referencesAny(value, derived)) derived.add(nameNode.text) + } else if (n.type === 'assignment_expression') { + const left = n.childForFieldName('left') + const right = n.childForFieldName('right') + if (left?.type === 'identifier' && right && referencesAny(right, derived)) derived.add(left.text) + } + } + for (const c of n.namedChildren) visit(c) + } + visit(body) + return derived +} + +/** A return, or a break not enclosed by a switch, marks the bounded-lookback + * search idiom (per-candidate probe, exit on first hit) — never N+1. */ +function loopBodyHasEarlyExit(body: SyntaxNode, loopNode: SyntaxNode, lang: SastLanguage): boolean { + let exits = false + const visit = (n: SyntaxNode) => { + if (exits) return + if (n !== body && isFunctionNode(n, lang)) return + if (n.type === 'return_statement') { + exits = true + return + } + if (n.type === 'break_statement' && !breakEnclosedBySwitch(n, loopNode)) { + exits = true + return + } + for (const c of n.namedChildren) visit(c) + } + visit(body) + return exits +} + +function breakEnclosedBySwitch(brk: SyntaxNode, loopNode: SyntaxNode): boolean { + // web-tree-sitter wrappers are not identity-stable — compare by node id. + let cur = brk.parent + for (let i = 0; i < 40 && cur && cur.id !== loopNode.id; i++) { + if (cur.type === 'switch_statement') return true + cur = cur.parent + } + return false +} + +// ---- mass assignment -------------------------------------------------------- +const MA_WRITE_METHODS = new Set(['create', 'update', 'upsert', 'updatemany', 'createmany']) +/** DB_RECEIVER extended with transaction-callback names for WRITE receivers. */ +const MA_TX_RECEIVER = /(^|[._])(tx|trx)([._]|$)/i +/** Rule-local accepted request roots/props — narrower than the taint engine's: + * headers/url/data/args/params/searchParams are deliberately excluded + * (schema-declared or server-to-server, not arbitrary-key client payloads). */ +const MA_REQUEST_ROOTS = new Set(['req', 'request', 'ctx', 'context', 'httprequest', 'httpcontext']) +const MA_REQUEST_PROPS = new Set(['body', 'rawbody', 'form', 'formdata', 'files', 'post', 'query']) +/** An explicit open-record annotation: the author declaring arbitrary-key + * pass-through. Named model types, Partial, and field literals never match. */ +const OPEN_RECORD_TYPE = /^(any|unknown|object|Record)$|^\{\[/ + +function isDbWriteReceiver(name: string | null): boolean { + if (!name) return false + return DB_RECEIVER.test(name) || MA_TX_RECEIVER.test(name) +} + +/** Arm (a): the expression IS the raw request payload — req.json()/req.body. */ +function isDirectRequestSource(node: SyntaxNode, lang: SastLanguage): boolean { + const n = unwrap(node, lang) + const call = asCall(n, lang) + if (call) { + const m = lc(call.method) + if (m !== 'json' && m !== 'text' && m !== 'formdata') return false + const recv = lc(call.receiverName ?? '') + const last = recv.slice(recv.lastIndexOf('.') + 1) + return last === 'req' || last === 'request' + } + const dotted = lc(dottedName(n, lang) ?? '') + const segs = dotted.split('.') + if (segs.length < 2) return false + return MA_REQUEST_ROOTS.has(segs[0]) && MA_REQUEST_PROPS.has(segs[segs.length - 1]) +} + +/** Payload expressions in mass-assignment sink position for this call. */ +function massAssignPayloads(call: NCall, lang: SastLanguage): SyntaxNode[] { + const m = lc(call.method) + const out: SyntaxNode[] = [] + // P1 prisma-style: db.model.update({ …, data: E }) (upsert also create/update) + if (MA_WRITE_METHODS.has(m) && isDbWriteReceiver(call.receiverName)) { + const arg0 = call.args[0] + if (arg0?.type === 'object') { + const keys = m === 'upsert' ? ['data', 'create', 'update'] : ['data'] + for (const child of arg0.namedChildren) { + if (child.type === 'pair') { + const key = lc(child.childForFieldName('key')?.text ?? '').replace(/['"]/g, '') + const value = child.childForFieldName('value') + if (value && keys.includes(key)) out.push(value) + } else if (child.type === 'shorthand_property_identifier' && keys.includes(lc(child.text))) { + out.push(child) + } + } + } + } + // P2 drizzle-style: db.insert(t).values(E) / db.update(t).set(E) + if (m === 'values' || m === 'set') { + const recvCall = call.receiver ? asCall(unwrap(call.receiver, lang), lang) : null + const rm = recvCall ? lc(recvCall.method) : '' + if (recvCall && (rm === 'insert' || rm === 'update') && isDbWriteReceiver(recvCall.receiverName) && call.args[0]) { + out.push(call.args[0]) + } + } + // P3 supabase/knex-style: supabase.from('t').update(E) / db('t').update(E) + if (m === 'insert' || m === 'update' || m === 'upsert') { + const recvCall = call.receiver ? asCall(unwrap(call.receiver, lang), lang) : null + if (((recvCall && lc(recvCall.method) === 'from') || isDbWriteReceiver(call.receiverName)) && call.args[0]) { + out.push(call.args[0]) + } + } + return out +} + +interface MassAssignBinding { + kind: 'defs' | 'param' + /** Value nodes of every def in the binding scope (kind 'defs'). */ + defs: SyntaxNode[] + /** The binding function node; null when bound at the program root. */ + fnNode: SyntaxNode | null +} + +/** + * Lexical binding resolution for a bare identifier: walk enclosing function + * scopes innermost→outermost (program root last); the first scope binding the + * name — via defs (collectAssignments semantics) or as a declared parameter — + * is the binding scope. Defs take precedence over a same-named parameter. + */ +function resolveMassAssignBinding( + from: SyntaxNode, + name: string, + lang: SastLanguage, +): MassAssignBinding | null { + let cur: SyntaxNode | null = from.parent + for (let i = 0; i < 60 && cur; i++) { + const isRoot = cur.parent === null + if (isFunctionNode(cur, lang) || isRoot) { + const fn = isFunctionNode(cur, lang) ? asFunction(cur, lang) : null + const body = isRoot && !fn ? cur : fn?.body + const defs = body + ? collectAssignments(body, lang).filter((a) => a.target === name).map((a) => a.value) + : [] + if (defs.length > 0) return { kind: 'defs', defs, fnNode: fn ? cur : null } + if (fn && fn.params.includes(name)) return { kind: 'param', defs: [], fnNode: cur } + } + cur = cur.parent + } + return null +} + +/** Whether the named parameter of fnNode carries an explicit open-record type + * annotation. A MISSING annotation never fires (plain-JS helpers). */ +function hasOpenRecordAnnotation(fnNode: SyntaxNode, name: string): boolean { + const params = fnNode.childForFieldName('parameters') + if (!params) return false + for (const p of params.namedChildren) { + const pattern = p.childForFieldName('pattern') ?? p.childForFieldName('name') + if (pattern?.type !== 'identifier' || pattern.text !== name) continue + const typeNode = p.childForFieldName('type') + if (!typeNode) return false + const normalized = typeNode.text.replace(/^:\s*/, '').replace(/\s+/g, '') + return OPEN_RECORD_TYPE.test(normalized) + } + return false +} + +/** Which arm (if any) marks this payload expression as a raw request object. */ +function massAssignArm(s: SyntaxNode, sinkNode: SyntaxNode, lang: SastLanguage): string | null { + const n = unwrap(s, lang) + // (a) inline request source: data: req.body / data: await req.json() + if (isDirectRequestSource(n, lang)) return 'request source' + // A shorthand `data` property references the binding of the same name. + const name = identifierName(n, lang) ?? (n.type === 'shorthand_property_identifier' ? n.text : null) + if (!name) return null // call results, arrays, ternaries, … never fire + const binding = resolveMassAssignBinding(sinkNode, name, lang) + if (!binding) return null + if (binding.kind === 'defs') { + // (b) flow-insensitive and FN-biased: EVERY def must be a direct request + // source — any validating reassignment (schema.parse) suppresses entirely. + return binding.defs.every((v) => isDirectRequestSource(v, lang)) ? 'request-sourced local' : null + } + // (c) open-record parameter: the annotation itself declares arbitrary-key + // pass-through into a write — the report-worthy contract. + return binding.fnNode && hasOpenRecordAnnotation(binding.fnNode, name) ? 'open-record parameter' : null +} + +export const PATTERN_RULES: PatternRule[] = [ + // ---- Server secret inlined into the browser bundle ---- + { + id: 'client-exposed-secret', + cweKey: 'CLIENTSECRET', + severity: 'HIGH', + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Server secret exposed to the browser bundle', + message: + 'A server-only credential is read through a public build prefix, so the bundler inlines its value into JavaScript every visitor downloads. Prefixing a variable to silence an "undefined" error publishes the secret.', + remediation: + 'Drop the public prefix and read this variable only in server code (route handler, server component, or server action). Rotate the credential — assume any value already shipped in a bundle is compromised.', + test(node, lang) { + if (!asMember(node, lang)) return [] + const dotted = dottedName(node, lang) + if (!dotted) return [] + const last = dotted.slice(dotted.lastIndexOf('.') + 1) + // Case-sensitive: these prefixes are uppercase by build-tool convention. + if (!CLIENT_ENV_PREFIX.test(last) || !SERVER_ONLY_ENV_NAME.test(last)) return [] + return [{ node, line: node.startPosition.row + 1, detail: last }] + }, + }, + + // ---- Insecure deserialization (dangerous-by-format) ---- + { + id: 'insecure-deserialization', + cweKey: 'DESER', + severity: 'HIGH', + languages: ALL, + title: 'Insecure deserialization', + message: 'An inherently unsafe deserializer is used. These formats can instantiate arbitrary types on load, enabling remote code execution if the data is ever attacker-influenced.', + remediation: 'Use a safe format (JSON) or the safe API variant (yaml.safe_load, allowed_classes, a validating binder); never deserialize untrusted data with these.', + test(node, lang) { + const call = asCall(node, lang) + if (!call) return [] + const full = lc(call.fullName) + const m = lc(call.method) + const recv = lc(call.receiverName) + // Python + if (/^(pickle|cpickle|_pickle|dill)\.(loads?|load)$/.test(full)) return [{ node, line: call.line, detail: 'pickle' }] + if (/marshal\.loads?$/.test(full)) return [{ node, line: call.line, detail: 'marshal' }] + if (full === 'yaml.load') { + // safe only when an explicit SafeLoader is passed + const loader = findOption(call, lang, 'loader') + const loaderText = loader ? lc(loader.text) : lc(call.args[1]?.text) + if (!/safeloader|safe_load/.test(loaderText)) return [{ node, line: call.line, detail: 'yaml.load' }] + return [] + } + // Java — ObjectInputStream.readObject() (any receiver; readObject is specific) + if (m === 'readobject' && call.receiverName) return [{ node, line: call.line, detail: 'ObjectInputStream' }] + // Ruby + if (full === 'marshal.load' || (m === 'load' && recv === 'yaml' && lang === 'ruby')) return [{ node, line: call.line, detail: recv }] + // C# — dangerous formatters (construct or Deserialize call) + if (call.isConstruct && /binaryformatter|soapformatter|netdatacontractserializer|losformatter|objectstateformatter/i.test(call.fullName)) { + return [{ node, line: call.line, detail: call.fullName }] + } + if (m === 'deserialize' && /binaryformatter|soapformatter|losformatter/.test(recv)) { + return [{ node, line: call.line, detail: call.receiverName ?? '' }] + } + return [] + }, + }, + + // ---- Weak hash ---- + { + id: 'weak-hash', + cweKey: 'WEAKHASH', + severity: 'MEDIUM', + languages: ALL, + title: 'Weak cryptographic hash (MD5/SHA-1)', + message: 'MD5/SHA-1 are broken for security purposes (collisions are practical). Using them for signatures, integrity, or password hashing is unsafe.', + remediation: 'Use SHA-256/SHA-3 for integrity, and a dedicated password hash (bcrypt/scrypt/Argon2) for passwords.', + test(node, lang) { + const call = asCall(node, lang) + if (!call) return [] + const m = lc(call.method) + const recv = lc(call.receiverName) + const arg0 = call.args[0] ? lc(stringLiteralValue(call.args[0], lang) ?? '') : '' + const isWeak = + (m === 'createhash' && /^(md5|sha1|sha-1)$/.test(arg0)) || + (m === 'getinstance' && /^(md5|sha-?1)$/.test(arg0)) || + ((m === 'md5' || m === 'sha1') && (recv === 'hashlib' || recv === '' || recv === 'digest')) || + /^(md5|sha1)\.create$/.test(lc(call.fullName)) + if (!isWeak) return [] + // Suppress non-security content hashing (fingerprint/dedup/ETag/identity). + if (isNonSecurityHash(node, lang)) return [] + const detail = m === 'createhash' || m === 'getinstance' ? arg0 : m + return [{ node, line: call.line, detail }] + }, + }, + + // ---- Weak cipher (DES / ECB / RC4) ---- + { + id: 'weak-cipher', + cweKey: 'WEAKCIPHER', + severity: 'MEDIUM', + languages: ALL, + title: 'Weak or misused cipher (DES / ECB / RC4)', + message: 'A broken cipher (DES/RC4) or the ECB mode is selected. ECB leaks plaintext structure; DES/RC4 are cryptographically broken.', + remediation: 'Use AES-256 in an authenticated mode (GCM) with a random IV/nonce.', + test(node, lang) { + const call = asCall(node, lang) + if (!call) return [] + const m = lc(call.method) + if (!/createcipher|createcipheriv|createdecipheriv|getinstance|new/.test(m) && !call.isConstruct) return [] + const arg0 = stringLiteralValue(call.args[0], lang) + if (!arg0) return [] + if (/(^|[-/_])(des|des-ede|rc4|ecb)([-/_]|$)/i.test(arg0)) return [{ node, line: call.line, detail: arg0 }] + return [] + }, + }, + + // ---- Insecure PRNG for a secret ---- + { + id: 'insecure-randomness', + cweKey: 'WEAKRNG', + severity: 'MEDIUM', + languages: ALL, + title: 'Insecure randomness used for a secret', + message: 'A non-cryptographic PRNG (Math.random / random / rand) is used to generate a security-sensitive value. Its output is predictable and can be brute-forced.', + remediation: 'Use a CSPRNG: crypto.randomBytes / secrets.token_bytes / RNGCryptoServiceProvider / SecureRandom.', + test(node, lang) { + const call = asCall(node, lang) + if (!call) return [] + const full = lc(call.fullName) + const isWeak = + full === 'math.random' || + /^random\.(random|randint|randrange|getrandbits|choice)$/.test(full) || + (call.isConstruct && /^random$/i.test(call.fullName)) || + full === 'rand' + if (!isWeak) return [] + const ctx = nameContext(node, lang, RNG_NAME) + return ctx ? [{ node, line: call.line, detail: ctx }] : [] + }, + }, + + // ---- Disabled TLS / certificate validation ---- + { + id: 'disabled-tls-validation', + cweKey: 'TLS', + severity: 'HIGH', + languages: ALL, + title: 'Disabled TLS certificate validation', + message: 'TLS certificate/hostname verification is turned off, exposing the connection to man-in-the-middle attacks.', + remediation: 'Never disable certificate validation. Fix the trust store / pin the correct CA instead.', + test(node, lang) { + const nv = asNamedValue(node, lang) + if (nv) { + const n = nv.name + if (n === 'rejectunauthorized' && boolLiteralValue(nv.value, lang) === false) return [{ node, line: node.startPosition.row + 1, detail: n }] + if (n === 'insecureskipverify' && boolLiteralValue(nv.value, lang) === true) return [{ node, line: node.startPosition.row + 1, detail: n }] + if (n === 'verify' && boolLiteralValue(nv.value, lang) === false && lang === 'python') return [{ node, line: node.startPosition.row + 1, detail: n }] + // C#: xyz.ServerCertificateValidationCallback = (…) => true + if (/servercertificatevalidationcallback|remotecertificatevalidationcallback|servercertificatecustomvalidationcallback/.test(n)) { + if (/=>\s*true|return\s+true/.test(nv.value.text)) return [{ node, line: node.startPosition.row + 1, detail: n }] + } + } + return [] + }, + }, + + // ---- Hard-coded credentials (complements the regex secrets analyzer) ---- + { + id: 'hardcoded-credentials', + cweKey: 'HARDCRED', + severity: 'HIGH', + languages: ALL, + title: 'Hard-coded credential', + message: 'An API key / token / secret is assigned a literal string in source. Committed credentials must be treated as compromised.', + remediation: 'Load the credential from environment/secret storage and rotate the exposed value.', + test(node, lang) { + const nv = asNamedValue(node, lang) + if (!nv) return [] + if (!SECRET_NAME.test(nv.name)) return [] + const val = stringLiteralValue(nv.value, lang) + if (!val || val.length < 8) return [] + if (CRED_PLACEHOLDER.test(val) || CRED_PLACEHOLDER.test(nv.value.text)) return [] + // A credential is a single token; internal whitespace means display text + // (an error message keyed by credential name), not a secret. + if (/\s/.test(val)) return [] + if (CRED_LABEL_VALUE.test(val)) return [] // header/param NAME or bare prefix, not a secret + if (isNonCredentialUrlLiteral(val)) return [] + return [{ node, line: node.startPosition.row + 1, detail: nv.name }] + }, + }, + + // ---- XXE ---- + { + id: 'xxe', + cweKey: 'XXE', + severity: 'HIGH', + languages: new Set(['csharp', 'python', 'java']), + title: 'XML external entity (XXE) processing enabled', + message: 'An XML parser is configured to resolve external entities/DTDs, allowing file disclosure and SSRF via crafted XML.', + remediation: 'Disable DTD/external-entity processing (DtdProcessing.Prohibit, resolve_entities=False, XMLConstants FEATURE_SECURE_PROCESSING).', + test(node, lang) { + // C#: DtdProcessing.Parse + const m = asMember(node, lang) + if (m && lang === 'csharp') { + const dotted = lc(dottedName(node, lang) ?? '') + if (dotted === 'dtdprocessing.parse') return [{ node, line: node.startPosition.row + 1, detail: dotted }] + } + // Python lxml: resolve_entities=True + const nv = asNamedValue(node, lang) + if (nv && nv.name === 'resolve_entities' && boolLiteralValue(nv.value, lang) === true) { + return [{ node, line: node.startPosition.row + 1, detail: nv.name }] + } + // C#: XmlResolver = new XmlUrlResolver() + if (nv && nv.name === 'xmlresolver' && /xmlurlresolver/i.test(nv.value.text)) { + return [{ node, line: node.startPosition.row + 1, detail: nv.name }] + } + return [] + }, + }, + + // ---- Insecure cookie flags ---- + { + id: 'insecure-cookie', + cweKey: 'COOKIE', + severity: 'MEDIUM', + languages: ALL, + title: 'Insecure cookie flag', + message: 'A cookie is set with HttpOnly or Secure explicitly disabled, exposing it to theft via XSS or plaintext interception.', + remediation: 'Set cookies with httpOnly: true and secure: true (and SameSite) for session/auth cookies.', + test(node, lang) { + const nv = asNamedValue(node, lang) + if (!nv) return [] + if ((nv.name === 'httponly' || nv.name === 'secure') && boolLiteralValue(nv.value, lang) === false) { + if (callContext(node, lang, /cookie|session/i)) return [{ node, line: node.startPosition.row + 1, detail: nv.name }] + } + return [] + }, + }, + + // ---- Timing-unsafe secret comparison ---- + { + id: 'timing-unsafe-compare', + cweKey: 'TIMING', + severity: 'LOW', + languages: ALL, + title: 'Timing-unsafe secret comparison', + message: 'A secret (key, signature, password) is compared with a standard equality operator. String comparison short-circuits on the first differing byte, letting an attacker recover the value byte-by-byte from response timing.', + remediation: 'Compare with a constant-time primitive — crypto.timingSafeEqual (Node), hmac.compare_digest (Python) — after a length guard, or compare HMAC digests of both sides.', + test(node, lang) { + const t = node.type + if (t !== 'binary_expression' && t !== 'comparison_operator' && t !== 'binary') return [] + const op = + node.childForFieldName('operator')?.text ?? + node.children.find((c) => EQ_OPS.has(c.type))?.text + if (!op || !EQ_OPS.has(op)) return [] + const left = node.childForFieldName('left') ?? node.namedChildren[0] + const right = node.childForFieldName('right') ?? node.namedChildren[1] + if (!left || !right) return [] + if (isTrivialCompareOperand(left, lang) || isTrivialCompareOperand(right, lang)) return [] + const hit = secretCompareSide(left, lang) ?? secretCompareSide(right, lang) + if (!hit) return [] + // Only password-ish hits get the same-user idiom passes, so a secret + // like confirmationToken elsewhere keeps its protection. + if (PASSWORD_NAME.test(hit)) { + const sides = [left, right].map((side) => { + const u = unwrap(side, lang) + return dottedName(u, lang) ?? identifierName(u, lang) ?? '' + }) + if (sides.some((sideName) => CONFIRMATION_NAME.test(sideName))) return [] + if (sides.some((sideName) => FORM_SIBLING_FIELD.test(sideName))) return [] + if ( + sides.every((sideName) => PASSWORD_NAME.test(sideName)) && + sides.some((sideName) => SAME_USER_PASSWORD_PREFIX.test(sideName)) + ) + return [] + } + return [{ node, line: node.startPosition.row + 1, detail: hit }] + }, + }, + + // ---- Weak RSA/DSA key size ---- + { + id: 'weak-key-size', + cweKey: 'WEAKCIPHER', + severity: 'MEDIUM', + languages: ALL, + title: 'Weak asymmetric key size', + message: 'An RSA/DSA key smaller than 2048 bits is generated; such keys are considered breakable.', + remediation: 'Generate RSA/DSA keys of at least 2048 bits (prefer 3072+), or use an elliptic-curve key.', + test(node, lang) { + const call = asCall(node, lang) + if (!call) return [] + const m = lc(call.method) + if (!/initialize|generatekeypair|generate|moduluslength/.test(m) && !/modulus_size|key_size/.test(lc(call.fullName))) return [] + for (const a of call.args) { + const n = numberLiteralValue(a, lang) + if (n !== null && n > 0 && n < 2048) { + // Require a genuine key-generation context. A bare `generat` match flags + // business helpers like generateCoupon(15)/generateOtp(6)/generateId(8) + // where the number is a percent/length, not a key modulus. + if (/key|rsa|dsa|modulus/i.test(call.fullName) || m === 'initialize') return [{ node, line: call.line, detail: String(n) }] + } + } + return [] + }, + }, + + // ---- Un-awaited database write (floating promise) ---- + { + id: 'unawaited-persistence', + cweKey: 'UNAWAITED', + severity: 'MEDIUM', + // JS family only: sync SQLAlchemy `session.commit()` is byte-identical to + // the async-bug shape and legitimate — Python is not deterministically + // separable, so it stays out. + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Un-awaited database write (floating promise)', + message: + 'A database write is started and its promise is discarded — neither awaited, returned, handled with .then/.catch, nor explicitly voided. If the write fails the error is silently lost and the caller proceeds as if it succeeded; the surrounding function can return before the write completes. For lazy clients (Drizzle, Prisma $executeRaw, Convex) the query may never execute at all.', + remediation: + 'Await the call (or return the promise to a caller that awaits it). If fire-and-forget is genuinely intended, make it explicit: attach .catch() with error logging, or prefix the statement with the void operator after attaching a rejection handler.', + test(node, lang) { + if (node.type !== 'expression_statement') return [] + // The FIRST NAMED CHILD must be the call itself. unwrap() is deliberately + // NOT used: it strips await_expression, and `await` is exactly what must + // keep excluding. Return position, assignment, void, yield, argument + // position and implicit arrow returns all fail this anchor structurally. + const child = node.namedChildren[0] + if (!child || child.type !== 'call_expression') return [] + const call = asCall(child, lang) + if (!call || call.isConstruct) return [] + const m = lc(call.method) + const recv = lc(call.receiverName ?? '') + const recvCore = stripContextSegment(recv) + const fullCore = stripContextSegment(lc(call.fullName)) + const hit = [{ node: child, line: call.line, detail: call.fullName }] + + // A. ORM-model write (Prisma/Mongo style): db..(…) + const segs = recvCore.split('.') + if ( + segs.length >= 2 && + FLOATING_DB_ROOTS.has(segs[0]) && + ORM_WRITE_METHODS.has(m) && + receiverChainCallFree(call.receiver, lang) + ) { + if (!DELETEISH_METHODS.has(m) || call.args.length === 0 || call.args[0].type === 'object') return hit + } + // B. Lazy query-builder chain (Drizzle/Knex): a write verb called + // DIRECTLY on the db root plus at least one chained call — these + // builders are lazy thenables, so a floating chain never executes. + if (/^(db|tx|trx|database)\.(insert|update|delete)\.[a-z_$]/.test(fullCore) && !PROMISE_TERMINALS.has(m)) { + return hit + } + // C. Prisma raw write escape hatch — PrismaPromise is lazy, a floating + // $executeRaw never runs. (asMember strips the leading '$'.) + if (segs.length === 1 && FLOATING_DB_ROOTS.has(segs[0]) && (m === 'executeraw' || m === 'executerawunsafe')) { + return hit + } + // D. Convex mutations, gated to the literal `ctx.db` convention so + // `this.db.delete(key)` on a Map-holding class can never fire. + if (recv === 'ctx.db' && CONVEX_WRITE_METHODS.has(m)) return hit + // E. Raw driver write: `pool.query("INSERT …")` with the promise dropped. + // The commonest floating-write shape in Express/pg code, and invisible to + // the ORM arms above. Four independent gates keep it exact: a DB-shaped + // receiver, a SQL string whose first keyword is a WRITE (a dropped SELECT + // wastes a read; it does not lose data), no function argument — a callback + // means the callback driver API, which returns nothing to await — and no + // call inside the receiver chain. + if ( + RAW_QUERY_METHODS.has(m) && + DB_RECEIVER.test(call.receiverName ?? '') && + isSqlWriteArgument(call.args[0], lang) && + !call.args.some((argument) => isFunctionNode(unwrap(argument, lang), lang)) && + receiverChainCallFree(call.receiver, lang) + ) { + return hit + } + return [] + }, + }, + + // ---- Swallowed error (empty catch block) ---- + { + id: 'swallowed-error', + cweKey: 'SWALLOW', + severity: 'LOW', + languages: new Set(['javascript', 'typescript', 'tsx', 'python']), + title: 'Swallowed error (empty catch block)', + message: + 'A catch/except block discards the error with no handling, no logging, and no comment. Failures inside the try body vanish silently — the operation can fail with no signal to callers, logs, or users, which hides real defects and data loss.', + remediation: + 'Handle or at least log the error. If ignoring it is deliberate, document that with a comment inside the catch block (or catch the specific expected exception type in Python) — that records intent and silences this finding.', + test(node, lang) { + if (lang === 'python') { + if (node.type !== 'except_clause') return [] + // A comment before `pass` attaches to the except_clause itself, not to + // the block — any comment under the clause is documented intent. + if (node.namedChildren.some((c) => c.type === 'comment')) return [] + const block = node.namedChildren.find((c) => c.type === 'block') + if (!block) return [] + const stmts = block.namedChildren + // A trailing `pass # ok` comment lands inside the block → length 2. + if (stmts.length !== 1 || stmts[0].type !== 'pass_statement') return [] + const typeNode = node.namedChildren.find((c) => c.type !== 'block' && c.type !== 'comment') + if (typeNode && !isBroadExceptType(typeNode)) return [] + const tryNode = node.parent + if (tryNode) { + if (tryHasAdjacentComment(tryNode, node.endPosition.row)) return [] + if (isBestEffortCleanup(tryNode, lang)) return [] + } + const detail = tryNode ? firstTryBodyCall(tryNode, lang)?.fullName : undefined + return [{ node, line: node.startPosition.row + 1, detail }] + } + if (node.type !== 'catch_clause') return [] + const body = node.childForFieldName('body') + if (!body) return [] + // Comments are named nodes in this grammar: `catch { // ignore }` is NOT + // empty, so comment-only catches (the dominant legit idiom) never fire. + // `catch {;}` contains only empty_statement nodes and is still empty. + if (!body.namedChildren.every((c) => c.type === 'empty_statement')) return [] + const tryNode = node.parent + if (tryNode) { + if (tryHasAdjacentComment(tryNode, node.endPosition.row)) return [] + if (isBestEffortCleanup(tryNode, lang)) return [] + } + const detail = tryNode ? firstTryBodyCall(tryNode, lang)?.fullName : undefined + return [{ node, line: node.startPosition.row + 1, detail }] + }, + }, + + // ---- Loose equality against a coercion-prone literal ---- + { + id: 'loose-equality', + cweKey: 'LOOSEEQ', + severity: 'MEDIUM', + // Python `==` has no coercion — excluded. + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Loose equality against a coercion-prone literal', + message: + "A value is compared to a number, empty-string, numeric-string, or boolean literal with == or !=, which applies JavaScript type coercion before comparing: '0' == 0, '' == 0, false == 0, '1' == true, and 0 == '' are all true. A string that arrives where a number was expected — a query parameter, JSON field, or form value — silently passes or fails this check, so a guard like `amount == 0` treats the string '0' (and '' and false) as zero.", + remediation: + 'Use strict equality (=== / !==) so the type is checked along with the value. If both representations are genuinely expected, normalize explicitly first — Number(x) === 0 — instead of relying on coercion. The one deliberate loose comparison, x == null (matching null and undefined together), is recognized and never flagged.', + test(node, lang) { + if (node.type !== 'binary_expression') return [] + const op = + node.childForFieldName('operator')?.text ?? + node.children.find((c) => LOOSE_EQ_OPS.has(c.type))?.text + if (!op || !LOOSE_EQ_OPS.has(op)) return [] + const leftRaw = node.childForFieldName('left') ?? node.namedChildren[0] + const rightRaw = node.childForFieldName('right') ?? node.namedChildren[1] + if (!leftRaw || !rightRaw) return [] + const left = unwrap(leftRaw, lang) + const right = unwrap(rightRaw, lang) + // `x == null` / `x != undefined` is the one deliberate loose idiom. + if (NULLISH_TEXT.test(left.text) || NULLISH_TEXT.test(right.text)) return [] + const leftHazard = isCoercionHazardousLiteral(left, lang) + const rightHazard = isCoercionHazardousLiteral(right, lang) + if (leftHazard === rightHazard) return [] // zero or two hazardous sides + const literal = leftHazard ? left : right + const other = leftHazard ? right : left + if (isAnyLiteral(other, lang)) return [] // constant folding, not a data-coercion bug + if (/^typeof\b/.test(other.text)) return [] // typeof yields a string — type-safe + const otherName = dottedName(other, lang) + if (otherName && GUARANTEED_NUMBER_LEAF.test(otherName.slice(otherName.lastIndexOf('.') + 1))) return [] + return [{ node, line: node.startPosition.row + 1, detail: `${op} ${literal.text}` }] + }, + }, + + // ---- Database query inside a loop (N+1) ---- + { + id: 'db-call-in-loop', + cweKey: 'NPLUSONE', + severity: 'MEDIUM', + // Python deferred: zero validation evidence in the sweep, and + // `session.get` collides with dict/web-session .get. + languages: new Set(['javascript', 'typescript', 'tsx']), + excludePath: /(^|\/)(seeds?|migrations?|scripts?|tools|bin)\/|(^|\/)seed\.(ts|js|mjs)$/i, + title: 'Database query inside a loop (N+1)', + message: + 'An awaited database read runs inside a loop and its arguments change every iteration — one query per element instead of one batched query (the N+1 pattern). Latency grows linearly with the collection size and the database absorbs N sequential round-trips. This is a signature mistake of AI-generated code that fetches a related record per item.', + remediation: + 'Batch the per-item lookups into a single query before the loop — findMany with `where: { id: { in: ids } }` (or a groupBy/aggregate) — then join in memory with a Map. If iterations are truly independent and no batch API exists, run them concurrently with Promise.all. If the loop is deliberately sequential over a small bounded set, dismiss this finding; dismissals persist across scans.', + test(node, lang) { + const call = asCall(node, lang) + if (!call || call.isConstruct) return [] + // Read-method gate: ORM reads, or raw-SQL SELECT literals only. + const m = lc(call.method).replace(/^\$/, '') + const isRead = + DB_READ_METHODS.has(m) || + ((m === 'query' || m === 'execute') && /^\s*select\b/i.test(stringLiteralValue(call.args[0], lang) ?? '')) + if (!isRead) return [] + // Receiver gate. DB_RECEIVER does not match tx/trx, so Prisma + // $transaction callback bodies (intentional-sequential) never fire. + if (!call.receiverName || !DB_RECEIVER.test(call.receiverName)) return [] + // Await gate: promises pushed for a later Promise.all never fire. + let parent = node.parent + while (parent && parent.type === 'parenthesized_expression') parent = parent.parent + if (!parent || parent.type !== 'await_expression') return [] + // Loop gate: a function boundary before the loop means the call belongs + // to a callback (.map/Promise.all), not the loop body. + let loop: SyntaxNode | null = null + let prev: SyntaxNode = node + let cur = node.parent + for (let i = 0; i < 40 && cur; i++) { + if (isFunctionNode(cur, lang)) return [] + if (LOOP_TYPES.has(cur.type)) { + loop = cur + break + } + prev = cur + cur = cur.parent + } + if (!loop) return [] + // while/do are cursor-pagination/polling/retry idioms; for-await is the + // async-iterator row-at-a-time idiom. + if (loop.type === 'while_statement' || loop.type === 'do_statement') return [] + if (loop.type === 'for_in_statement' && loop.children.some((c) => c.type === 'await')) return [] + // Body-position gate: a query in the for CONDITION (slug-probe idiom) + // never fires. Wrapper objects are not identity-stable — compare ids. + const body = loop.childForFieldName('body') + if (!body || prev.id !== body.id) return [] + let loopVars: string[] = [] + if (loop.type === 'for_in_statement') { + const right = loop.childForFieldName('right') + if (right && unwrap(right, lang).type === 'array') return [] // constant bounded iteration + const left = loop.childForFieldName('left') + loopVars = left ? identifierLeaves(left) : [] + if (loopVars.some((v) => BATCH_LOOP_VAR.test(v))) return [] // deliberate batching + } else { + // for(;;): a numeric-literal bound (`race < 5`) is bounded retry/backoff. + const condition = loop.childForFieldName('condition') + if (condition && containsNumberLiteral(condition, lang)) return [] + const initializer = loop.childForFieldName('initializer') + if (initializer) { + walk(initializer, (n) => { + if (n.type === 'variable_declarator') { + const nameNode = n.childForFieldName('name') + if (nameNode?.type === 'identifier') loopVars.push(nameNode.text) + } + }) + } + } + // Per-iteration dependence: a loop-invariant query (retry re-read, + // constant probe) never fires. + const derived = perIterationNames(body, loopVars, node, lang) + if (!call.args.some((a) => referencesAny(a, derived))) return [] + // Early-exit suppression: bounded-lookback search, not N+1. + if (loopBodyHasEarlyExit(body, loop, lang)) return [] + return [{ node, line: call.line, detail: call.fullName }] + }, + }, + + // ---- Read-modify-write race on a database row (lost update) ---- + { + id: 'read-modify-write-race', + cweKey: 'RMWRACE', + severity: 'MEDIUM', + // JS family only: the pair search reads Prisma/Drizzle-shaped object + // arguments, and the key-equality test is unreliable over string-built SQL. + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Read-modify-write on a database row without a transaction', + message: + 'A row is read, a value from it is recomputed in application code, and the result is written back to the same row in a separate statement. Two requests that interleave between the read and the write both start from the same value, so one update overwrites the other and its change is silently lost — a balance credited twice reads as credited once. Nothing here closes that window: no transaction wraps the pair, and the write sends a computed number rather than an atomic operator.', + remediation: + 'Let the database do the arithmetic in one statement: Prisma `data: { balance: { increment: amount } }` (or `{ decrement }`), Drizzle or raw SQL `SET balance = balance + $1`. When the new value cannot be expressed as an operator, wrap the read and the write in one interactive transaction that locks the row (`$transaction` plus `SELECT … FOR UPDATE` or an advisory lock), or make the write a compare-and-set — `updateMany({ where: { id, balance: previous }, data: … })` — and retry when it matches zero rows. If this field is deliberately last-write-wins, dismiss this finding; dismissals persist across scans.', + test(node, lang) { + return readModifyWriteHits(node, lang, DB_RECEIVER) + }, + }, + + // ---- Mass assignment: request body written wholesale to a DB record ---- + { + id: 'mass-assignment', + cweKey: 'MASSASSIGN', + severity: 'HIGH', + // No Python: Model(**x) is statically indistinguishable between validating + // (pydantic/sqlmodel) and raw constructors — firing would FP on the + // dominant FastAPI idiom. + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Mass assignment: request body written wholesale to a database record', + message: + 'An object built directly from the raw request body is spread (or passed whole) into a database write payload. Every key the client sends becomes a column update, so an attacker can set fields they should never control (role, orgId, isAdmin, balance) just by adding properties to the JSON body.', + remediation: + 'Never pass the raw body through to a write. Validate with a schema that strips unknown keys (zod .parse on a non-passthrough object schema) or pick allowed fields explicitly ({ name: body.name, email: body.email }). Do not rely on deleting known-bad keys — a blocklist misses new columns. If a helper must accept partial updates, type it with the model\'s input type, not Record.', + test(node, lang) { + // Arms (a)/(b) only: an actual request object reaches the write. The + // open-record-parameter arm carries no request evidence and reports + // separately below, at the severity its evidence supports. + return massAssignHit(node, lang, (arm) => arm !== 'open-record parameter') + }, + }, + + // ---- Open-record payload contract on a database write ---- + { + id: 'open-record-write', + cweKey: 'MASSASSIGN', + severity: 'MEDIUM', + languages: new Set(['javascript', 'typescript', 'tsx']), + title: 'Database write helper accepts an open-record payload', + message: + 'A parameter typed as any/Record is written wholesale into a database record. The type contract lets every caller-supplied key become a column update, so the helper invites mass assignment even if today’s callers pass safe literals — one new call site fed by request data makes it exploitable.', + remediation: + 'Type the parameter with the model’s input type (e.g. Prisma.InvoiceUpdateInput) or validate inside the helper with a schema that strips unknown keys. If only fixed fields are ever updated, pick them explicitly instead of spreading the argument.', + test(node, lang) { + return massAssignHit(node, lang, (arm) => arm === 'open-record parameter') + }, + }, +] + +/** Shared scan for the two mass-assignment-family rules above. */ +function massAssignHit( + node: SyntaxNode, + lang: SastLanguage, + wanted: (arm: string) => boolean, +): PatternHit[] { + const call = asCall(node, lang) + if (!call || call.isConstruct) return [] + for (const payload of massAssignPayloads(call, lang)) { + const e = unwrap(payload, lang) + if (e.type === 'object') { + // Only DIRECT spread elements count; nested objects and plain pairs + // (explicit field picking) are ignored. + for (const child of e.namedChildren) { + if (child.type !== 'spread_element') continue + const inner = child.namedChildren[0] + if (!inner) continue + const arm = massAssignArm(inner, node, lang) + if (arm && wanted(arm)) { + const name = dottedName(unwrap(inner, lang), lang) ?? inner.text.slice(0, 40) + return [{ node, line: call.line, detail: `${name} (${arm})` }] + } + } + } else { + const arm = massAssignArm(e, node, lang) + if (arm && wanted(arm)) { + const name = dottedName(e, lang) ?? e.text.slice(0, 40) + return [{ node, line: call.line, detail: `${name} (${arm})` }] + } + } + } + return [] +} + +export function ruleAppliesTo(rule: RuleMeta, lang: SastLanguage): boolean { + return rule.languages === null || rule.languages.has(lang) +} diff --git a/packages/analyzer-engine/src/security/taint.ts b/packages/analyzer-engine/src/security/taint.ts new file mode 100644 index 0000000..53e6d1c --- /dev/null +++ b/packages/analyzer-engine/src/security/taint.ts @@ -0,0 +1,473 @@ +import type { SastLanguage, SyntaxNode } from './lang' +import { + asCall, + asMember, + collectAssignments, + concatOperands, + dottedName, + identifierName, + interpolationExprs, + stringLiteralValue, + subscriptBase, + unwrap, + walk, + type NCall, + type NFunc, +} from './normalize' + +/** + * Intraprocedural taint analysis with a one-hop interprocedural extension. + * + * Approach: within a function we compute, by bounded fixpoint, the set of local + * variables that carry untrusted data — seeded from SOURCE expressions + * (request/query/body, argv, env, network reads, …) and propagated through + * assignments, string concatenation, interpolation, subscripts and non- + * sanitizing calls. SANITIZERS (parameterizers, escapers, validators, path + * normalizers) clear taint. We then check every call/construct against the SINK + * registry: a sink fires only when its dangerous argument is taint-influenced, + * which is what keeps injection rules high-precision (a constant or parameter- + * ized argument is never flagged). + * + * The analysis is flow-insensitive (a variable assigned a tainted value ANY- + * where is treated as tainted) — the standard, deterministic lightweight-SAST + * trade-off. It over-approximates slightly (a var later overwritten with a + * constant), which is disclosed as a known false-positive vector. + * + * Interprocedural: for each function we also record which *parameters* reach a + * sink; a call passing a tainted argument into such a parameter is reported at + * the call site with an interprocedural flow. Exactly one hop — bounded and + * deterministic. + */ + +// ---- origins & taint state ------------------------------------------------ + +export type Origin = + | { kind: 'source'; sourceKind: string; node: SyntaxNode } + | { kind: 'param'; index: number; name: string; node: SyntaxNode } + +/** A variable's taint = the set of origins that can flow into it. */ +type Origins = Origin[] + +function mergeOrigins(a: Origins, b: Origins): Origins { + if (a.length === 0) return b + if (b.length === 0) return a + const out = a.slice() + for (const o of b) { + const dup = out.some((x) => + x.kind === o.kind && + (x.kind === 'source' ? x.sourceKind === (o as { sourceKind: string }).sourceKind : (x as { index: number }).index === (o as { index: number }).index), + ) + if (!dup) out.push(o) + } + return out +} + +// ---- source / sanitizer detection ----------------------------------------- + +/** Member-access roots that denote an HTTP request / untrusted-input object. */ +const REQUEST_ROOTS = new Set([ + 'req', 'request', 'ctx', 'context', 'httprequest', 'httpcontext', 'event', 'args', + '$_get', '$_post', '$_request', '$_cookie', '$_server', 'params', 'searchparams', +]) +/** Member properties on a request object that expose untrusted data. */ +const REQUEST_PROPS = new Set([ + 'query', 'body', 'params', 'cookies', 'headers', 'form', 'get', 'post', 'data', + 'url', 'originalurl', 'rawbody', 'files', 'querystring', 'formdata', 'getparameter', +]) +/** + * Whole dotted names that are globally untrusted. + * + * NOTE: `process.env` (and the equivalent env-read APIs below) are deliberately + * NOT sources. Environment variables are operator/deployment configuration, not + * attacker-controlled input — treating them as taint flags a redirect/URL built + * from config on nearly every real app (SSRF/open-redirect false positives). + */ +const GLOBAL_SOURCES = new Set([ + 'process.argv', 'location.search', 'location.hash', 'location.href', + 'window.name', 'document.cookie', 'document.url', 'document.referrer', 'sys.argv', + '$_get', '$_post', '$_request', '$_cookie', '$_server', '$_files', +]) +/** + * Go request/context variable names. Go-only: `r`/`c`/`g` are ubiquitous + * loop and temp names elsewhere, so these are gated behind an accessor + * whitelist (a member field or method that only a request/context exposes). + */ +const GO_REQUEST_ROOTS = new Set(['r', 'req', 'request', 'c', 'ctx', 'gctx', 'gc', 'ec', 'g']) +/** + * net/http.Request struct fields that carry untrusted data. Gated on a + * request-shaped root so `rows.URL`-style false matches cannot occur (these + * names are specific to *http.Request anyway). + */ +const GO_REQUEST_MEMBERS = new Set([ + 'url', 'header', 'body', 'form', 'postform', 'multipartform', 'trailer', 'host', 'requesturi', +]) +/** + * Request/context accessor methods returning untrusted data, gated on a + * request-shaped receiver. Covers net/http (FormValue, Cookie) and the + * common frameworks gin/echo/fiber (Query, Param, PostForm, GetHeader). + */ +const GO_REQUEST_METHODS = new Set([ + 'formvalue', 'postformvalue', 'cookie', 'referer', 'useragent', + 'query', 'defaultquery', 'querystring', 'param', 'params', 'postform', 'getheader', 'getstring', +]) +/** Call names whose *return value* is untrusted input. Matched on last segment. */ +const SOURCE_CALL_METHODS = new Set([ + 'input', // python input() + 'getparameter', 'getheader', 'getquerystring', 'getinputstream', // java servlet + 'readline', 'readtoend', 'nextline', 'readstring', // console / stream reads +]) +/** Full dotted call names that are untrusted sources. */ +const SOURCE_CALL_FULLNAMES = new Set([ + 'sys.stdin.read', 'sys.stdin.readline', + 'console.readline', 'console.in.readline', + 'request.args.get', 'request.form.get', 'request.values.get', 'request.get_json', + 'request.getparameter', 'request.getheader', +]) +/** + * Reader methods that pull raw bytes/ints off an untrusted stream — the taint + * source for the CardShopCoop unbounded-BinaryReader deserialization case. + */ +const READER_SOURCE_METHODS = new Set([ + 'readint32', 'readint16', 'readint64', 'readuint32', 'readuint16', 'readbyte', 'readbytes', 'readstring', +]) +/** + * Receiver names that denote a binary reader/stream. Includes the idiomatic + * short abbreviations (`br`, `rdr`) that real C#/Java code uses for a + * BinaryReader — without these the unbounded-read taint never seeds when the + * reader isn't literally named "reader"/"stream" (CardShopCoop `var br = ...`). + */ +const READER_RECEIVER = /reader|stream|buffer|^(br|rdr)$/ + +/** If the expression is an untrusted source, return its kind label. */ +export function sourceKindOf(node: SyntaxNode, lang: SastLanguage): string | null { + // `await params` / `await searchParams` — Next 15+ async route/page props. + // Checked on the RAW node because unwrap() strips the await. Awaiting a bare + // identifier with exactly these names is a strong framework signal; ordinary + // promises are not named `params`. + if (node.type === 'await_expression') { + const inner = identifierName(unwrap(node, lang), lang)?.toLowerCase() + if (inner === 'params' || inner === 'searchparams') return `await ${inner}` + } + const n = unwrap(node, lang) + + // call sources + const call = asCall(n, lang) + if (call) { + const full = call.fullName.toLowerCase() + const method = call.method.toLowerCase() + if (SOURCE_CALL_FULLNAMES.has(full)) return full + if (READER_SOURCE_METHODS.has(method) && READER_RECEIVER.test(call.receiverName?.toLowerCase() ?? '')) { + return 'network-bytes' + } + // Fetch-API/Next.js request body readers — req.json()/req.text()/req.formData(). + // Gated on a request-shaped receiver: a fetch RESPONSE's .json() is the + // server's own upstream call, not attacker input (flagging it buries real + // findings in internal-API noise). + if ( + (method === 'json' || method === 'text' || method === 'formdata') && + call.receiverName && + REQUEST_ROOTS.has(call.receiverName.toLowerCase()) + ) { + return `${call.receiverName}.${method}()` + } + // next/headers cookies() — every cookie value is attacker-controlled. + if ( + method === 'cookies' && + !call.receiverName && + !call.isConstruct && + (lang === 'javascript' || lang === 'typescript' || lang === 'tsx') + ) { + return 'cookies()' + } + // Getter calls on a request-shaped root: searchParams.get('q'), + // params.get(…). The member branch below already treats `.anything` + // as a source, so this adds no exposure the member form doesn't have. + if (method === 'get' && call.receiverName && REQUEST_ROOTS.has(call.receiverName.toLowerCase())) { + return `${call.receiverName}.get` + } + // bare input() in python + if (lang === 'python' && method === 'input' && !call.receiverName) return 'stdin' + if (SOURCE_CALL_METHODS.has(method) && call.receiverName) return `${call.receiverName}.${method}` + if (lang === 'go') { + const recv = call.receiverName?.toLowerCase() + // r.FormValue(), c.Query(), c.Param(), r.Cookie(), c.GetHeader()... + if (recv && GO_REQUEST_ROOTS.has(recv) && GO_REQUEST_METHODS.has(method)) { + return `${recv}.${method}` + } + // gorilla/mux path variables: mux.Vars(r) → map[string]string + if (recv === 'mux' && method === 'vars') return 'mux.vars' + } + return null + } + + // member / subscript sources + const dotted = dottedName(n, lang)?.toLowerCase() + if (dotted && GLOBAL_SOURCES.has(dotted)) return dotted + const m = asMember(n, lang) ?? (subscriptBase(n) ? asMember(subscriptBase(n)!, lang) : null) + if (m) { + const rootName = rootIdentifier(n, lang)?.toLowerCase() + const prop = m.property.toLowerCase() + if (rootName && REQUEST_ROOTS.has(rootName) && REQUEST_PROPS.has(prop)) return `${rootName}.${prop}` + if (rootName && REQUEST_ROOTS.has(rootName)) return rootName // req..x + // Go: r.URL / r.Header / r.Body etc. — request fields, gated on a + // request-shaped root so a *sql.Rows named `r` is not caught. + if (lang === 'go' && rootName && GO_REQUEST_ROOTS.has(rootName) && GO_REQUEST_MEMBERS.has(prop)) { + return `${rootName}.${prop}` + } + // php superglobal subscript: $_GET['x'] + if (rootName && GLOBAL_SOURCES.has(rootName)) return rootName + } + // bare php superglobal reference $_GET / subscript base + const rid = rootIdentifier(n, lang)?.toLowerCase() + if (rid && GLOBAL_SOURCES.has(rid)) return rid + return null +} + +/** Left-most identifier of a member/subscript chain. */ +function rootIdentifier(node: SyntaxNode, lang: SastLanguage): string | null { + let cur: SyntaxNode | null = unwrap(node, lang) + for (let i = 0; i < 20 && cur; i++) { + const m = asMember(cur, lang) + if (m) { + cur = m.object + continue + } + const sub = subscriptBase(cur) + if (sub) { + cur = sub + continue + } + const call = asCall(cur, lang) + if (call && call.receiver) { + cur = call.receiver + continue + } + return identifierName(cur, lang) ?? (cur.type === 'identifier' ? cur.text : cur.text?.replace(/^\$/, '') ?? null) + } + return null +} + +/** Call names that neutralize taint (parameterizers, escapers, validators). */ +const SANITIZER_METHODS = new Set([ + // parameterization / prepared statements + 'escape', 'escapestring', 'quote', 'mysql_real_escape_string', 'real_escape_string', + 'parameterize', 'prepare', 'preparestatement', + // encoding / escaping + 'htmlescape', 'escapehtml', 'encodeuricomponent', 'encodeuri', 'quotemeta', 'shlex_quote', 'quote_plus', + // path + 'basename', 'normalize', 'resolve', 'realpath', 'abspath', 'safe_join', + // validation / allow-list + 'validate', 'sanitize', 'clean', 'allowlist', 'whitelist', 'iselement', + 'isalnum', 'isnumeric', 'isdigit', 'parseint', 'parsefloat', 'tonumber', 'int', 'uuid', +]) +const SANITIZER_FULLNAMES = new Set([ + 'path.basename', 'path.normalize', 'path.resolve', 'os.path.basename', 'os.path.abspath', + 'shlex.quote', 'html.escape', 'urllib.parse.quote', 'werkzeug.utils.secure_filename', + 'secure_filename', 'number', 'boolean', +]) + +function isSanitizer(call: NCall): boolean { + return SANITIZER_METHODS.has(call.method.toLowerCase()) || SANITIZER_FULLNAMES.has(call.fullName.toLowerCase()) +} + +// ---- expression taint evaluation ------------------------------------------ + +interface Env { + lang: SastLanguage + taint: Map + /** Remaining node-visit budget for the current phase (fixpoint solve, then + * sink queries). Each evalOrigins visit accesses web-tree-sitter node + * properties (namedChildren, text) that allocate in the grow-only WASM heap; + * unbounded recursion over a JSX-heavy fixpoint churns hundreds of MB per + * file. When exhausted the phase degrades to no-taint for the rest of the + * function — a partial result that `exhausted` makes reportable. */ + budget: number + /** True once any phase ran out of budget: taint results for this function + * may be incomplete, so absence of a finding must not be presented as a + * fully-analyzed file. */ + exhausted: boolean +} + +// A normal function solves in well under this; it only bounds pathological +// JSX/expression fixpoints that would otherwise OOM the scan. +const TAINT_VISIT_BUDGET = 12_000 + +/** Origins that can flow out of an expression given the current taint map. */ +function evalOrigins(node: SyntaxNode, env: Env, depth = 0): Origins { + if (depth > 60) return [] + if (--env.budget < 0) { + env.exhausted = true + return [] + } + const lang = env.lang + const n = unwrap(node, lang) + + // 1. direct source? (raw node — sourceKindOf needs to see `await params` + // before unwrap strips the await; it unwraps internally for everything else) + const src = sourceKindOf(node, lang) + if (src) return [{ kind: 'source', sourceKind: src, node: n }] + + // 2. calls + const call = asCall(n, lang) + if (call) { + if (isSanitizer(call)) return [] // sanitized → clean + // NOTE: `new URL('', base)` deliberately does NOT propagate taint. + // A compile-time-constant first argument fixes the target; the (often + // tainted) second argument only supplies the absolute-URL base to resolve + // against — the standard Next.js `NextResponse.redirect(new URL('/login', + // req.url))` idiom, not an open redirect or SSRF. Trade-off: the rare + // `new URL('/fixed', attackerControlledBase)` host injection is suppressed too. + if (call.fullName.toLowerCase() === 'url' && stringLiteralValue(call.args[0], lang) !== null) return [] + let acc: Origins = [] + if (call.receiver) acc = mergeOrigins(acc, evalOrigins(call.receiver, env, depth + 1)) + for (const a of call.args) acc = mergeOrigins(acc, evalOrigins(a, env, depth + 1)) + return acc + } + + // 3. string concatenation + const parts = concatOperands(n, lang) + if (parts) { + let acc: Origins = [] + for (const p of parts) acc = mergeOrigins(acc, evalOrigins(p, env, depth + 1)) + return acc + } + + // 4. string interpolation (template literals / f-strings / interpolated strings) + const interp = interpolationExprs(n, lang) + if (interp) { + let acc: Origins = [] + for (const e of interp) acc = mergeOrigins(acc, evalOrigins(e, env, depth + 1)) + return acc + } + + // 5. subscript / index — taint of the base container + const sub = subscriptBase(n) + if (sub) return evalOrigins(sub, env, depth + 1) + + // 6. member access — taint of the object + const m = asMember(n, lang) + if (m) return evalOrigins(m.object, env, depth + 1) + + // 7. bare identifier + const id = identifierName(n, lang) + if (id) return env.taint.get(id) ?? [] + + // 8. fall back to a bounded union over subexpressions (ternary, casts, etc.) + if (n.namedChildCount > 0 && n.namedChildCount <= 8) { + let acc: Origins = [] + for (const c of n.namedChildren) acc = mergeOrigins(acc, evalOrigins(c, env, depth + 1)) + return acc + } + return [] +} + +// ---- function-level analysis ---------------------------------------------- + +export interface TaintFlow { + sourceKind: string + sourceNode: SyntaxNode + interprocedural: boolean + /** Intermediate variable names the value passed through, in order. */ + via: string[] +} + +/** Which param indexes of a function reach a sink (for the interprocedural hop). */ +export interface FnSummary { + name: string + params: string[] + /** param index → true if that param flows into any sink in this function. */ + sinkParams: Set +} + +export interface FunctionTaint { + fn: NFunc + /** Final taint map: variable → origins. */ + env: Env + summary: FnSummary +} + +/** + * Run the fixpoint taint solver over one function body. Seeds params as taint + * origins (for the interprocedural summary) and lets real sources seed + * themselves through assignments. + */ +export function analyzeFunction(fn: NFunc, lang: SastLanguage): FunctionTaint { + const taint = new Map() + const env: Env = { lang, taint, budget: TAINT_VISIT_BUDGET, exhausted: false } + + // Seed parameters so we can learn which ones reach sinks. + fn.params.forEach((p, i) => { + if (p) taint.set(p, [{ kind: 'param', index: i, name: p, node: fn.node }]) + }) + + const summary: FnSummary = { name: fn.name, params: fn.params, sinkParams: new Set() } + if (!fn.body) return { fn, env, summary } + + const assigns = collectAssignments(fn.body, lang) + // Fixpoint: propagate through assignments until stable (bounded). + for (let iter = 0; iter < 8; iter++) { + let changed = false + for (const a of assigns) { + const before = taint.get(a.target) ?? [] + const rhs = evalOrigins(a.value, env) + if (rhs.length === 0) continue + const merged = mergeOrigins(before, rhs) + if (merged.length !== before.length) { + taint.set(a.target, merged) + changed = true + } + } + if (!changed) break + } + // The solve and the later sink queries (taintOf) share this env. Give the + // query phase its own fresh budget so a churn-heavy solve cannot silently + // blind every subsequent sink check; `exhausted` keeps the drained solve + // reportable as degraded coverage. + env.budget = TAINT_VISIT_BUDGET + return { fn, env, summary } +} + +/** True if any taint phase for this function ran out of visit budget — its + * results may be incomplete and the file must be reported as truncated. */ +export function taintBudgetExhausted(ft: FunctionTaint): boolean { + return ft.env.exhausted +} + +/** Origins reaching a specific expression, using a solved function environment. */ +export function taintOf(node: SyntaxNode, ft: FunctionTaint): Origins { + return evalOrigins(node, ft.env) +} + +/** True if the given origins include a real (non-parameter) untrusted source. */ +export function hasRealSource(origins: Origins): boolean { + return origins.some((o) => o.kind === 'source') +} + +/** Param indexes present in the origins (for interprocedural summaries). */ +export function paramIndexes(origins: Origins): number[] { + return origins.filter((o): o is Extract => o.kind === 'param').map((o) => o.index) +} + +/** First real source origin, for flow reporting. */ +export function firstSource(origins: Origins): Extract | null { + return origins.find((o): o is Extract => o.kind === 'source') ?? null +} + +/** + * Whether a call site may bind to a same-file function summary. + * + * Interprocedural summaries are keyed by a function's last-segment name, so a + * bare name collision — e.g. `logger.run(x)` at the call site vs. a local + * `function run(cmd) { exec(cmd) }` — would otherwise resolve to the wrong + * callee and mis-report a one-hop flow. A call binds to a local-function + * summary only when it is a direct call (`run(x)`) or an in-object self call + * (`this.run(x)` / `self.run(x)`); a member call on any other receiver is a + * different function that merely shares a name, so it must not match. + */ +export function bindsToLocalFn(call: NCall): boolean { + if (!call.receiver) return true + const recv = call.receiverName?.toLowerCase() + return recv === 'this' || recv === 'self' +} + +export { walk } diff --git a/packages/analyzer-engine/src/security/types.ts b/packages/analyzer-engine/src/security/types.ts new file mode 100644 index 0000000..6f3c309 --- /dev/null +++ b/packages/analyzer-engine/src/security/types.ts @@ -0,0 +1,88 @@ +/** + * SAST engine public types. + * + * A finding is trustworthy only when it can show *why*: the source of the + * untrusted value, the sink it reached, and the path between them. Every taint + * finding carries a {@link Flow}; pattern findings (dangerous API / insecure + * config) carry only the sink site because there is no dataflow to show. + */ + +import type { SastLanguage } from './lang' + +export type Severity = 'LOW' | 'MEDIUM' | 'HIGH' | 'CRITICAL' + +/** How a finding was proven — dataflow vs. a dangerous API/config match. */ +export type FindingKind = 'taint' | 'pattern' + +export interface CodeLocation { + filePath: string + line: number + /** 1-based column, best effort. */ + column?: number + /** A short, single-line snippet of the offending code (never secrets/values). */ + snippet?: string +} + +/** + * The proof for a taint finding: where the untrusted value entered, where it + * landed, and a human-readable summary of the hops between them. `steps` + * includes source and sink as the first and last entries. + */ +export interface Flow { + source: CodeLocation & { label: string } + sink: CodeLocation & { label: string } + steps: Array + /** One-line English summary, e.g. "req.query.id → id → db.query()". */ + summary: string + /** True when the value crossed a function boundary (one-hop interprocedural). */ + interprocedural: boolean +} + +export interface SastFinding { + ruleId: string + kind: FindingKind + /** e.g. "CWE-89". */ + cwe: string + /** e.g. "A03:2021 Injection". */ + owasp: string + severity: Severity + /** Short, specific human-readable title. */ + title: string + /** Longer explanation of the risk and why this instance is flagged. */ + message: string + language: SastLanguage + filePath: string + line: number + column?: number + /** Present for taint findings; absent for pattern findings. */ + flow?: Flow + /** Remediation guidance. */ + remediation: string + /** Stable, sorted metadata for the report layer. */ + metadata?: Record +} + +/** Per-scan diagnostics so callers can be honest about coverage. */ +export interface SastDiagnostics { + /** Files handed to the engine after production/test filtering. */ + inputFiles: number + filesScanned: number + filesSkipped: number + /** Languages that could not be parsed (grammar unavailable) — SAST degraded. */ + degradedLanguages: SastLanguage[] + /** Files whose scan hit the per-file budget and returned partial results. */ + truncatedFiles: number + /** The global finding cap was reached and later findings were omitted. */ + findingsTruncated: boolean + /** A process resource bound stopped a batch before every file was scanned. */ + resourceLimitReached?: boolean + /** The isolated scan stopped before the enclosing job's wall-clock limit. */ + budgetExceeded?: boolean + /** A systemic child-runtime failure stopped later batches from starting. */ + failureReason?: string +} + +export interface SastResult { + findings: SastFinding[] + diagnostics: SastDiagnostics +} diff --git a/packages/analyzer-engine/src/structure.ts b/packages/analyzer-engine/src/structure.ts index 4e84591..2c941f8 100644 --- a/packages/analyzer-engine/src/structure.ts +++ b/packages/analyzer-engine/src/structure.ts @@ -1,4 +1,6 @@ -import type { Analyzer, AnalyzerFinding } from './types' +import { nodePackageManager } from './detect' +import { ciWorkflowFix, readmeStarterFix } from './fixes' +import type { Analyzer, AnalyzerFinding, RepoIndex } from './types' /** * Code-file count above which cross-file views (knowledge graph, duplication) @@ -7,6 +9,23 @@ import type { Analyzer, AnalyzerFinding } from './types' */ const LARGE_REPO_CODE_FILES = 1500 +/** Install command per Node package manager, for the README quick start. */ +const INSTALL_COMMAND = { pnpm: 'pnpm install', yarn: 'yarn install', npm: 'npm install', bun: 'bun install' } as const + +/** CI steps worth generating, in the order a pipeline should run them. */ +const CI_SCRIPT_ORDER = ['lint', 'build', 'test'] as const + +function rootPackageJson(index: RepoIndex): Record | undefined { + const raw = index.files.find((file) => file.path === 'package.json')?.content + if (!raw) return undefined + try { + const parsed: unknown = JSON.parse(raw) + return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed as Record : undefined + } catch { + return undefined + } +} + /** Project structure & hygiene: missing README, tests, CI, licenses, huge dirs. */ export const structureAnalyzer: Analyzer = { id: 'structure', @@ -16,6 +35,8 @@ export const structureAnalyzer: Analyzer = { const findings: AnalyzerFinding[] = [] const paths = index.files.map((f) => f.path) const has = (test: (p: string) => boolean) => paths.some(test) + const manager = nodePackageManager(index.files) + const packageJson = rootPackageJson(index) if (!has((p) => /^readme(\.md|\.rst|\.txt)?$/i.test(p))) { findings.push({ @@ -24,18 +45,39 @@ export const structureAnalyzer: Analyzer = { title: 'Missing README', description: 'The repository has no root README. New contributors and clients have no entry point to understand the project.', suggestion: 'Add a README covering purpose, setup, environment variables, and deployment.', + // The declared package name, never the directory name: analyzers run + // against materialized snapshots whose directory is a temporary id. + fix: readmeStarterFix({ + projectName: typeof packageJson?.name === 'string' && packageJson.name.trim() ? packageJson.name.trim() : 'Project', + installCommand: manager ? INSTALL_COMMAND[manager] : undefined, + }), impactScore: 60, effort: 'low', }) } if (!has((p) => p.startsWith('.github/workflows/') || p.includes('.gitlab-ci') || p.includes('.circleci'))) { + // A starter workflow only where the repository names the commands it + // would run. Inventing `npm test` for a project with no test script + // ships a pipeline that fails on its first run. + const scripts = packageJson?.scripts && typeof packageJson.scripts === 'object' + ? packageJson.scripts as Record + : {} + const fix = manager + ? ciWorkflowFix({ + manager, + hasLockfile: has((p) => ['pnpm-lock.yaml', 'yarn.lock', 'package-lock.json', 'bun.lockb', 'bun.lock'].includes(p)), + hasPackageManagerField: typeof packageJson?.packageManager === 'string', + scripts: CI_SCRIPT_ORDER.filter((script) => typeof scripts[script] === 'string'), + }) + : undefined findings.push({ category: 'STRUCTURE', severity: 'MEDIUM', title: 'No CI pipeline detected', description: 'No GitHub Actions, GitLab CI, or CircleCI configuration found. Changes are not automatically built or tested.', suggestion: 'Add a CI workflow that installs dependencies, builds, and runs tests on every pull request.', + ...(fix ? { fix } : {}), impactScore: 65, effort: 'medium', }) @@ -114,19 +156,27 @@ export const structureAnalyzer: Analyzer = { .filter((f) => f.kind === 'generated') .reduce((total, f) => total + (f.sizeBytes ?? 0), 0) const generatedKb = Math.round(generatedBytes / 1024) - if (generatedPaths.length > 0 && (generatedLoc >= 500 || generatedBytes >= 50_000)) { + // No size gate: an exclusion the reader never sees is a silent one, and a + // four-line "generated" stub is exactly where a hidden change hides best. + // Every excluded path is named, so nothing is skipped off the record. + if (generatedPaths.length > 0) { const plural = generatedPaths.length > 1 - const volume = `~${generatedLoc.toLocaleString()} LOC, ${generatedKb.toLocaleString()} KB` + // KB understates a four-line stub as "0 KB"; bytes understates a 375KB + // bundle. Report each in the unit that does not lie about it. + const size = generatedBytes >= 1024 + ? `${generatedKb.toLocaleString()} KB` + : `${generatedBytes.toLocaleString()} bytes` + const volume = `~${generatedLoc.toLocaleString()} LOC, ${size}` findings.push({ category: 'STRUCTURE', severity: 'LOW', - title: `Generated code excluded from analysis (${generatedPaths.length} file${plural ? 's' : ''}, ${generatedKb.toLocaleString()} KB)`, - description: `CodeTruss detected ${generatedPaths.length} machine-generated or minified file${plural ? 's' : ''} (${volume}, e.g. \`${generatedPaths[0]}\`) and excluded ${plural ? 'them' : 'it'} from LOC totals, scores, and the architecture graph so ${plural ? 'they' : 'it'} don't inflate metrics or produce spurious "oversized file" / "duplicated logic" findings. Minified bundles report few lines for their size, so the KB figure is the honest measure of what was skipped.`, + title: `Generated code excluded from analysis (${generatedPaths.length} file${plural ? 's' : ''}, ${size})`, + description: `CodeTruss detected ${generatedPaths.length} machine-generated or minified file${plural ? 's' : ''} (${volume}) and excluded ${plural ? 'them' : 'it'} from LOC totals, scores, and the architecture graph so ${plural ? "they don't" : "it doesn't"} inflate metrics or produce spurious "oversized file" / "duplicated logic" findings. Secret scanning still reads ${plural ? 'them' : 'it'} in full — a committed credential is reported wherever it lives. Excluded: ${generatedPaths.map((path) => `\`${path}\``).join(', ')}. Minified bundles report few lines for their size, so the byte figure is the honest measure of what was skipped.`, filePath: generatedPaths[0], suggestion: 'Keep generated and vendored bundles out of review scope — regenerate them at build time, or mark them linguist-generated in .gitattributes.', impactScore: 20, effort: 'low', - metadata: { files: generatedPaths.length, loc: generatedLoc, bytes: generatedBytes }, + metadata: { files: generatedPaths.length, loc: generatedLoc, bytes: generatedBytes, paths: generatedPaths }, }) } diff --git a/packages/analyzer-engine/src/types.ts b/packages/analyzer-engine/src/types.ts index ace0118..b39fc75 100644 --- a/packages/analyzer-engine/src/types.ts +++ b/packages/analyzer-engine/src/types.ts @@ -26,6 +26,17 @@ export interface IndexedFile { loc: number sha: string | null content: string | null + /** + * Text of a file the indexer excluded from analysis by classification + * (generated, minified). `content` stays null so LOC totals, the knowledge + * graph, and every quality analyzer keep skipping it — that exclusion exists + * to stop machine-written output from producing spurious findings. + * + * Secret scanning is the one pass that must never be exempted by provenance: + * a committed credential is a credential whether or not a generator wrote the + * line, so `secretsAnalyzer` reads this instead. + */ + excludedContent?: string | null } export interface IndexCoverage { @@ -55,6 +66,30 @@ export interface RepoIndex { coverage?: IndexCoverage } +/** + * A concrete, reviewable change that would resolve one finding. + * + * SUGGESTION ONLY. CodeTruss never applies, writes, or executes it, and never + * presents it as required — a change derived from a single matched line cannot + * know the rest of the codebase. An analyzer attaches one only when the + * finding's own evidence (path, line, matched text, index facts) makes the + * change correct for that exact instance. When the right fix is ambiguous the + * prose `suggestion` stays the whole answer and no `fix` is attached: a wrong + * autofix is a false positive with extra damage. + */ +export interface FindingFix { + /** What the change does, in one line. */ + description: string + /** `diff` is unified-diff text; `snippet` is replacement or starter content. */ + kind: 'diff' | 'snippet' + /** Fenced-code language for rendering — `diff`, `sh`, `yaml`, `markdown`, … */ + language: string + /** The suggested text itself. */ + content: string + /** What the reader must confirm before applying. Never empty. */ + safetyNote: string +} + export interface AnalyzerFinding { category: FindingCategory severity: FindingSeverity @@ -63,6 +98,8 @@ export interface AnalyzerFinding { filePath?: string line?: number suggestion?: string + /** Optional ready-to-review change. Absent whenever the correct fix is ambiguous. */ + fix?: FindingFix impactScore: number effort?: 'low' | 'medium' | 'high' metadata?: Record @@ -118,6 +155,18 @@ export function analyzerResult(output: AnalyzerFinding[]): AnalyzerRunResult { export interface AnalyzerContext { /** The SAST pass (security rules + taint tracking) runs for this analysis. */ sast: boolean + /** + * Vulnerability classes the SAST pass did NOT check in this analysis, when it + * ran with a reduced rule set. + * + * The CLI runs a precision-validated SUBSET of the rule pack behind its + * zero-dependency parser: real injection and AI-agent-defect coverage, but not + * the classes that still need the hosted graph. "The pass ran" and "every + * class was checked" are different claims, and a receipt that conflated them + * would be the same blind spot the coverage analyzer exists to close — just + * one level down. + */ + sastUncheckedClasses?: readonly string[] } /** diff --git a/packages/cli/CHANGELOG.md b/packages/cli/CHANGELOG.md index ead7df3..bc18773 100644 --- a/packages/cli/CHANGELOG.md +++ b/packages/cli/CHANGELOG.md @@ -5,6 +5,175 @@ checksums are published at [file.path, file.oldPath].filter(Boolean) as string[])) + const changed = new Set((files.flatMap((file) => [file.path, file.oldPath].filter(Boolean) as string[])).map(posixPath)) return findings.filter((finding) => { - const filePath = finding.filePath + const filePath = finding.filePath ? posixPath(finding.filePath) : undefined return Boolean(filePath && [...changed].some((path) => filePath === path || filePath.startsWith(`${path}/`) || path.startsWith(`${filePath}/`))) }) } @@ -187,10 +202,32 @@ export function computeVerdict(input: { for (const issue of input.baselineEvidenceIssues ?? []) review.push(`baseline evidence limitation resolved in the final tree: ${issue}`) for (const issue of input.advisoryEvidenceIssues ?? []) review.push(`index coverage was partial but authoritative: ${issue}`) for (const verification of input.verifications.filter((item) => item.exitCode !== 0)) failed.push(`verification command failed: ${verification.command}`) - const blocking = input.findings.filter((finding) => severityRank[finding.severity] >= severityRank.HIGH && (finding.category === 'SECURITY_HYGIENE' || finding.category === 'DEPENDENCY')) + /** + * The local security pass reports; it does not block — yet. + * + * Its findings land in SECURITY_HYGIENE at HIGH/CRITICAL, which the rule below + * would otherwise turn into FAILED, and a FAILED verdict at `Stop` halts the + * developer's agent mid-turn. One false positive doing that is how a security + * tool gets uninstalled, so this pass surfaces as REVIEW_REQUIRED on its first + * release. Precision earns the right to block after real-repo soak; severity + * does not grant it. Promotion is a deliberate later change, not a default. + */ + const isLocalSast = (finding: AnalyzerFinding) => finding.analyzerId === LOCAL_SAST_PASS_ID + const blocking = input.findings.filter( + (finding) => + severityRank[finding.severity] >= severityRank.HIGH && + (finding.category === 'SECURITY_HYGIENE' || finding.category === 'DEPENDENCY') && + !isLocalSast(finding), + ) if (blocking.length) failed.push(`${blocking.length} high/critical security or dependency finding(s) affect changed files`) + const localSastFindings = input.findings.filter(isLocalSast) + if (localSastFindings.length) { + const rules = [...new Set(localSastFindings.map((finding) => String(finding.metadata?.ruleId ?? 'security')))].sort() + review.push(`${localSastFindings.length} local security finding(s) affect changed files (${rules.join(', ')})`) + } const denied = input.files.filter((file) => file.classification === 'denied') const unexpected = input.files.filter((file) => file.classification === 'unexpected') + const inferred = input.files.filter((file) => file.classification === 'inferred') const sensitive = input.files.filter((file) => file.sensitive) const deps = input.files.filter((file) => file.dependency) if (denied.length) review.push(`${denied.length} file(s) changed in denied paths: ${denied.slice(0, 5).map((file) => file.path).join(', ')}`) @@ -198,7 +235,9 @@ export function computeVerdict(input: { if (sensitive.length) review.push(`sensitive surfaces changed: ${sensitive.slice(0, 5).map((file) => `${file.path} (${file.sensitive})`).join(', ')}`) if (deps.length) review.push(`dependency manifests or lockfiles changed: ${deps.slice(0, 5).map((file) => file.path).join(', ')}`) if (input.startDirty) review.push('the working tree was dirty at session start, so exact agent attribution is uncertain') - const reviewFindings = input.findings.filter((finding) => severityRank[finding.severity] >= severityRank.MEDIUM && !blocking.includes(finding)) + const reviewFindings = input.findings.filter( + (finding) => severityRank[finding.severity] >= severityRank.MEDIUM && !blocking.includes(finding) && !isLocalSast(finding), + ) if (reviewFindings.length) review.push(`${reviewFindings.length} medium-or-higher analyzer finding(s) affect changed files`) if (input.llm?.diffCoverage?.truncated) { review.push(`local ${input.llm.provider} review covered ${input.llm.diffCoverage.reviewedBytes} of ${input.llm.diffCoverage.totalBytes} diff bytes`) @@ -207,7 +246,14 @@ export function computeVerdict(input: { if (!input.verifications.length) notes.push('no verification commands were configured') else if (!input.verifications.some((item) => item.exitCode !== 0)) notes.push(`all ${input.verifications.length} verification command(s) passed`) if (!input.files.length) notes.push('no repository files changed') - else if (!denied.length && !unexpected.length) notes.push(`all ${input.files.length} changed file(s) are within approved scope`) + // Scope reached by inference is in scope, but it is not scope the repository + // approved. Saying "all files are within approved scope" over it would be the + // one wrong sentence on an otherwise honest receipt. + else if (!denied.length && !unexpected.length) { + notes.push(inferred.length + ? `${input.files.length - inferred.length} changed file(s) are within approved scope; ${inferred.length} more matched scope inferred from this turn and disclosed on the receipt` + : `all ${input.files.length} changed file(s) are within approved scope`) + } if (failed.length) return { verdict: 'FAILED', reasons: [...failed, ...review] } if (review.length) return { verdict: 'REVIEW_REQUIRED', reasons: review } return { verdict: 'PASS', reasons: notes } diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 22461e5..6ae1e3a 100644 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -27,12 +27,14 @@ import { readHookTurnContext, type AgentHookSurface, } from './hook-runtime.js' +import { topFixSuggestion } from './fix-suggestions.js' import { CODETRUSS_PRE_COMMIT_ENV, doctorHooks, hookStatus, inspectLocalHookHealth, installHooks, uninstallHooks } from './hooks.js' import { hostedAuthStatus, loginHosted, logoutHosted } from './hosted-auth.js' import { parseInternalHookResultRequest, writeInternalHookResult } from './hook-result.js' import { reviewWithLlm } from './llm.js' import { collectLocalMetrics, renderLocalMetrics } from './metrics.js' import { classifyPath, isDependencyFile, sensitiveCategory } from './policy.js' +import { applyInferredScope, inferTurnScope } from './scope-inference.js' import { policyFingerprint } from './policy-fingerprint.js' import { CODETRUSS_EVIDENCE_OBJECT_DIRECTORY_ENV, @@ -310,7 +312,7 @@ Usage: codetruss verify [id|latest] codetruss sync [id|latest] [--dry-run] codetruss auth login|status|logout - codetruss verify-policy [status|trust|revoke] + codetruss verify-policy [status|trust|trust-key|revoke] codetruss hooks install|status|doctor|uninstall [pre-commit|claude|codex|all] Exit codes: PASS=0, REVIEW_REQUIRED=1, FAILED=2, usage/environment=3.` @@ -436,13 +438,19 @@ async function executeReview(parsed: Parsed, root: string, liveConfig: CliConfig gitEnvironment: immutableTarget.gitEnvironment, }) - const files = await changedFiles( + const approvedScopeFiles = await changedFiles( root, immutableTarget.baselineTreeish, false, (path, oldPath) => classifyPath(path, oldPath, options.allow, options.deny), sensitiveCategory, isDependencyFile, immutableTarget.finalTreeish, { env: immutableTarget.gitEnvironment }, ) + // Scope inference needs the whole changed set, so it runs after + // classification rather than inside it. Deny and sensitive surfaces are + // already decided at this point and inference cannot reopen them. + const { files, roots: inferredScope } = applyInferredScope(approvedScopeFiles, inferTurnScope({ + task, files: approvedScopeFiles, allow: options.allow, deny: options.deny, + })) const diff = await captureDiffEvidence(root, immutableTarget.baselineTreeish, false, files, { targetTreeish: immutableTarget.finalTreeish, env: immutableTarget.gitEnvironment, @@ -563,7 +571,12 @@ async function executeReview(parsed: Parsed, root: string, liveConfig: CliConfig git: { baselineTree: baselineSnapshot.tree, finalTree: finalSnapshot.tree }, policy: { sha256: policyFingerprint(options, config) }, startDirty, startDirtyFiles, agent, - scope: { allow: options.allow, deny: options.deny }, files, + scope: { + allow: options.allow, + deny: options.deny, + ...(inferredScope.length ? { inferred: inferredScope } : {}), + }, + files, diff: { sha256: sha256(diff.patch), bytes: diff.capturedBytes, @@ -609,6 +622,9 @@ async function executeReview(parsed: Parsed, root: string, liveConfig: CliConfig verdict: receipt.verdict, receiptPath: resolve(paths.markdown), reasons: receipt.reasons, + // Give the agent the one change it can make next turn, before a + // person ever opens the receipt. Suggestion only — the wording says so. + suggestion: topFixSuggestion(receipt.analyzers.findings), }, ) } diff --git a/packages/cli/src/config.ts b/packages/cli/src/config.ts index d25bd1e..95ba0ab 100644 --- a/packages/cli/src/config.ts +++ b/packages/cli/src/config.ts @@ -200,7 +200,30 @@ export async function initialize(root: string, force = false, options: Initializ return path } +/** + * Why auto-detection found nothing, when the answer is actionable. Setup used + * to print a flat "no commands were detected", which reads as "this repository + * has no tests" even when the repository has `"test": "vitest run"` and is one + * `npm install` away from a lockfile. + */ +export type VerifyDetectionBlocker = 'missing-lockfile' | 'package-manager-unavailable' + +export interface VerifyDetection { + /** Commands safe to record. Empty whenever a blocker is set. */ + commands: string[] + blocker?: VerifyDetectionBlocker + /** + * What was found: full commands when the package manager is known, bare + * script names under `missing-lockfile`, where no manager could be resolved. + */ + candidates: string[] +} + async function detectVerify(root: string): Promise { + return (await detectVerifyCommands(root)).commands +} + +export async function detectVerifyCommands(root: string): Promise { const exists = async (name: string) => access(join(root, name)).then(() => true, () => false) // Detect commands without executing repository-controlled code. This also // recognizes Windows package-manager shims such as `pnpm.cmd`. @@ -234,22 +257,35 @@ async function detectVerify(root: string): Promise { throw new Error(`could not inspect package.json: ${error instanceof Error ? error.message : String(error)}`) } } - if (await exists('pnpm-lock.yaml')) { - const scripts = await packageScripts() - return await available('pnpm') ? ['lint', 'test'].filter((name) => typeof scripts[name] === 'string').map((name) => `pnpm ${name}`) : [] - } - if (await exists('package-lock.json')) { + /** The same [lint, test] collection for every Node package manager. */ + const nodeScripts = async ( + manager: 'pnpm' | 'npm' | 'yarn', + format: (name: string) => string, + ): Promise => { const scripts = await packageScripts() - return await available('npm') && typeof scripts.test === 'string' ? ['npm test'] : [] + const candidates = ['lint', 'test'].filter((name) => typeof scripts[name] === 'string').map(format) + if (!candidates.length) return { commands: [], candidates: [] } + return await available(manager) + ? { commands: candidates, candidates } + : { commands: [], candidates, blocker: 'package-manager-unavailable' } } - if (await exists('yarn.lock')) { - const scripts = await packageScripts() - return await available('yarn') && typeof scripts.test === 'string' ? ['yarn test'] : [] + + if (await exists('pnpm-lock.yaml')) return nodeScripts('pnpm', (name) => `pnpm ${name}`) + // `npm lint` is not a command; only lifecycle names run without `run`. + if (await exists('package-lock.json')) return nodeScripts('npm', (name) => `npm run ${name}`) + if (await exists('yarn.lock')) return nodeScripts('yarn', (name) => `yarn ${name}`) + if ((await exists('go.mod')) && await available('go')) return { commands: ['go test ./...'], candidates: ['go test ./...'] } + if ((await exists('Cargo.toml')) && await available('cargo')) return { commands: ['cargo test'], candidates: ['cargo test'] } + if (((await exists('pyproject.toml')) || (await exists('requirements.txt'))) && await available('pytest')) { + return { commands: ['pytest'], candidates: ['pytest'] } } - if ((await exists('go.mod')) && await available('go')) return ['go test ./...'] - if ((await exists('Cargo.toml')) && await available('cargo')) return ['cargo test'] - if (((await exists('pyproject.toml')) || (await exists('requirements.txt'))) && await available('pytest')) return ['pytest'] - return [] + // A package.json with runnable scripts and no lockfile is the common case + // where detection legitimately fails: CodeTruss cannot tell which package + // manager to invoke. Say that, rather than implying there is nothing to run. + const scripts = await packageScripts() + const candidates = ['lint', 'test'].filter((name) => typeof scripts[name] === 'string') + if (candidates.length) return { commands: [], candidates, blocker: 'missing-lockfile' } + return { commands: [], candidates: [] } } /** diff --git a/packages/cli/src/fix-suggestions.ts b/packages/cli/src/fix-suggestions.ts new file mode 100644 index 0000000..04a9596 --- /dev/null +++ b/packages/cli/src/fix-suggestions.ts @@ -0,0 +1,109 @@ +import type { AnalyzerFinding } from '@codetruss/analyzer-engine' + +/** + * Rendering for `AnalyzerFinding.fix`. + * + * One rule governs every string in this module: a fix is a SUGGESTION. It was + * not applied, it is not required, and nothing here may read as though CodeTruss + * changed a file. The framing is repeated at the block level and again per fix + * because the two are read in different places — the receipt section header by a + * person, the single-line summary by an agent that may act on it immediately. + */ + +const SEVERITY_RANK = { INFO: 0, LOW: 1, MEDIUM: 2, HIGH: 3, CRITICAL: 4 } as const + +/** Receipts stay readable; the signed JSON always carries every fix. */ +const MAX_RENDERED_FIXES = 10 + +/** Bound one summary line well inside the hook result's 2,000-character cap. */ +const MAX_SUMMARY_CHARS = 1_200 + +export const FIX_DISCLAIMER = + 'Suggested changes only. CodeTruss did not apply, write, or run any of them, and a suggestion derived from one ' + + 'finding cannot see the rest of the codebase — review each change before using it.' + +type FixFinding = AnalyzerFinding & { fix: NonNullable } + +function hasFix(finding: AnalyzerFinding): finding is FixFinding { + return finding.fix !== undefined +} + +/** Highest severity first; ties keep analyzer order so output stays stable. */ +export function findingsWithFixes(findings: AnalyzerFinding[]): FixFinding[] { + return findings + .filter(hasFix) + .map((finding, order) => ({ finding, order })) + .sort((left, right) => ( + SEVERITY_RANK[right.finding.severity] - SEVERITY_RANK[left.finding.severity] || left.order - right.order + )) + .map((entry) => entry.finding) +} + +/** + * A fence longer than any backtick run inside the content. Fix content is often + * Markdown that contains its own fences, and a three-backtick block would end + * early and spill raw snippet text into the receipt body. + */ +function fenceFor(content: string): string { + const longest = Math.max(0, ...[...content.matchAll(/`+/g)].map((match) => match[0].length)) + return '`'.repeat(Math.max(3, longest + 1)) +} + +function location(finding: AnalyzerFinding): string { + if (!finding.filePath) return 'repository' + return `\`${finding.filePath}${finding.line ? `:${finding.line}` : ''}\`` +} + +/** + * The receipt's suggested-fix section, or NO LINES AT ALL when nothing carries + * a fix. The empty case is load-bearing: receipts signed before fixes existed + * must still render byte-for-byte, so this block may never emit a header it + * cannot fill. + */ +export function suggestedFixLines(findings: AnalyzerFinding[]): string[] { + const withFixes = findingsWithFixes(findings) + if (withFixes.length === 0) return [] + const rendered = withFixes.slice(0, MAX_RENDERED_FIXES) + const lines = [ + `## Suggested fixes (${withFixes.length})`, + '', + FIX_DISCLAIMER, + '', + ] + for (const finding of rendered) { + const fence = fenceFor(finding.fix.content) + lines.push( + `### ${finding.severity} — ${finding.title.replaceAll('\n', ' ')} (${location(finding)})`, + '', + finding.fix.description, + '', + `${fence}${finding.fix.language}`, + ...finding.fix.content.replace(/\n$/, '').split('\n'), + fence, + '', + `Before applying: ${finding.fix.safetyNote}`, + '', + ) + } + if (withFixes.length > rendered.length) { + lines.push(`${withFixes.length - rendered.length} further finding(s) carry a suggested change; the signed JSON receipt has all of them.`, '') + } + return lines +} + +/** + * One line for the agent-facing Stop summary: the highest-severity finding's + * suggestion, so the agent can correct the change before a person ever reads + * the receipt. Returns undefined when nothing carries a fix. + */ +export function topFixSuggestion(findings: AnalyzerFinding[]): string | undefined { + const finding = findingsWithFixes(findings)[0] + if (!finding) return undefined + const where = finding.filePath ? ` at ${finding.filePath}${finding.line ? `:${finding.line}` : ''}` : '' + return [ + `Suggested fix (NOT applied — review before using) for ${finding.severity} "${finding.title.replaceAll('\n', ' ')}"${where}:`, + finding.fix.description, + finding.fix.safetyNote, + 'The exact suggested change is in the receipt above.', + ].join(' ').slice(0, MAX_SUMMARY_CHARS) +} diff --git a/packages/cli/src/git-snapshot.ts b/packages/cli/src/git-snapshot.ts index ae49cc9..fcd9dc3 100644 --- a/packages/cli/src/git-snapshot.ts +++ b/packages/cli/src/git-snapshot.ts @@ -139,13 +139,22 @@ interface SourceIdentity { target?: Buffer } -class WorkingTreeChangedError extends Error { +export class WorkingTreeChangedError extends Error { constructor(path: string) { super(`working tree changed while snapshotting ${JSON.stringify(path)}`) this.name = 'WorkingTreeChangedError' } } +export function isWorkingTreeChangedError(error: unknown): boolean { + let current = error + for (let depth = 0; depth < 4 && current instanceof Error; depth++) { + if (current instanceof WorkingTreeChangedError) return true + current = (current as Error & { cause?: unknown }).cause + } + return false +} + function splitNulUtf8(output: Buffer): string[] { const decoder = new TextDecoder('utf-8', { fatal: true }) const paths: string[] = [] diff --git a/packages/cli/src/hook-baseline.ts b/packages/cli/src/hook-baseline.ts index 502d9d4..11968c2 100644 --- a/packages/cli/src/hook-baseline.ts +++ b/packages/cli/src/hook-baseline.ts @@ -3,7 +3,7 @@ import { lstat, readdir, readlink } from 'node:fs/promises' import { join } from 'node:path' import { spawn } from 'node:child_process' import { dirtyFiles, head } from './git.js' -import { materializeWorkingTreeSnapshot } from './git-snapshot.js' +import { materializeWorkingTreeSnapshot, WorkingTreeChangedError } from './git-snapshot.js' import { gitCommandArguments, runGit, runGitText } from './git-process.js' import type { PrivateGitObjectStore } from './private-git-object-store.js' @@ -138,12 +138,12 @@ function assertGitCaptureState( expectedDirtyFiles: readonly string[], expectedGitlinks: Map, ): void { - if (head(repoRoot) !== expectedHead) throw new Error('HEAD changed while the hook baseline was being captured') + if (head(repoRoot) !== expectedHead) throw new WorkingTreeChangedError('HEAD') if (JSON.stringify(dirtyFiles(repoRoot)) !== JSON.stringify(expectedDirtyFiles)) { - throw new Error('Git status changed while the hook baseline was being captured') + throw new WorkingTreeChangedError('Git status') } if (!mapsEqual(resolvedGitlinks(repoRoot), expectedGitlinks)) { - throw new Error('Gitlink state changed while the hook baseline was being captured') + throw new WorkingTreeChangedError('Gitlink state') } } diff --git a/packages/cli/src/hook-result.ts b/packages/cli/src/hook-result.ts index 27fa315..c8960c0 100644 --- a/packages/cli/src/hook-result.ts +++ b/packages/cli/src/hook-result.ts @@ -19,6 +19,12 @@ export interface InternalHookResult { verdict: 'PASS' | 'REVIEW_REQUIRED' | 'FAILED' receiptPath: string reasons: string[] + /** + * The highest-severity finding's suggested change, carried separately from + * `reasons` so the display cap on reasons can never drop it. A suggestion is + * not a reason for the verdict; it is the next action the agent can take. + */ + suggestion?: string } function present(value: string | undefined): boolean { @@ -109,6 +115,10 @@ export async function writeInternalHookResult( const reasons = result.reasons .slice(0, MAX_RESULT_REASONS) .map((reason) => reason.slice(0, MAX_RESULT_REASON_CHARS)) + if (result.suggestion !== undefined && typeof result.suggestion !== 'string') { + throw new Error('CodeTruss hook result suggestion must be a string') + } + const suggestion = result.suggestion?.slice(0, MAX_RESULT_REASON_CHARS) const value = `${JSON.stringify({ version: 1, @@ -116,6 +126,7 @@ export async function writeInternalHookResult( verdict: result.verdict, receiptPath: result.receiptPath, reasons, + ...(suggestion ? { suggestion } : {}), })}\n` if (Buffer.byteLength(value) > MAX_RESULT_BYTES) { throw new Error(`CodeTruss hook result exceeds ${MAX_RESULT_BYTES} bytes`) diff --git a/packages/cli/src/hook-runtime.ts b/packages/cli/src/hook-runtime.ts index eddb61b..ac5b1df 100644 --- a/packages/cli/src/hook-runtime.ts +++ b/packages/cli/src/hook-runtime.ts @@ -4,7 +4,9 @@ import { basename, dirname, isAbsolute, join, relative, resolve } from 'node:pat import type { CliConfig } from './types.js' import { receiptDir } from './config.js' import { createExactSnapshotCommit, deleteLegacyHookBaseline, type ExactSnapshotCommit } from './hook-baseline.js' +import { isWorkingTreeChangedError } from './git-snapshot.js' import { classifyPath, isDependencyFile, sensitiveCategory } from './policy.js' +import { applyInferredScope, inferTurnScope } from './scope-inference.js' import { runGit, runGitText } from './git-process.js' import { runLocalCommand } from './local-command.js' import { @@ -111,6 +113,7 @@ const MAX_SELECTOR_BYTES = 16 * 1024 const MAX_STATE_BYTES = 256 * 1024 const MAX_REVIEW_OUTPUT_CHARS = 6_000 const REVIEW_TIMEOUT_MS = 5 * 60 * 1_000 +const MAX_EXACT_CAPTURE_ATTEMPTS = 3 const STATE_VERSION_DIR = 'v2' const LEGACY_STATE_VERSION_DIR = 'v1' const PATH_KEY_HEX_CHARS = 24 @@ -158,6 +161,18 @@ function systemMessage(message: string): HookOutput { return { systemMessage: message.slice(0, 10_000) } } +/** + * Baseline capture is instrumentation, not a verdict. Blocking UserPromptSubmit + * fails closed against a *person*: their prompt is erased and they are told to + * try again, because CodeTruss could not take a snapshot. Never do that. Emit a + * note explaining the turn is unreceipted and let the prompt through — Stop + * remains the enforcement point, and it still fails closed when a turn reaches + * it without a provable baseline, so an agent cannot silently finish unreviewed. + */ +function captureNotice(message: string): HookOutput { + return systemMessage(message) +} + /** * Stop failures get one blocking turn so an agent cannot silently finish * without a receipt. Claude marks the resulting continuation with @@ -1239,10 +1254,37 @@ function readableTask(prompt: string): string { return summary ? `[${tag}] ${summary}` : `[${tag}]` } -function inputTask(input: HookInput): string | undefined { +/** + * Agent harnesses deliver turns that carry no prompt at all: background-task + * notifications, hook feedback continuations, and resumed agents all reach + * UserPromptSubmit with an absent or empty `prompt` field. That is a legitimate + * turn shape, not malformed input — an exact baseline is a snapshot of the + * working tree, and the tree exists whether or not a human typed anything. + * Label those turns honestly and capture them like any other, so the turn still + * earns a receipt instead of stranding Stop without a baseline. + */ +const PROMPTLESS_TASK = '[no prompt] agent turn with no submitted prompt text' + +function inputTask(input: HookInput): string { const prompt = asNonEmptyString(input.prompt) - if (!prompt) return undefined - return asNonEmptyString(readableTask(prompt).slice(0, MAX_TASK_CHARS)) + if (!prompt) return PROMPTLESS_TASK + return asNonEmptyString(readableTask(prompt).slice(0, MAX_TASK_CHARS)) ?? PROMPTLESS_TASK +} + +async function captureExactSnapshot( + root: string, + snapshotParent: string, + objectStore: PrivateGitObjectStore, + capture: NonNullable, +): Promise { + for (let attempt = 1; attempt <= MAX_EXACT_CAPTURE_ATTEMPTS; attempt++) { + try { + return await capture(root, snapshotParent, objectStore) + } catch (error) { + if (!isWorkingTreeChangedError(error) || attempt === MAX_EXACT_CAPTURE_ATTEMPTS) throw error + } + } + throw new Error('exact snapshot capture exhausted its retry bound') } type NamedLockName = 'capture.lock' | 'stop.lock' | 'migration.lock' @@ -1378,18 +1420,17 @@ async function capturePromptBaseline( ): Promise { const sessionId = asNonEmptyString(input.session_id) const task = inputTask(input) - if (!sessionId) return blockDecision('CodeTruss could not capture an exact turn baseline: hook input is missing session_id.') - if (!task) return blockDecision('CodeTruss could not capture an exact turn baseline: hook input is missing the submitted prompt.') + if (!sessionId) return captureNotice('CodeTruss could not capture an exact turn baseline: hook input is missing session_id.') let turnId: string | undefined try { turnId = inputTurnId(input) } catch (error) { - return blockDecision(`CodeTruss could not capture an exact turn baseline: ${safeError(error)}.`) + return captureNotice(`CodeTruss could not capture an exact turn baseline: ${safeError(error)}.`) } const prepared = await gitStateRoot(root, surface, sessionId) if (prepared.legacyBlockReason) { - return blockDecision(`CodeTruss could not safely migrate private hook evidence: ${prepared.legacyBlockReason}.`) + return captureNotice(`CodeTruss could not safely migrate private hook evidence: ${prepared.legacyBlockReason}.`) } const base = prepared.base const sessionDir = sessionStateDir(base, surface, sessionId) @@ -1403,7 +1444,7 @@ async function capturePromptBaseline( await ensurePrivateDirectory(turnDir) const now = dependencies.now?.() ?? new Date() const lock = await acquireNamedLock(turnDir, 'capture.lock', now) - if (!lock) return blockDecision('CodeTruss exact baseline capture is already running for this agent turn.') + if (!lock) return captureNotice('CodeTruss exact baseline capture is already running for this agent turn.') try { const contextPath = turnContextPath(turnDir) const storePath = turnObjectStorePath(turnDir) @@ -1411,14 +1452,14 @@ async function capturePromptBaseline( try { existing = await readBoundedRegularJson(statePath, MAX_STATE_BYTES) } catch (error) { - return blockDecision(`CodeTruss refused to replace unsafe or invalid existing turn state: ${safeError(error)}.`) + return captureNotice(`CodeTruss refused to replace unsafe or invalid existing turn state: ${safeError(error)}.`) } if (existing && (!exactStateIdentity(existing, surface, hash(sessionId), [turnKey], prepared.worktreeIdentity) || existing.turnId !== turnId)) { - return blockDecision('CodeTruss refused to replace hook evidence whose session, surface, turn, or stable Git worktree ownership cannot be proven.') + return captureNotice('CodeTruss refused to replace hook evidence whose session, surface, turn, or stable Git worktree ownership cannot be proven.') } if (existing && existing.taskHash !== hash(task)) { - return blockDecision('CodeTruss refused to reuse exact turn evidence for a different prompt task.') + return captureNotice('CodeTruss refused to reuse exact turn evidence for a different prompt task.') } if (exactStateIdentity(existing, surface, hash(sessionId), [turnKey], prepared.worktreeIdentity) && existing.turnId === turnId && existing.status === 'ready' && existing.objectStoreVersion === 1 @@ -1427,7 +1468,7 @@ async function capturePromptBaseline( await openPrivateGitObjectStore(root, storePath) const context = await readHookTurnContext(contextPath, existing.contextSha256) if (context.task !== task || existing.taskHash !== hash(context.task)) { - return blockDecision('CodeTruss refused to reuse an exact baseline for a different prompt task.') + return captureNotice('CodeTruss refused to reuse an exact baseline for a different prompt task.') } await writePrivateJson(currentPath, { version: 1, turnKey, ...(turnId ? { turnId } : {}) } satisfies CurrentTurn) return undefined @@ -1457,7 +1498,12 @@ async function capturePromptBaseline( let objectStore: PrivateGitObjectStore | undefined try { objectStore = await initializePrivateGitObjectStore(root, storePath) - const baseline = await (dependencies.captureBaseline ?? createExactSnapshotCommit)(root, join(turnDir, 's'), objectStore) + const baseline = await captureExactSnapshot( + root, + join(turnDir, 's'), + objectStore, + dependencies.captureBaseline ?? createExactSnapshotCommit, + ) const context: HookTurnContext = { version: 1, surface, @@ -1492,7 +1538,7 @@ async function capturePromptBaseline( error: message, updatedAt: (dependencies.now?.() ?? new Date()).toISOString(), } satisfies HookState) - return blockDecision(message) + return captureNotice(message) } } finally { await releaseNamedLock(lock) @@ -1584,15 +1630,27 @@ async function fastPathFeedback( const normalized = rawPaths.map((path) => normalizeHookPath(root, cwd, path)) const outside = [...new Set(normalized.flatMap((item) => item.outside ? [item.outside] : []))] const paths = [...new Set(normalized.flatMap((item) => item.path ? [item.path] : []))].sort() + const observed = paths.map((path) => ({ + path, + classification: classifyPath(path, undefined, config.allow, config.deny), + sensitive: sensitiveCategory(path), + dependency: isDependencyFile(path), + })) + // This check sees only the paths in one tool call, so it can infer strictly + // less than the Stop-time receipt — never more. A path it still calls out may + // yet be in scope by inference once the whole turn is visible, which is what + // the closing line about the full receipt already says. Staying quiet about a + // path inference has already covered is what keeps a first session readable; + // the receipt, not this line, is where inference is disclosed. + const { files: checked } = applyInferredScope(observed, inferTurnScope({ + task: '', files: observed, allow: config.allow, deny: config.deny, + })) const warnings: string[] = [] - for (const path of paths) { - const classification = classifyPath(path, undefined, config.allow, config.deny) - const sensitive = sensitiveCategory(path) - const dependency = isDependencyFile(path) - if (classification === 'denied') warnings.push(`${path}: denied by the task scope`) - else if (classification === 'unexpected') warnings.push(`${path}: outside the allowed task scope`) - if (sensitive) warnings.push(`${path}: sensitive ${sensitive} surface`) - if (dependency) warnings.push(`${path}: dependency or lockfile surface`) + for (const file of checked) { + if (file.classification === 'denied') warnings.push(`${file.path}: denied by the task scope`) + else if (file.classification === 'unexpected') warnings.push(`${file.path}: outside the allowed task scope`) + if (file.sensitive) warnings.push(`${file.path}: sensitive ${file.sensitive} surface`) + if (file.dependency) warnings.push(`${file.path}: dependency or lockfile surface`) } for (const path of outside) warnings.push(`${path}: resolves outside the repository`) if (warnings.length === 0) return undefined @@ -1651,6 +1709,8 @@ interface HookReviewResultDocument { verdict: 'PASS' | 'REVIEW_REQUIRED' | 'FAILED' receiptPath: string reasons: string[] + /** Optional: absent from every result written before fix suggestions existed. */ + suggestion?: string } function isContainedPath(parent: string, candidate: string): boolean { @@ -1661,7 +1721,13 @@ function isContainedPath(parent: string, candidate: string): boolean { function exactReviewResultDocument(value: unknown, attemptId: string): value is HookReviewResultDocument { if (!value || typeof value !== 'object' || Array.isArray(value)) return false const document = value as Record - return Object.keys(document).sort().join(',') === 'attemptId,reasons,receiptPath,verdict,version' + // Two exact key sets, not a loose superset: a result carrying a suggestion, + // and one written by a CLI that predates them. Anything else is rejected. + const keys = Object.keys(document).sort().join(',') + return (keys === 'attemptId,reasons,receiptPath,verdict,version' + || keys === 'attemptId,reasons,receiptPath,suggestion,verdict,version') + && (document.suggestion === undefined + || (typeof document.suggestion === 'string' && document.suggestion.length > 0 && document.suggestion.length <= 2_000)) && document.version === 1 && document.attemptId === attemptId && /^[0-9a-f]{64}$/.test(document.attemptId) @@ -1793,7 +1859,10 @@ async function reviewSummary( } } const reasons = document.reasons.slice(0, 5).map((reason) => `- ${reason}`).join('\n') - const message = `CodeTruss ${document.verdict}. Receipt: ${document.receiptPath}${reasons ? `\n${reasons}` : ''}` + // The suggestion follows the reasons and is never subject to their display + // cap: it is the one line the agent can act on before a person reads this. + const suggestion = document.suggestion ? `\n${document.suggestion}` : '' + const message = `CodeTruss ${document.verdict}. Receipt: ${document.receiptPath}${reasons ? `\n${reasons}` : ''}${suggestion}` return { verdict: document.verdict, receiptPath: document.receiptPath, @@ -1997,7 +2066,12 @@ async function reviewAtStop( if (state.finalCommit || state.finalHead || state.reviewAttemptId) { throw new Error('ready hook state contains an unexpected partial final review identity') } - const final = await (dependencies.captureBaseline ?? createExactSnapshotCommit)(root, join(turnDir, 'f'), objectStore) + const final = await captureExactSnapshot( + root, + join(turnDir, 'f'), + objectStore, + dependencies.captureBaseline ?? createExactSnapshotCommit, + ) objectStore.assertObjectId(final.commit, 'final snapshot commit') finalCommit = final.commit finalHead = final.head diff --git a/packages/cli/src/hooks.ts b/packages/cli/src/hooks.ts index 0784649..2575eb9 100644 --- a/packages/cli/src/hooks.ts +++ b/packages/cli/src/hooks.ts @@ -1,9 +1,10 @@ import { randomUUID } from 'node:crypto' import { constants as fsConstants } from 'node:fs' -import { access, chmod, lstat, mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises' -import { basename, delimiter, dirname, isAbsolute, join, resolve } from 'node:path' +import { access, chmod, lstat, mkdir, readFile, realpath, rename, rm, writeFile } from 'node:fs/promises' +import { basename, delimiter, dirname, isAbsolute, join, parse as parsePath, resolve } from 'node:path' import { loadConfig } from './config.js' import { runGitText } from './git-process.js' +import { CLI_VERSION } from './version.js' import { verifyCommandTrustStatus } from './verify-trust.js' type HookHandler = { type?: string; command?: string; args?: string[]; [key: string]: unknown } @@ -535,6 +536,58 @@ async function executablePath(root: string): Promise { return undefined } +/** + * Version of the @codetruss/cli install that owns `executable`, read from its + * package manifest. Never executes the binary: resolving a shadowing install + * must not run whatever happens to be first on PATH. + */ +export async function installedCliVersion(executable: string): Promise { + let cursor: string + try { + cursor = dirname(await realpath(executable)) + } catch { + return undefined + } + const { root } = parsePath(cursor) + while (true) { + try { + const manifest = JSON.parse(await readFile(join(cursor, 'package.json'), 'utf8')) as { + name?: unknown + version?: unknown + } + if (manifest.name === '@codetruss/cli' && typeof manifest.version === 'string') return manifest.version + } catch { + // no manifest here, or unreadable — keep walking up + } + if (cursor === root) return undefined + const parent = dirname(cursor) + if (parent === cursor) return undefined + cursor = parent + } +} + +/** + * The hooks invoke `codetruss` by name, so they run whatever PATH resolves — + * which is not always what was just installed. A stale binary earlier in PATH + * silently shadows the new one, and the installer's own "Ready" message used to + * hide it. Reported as a warning: the hooks still work, they just are not this + * version. Determined by manifest version, so a repository-local install of the + * same version is not mistaken for a shadow. + */ +async function inspectExecutableShadow( + executable: string, + add: (check: HookDoctorCheck) => void, +): Promise { + const resolved = await installedCliVersion(executable) + if (!resolved || resolved === CLI_VERSION) return + add({ + level: 'warning', + target: 'runtime', + message: `installed hooks resolve codetruss ${resolved}, but this CLI is ${CLI_VERSION}; put the intended install first on PATH or remove the older one`, + path: executable, + }) +} + function exactAgentHandler(handler: HookHandler, expected: HookHandler): boolean { return handler.type === expected.type && handler.command === expected.command @@ -697,8 +750,10 @@ export async function inspectHookDoctor(root: string, target: string): Promise a.filePath.localeCompare(b.filePath)) + return inputs +} + +export interface LocalSastResult { + findings: AnalyzerFinding[] + pass: AnalyzerPass +} + +export async function runLocalSast(index: RepoIndex): Promise { + const inputs = localSastInputs(index) + if (inputs.length === 0) { + return { + findings: [], + pass: { + id: LOCAL_SAST_PASS_ID, + result: { + findings: [], + complete: true, + metrics: { inputFiles: 0, filesScanned: 0, filesSkipped: 0, rules: CLI_SAST_RULE_IDS.size }, + }, + }, + } + } + + try { + const result = await scanFiles(inputs, zeroDependencyJsParser, { ruleIds: CLI_SAST_RULE_IDS }) + const diagnostics = result.diagnostics + // A file the parser could not read is coverage lost, and the receipt has to + // say so rather than let silence read as "nothing found there". + const truncated = + diagnostics.filesSkipped > 0 || diagnostics.truncatedFiles > 0 || diagnostics.findingsTruncated + const findings = result.findings.map((finding) => ({ + ...mapSastFinding(finding), + analyzerId: LOCAL_SAST_PASS_ID, + })) + return { + findings, + pass: { + id: LOCAL_SAST_PASS_ID, + result: { + findings, + complete: !truncated && diagnostics.degradedLanguages.length === 0, + truncated, + detail: truncated + ? `${diagnostics.filesSkipped} file(s) could not be parsed locally and were not analyzed` + : undefined, + metrics: { + inputFiles: diagnostics.inputFiles, + filesScanned: diagnostics.filesScanned, + filesSkipped: diagnostics.filesSkipped, + rules: CLI_SAST_RULE_IDS.size, + }, + }, + }, + } + } catch (error) { + const detail = error instanceof Error ? error.message : String(error) + return { + findings: [], + pass: { id: LOCAL_SAST_PASS_ID, result: { findings: [], complete: false, detail }, error: detail }, + } + } +} diff --git a/packages/cli/src/policy.ts b/packages/cli/src/policy.ts index 84c83e2..5d8893b 100644 --- a/packages/cli/src/policy.ts +++ b/packages/cli/src/policy.ts @@ -11,7 +11,10 @@ export function classifyPath(path: string, oldPath: string | undefined, allow: s const current = one(path) if (!oldPath) return current const previous = one(oldPath) - const rank: Record = { allowed: 0, unexpected: 1, denied: 2 } + // `inferred` is never produced here; scope inference runs over the whole + // changed set afterwards. The rank still orders it, so a rename that touches + // an inferred origin can never be reported as plainly allowed. + const rank: Record = { allowed: 0, inferred: 1, unexpected: 2, denied: 3 } return rank[previous] > rank[current] ? previous : current } diff --git a/packages/cli/src/receipt.ts b/packages/cli/src/receipt.ts index ecc9a0f..f4810d0 100644 --- a/packages/cli/src/receipt.ts +++ b/packages/cli/src/receipt.ts @@ -1,5 +1,6 @@ import { chmod, mkdir, readFile, readdir, rename, unlink, writeFile } from 'node:fs/promises' import { basename, join } from 'node:path' +import { suggestedFixLines } from './fix-suggestions.js' import { loadSigningKey, normalizePublicKey, @@ -8,7 +9,7 @@ import { signBytes, verifyBytes, } from './signing.js' -import type { Receipt, SyncEnvelope, Verdict } from './types.js' +import type { InferredScopeBasis, Receipt, SyncEnvelope, Verdict } from './types.js' const SYNC_REDACTION = '[redacted unrelated path]' @@ -82,26 +83,77 @@ function legacyScoreLines(receipt: Receipt): string[] { } /** - * The omitted passes named as detection gaps, not as a scoring footnote. + * What ran and what did not, named as detection rather than as a scoring + * footnote — and dispatched on the receipt's own profile version. * - * Earlier wording reported the omission only through "Hosted Health scores: - * N/A", which a developer reads as "no score" — not as "injection was never - * checked". A receipt is evidence of what ran, so the absent passes get their - * own section, named as the detection they cost rather than the number. + * A receipt is evidence about one execution, so its rendering must describe the + * pass set THAT execution had. A `local-registry-v1` receipt was signed when no + * security pass ran locally at all; re-rendering it with v2's wording would + * claim coverage that never happened, and would break its signature check. */ function analysisProfileLines(receipt: Receipt): string[] { const current = 'analysisProfile' in receipt.analyzers && receipt.analyzers.analysisProfile + if (!current) { + return [ + '## Analysis profile', + '', + 'Legacy local receipt. Earlier CLI versions emitted numeric scores without hosted graph and SAST; those values are suppressed.', + '', + ...whatDidNotRunV1(receipt), + ] + } + if (current.id === 'local-registry-v1') return omittedSastProfileLines(receipt, current.id) + return [ '## Analysis profile', '', - ...(current ? [ - `Profile: \`${current.id}\`.`, - '', - 'The 13 deterministic registry analyzers ran locally on this machine.', - ] : [ - 'Legacy local receipt. Earlier CLI versions emitted numeric scores without hosted graph and SAST; those values are suppressed.', + `Profile: \`${current.id}\`.`, + '', + 'The 13 deterministic registry analyzers ran locally on this machine, plus a local security pass: the shared SAST engine — the same rules and the same source-to-sink taint tracking as the hosted audit — over the JavaScript, TypeScript and TSX in this repository.', + '', + '### What the local security pass checked', + '', + '- **SQL injection (CWE-89).** Untrusted input tracked from request sources through string building into query execution.', + '- **Mass assignment (CWE-915).** A raw request body spread into a database write, and write helpers whose payload type accepts arbitrary keys.', + '- **Un-awaited database writes, swallowed errors, coercion-prone `==` comparisons, and N+1 queries in loops** — the defect classes coding agents most often introduce.', + '', + '### What did not run', + '', + '- **The rest of the security rule pack.** Command injection, code injection, path traversal, SSRF, open redirect, XSS and insecure deserialization were **not** checked here. Those rules run in a hosted scan; absence of a finding in those classes means they were not analyzed, not that the code is clean.', + '- **Non-JavaScript languages.** The local pass covers JavaScript, TypeScript and TSX only. Python, Go, Java, C#, PHP, Ruby and Rust in this repository received secret scanning and the other registry passes, but no security rule or taint analysis.', + '- **Hosted symbol graph.** No cross-file call or data-flow graph was built, so architecture and dead-code conclusions cover only what the local passes can see in isolation.', + ...(receipt.llm ? [] : [ + '- **Optional LLM review.** No model read this diff. It is opt-in via `--llm` and is force-disabled under agent hooks, so a hook receipt is always deterministic evidence only.', ]), + '- **Hosted Health scores.** Not calculated, reported as **N/A**. The scores are defined over the graph and the complete SAST pass; a number derived from this pass set would overstate what ran.', + '', + 'Local security findings are reported for review and do not fail the verdict on their own.', '', + 'A PASS verdict means the passes listed above never ran and the passes that did run found nothing new. It is not a statement that this change is secure.', + '', + '[Run a hosted full audit](https://codetruss.com/dashboard/repos/new?source=cli-receipt).', + ] +} + +/** + * The profile block exactly as CLI 0.2.29–0.2.34 wrote it, when the SAST pass + * was omitted from local analysis entirely. Frozen so those receipts still + * reproduce byte-for-byte. + */ +function omittedSastProfileLines(receipt: Receipt, profileId: string): string[] { + return [ + '## Analysis profile', + '', + `Profile: \`${profileId}\`.`, + '', + 'The 13 deterministic registry analyzers ran locally on this machine.', + '', + ...whatDidNotRunV1(receipt), + ] +} + +function whatDidNotRunV1(receipt: Receipt): string[] { + return [ '### What did not run', '', '- **Security static analysis (SAST).** No injection or taint analysis was performed. SQL injection, command injection, code injection, path traversal, SSRF, open redirect, XSS and insecure deserialization were never checked, so this receipt says nothing either way about those classes.', @@ -141,6 +193,47 @@ function priorProfileLines(receipt: Receipt): string[] { ] } +const INFERRED_BASIS_LABELS: Record = { + 'task-reference': 'named in the task', + 'working-set': 'working set for this turn', + 'sibling-test': 'test beside changed source', +} + +function scopeCell(file: Receipt['files'][number]): string { + return file.classification === 'inferred' ? 'allowed (inferred)' : file.classification +} + +/** + * Inferred scope, disclosed as what it is. + * + * A reviewer must never have to wonder whether a path was in scope because the + * repository approved it or because CodeTruss worked it out. This block names + * the weaker allowances, what each was read from, and the approved roots they + * sit beside — and it renders only when a turn actually used one, so receipts + * signed before inference existed still reproduce byte for byte. + */ +function inferredScopeLines(receipt: Receipt): string[] { + const inferred = receipt.scope.inferred ?? [] + if (inferred.length === 0) return [] + const covered = receipt.files.filter((file) => file.classification === 'inferred').length + return [ + '', + `## Inferred scope (${inferred.length})`, + '', + `${covered} changed file(s) matched no approved allow root. CodeTruss read the allowances below from this turn's own task text and changed files, and applied them to this turn only. They were not written to \`.codetruss.yml\` and do not carry forward.`, + '', + '| Inferred root | Read from | Evidence |', + '|---|---|---|', + ...inferred.map((root) => `| \`${root.root.replaceAll('|', '\\|')}\` | ${INFERRED_BASIS_LABELS[root.basis]} | ${root.evidence.map((item) => `\`${item.replaceAll('|', '\\|')}\``).join(', ')} |`), + '', + receipt.scope.allow.length + ? `Approved allow roots: ${receipt.scope.allow.map((glob) => `\`${glob}\``).join(', ')}.` + : 'This repository has no approved allow roots, so its scope for this turn was inferred entirely.', + '', + 'Denied paths, sensitive surfaces, and dependency manifests are never inferable. Each allowance above is a function of the task and changed-file list in this receipt, so the same result can be recomputed from these bytes.', + ] +} + /** Which historical rendering of the analysis block to reproduce. */ type ReceiptMarkdownVariant = 'current' | 'legacy-scores' | 'prior-profile' @@ -172,7 +265,8 @@ function renderMarkdownInternal(receipt: Receipt, variant: ReceiptMarkdownVarian '', '| Path | Change | Scope | Sensitive | Lines |', '|---|---|---|---|---:|', - ...receipt.files.map((file) => `| \`${file.path.replaceAll('|', '\\|')}\` | ${file.change} | ${file.classification} | ${file.sensitive ?? (file.dependency ? 'dependency' : '—')} | +${file.additions}/−${file.deletions} |`), + ...receipt.files.map((file) => `| \`${file.path.replaceAll('|', '\\|')}\` | ${file.change} | ${scopeCell(file)} | ${file.sensitive ?? (file.dependency ? 'dependency' : '—')} | +${file.additions}/−${file.deletions} |`), + ...inferredScopeLines(receipt), '', `## Introduced or worsened analyzer findings (${receipt.analyzers.findings.length})`, '', @@ -180,6 +274,9 @@ function renderMarkdownInternal(receipt: Receipt, variant: ReceiptMarkdownVarian '|---|---|---|---|', ...receipt.analyzers.findings.slice(0, 100).map((finding) => `| ${finding.severity} | ${finding.analyzerId ?? 'unknown'} | ${finding.filePath ? `\`${finding.filePath}${finding.line ? `:${finding.line}` : ''}\`` : 'repository'} | ${finding.title.replaceAll('|', '\\|')} |`), '', + // Emits nothing when no finding carries a fix, so every receipt signed + // before suggestions existed still renders to its original bytes. + ...suggestedFixLines(receipt.analyzers.findings), ...analysisLines(receipt, variant), ...(receipt.analyzers.delta ? [ `Finding delta: ${receipt.analyzers.delta.introduced} introduced, ${receipt.analyzers.delta.worsened} worsened, ${receipt.analyzers.delta.recurring} recurring, ${receipt.analyzers.delta.resolved} resolved.`, @@ -303,9 +400,14 @@ export async function verifyReceipt(dir: string, id = 'latest', pinnedPublicKey? // Every rendering this exact signed JSON could legitimately have produced. // Rewording a disclosure must not invalidate receipts already on disk, so // each superseded wording stays reproducible for verification only. + const profile = 'analysisProfile' in receipt.analyzers ? receipt.analyzers.analysisProfile : null const accepted = [ renderMarkdown(receipt), - 'analysisProfile' in receipt.analyzers ? renderPriorProfileMarkdown(receipt) : renderLegacyMarkdown(receipt), + // Superseded wordings are accepted only for the profile version that could + // have produced them. A `local-registry-v2` receipt was never written by a + // CLI that omitted SAST, so its Markdown must not be allowed to say so. + ...(!profile ? [renderLegacyMarkdown(receipt)] : []), + ...(profile?.id === 'local-registry-v1' ? [renderPriorProfileMarkdown(receipt)] : []), ] if (!accepted.includes(markdown)) throw new Error('Markdown receipt does not match the signed JSON') if (sha256(markdown) !== receipt.evidence.markdownSha256) throw new Error('Markdown receipt hash does not match') @@ -374,6 +476,9 @@ export async function createSyncEnvelope(receipt: Receipt): Promise ({ ...item, command: '[redacted for sync]', output: '' })) @@ -384,7 +489,7 @@ export async function createSyncEnvelope(receipt: Receipt): Promise haystack.length) return false + for (let start = 0; start + needle.length <= haystack.length; start += 1) { + if (needle.every((word, offset) => haystack[start + offset] === word)) return true + } + return false +} + +/** Every ancestor directory of a path, repository root excluded. */ +function ancestorDirectories(path: string): string[] { + const parts = normalizePath(path).split('/').filter(Boolean) + const directories: string[] = [] + for (let depth = 1; depth < parts.length; depth += 1) directories.push(parts.slice(0, depth).join('/')) + return directories +} + +function isTestPath(path: string): boolean { + const parts = normalizePath(path).split('/') + const name = (parts.at(-1) ?? '').toLowerCase() + return parts.slice(0, -1).some((part) => TEST_DIRECTORY_NAMES.has(part.toLowerCase())) + || /\.(test|spec)\.[a-z0-9]+$/.test(name) + || /_test\.[a-z0-9]+$/.test(name) + || name.startsWith('test_') +} + +/** `reset.test.ts`, `reset_test.go`, and `test_reset.py` all stem to `reset`. */ +function fileStem(path: string): string { + return (normalizePath(path).split('/').at(-1) ?? '') + .toLowerCase() + .replace(/\.[a-z0-9]+$/, '') + .replace(/\.(test|spec)$/, '') + .replace(/_test$/, '') + .replace(/^test_/, '') +} + +/** The literal directory prefix an allow glob commits to: `src/auth/**` → `src/auth`. */ +function globPrefix(glob: string): string { + const literal: string[] = [] + for (const part of normalizePath(glob.trim()).split('/')) { + if (GLOB_MAGIC.test(part)) break + literal.push(part) + } + return literal.join('/').replace(/\/+$/, '') +} + +/** + * An explicit allow root is a decision, not a default. If the repository + * approved `src/auth/**`, an inferred `src` would quietly replace that + * narrowing with its parent, so any inferred root that is an ancestor of — or + * equal to — an approved root is discarded. Inference may name a sibling this + * turn worked in; it may never climb above a line the user drew. + */ +function widensExplicitScope(root: string, explicitPrefixes: string[]): boolean { + return explicitPrefixes.some((prefix) => prefix === root || prefix.startsWith(`${root}/`)) +} + +function taskPathTokens(task: string): string[] { + const tokens: string[] = [] + for (const raw of task.match(TASK_PATH_TOKEN) ?? []) { + const token = normalizePath(raw).replace(/\/+$/, '') + if (!token || token.startsWith('/') || /^[a-z]:/i.test(token)) continue + if (token.split('/').some((part) => part === '..' || part === '.')) continue + if (!tokens.includes(token)) tokens.push(token) + } + return tokens +} + +/** + * Derive this turn's inferred allowances, in trust order: + * + * 1. `task-reference` — the task names a path, or names a directory by its own + * feature name, that the turn actually changed files under. + * 2. `working-set` — changed files cluster under one shared parent directory. + * The parent is the file's immediate directory, never a climb toward the + * repository root, and never the repository root itself. With approved allow + * roots present a cluster needs two files; with none configured a single + * file establishes its directory, because there is no approved scope for it + * to drift from and the receipt says the scope was inferred entirely. + * 3. `sibling-test` — a test file whose name mirrors a source file already in + * scope. Only that exact file is admitted, never its directory. + */ +export function inferTurnScope(input: TurnScopeInput): InferredScopeRoot[] { + const candidates = input.files.filter(inferable) + if (candidates.length === 0) return [] + const explicitPrefixes = input.allow.map(globPrefix).filter(Boolean) + const paths = [...new Set(candidates.map((file) => normalizePath(file.path)))].sort() + const roots: InferredScopeRoot[] = [] + const covered = (path: string) => roots.some((root) => covers(root.root, path)) + const admit = (root: InferredScopeRoot): void => { + if (roots.length >= MAX_INFERRED_ROOTS) return + if (!root.root || root.root === '.') return + if (roots.some((existing) => existing.root === root.root)) return + if (widensExplicitScope(root.root, explicitPrefixes)) return + if (!paths.some((path) => covers(root.root, path) && !covered(path))) return + roots.push(root) + } + + for (const token of taskPathTokens(input.task)) { + admit({ root: token, basis: 'task-reference', evidence: [token] }) + } + + const taskWords = words(input.task) + for (const path of paths) { + for (const directory of ancestorDirectories(path)) { + const name = directory.split('/').at(-1) ?? '' + if (GENERIC_DIRECTORY_NAMES.has(name.toLowerCase())) continue + const nameWords = words(name) + if (!nameWords.some((word) => word.length >= 3)) continue + if (!containsWordSequence(taskWords, nameWords)) continue + admit({ root: directory, basis: 'task-reference', evidence: [nameWords.join(' ')] }) + } + } + + const cohesion = explicitPrefixes.length === 0 ? 1 : 2 + const groups = new Map() + for (const path of paths) { + if (covered(path)) continue + const parent = parentDirectory(path) + if (!parent) continue + groups.set(parent, [...(groups.get(parent) ?? []), path]) + } + const ordered = [...groups].sort(([leftRoot, left], [rightRoot, right]) => ( + right.length - left.length || leftRoot.localeCompare(rightRoot) + )) + for (const [parent, members] of ordered) { + if (members.length < cohesion) continue + admit({ root: parent, basis: 'working-set', evidence: members }) + } + + const sourceStems = new Map() + for (const file of input.files) { + const path = normalizePath(file.path) + if (isTestPath(path) || file.classification === 'denied' || file.sensitive) continue + if (file.classification === 'allowed' || covered(path)) sourceStems.set(fileStem(path), path) + } + for (const path of paths) { + if (covered(path) || !isTestPath(path)) continue + const source = sourceStems.get(fileStem(path)) + if (!source) continue + admit({ root: path, basis: 'sibling-test', evidence: [source] }) + } + + return roots +} + +/** + * Apply inferred allowances and report only the roots that actually covered a + * file, so the receipt never advertises an allowance that did nothing. + */ +export function applyInferredScope( + files: T[], + roots: InferredScopeRoot[], +): { files: T[]; roots: InferredScopeRoot[] } { + if (roots.length === 0) return { files, roots } + const used = new Set() + const applied = files.map((file) => { + if (!inferable(file)) return file + // A rename is in scope only when its destination AND its origin are: the + // origin is exactly where drift hides behind a plausible-looking new path. + const touched = [file.path, ...(file.oldPath ? [file.oldPath] : [])] + const matched = roots.filter((root) => touched.every((path) => covers(root.root, path))) + if (matched.length === 0) return file + for (const root of matched) used.add(root.root) + return { ...file, classification: 'inferred' as const } + }) + return { files: applied, roots: roots.filter((root) => used.has(root.root)) } +} diff --git a/packages/cli/src/setup.ts b/packages/cli/src/setup.ts index 3ebbbca..385f4ca 100644 --- a/packages/cli/src/setup.ts +++ b/packages/cli/src/setup.ts @@ -2,7 +2,7 @@ import { access, lstat } from 'node:fs/promises' import { join } from 'node:path' import { createInterface } from 'node:readline/promises' import type { Readable, Writable } from 'node:stream' -import { CONFIG_FILE, initialize, loadConfig } from './config.js' +import { CONFIG_FILE, detectVerifyCommands, initialize, loadConfig, type VerifyDetection } from './config.js' import { inspectHookDoctor, installHooks } from './hooks.js' import { ensureLocalEvidenceProtected } from './local-evidence.js' import { trustVerifyCommands, verifyCommandTrustStatus } from './verify-trust.js' @@ -27,6 +27,15 @@ const SUGGESTED_SCOPE_DIRECTORIES = [ const HOOK_TARGETS = ['all', 'pre-commit', 'claude', 'codex', 'none'] as const type SetupHookTarget = typeof HOOK_TARGETS[number] +/** + * The files setup itself writes. Left out of the allow list, setup's own + * footprint made every user's FIRST commit REVIEW_REQUIRED for "changed outside + * approved scope" — CodeTruss flagging CodeTruss, on the one run where the tool + * has to earn trust. These paths are in scope by default; `.codetruss.yml` + * remains a sensitive policy surface, which is the point of reviewing it. + */ +export const SETUP_FOOTPRINT_GLOBS = ['.codetruss.yml', '.claude/**', '.codex/**', '.githooks/**'] as const + export interface GuidedSetupOptions { allow?: string[] deny?: string[] @@ -133,6 +142,60 @@ async function resolveHookTarget( return hookTarget(answer || 'all')! } +function detectionLines(detection: VerifyDetection, withheld: boolean): string[] { + const list = detection.candidates.map((entry) => ` - ${entry}\n`) + if (detection.commands.length) { + return [ + 'Detected repository verification commands but did NOT record them:\n', + ...list, + ...(withheld + ? ['They execute repository code, and an unattended run must not approve that on your behalf.\n'] + : []), + `To enable them: add them under verify: in ${CONFIG_FILE}, then run codetruss verify-policy trust\n`, + ] + } + if (detection.blocker === 'missing-lockfile') { + return [ + 'Found package.json scripts but recorded no verification commands:\n', + ...list, + 'No lockfile is committed, so CodeTruss cannot tell which package manager runs them.\n', + `To enable them: commit a lockfile and rerun setup, or add the exact commands under verify: in ${CONFIG_FILE}, then run codetruss verify-policy trust\n`, + ] + } + if (detection.blocker === 'package-manager-unavailable') { + return [ + 'Detected repository verification commands but their package manager is not on PATH:\n', + ...list, + `To enable them: install the package manager and rerun setup, or add the exact commands under verify: in ${CONFIG_FILE}, then run codetruss verify-policy trust\n`, + ] + } + return [`No repository verification commands were detected; add trusted checks to verify: in ${CONFIG_FILE} when ready.\n`] +} + +function withSetupFootprint(globs: string[]): string[] { + return [...globs, ...SETUP_FOOTPRINT_GLOBS.filter((glob) => !globs.includes(glob))] +} + +/** + * Does a stored allow list already express this request? Setup appends its own + * footprint, so the saved policy is the request plus those globs — and a policy + * saved by an earlier CLI carries the request alone. Both are the same ask, and + * a rerun of the identical command must stay idempotent rather than error. + */ +function allowPolicyMatches(requested: string[], existing: string[]): boolean { + const stored = JSON.stringify(existing) + return stored === JSON.stringify(requested) || stored === JSON.stringify(withSetupFootprint(requested)) +} + +/** Report what detection actually found, including what setup chose to withhold. */ +async function writeVerifyDetectionTruth( + root: string, + withheld: boolean, + write: (value: string) => void, +): Promise { + for (const line of detectionLines(await detectVerifyCommands(root), withheld)) write(line) +} + /** * One guided, resumable setup path. Repository verification commands are * displayed before their exact path-bound fingerprint is trusted. `--yes` @@ -146,6 +209,7 @@ export async function guidedSetup(root: string, options: GuidedSetupOptions = {} const deny = normalizeGlobs(options.deny, 'deny') const requestedHooks = hookTarget(options.hooks) const yes = options.yes === true + let withheldVerify = false let readline: ReturnType | undefined let answers: AsyncIterableIterator | undefined const ask = options.ask ?? (async (question: string) => { @@ -166,14 +230,14 @@ export async function guidedSetup(root: string, options: GuidedSetupOptions = {} const hasConfig = await exists(configPath) if (hasConfig) { const existing = await loadConfig(root) - const allowChanged = allow !== undefined && JSON.stringify(allow) !== JSON.stringify(existing.allow) + const allowChanged = allow !== undefined && !allowPolicyMatches(allow, existing.allow) const denyChanged = deny !== undefined && JSON.stringify(deny) !== JSON.stringify(existing.deny) if (allowChanged || denyChanged) { throw new Error(`${CONFIG_FILE} already exists with a different ${allowChanged ? 'allow' : 'deny'} policy; edit and review it directly, then rerun setup`) } write(`${allow !== undefined || deny !== undefined ? 'Requested policy matches' : 'Using existing'} ${CONFIG_FILE}.\n`) } else { - const selectedAllow = await resolveAllowGlobs(root, allow, yes, ask, write) + const selectedAllow = withSetupFootprint(await resolveAllowGlobs(root, allow, yes, ask, write)) const selectedDeny = deny ?? [] // Unattended setup must never record commands it has no permission to // run: an untrusted verify list makes every later review exit 3 with no @@ -181,13 +245,14 @@ export async function guidedSetup(root: string, options: GuidedSetupOptions = {} // Only when hooks will actually be installed: with --hooks none there is // nothing to block, and recording untrusted commands for later inspection // is the point of that flow. - const withheldVerify = yes && !options.trustVerify && requestedHooks !== 'none' + withheldVerify = yes && !options.trustVerify && requestedHooks !== 'none' const path = await initialize(root, false, { allow: selectedAllow, deny: selectedDeny, ...(withheldVerify ? { verify: [] } : {}), }) write(`Saved policy: ${path}\n`) + write(`Commit ${CONFIG_FILE} so this policy is reviewable.\n`) } const config = await loadConfig(root) @@ -202,7 +267,11 @@ export async function guidedSetup(root: string, options: GuidedSetupOptions = {} for (const command of config.verify) write(` - ${command}\n`) write('These commands execute repository code, each in its own isolated snapshot.\n') } else { - write('No repository verification commands were detected; add trusted checks to verify: in .codetruss.yml when ready.\n') + // "No commands were detected" was a lie in the two most common cases: an + // unattended run deliberately withheld them, or detection found the + // scripts and could not resolve a package manager. Report what was + // actually found and what clears it. + await writeVerifyDetectionTruth(root, withheldVerify, write) } const selectedHooks = await resolveHookTarget(requestedHooks, yes, ask) diff --git a/packages/cli/src/types.ts b/packages/cli/src/types.ts index 0cee623..9c8586a 100644 --- a/packages/cli/src/types.ts +++ b/packages/cli/src/types.ts @@ -1,7 +1,23 @@ import type { AnalyzerFinding, AnalyzerPass, RepoIndex, Scores } from '@codetruss/analyzer-engine' export type Verdict = 'PASS' | 'REVIEW_REQUIRED' | 'FAILED' -export type ScopeClassification = 'allowed' | 'denied' | 'unexpected' +/** + * `inferred` is in scope on weaker evidence than `allowed`: the repository + * approved nothing that covers the path, but this turn's own task text and + * changed files did. It is always disclosed on the receipt, never silent. + */ +export type ScopeClassification = 'allowed' | 'denied' | 'unexpected' | 'inferred' +/** In descending trust order; see packages/cli/src/scope-inference.ts. */ +export const INFERRED_SCOPE_BASES = ['task-reference', 'working-set', 'sibling-test'] as const +export type InferredScopeBasis = typeof INFERRED_SCOPE_BASES[number] + +export interface InferredScopeRoot { + /** Repository-relative directory, or a single file for `sibling-test`. */ + root: string + basis: InferredScopeBasis + /** What this allowance was read from: the task phrase, or the changed paths. */ + evidence: string[] +} export const RECEIPT_INVOCATION_KINDS = ['manual_run', 'manual_review', 'pre_commit', 'agent_hook'] as const export type ReceiptInvocationKind = typeof RECEIPT_INVOCATION_KINDS[number] export const AGENT_HOOK_SURFACES = ['claude', 'codex'] as const @@ -13,14 +29,33 @@ export const CONFIG_LLM_PROVIDERS = [...LLM_PROVIDERS, 'codex'] as const export type ConfiguredLlmProvider = typeof CONFIG_LLM_PROVIDERS[number] export const MAX_LLM_DIFF_BYTES = 2_000_000 -/** Honest local-analysis contract: registry passes run locally; hosted-only passes and scores do not. */ +/** + * Honest local-analysis contract: which passes ran on this machine, which did + * not, and whether scores may be inferred. + * + * `local-registry-v2` supersedes `local-registry-v1`, in which SAST was omitted + * entirely. A reduced security pass now runs locally, so `omittedPasses` no + * longer names it and `localPasses` names what took its place. The id is bumped + * rather than the tuple loosened: this shape is inside signed receipts, and + * every v1 receipt must keep verifying byte-for-byte against the wording it was + * signed with. + */ export const LOCAL_ANALYSIS_PROFILE = { - id: 'local-registry-v1', - omittedPasses: ['graph', 'sast'], + id: 'local-registry-v2', + omittedPasses: ['graph'], + localPasses: ['local-sast'], scoreStatus: 'not-computed', } as const export type LocalAnalysisProfile = typeof LOCAL_ANALYSIS_PROFILE +/** The v1 shape, retained so receipts signed by CLI ≤ 0.2.34 still parse. */ +export interface LegacyLocalAnalysisProfileV1 { + id: 'local-registry-v1' + omittedPasses: readonly ['graph', 'sast'] + scoreStatus: 'not-computed' +} +export type AnyLocalAnalysisProfile = LocalAnalysisProfile | LegacyLocalAnalysisProfileV1 + export interface CliConfig { version: 1 allow: string[] @@ -92,7 +127,7 @@ interface AnalyzerReceiptEvidence { export type AnalyzerReceipt = AnalyzerReceiptEvidence & ( | { /** Current local receipts never infer hosted Health scores from an incomplete pass set. */ - analysisProfile: LocalAnalysisProfile + analysisProfile: AnyLocalAnalysisProfile scores?: never baselineScores?: never } @@ -130,7 +165,13 @@ export interface Receipt { startDirty: boolean startDirtyFiles: string[] agent?: { command: string[]; exitCode: number; durationMs: number; startError?: string } - scope: { allow: string[]; deny: string[] } + /** + * `allow`/`deny` are the approved policy. `inferred` records the weaker, + * turn-only allowances that covered paths the policy did not, and is present + * only when this turn actually used one — which is what keeps receipts signed + * before inference existed rendering, and verifying, byte for byte. + */ + scope: { allow: string[]; deny: string[]; inferred?: InferredScopeRoot[] } files: ChangedFile[] diff: { sha256: string; bytes: number; totalBytes?: number; truncated: boolean } analyzers: AnalyzerReceipt diff --git a/packages/cli/src/verify-trust.ts b/packages/cli/src/verify-trust.ts index f86d344..059e133 100644 --- a/packages/cli/src/verify-trust.ts +++ b/packages/cli/src/verify-trust.ts @@ -1,10 +1,33 @@ +import { existsSync } from 'node:fs' import { chmod, mkdir, readFile, realpath, rename, unlink, writeFile } from 'node:fs/promises' import { homedir } from 'node:os' -import { dirname, join, resolve } from 'node:path' +import { dirname, isAbsolute, join, resolve } from 'node:path' import { sha256 } from './signing.js' const TRUST_VERSION = 1 as const -export const DEFAULT_VERIFY_TRUST_FILE = join(homedir(), '.config', 'codetruss', 'verify-command-trust.json') +const TRUST_FILE_NAME = 'verify-command-trust.json' + +/** + * Where the user-local trust store lives. Aligned with cliAuthFilePath(): both + * are user config under the same root, and honouring XDG_CONFIG_HOME in one but + * not the other silently split a single user's CodeTruss state across two + * directories. + * + * Migration: an existing store at the legacy `~/.config` path keeps being used + * even once XDG_CONFIG_HOME is set, so setting that variable never orphans + * approvals someone already made. + */ +export function verifyTrustFilePath(env: NodeJS.ProcessEnv = process.env): string { + const configured = env.XDG_CONFIG_HOME?.trim() + if (configured && !isAbsolute(configured)) { + throw new Error('XDG_CONFIG_HOME must be an absolute user config path') + } + const legacy = join(homedir(), '.config', 'codetruss', TRUST_FILE_NAME) + if (!configured) return legacy + const preferred = join(configured, 'codetruss', TRUST_FILE_NAME) + if (existsSync(preferred) || !existsSync(legacy)) return preferred + return legacy +} interface VerifyTrustStore { version: typeof TRUST_VERSION @@ -84,7 +107,7 @@ async function writeStore(path: string, store: VerifyTrustStore): Promise export async function verifyCommandTrustStatus( root: string, commands: string[], - trustFile = DEFAULT_VERIFY_TRUST_FILE, + trustFile = verifyTrustFilePath(), ): Promise { const hash = await verifyCommandTrustHash(root, commands) const store = await readStore(trustFile) @@ -95,7 +118,7 @@ export async function verifyCommandTrustStatus( export async function trustVerifyCommands( root: string, commands: string[], - trustFile = DEFAULT_VERIFY_TRUST_FILE, + trustFile = verifyTrustFilePath(), now = new Date(), ): Promise { const hash = await verifyCommandTrustHash(root, commands) @@ -108,7 +131,7 @@ export async function trustVerifyCommands( export async function revokeVerifyCommands( root: string, commands: string[], - trustFile = DEFAULT_VERIFY_TRUST_FILE, + trustFile = verifyTrustFilePath(), ): Promise { const hash = await verifyCommandTrustHash(root, commands) const store = await readStore(trustFile) diff --git a/packages/cli/test/analysis-profile.test.ts b/packages/cli/test/analysis-profile.test.ts index 1733652..02f5204 100644 --- a/packages/cli/test/analysis-profile.test.ts +++ b/packages/cli/test/analysis-profile.test.ts @@ -13,7 +13,7 @@ afterEach(async () => { }) describe('honest local analysis profile', () => { - it('does not emit a perfect security score when graph and SAST never ran', async () => { + it('finds the injection the hosted pass used to be needed for, and still withholds scores', async () => { const root = await mkdtemp(join(tmpdir(), 'codetruss-analysis-profile-')) cleanup.push(root) await mkdir(join(root, 'src')) @@ -28,27 +28,33 @@ describe('honest local analysis profile', () => { ].join('\n')) const analysis = await analyzeRepository(root) - expect(analysis.passes).toHaveLength(13) + // 13 registry analyzers plus the local security pass, which is deliberately + // NOT in the registry so "13 deterministic analyzers" stays true. + expect(analysis.passes).toHaveLength(14) + expect(analysis.passes.at(-1)?.id).toBe('local-sast') - // This is the exact misleading value earlier CLI versions inferred from - // registry-only findings even though the synthetic SQL injection was never - // examined by the hosted SAST pass. - expect(computeScores(analysis.index, analysis.findings).security).toBe(100) + const injection = analysis.findings.find((finding) => finding.metadata?.ruleId === 'sql-injection') + expect(injection).toBeDefined() + expect(injection?.severity).toBe('CRITICAL') + expect(injection?.filePath).toBe('src/users.ts') + expect(injection?.analyzerId).toBe('local-sast') + // Scores remain withheld: the graph pass and the rest of the rule pack are + // still absent, so any number would overstate what ran. + expect(computeScores(analysis.index, analysis.findings).security).toBeLessThan(100) const evidence = analyzerReceipt(analysis) expect(evidence.analysisProfile).toEqual(LOCAL_ANALYSIS_PROFILE) + expect(LOCAL_ANALYSIS_PROFILE.omittedPasses).toEqual(['graph']) expect(evidence).not.toHaveProperty('scores') expect(evidence).not.toHaveProperty('baselineScores') - expect(JSON.stringify(evidence)).not.toContain('"security"') }) - it('discloses the absent SAST pass on the TypeScript repos where it used to stay silent', async () => { + it('discloses the rule classes the local security pass still does not check', async () => { const root = await mkdtemp(join(tmpdir(), 'codetruss-local-sast-gap-')) cleanup.push(root) await mkdir(join(root, 'src')) // A repo large enough to draw coverage conclusions, entirely in a language - // the SAST engine covers — the exact shape that produced zero coverage - // findings while the injection rules never ran. + // the local pass covers. for (let unit = 0; unit < 8; unit += 1) { const body = Array.from({ length: 50 }, (_, line) => ` const value${line} = ${line} * ${unit + 1}`) await writeFile( @@ -63,11 +69,14 @@ describe('honest local analysis profile', () => { const finding = coverage!.result.findings[0] expect(finding.category).toBe('SECURITY_HYGIENE') expect(finding.severity).toBe('INFO') - expect(finding.title).toMatch(/did not run/i) - expect(finding.description).toMatch(/SQL injection/) - expect(finding.metadata).toMatchObject({ sastPassRan: false, sastLanguages: ['TypeScript'] }) - // A disclosure, never a blocking accusation: the local verdict must not - // fail a change because a pass was absent. + expect(finding.title).toMatch(/reduced rule set/i) + // "The pass ran" must never be allowed to read as "every class was checked". + expect(finding.description).toMatch(/command injection/) + expect(finding.description).toMatch(/path traversal/) + expect(finding.description).toContain('means "not checked", not "clean"') + expect(finding.metadata).toMatchObject({ sastPassRan: true }) + expect(finding.metadata?.sastUncheckedClasses).toContain('SSRF') + // A disclosure, never a blocking accusation. expect(finding.severity).not.toBe('HIGH') expect(finding.severity).not.toBe('CRITICAL') }) diff --git a/packages/cli/test/command-e2e.test.ts b/packages/cli/test/command-e2e.test.ts index f478bff..d9f104c 100644 --- a/packages/cli/test/command-e2e.test.ts +++ b/packages/cli/test/command-e2e.test.ts @@ -115,6 +115,96 @@ describe('CLI snapshot and delta enforcement', () => { expect(`${setup.stderr}${setup.stdout}`).toContain('--allow') }, 30_000) + it('names the verification commands unattended setup withheld instead of claiming none exist', async () => { + // D2/D3: with `"test": "vitest run"` and a lockfile on disk, setup printed + // "No repository verification commands were detected" — which reads as + // "this repository has no tests" and is simply false. The commands were + // detected and deliberately withheld; say that, and say what enables them. + const root = await repository() + await mkdir(join(root, 'src')) + await installPersistentCliFixture(root) + await writeFile(join(root, '.gitignore'), '/node_modules\n') + await writeFile( + join(root, 'package.json'), + `${JSON.stringify({ private: true, scripts: { lint: 'eslint .', test: 'vitest run' } }, null, 2)}\n`, + ) + await writeFile(join(root, 'pnpm-lock.yaml'), 'lockfileVersion: 9\n') + + const setup = runCli(root, ['setup', '--yes', '--allow', 'src/**', '--hooks', 'all']) + + expect(setup.status, `${setup.stderr}\n${setup.stdout}`).toBe(0) + expect(setup.stdout).not.toContain('No repository verification commands were detected') + expect(setup.stdout).toContain('did NOT record them') + expect(setup.stdout).toContain('pnpm lint') + expect(setup.stdout).toContain('pnpm test') + expect(setup.stdout).toContain('unattended run must not approve that on your behalf') + expect(setup.stdout).toContain('codetruss verify-policy trust') + // The withholding itself is unchanged: nothing untrusted lands in policy. + expect(await readFile(join(root, '.codetruss.yml'), 'utf8')).not.toMatch(/^verify:\n\s+-/m) + }, 30_000) + + it('blames the missing lockfile when detection genuinely finds nothing to run', async () => { + // D3: the dependencies analyzer already knows a lockfile is missing. Setup + // saying only "none detected" sends the user looking for the wrong problem. + const root = await repository() + await mkdir(join(root, 'src')) + await installPersistentCliFixture(root) + await writeFile(join(root, '.gitignore'), '/node_modules\n') + await writeFile( + join(root, 'package.json'), + `${JSON.stringify({ private: true, scripts: { lint: 'eslint .', test: 'vitest run' } }, null, 2)}\n`, + ) + + const setup = runCli(root, ['setup', '--yes', '--allow', 'src/**', '--hooks', 'all']) + + expect(setup.status, `${setup.stderr}\n${setup.stdout}`).toBe(0) + expect(setup.stdout).toContain('No lockfile is committed') + expect(setup.stdout).toContain('which package manager runs them') + expect(setup.stdout).toMatch(/- lint\n/) + expect(setup.stdout).toMatch(/- test\n/) + }, 30_000) + + it('keeps setup’s own footprint inside approved scope on the first commit', async () => { + // D4: `.claude/settings.json`, `.codetruss.yml` and `.codex/hooks.json` are + // written BY setup, then flagged "changed outside approved scope" on the + // user's very first review — CodeTruss failing its own installation. + const root = await repository() + await mkdir(join(root, 'src')) + await installPersistentCliFixture(root) + await writeFile(join(root, '.gitignore'), '/node_modules\n') + await writeFile(join(root, 'src', 'value.ts'), 'export const value = 1\n') + git(root, 'add', '.') + git(root, 'commit', '--quiet', '-m', 'baseline') + + const setup = runCli(root, ['setup', '--yes', '--allow', 'src/**', '--hooks', 'all']) + expect(setup.status, `${setup.stderr}\n${setup.stdout}`).toBe(0) + expect(setup.stdout).toContain('Commit .codetruss.yml so this policy is reviewable.') + + git(root, 'add', '.') + const review = runCli(root, ['review', '--staged', '--task', 'Install CodeTruss', '--no-verify']) + const receipt = await latestReceipt(root) + const footprint = ['.codetruss.yml', '.claude/settings.json', '.codex/hooks.json'] + for (const path of footprint) { + const file = receipt.files.find((entry) => entry.path === path) + expect(file, `${path} missing from ${JSON.stringify(receipt.files.map((f) => f.path))}`).toBeDefined() + expect(file!.classification, path).toBe('allowed') + } + expect(receipt.reasons.join('\n')).not.toContain('outside approved scope') + // `.codetruss.yml` stays a sensitive policy surface — reviewing a policy + // change is the point — so the verdict is still REVIEW_REQUIRED, now for a + // reason that is about the user's repository rather than CodeTruss's own files. + expect(review.status).toBe(1) + expect(receipt.reasons.join('\n')).toContain('.codetruss.yml (policy)') + }, 30_000) + + it('lists trust-key in --help, since blocked commits tell you to run it', async () => { + // D8: the error a blocked teammate hits names `verify-policy trust-key`, + // and --help did not admit the subcommand existed. + const root = await repository() + const help = runCli(root, ['--help']) + expect(`${help.stdout}${help.stderr}`).toContain('verify-policy [status|trust|trust-key|revoke]') + }, 30_000) + it('completes idempotent local-only setup and keeps generated evidence out of normal staging', async () => { const root = await repository() await mkdir(join(root, 'src')) @@ -600,6 +690,50 @@ describe('CLI snapshot and delta enforcement', () => { expect(receipt.analyzers.findings.some((finding) => finding.title.includes('AWS access key'))).toBe(true) expect(receipt.diff.truncated).toBe(false) expect(await readFile(join(root, '.codetruss', 'receipts', receipt.evidence.patchFile!), 'utf8')).toContain(SYNTHETIC_AWS_KEY) + // The rendered receipt carries the move-to-env suggestion, and the credential + // stays out of it: the captured patch is the only place the value appears. + const markdown = await readFile(join(root, '.codetruss', 'receipts', `${receipt.sessionId}.md`), 'utf8') + expect(markdown).toContain('## Suggested fixes (1)') + expect(markdown).toContain('+export const key = process.env.KEY') + expect(markdown).not.toContain(SYNTHETIC_AWS_KEY) + }, 20_000) + + it('does not let an AUTO-GENERATED banner buy a committed live key a PASS', async () => { + // D1: generated-file classification exists to suppress FALSE quality + // findings about machine-written code. It silently disabled EVERY analyzer, + // secrets included — so the four lines below produced a signed PASS while + // the identical file without the banner FAILED. One comment, whole scanner + // bypassed. + const root = await repository() + await mkdir(join(root, 'src')) + await writeFile(join(root, 'src', 'app.ts'), 'export const app = 1\n') + git(root, 'add', '.') + git(root, 'commit', '--quiet', '-m', 'baseline') + const stripeKey = `sk_live_51QN8${'a1B2c3D4e5F6g7H8'.repeat(2)}` + await writeFile( + join(root, 'src', 'generated-config.ts'), + '// AUTO-GENERATED FILE - DO NOT EDIT\n' + + '// Regenerate with: pnpm codegen\n\n' + + `export const STRIPE_SECRET = '${stripeKey}'\n`, + ) + git(root, 'add', 'src/generated-config.ts') + + const result = runCli(root, ['review', '--staged', '--task', 'Add generated config', '--allow', 'src/**', '--no-verify']) + + expect(result.status, `${result.stderr}\n${result.stdout}`).toBe(2) + const receipt = await latestReceipt(root) + expect(receipt.verdict).toBe('FAILED') + const secret = receipt.analyzers.findings.find((finding) => finding.title.includes('Stripe live secret key')) + expect(secret, JSON.stringify(receipt.analyzers.findings.map((f) => f.title))).toBeDefined() + expect(secret!.filePath).toBe('src/generated-config.ts') + expect(secret!.severity).toBe('HIGH') + // The value itself never reaches the receipt. + expect(JSON.stringify(receipt.analyzers)).not.toContain(stripeKey) + // …and the exclusion that suppressed the other analyzers is on the record, + // naming the exact file, at four lines and ~150 bytes. + const disclosure = receipt.analyzers.findings.find((finding) => finding.title.startsWith('Generated code excluded')) + expect(disclosure, JSON.stringify(receipt.analyzers.findings.map((f) => f.title))).toBeDefined() + expect(disclosure!.description).toContain('src/generated-config.ts') }, 20_000) it('runs verification against the exact staged snapshot instead of unstaged bytes', async () => { @@ -664,6 +798,60 @@ describe('CLI snapshot and delta enforcement', () => { expect(receipt.coverageNotes.at(-1)).toContain('installed Node dependencies') }, 20_000) + /** + * The first session a stranger ever runs. `setup --yes` adopts whatever + * conventional directories exist, the agent then does something entirely + * reasonable one directory over, and the signature detection has to read as + * signal rather than as a false alarm — without going quiet on real drift. + */ + it('does not raise scope drift on a plausible first turn, and still raises it on an unrelated one', async () => { + const root = await repository() + await mkdir(join(root, 'src')) + await writeFile(join(root, 'src', 'index.ts'), 'export const value = 1\n') + git(root, 'add', '.') + git(root, 'commit', '--quiet', '-m', 'baseline') + + const setup = runCli(root, ['setup', '--yes', '--hooks', 'none']) + expect(setup.status, `${setup.stderr}\n${setup.stdout}`).toBe(0) + expect(setup.stdout).toContain('Adopted detected allowed change roots: src/**') + git(root, 'add', '.codetruss.yml') + git(root, 'commit', '--quiet', '-m', 'adopt codetruss policy') + + await mkdir(join(root, 'server', 'auth'), { recursive: true }) + await writeFile(join(root, 'server', 'auth', 'password-reset.ts'), 'export const reset = () => true\n') + await writeFile(join(root, 'server', 'auth', 'tokens.ts'), 'export const token = () => "t"\n') + + const first = runCli(root, ['review', '--task', 'Add password reset', '--no-verify']) + expect(first.status, `${first.stderr}\n${first.stdout}`).toBe(0) + const firstReceipt = await latestReceipt(root) + expect(firstReceipt.verdict).toBe('PASS') + expect(firstReceipt.scope.inferred).toEqual([{ + root: 'server/auth', + basis: 'working-set', + evidence: ['server/auth/password-reset.ts', 'server/auth/tokens.ts'], + }]) + expect(firstReceipt.files.map((file) => file.classification)).toEqual(['inferred', 'inferred']) + const markdown = await readFile(join(root, '.codetruss', 'receipts', `${firstReceipt.sessionId}.md`), 'utf8') + expect(markdown).toContain('## Inferred scope (1)') + // The inferred root is only judgeable against the roots that WERE approved, + // so the receipt names every one of them — the detected source root and the + // footprint setup wrote for itself. + expect(markdown).toContain('Approved allow roots: `src/**`, `.codetruss.yml`, `.claude/**`, `.codex/**`, `.githooks/**`.') + expect(runCli(root, ['verify', firstReceipt.sessionId]).status).toBe(0) + + git(root, 'add', '.') + git(root, 'commit', '--quiet', '-m', 'password reset') + await mkdir(join(root, 'analytics'), { recursive: true }) + await writeFile(join(root, 'analytics', 'tracker.ts'), 'export const track = () => undefined\n') + + const drifted = runCli(root, ['review', '--task', 'Add password reset', '--no-verify']) + expect(drifted.status, `${drifted.stderr}\n${drifted.stdout}`).toBe(1) + const driftReceipt = await latestReceipt(root) + expect(driftReceipt.verdict).toBe('REVIEW_REQUIRED') + expect(driftReceipt.scope.inferred).toBeUndefined() + expect(driftReceipt.reasons).toContain('1 file(s) changed outside approved scope: analytics/tracker.ts') + }, 30_000) + it('does not fail a harmless edit because of an unchanged pre-existing finding', async () => { const root = await repository() await mkdir(join(root, 'src')) diff --git a/packages/cli/test/config.test.ts b/packages/cli/test/config.test.ts index cb3559d..77d2052 100644 --- a/packages/cli/test/config.test.ts +++ b/packages/cli/test/config.test.ts @@ -7,6 +7,7 @@ import { loadSigningKey, requireTrustedSigningKey } from '../src/signing.js' import { APPROVED_RECEIPT_DIR, PRODUCTION_SYNC_ORIGIN, + detectVerifyCommands, initialize, loadConfig, receiptDir, @@ -64,6 +65,46 @@ describe('initialization', () => { expect(configText).not.toContain('sync:') }) + it('collects lint and test for every Node package manager, not just pnpm', async () => { + // npm and yarn repositories were losing their lint script for no reason + // other than which lockfile they happen to commit. + const scripts = { scripts: { lint: 'eslint .', test: 'vitest run', dev: 'next dev' } } + const withLockfile = async (name: string) => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-detect-verify-')) + await writeFile(join(root, 'package.json'), `${JSON.stringify(scripts)}\n`) + await writeFile(join(root, name), '\n') + return detectVerifyCommands(root) + } + + expect((await withLockfile('pnpm-lock.yaml')).candidates).toEqual(['pnpm lint', 'pnpm test']) + // `npm lint` is not a command — only lifecycle names run without `run`. + expect((await withLockfile('package-lock.json')).candidates).toEqual(['npm run lint', 'npm run test']) + expect((await withLockfile('yarn.lock')).candidates).toEqual(['yarn lint', 'yarn test']) + }) + + it('names the missing lockfile as the reason detection found nothing', async () => { + // "No verification commands were detected" reads as "this repository has no + // tests" even when package.json defines them and only the lockfile is absent. + const root = await mkdtemp(join(tmpdir(), 'codetruss-detect-no-lockfile-')) + await writeFile( + join(root, 'package.json'), + `${JSON.stringify({ scripts: { lint: 'eslint .', test: 'vitest run' } })}\n`, + ) + + await expect(detectVerifyCommands(root)).resolves.toEqual({ + commands: [], + candidates: ['lint', 'test'], + blocker: 'missing-lockfile', + }) + }) + + it('claims no candidates when the repository genuinely defines none', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-detect-empty-')) + await writeFile(join(root, 'package.json'), `${JSON.stringify({ scripts: { dev: 'next dev' } })}\n`) + + await expect(detectVerifyCommands(root)).resolves.toEqual({ commands: [], candidates: [] }) + }) + it('never accepts a repository-selected bearer-token destination', async () => { const root = await mkdtemp(join(tmpdir(), 'codetruss-config-sync-')) await writeFile(join(root, '.codetruss.yml'), 'version: 1\nsync:\n url: https://collector.invalid\n') diff --git a/packages/cli/test/fix-suggestion-fixture.ts b/packages/cli/test/fix-suggestion-fixture.ts new file mode 100644 index 0000000..b57305b --- /dev/null +++ b/packages/cli/test/fix-suggestion-fixture.ts @@ -0,0 +1,89 @@ +import { createHash } from 'node:crypto' +import type { AnalyzerFinding } from '@codetruss/analyzer-engine' +import type { Receipt } from '../src/types.js' + +/** + * A fully-determined receipt: every field that reaches the Markdown renderer is + * fixed, so its rendering is a stable golden. Shared by the fix-suggestion tests + * and by the throwaway script that captured the pre-suggestion golden bytes. + */ +export function fixtureReceipt(findings: AnalyzerFinding[] = []): Receipt { + const patch = 'diff evidence' + return { + receiptVersion: 1, + sessionId: '20260712T210000123Z-fixture', + createdAt: '2026-07-12T21:00:00.123Z', + finishedAt: '2026-07-12T21:00:00.123Z', + durationMs: 0, + mode: 'review', + task: 'fix suggestion fixture', + repoRoot: '/repo', + startCommit: 'abc', + endCommit: 'abc', + git: { baselineTree: 'a'.repeat(40), finalTree: 'b'.repeat(40) }, + policy: { sha256: 'c'.repeat(64) }, + startDirty: false, + startDirtyFiles: [], + scope: { allow: ['src/**'], deny: [] }, + files: [], + diff: { + sha256: createHash('sha256').update(patch).digest('hex'), + bytes: Buffer.byteLength(patch), + totalBytes: Buffer.byteLength(patch), + truncated: false, + }, + analyzers: { + passes: [], + findings, + // Pinned to v1, not to whatever LOCAL_ANALYSIS_PROFILE currently is: the + // golden below captures bytes CLI 0.2.30 actually wrote, and the receipts + // already signed on disk are v1 receipts. Tracking the current profile + // would make this assert that today's renderer matches itself, which is + // not the backward-compatibility claim being made. + analysisProfile: { id: 'local-registry-v1', omittedPasses: ['graph', 'sast'], scoreStatus: 'not-computed' }, + index: { totalLoc: 0, languages: {}, primaryLanguage: null }, + }, + verifications: [], + coverageNotes: ['local'], + verdict: 'REVIEW_REQUIRED', + reasons: ['1 medium-or-higher analyzer finding(s) affect changed files'], + evidence: {}, + } +} + +/** + * `renderMarkdown(fixtureReceipt(fixtureFindings()))` as CLI 0.2.30 wrote it — + * captured by running the pre-suggestion renderer, not by re-recording current + * output. Fix suggestions must stay purely additive: a receipt whose findings + * carry no fix has to render to these exact bytes, or every receipt already + * signed on disk stops verifying. + */ +export const PRE_SUGGESTION_RECEIPT_MARKDOWN = "# CodeTruss receipt — REVIEW_REQUIRED\n\n- **Session:** `20260712T210000123Z-fixture`\n- **Task:** fix suggestion fixture\n- **Repository:** `/repo`\n- **Starting commit:** `abc`\n- **Evidence trees:** `aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa` → `bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb`\n- **Policy SHA-256:** `cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc`\n- **Mode:** review\n\n## Verdict: REVIEW_REQUIRED\n\n- 1 medium-or-higher analyzer finding(s) affect changed files\n\nDiff evidence: 13/13 bytes captured (complete), SHA-256 `7d9ef774aece6330…`.\n\n## Changed files (0)\n\n| Path | Change | Scope | Sensitive | Lines |\n|---|---|---|---|---:|\n\n## Introduced or worsened analyzer findings (2)\n\n| Severity | Analyzer | Location | Finding |\n|---|---|---|---|\n| HIGH | secrets | `src/config.ts:12` | Possible AWS access key committed in config.ts |\n| MEDIUM | dependencies | repository | No lockfile committed |\n\n## Analysis profile\n\nProfile: `local-registry-v1`.\n\nThe 13 deterministic registry analyzers ran locally on this machine.\n\n### What did not run\n\n- **Security static analysis (SAST).** No injection or taint analysis was performed. SQL injection, command injection, code injection, path traversal, SSRF, open redirect, XSS and insecure deserialization were never checked, so this receipt says nothing either way about those classes.\n- **Hosted symbol graph.** No cross-file call or data-flow graph was built, so architecture and dead-code conclusions cover only what the local passes can see in isolation.\n- **Optional LLM review.** No model read this diff. It is opt-in via `--llm` and is force-disabled under agent hooks, so a hook receipt is always deterministic evidence only.\n- **Hosted Health scores.** Not calculated, reported as **N/A**. The scores are defined over the graph and SAST passes; a number derived from this pass set would overstate what ran.\n\nA PASS verdict means the passes listed above never ran and the passes that did run found nothing new. It is not a statement that this change is secure.\n\n[Run a hosted full audit](https://codetruss.com/dashboard/repos/new?source=cli-receipt).\n\n## Verification\n\n- No verification commands configured.\n\n## Coverage and privacy\n\n- local\n\n_The signature proves these receipt bytes have not changed since signing. It does not prove trusted execution or that every analysis conclusion is correct._\n" + +/** One finding of each kind the renderer has to handle, without any fix. */ +export function fixtureFindings(): AnalyzerFinding[] { + return [ + { + category: 'SECURITY_HYGIENE', + severity: 'HIGH', + title: 'Possible AWS access key committed in config.ts', + description: 'Line 12 of src/config.ts appears to contain an AWS access key.', + filePath: 'src/config.ts', + line: 12, + suggestion: 'Rotate this credential immediately.', + impactScore: 95, + effort: 'low', + analyzerId: 'secrets', + }, + { + category: 'DEPENDENCY', + severity: 'MEDIUM', + title: 'No lockfile committed', + description: 'package.json exists but no lockfile is committed.', + suggestion: 'Commit the lockfile for your package manager.', + impactScore: 75, + effort: 'low', + analyzerId: 'dependencies', + }, + ] +} diff --git a/packages/cli/test/fix-suggestion-hook.test.ts b/packages/cli/test/fix-suggestion-hook.test.ts new file mode 100644 index 0000000..3a55c7c --- /dev/null +++ b/packages/cli/test/fix-suggestion-hook.test.ts @@ -0,0 +1,167 @@ +import { spawnSync } from 'node:child_process' +import { lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { DEFAULT_CONFIG } from '../src/config.js' +import { handleAgentHook, type HookReviewRequest } from '../src/hook-runtime.js' +import { writeInternalHookResult } from '../src/hook-result.js' +import type { CliConfig } from '../src/types.js' + +const cleanup: string[] = [] +const attemptId = 'b'.repeat(64) + +afterEach(async () => { + await Promise.all(cleanup.splice(0).map((path) => rm(path, { recursive: true, force: true }))) +}) + +function git(root: string, ...args: string[]): string { + const result = spawnSync('git', ['-C', root, ...args], { encoding: 'utf8' }) + if (result.status !== 0) throw new Error(result.stderr || `git ${args.join(' ')} failed`) + return result.stdout.trim() +} + +async function repo(): Promise { + const root = await mkdtemp(join(tmpdir(), 'codetruss-fix-hook-')) + cleanup.push(root) + git(root, 'init', '--quiet') + git(root, 'config', 'user.name', 'Hook Test') + git(root, 'config', 'user.email', 'hook@example.com') + await writeFile(join(root, 'README.md'), 'baseline\n') + git(root, 'add', 'README.md') + git(root, 'commit', '--quiet', '-m', 'baseline') + await writeFile(join(root, '.codetruss.yml'), 'version: 1\nallow:\n - src/**\ndeny:\n - vendor/**\nverify: []\n') + return root +} + +function config(allow = ['src/**']): CliConfig { + return { ...structuredClone(DEFAULT_CONFIG), allow, deny: ['vendor/**'] } +} + +const SUGGESTION = 'Suggested fix (NOT applied — review before using) for HIGH "Possible AWS access key committed in config.ts" at src/config.ts:12: ' + + 'Read the credential from `AWS_KEY` at runtime and document it in .env.example. Rotate this credential first.' + +/** More reasons than the Stop summary displays, so truncation is exercised. */ +const REASONS = Array.from({ length: 8 }, (_, position) => `verdict reason ${position + 1}`) + +/** + * Drive one real prompt/Stop turn and return whatever the agent is shown. + * The suggestion has to survive the same validator, the same reason cap, and + * the same message assembly a live hook uses — asserting on the builder alone + * would prove nothing about what the agent actually reads. + */ +async function stopOutput( + writeResult: (request: HookReviewRequest, receiptPath: string) => Promise, +): Promise { + const root = await repo() + const receiptPath = join(root, '.codetruss', 'receipts', 'hook.md') + await mkdir(join(root, '.codetruss', 'receipts'), { recursive: true }) + await writeFile(receiptPath, '# receipt\n') + const runReview = vi.fn(async (request: HookReviewRequest) => { + await writeResult(request, receiptPath) + return { status: 1 as const, stdout: '', stderr: '' } + }) + const dependencies = { runReview, now: () => new Date() } + await mkdir(join(root, 'src'), { recursive: true }) + await writeFile(join(root, 'src', 'config.ts'), 'export const value = "before"\n') + const prompt = { session_id: 'session-fix', turn_id: 'turn-1', hook_event_name: 'UserPromptSubmit', prompt: 'Wire up billing', cwd: root } + await handleAgentHook(root, 'codex', prompt, config(), dependencies) + await writeFile(join(root, 'src', 'config.ts'), 'export const value = "after"\n') + return handleAgentHook(root, 'codex', { ...prompt, hook_event_name: 'Stop', background_tasks: [] }, config(), dependencies) +} + +describe('the Stop hook hands the agent one suggestion', () => { + it('shows the suggestion even when the verdict reasons fill the display cap', async () => { + const output = await stopOutput(async (request, receiptPath) => { + await writeFile(request.resultPath, `${JSON.stringify({ + version: 1, + attemptId: request.attemptId, + verdict: 'REVIEW_REQUIRED', + receiptPath, + reasons: REASONS, + suggestion: SUGGESTION, + })}\n`, { mode: 0o600, flag: 'wx' }) + }) + + const message = JSON.stringify(output) + expect(message).toContain('CodeTruss REVIEW_REQUIRED') + expect(message).toContain('verdict reason 1') + // The reason list is capped at five; the suggestion is not part of it. + expect(message).not.toContain('verdict reason 6') + expect(JSON.parse(message).systemMessage).toContain(SUGGESTION) + expect(JSON.parse(message).systemMessage.indexOf(SUGGESTION)) + .toBeGreaterThan(JSON.parse(message).systemMessage.indexOf('verdict reason 5')) + }) + + it('still accepts a result written by a CLI that has no suggestions', async () => { + const output = await stopOutput(async (request, receiptPath) => { + await writeFile(request.resultPath, `${JSON.stringify({ + version: 1, + attemptId: request.attemptId, + verdict: 'REVIEW_REQUIRED', + receiptPath, + reasons: ['outside allowed scope'], + })}\n`, { mode: 0o600, flag: 'wx' }) + }) + expect(JSON.stringify(output)).toContain('outside allowed scope') + expect(JSON.stringify(output)).not.toContain('Suggested fix') + }) + + it('rejects a result whose suggestion is not a bounded string', async () => { + const output = await stopOutput(async (request, receiptPath) => { + await writeFile(request.resultPath, `${JSON.stringify({ + version: 1, + attemptId: request.attemptId, + verdict: 'REVIEW_REQUIRED', + receiptPath, + reasons: [], + suggestion: { description: 'structured payloads are not accepted here' }, + })}\n`, { mode: 0o600, flag: 'wx' }) + }) + expect(JSON.stringify(output)).toContain('invalid schema or attempt binding') + }) +}) + +describe('the internal hook result carries the suggestion separately from reasons', () => { + async function resultFixture(): Promise<{ contextPath: string; receiptPath: string; resultPath: string }> { + const root = await mkdtemp(join(tmpdir(), 'codetruss-fix-result-')) + cleanup.push(root) + const attempts = join(root, 'turn', 'attempts') + await mkdir(attempts, { recursive: true, mode: 0o700 }) + const contextPath = join(root, 'turn', 'turn-context.json') + const receiptPath = join(root, 'receipt.md') + await writeFile(contextPath, '{}\n', { mode: 0o600 }) + await writeFile(receiptPath, '# receipt\n', { mode: 0o600 }) + return { contextPath, receiptPath, resultPath: join(attempts, 'result.json') } + } + + it('writes the suggestion as its own bounded field', async () => { + const files = await resultFixture() + await writeInternalHookResult( + { path: files.resultPath, attemptId }, + files.contextPath, + { verdict: 'REVIEW_REQUIRED', receiptPath: files.receiptPath, reasons: ['outside allowed scope'], suggestion: SUGGESTION }, + ) + expect(JSON.parse(await readFile(files.resultPath, 'utf8'))).toEqual({ + version: 1, + attemptId, + verdict: 'REVIEW_REQUIRED', + receiptPath: files.receiptPath, + reasons: ['outside allowed scope'], + suggestion: SUGGESTION, + }) + if (process.platform !== 'win32') expect((await lstat(files.resultPath)).mode & 0o777).toBe(0o600) + }) + + it('omits the field entirely when nothing carried a fix', async () => { + const files = await resultFixture() + await writeInternalHookResult( + { path: files.resultPath, attemptId }, + files.contextPath, + { verdict: 'PASS', receiptPath: files.receiptPath, reasons: [] }, + ) + expect(Object.keys(JSON.parse(await readFile(files.resultPath, 'utf8')))).toEqual([ + 'version', 'attemptId', 'verdict', 'receiptPath', 'reasons', + ]) + }) +}) diff --git a/packages/cli/test/fix-suggestions.test.ts b/packages/cli/test/fix-suggestions.test.ts new file mode 100644 index 0000000..e1e22e1 --- /dev/null +++ b/packages/cli/test/fix-suggestions.test.ts @@ -0,0 +1,142 @@ +import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import type { AnalyzerFinding, FindingFix } from '@codetruss/analyzer-engine' +import { afterEach, describe, expect, it } from 'vitest' +import { FIX_DISCLAIMER, suggestedFixLines, topFixSuggestion } from '../src/fix-suggestions.js' +import { createSyncEnvelope, renderMarkdown, verifyReceipt, writeReceipt } from '../src/receipt.js' +import { PRE_SUGGESTION_RECEIPT_MARKDOWN, fixtureFindings, fixtureReceipt } from './fix-suggestion-fixture.js' + +const originalKey = process.env.CODETRUSS_SIGNING_KEY +const roots: string[] = [] + +afterEach(async () => { + if (originalKey === undefined) delete process.env.CODETRUSS_SIGNING_KEY + else process.env.CODETRUSS_SIGNING_KEY = originalKey + await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true }))) +}) + +function fix(overrides: Partial = {}): FindingFix { + return { + description: 'Read the credential from `AWS_KEY` at runtime.', + kind: 'diff', + language: 'diff', + content: '--- a/src/config.ts\n+++ b/src/config.ts\n@@ -12 +12 @@\n-const awsKey = ""\n+const awsKey = process.env.AWS_KEY\n', + safetyNote: 'Rotate this credential first — it is already in Git history.', + ...overrides, + } +} + +function withFixes(): AnalyzerFinding[] { + const [secret, lockfile] = fixtureFindings() + return [ + { ...lockfile, fix: fix({ kind: 'snippet', language: 'sh', content: 'pnpm install --lockfile-only\n', description: 'Generate and commit pnpm-lock.yaml with pnpm.', safetyNote: 'Review the generated lockfile before committing.' }) }, + { ...secret, fix: fix() }, + ] +} + +describe('suggested fixes in the signed receipt', () => { + it('renders nothing at all when no finding carries a fix', () => { + expect(suggestedFixLines(fixtureFindings())).toEqual([]) + }) + + it('reproduces the exact pre-suggestion Markdown for a fix-free receipt', () => { + // Backward verification: the golden was produced by the renderer that + // shipped before fixes existed. Any byte drift here invalidates every + // receipt already signed on disk. + expect(renderMarkdown(fixtureReceipt(fixtureFindings()))).toBe(PRE_SUGGESTION_RECEIPT_MARKDOWN) + }) + + it('verifies a receipt written before fixes existed and one written with them', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-fix-verify-')) + roots.push(root) + process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') + + const withoutFixes = fixtureReceipt(fixtureFindings()) + const dir = join(root, 'receipts') + await writeReceipt(dir, withoutFixes, 'diff evidence') + await expect(verifyReceipt(dir, withoutFixes.sessionId)).resolves.toMatchObject({ verdict: 'REVIEW_REQUIRED' }) + expect(await readFile(join(dir, `${withoutFixes.sessionId}.md`), 'utf8')).toBe(PRE_SUGGESTION_RECEIPT_MARKDOWN) + + const suggested = { ...fixtureReceipt(withFixes()), sessionId: '20260712T210000123Z-suggested' } + await writeReceipt(dir, suggested, 'diff evidence') + await expect(verifyReceipt(dir, suggested.sessionId)).resolves.toMatchObject({ verdict: 'REVIEW_REQUIRED' }) + }) + + it('shows each suggestion as a fenced block with its location and safety note', () => { + const markdown = renderMarkdown(fixtureReceipt(withFixes())) + expect(markdown).toContain('## Suggested fixes (2)') + // Highest severity first, regardless of finding order. + expect(markdown.indexOf('### HIGH —')).toBeLessThan(markdown.indexOf('### MEDIUM —')) + expect(markdown).toContain('### HIGH — Possible AWS access key committed in config.ts (`src/config.ts:12`)') + expect(markdown).toContain('```diff\n--- a/src/config.ts') + expect(markdown).toContain('```sh\npnpm install --lockfile-only\n```') + expect(markdown).toContain('Before applying: Rotate this credential first') + }) + + it('never presents a suggestion as applied, automatic, or mandatory', () => { + const markdown = renderMarkdown(fixtureReceipt(withFixes())) + expect(markdown).toContain(FIX_DISCLAIMER) + expect(FIX_DISCLAIMER).toMatch(/did not apply, write, or run/) + for (const forbidden of [ + /CodeTruss (?:has )?applied/i, + /automatically (?:applied|fixed|corrected)/i, + /(?:fix|change) (?:was|has been) applied/i, + /you must apply/i, + /required fix/i, + ]) { + expect(markdown).not.toMatch(forbidden) + } + }) + + it('widens the fence so a Markdown starter block cannot end it early', () => { + const starter = '# Project\n\n## Quick start\n\n```sh\npnpm install\n```\n' + const [finding] = fixtureFindings() + const markdown = renderMarkdown(fixtureReceipt([ + { ...finding, fix: fix({ kind: 'snippet', language: 'markdown', content: starter }) }, + ])) + expect(markdown).toContain('````markdown\n# Project') + expect(markdown).toContain('```sh\npnpm install\n```\n````') + }) + + it('keeps suggested fixes off the hosted sync copy', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-fix-sync-')) + roots.push(root) + process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') + const receipt = fixtureReceipt(withFixes()) + receipt.files = [{ path: 'src/config.ts', change: 'modified', classification: 'allowed', dependency: false, additions: 1, deletions: 1 }] + await writeReceipt(join(root, 'receipts'), receipt, 'diff evidence') + const envelope = await createSyncEnvelope(receipt) + const synced = JSON.parse(envelope.signedReceipt) as { analyzers: { findings: AnalyzerFinding[] }; coverageNotes: string[] } + + expect(synced.analyzers.findings.length).toBeGreaterThan(0) + for (const finding of synced.analyzers.findings) expect(finding).not.toHaveProperty('fix') + expect(envelope.signedReceipt).not.toContain('process.env.AWS_KEY') + expect(synced.coverageNotes.at(-1)).toContain('suggested-fix bodies') + // The local receipt keeps everything the sync copy dropped. + expect(receipt.analyzers.findings[1].fix).toBeDefined() + }) +}) + +describe('the agent-facing Stop summary line', () => { + it('returns nothing when no finding carries a fix', () => { + expect(topFixSuggestion(fixtureFindings())).toBeUndefined() + }) + + it('carries exactly the highest-severity suggestion, framed as not applied', () => { + const summary = topFixSuggestion(withFixes()) + expect(summary).toBeDefined() + expect(summary).toContain('Suggested fix (NOT applied — review before using) for HIGH') + expect(summary).toContain('"Possible AWS access key committed in config.ts" at src/config.ts:12') + expect(summary).toContain('Read the credential from `AWS_KEY` at runtime.') + expect(summary).toContain('Rotate this credential first') + // One suggestion, not a digest of every finding. + expect(summary).not.toContain('No lockfile committed') + }) + + it('stays inside the hook result per-reason character bound', () => { + const [finding] = fixtureFindings() + const summary = topFixSuggestion([{ ...finding, fix: fix({ safetyNote: 'x'.repeat(5_000) }) }]) + expect(summary!.length).toBeLessThanOrEqual(2_000) + }) +}) diff --git a/packages/cli/test/hooks.test.ts b/packages/cli/test/hooks.test.ts index 8fd9f3b..199bbec 100644 --- a/packages/cli/test/hooks.test.ts +++ b/packages/cli/test/hooks.test.ts @@ -19,8 +19,8 @@ import { type HookReviewRequest, } from '../src/hook-runtime.js' import { CODETRUSS_HOOK_RESULT_PATH_ENV, CODETRUSS_HOOK_REVIEW_ATTEMPT_ID_ENV } from '../src/hook-result.js' -import { materializeTreeSnapshot, materializeWorkingTreeSnapshot } from '../src/git-snapshot.js' -import { doctorHooks, hookStatus, inspectLocalHookHealth, installHooks, uninstallHooks } from '../src/hooks.js' +import { materializeTreeSnapshot, materializeWorkingTreeSnapshot, WorkingTreeChangedError } from '../src/git-snapshot.js' +import { doctorHooks, hookStatus, inspectHookDoctor, inspectLocalHookHealth, installedCliVersion, installHooks, uninstallHooks } from '../src/hooks.js' import { classifyPath, isDependencyFile, sensitiveCategory } from '../src/policy.js' import { hookSessionId } from '../src/receipt.js' import { @@ -438,6 +438,49 @@ describe('hook installation', () => { } }) + it('warns when an older install shadows the codetruss the hooks will run', async () => { + // D6: the installer's readiness check was a bare `command -v codetruss`, + // which succeeds just as happily when an older binary sits earlier in PATH. + // The hooks invoke `codetruss` by name, so that stale copy is what runs. + const root = await repo() + await writeConfig(root) + const shadow = await mkdtemp(join(tmpdir(), 'codetruss-shadow-install-')) + await writeFile( + join(shadow, 'package.json'), + `${JSON.stringify({ name: '@codetruss/cli', version: '0.0.1-older' })}\n`, + ) + const bin = join(root, 'node_modules', '.bin', process.platform === 'win32' ? 'codetruss.cmd' : 'codetruss') + await mkdir(dirname(bin), { recursive: true }) + await writeFile(bin, process.platform === 'win32' ? '@exit /b 0\r\n' : '#!/bin/sh\nexit 0\n') + await chmod(bin, 0o755) + await installHooks(root, 'pre-commit') + + // Same version: nothing to say. + await expect(installedCliVersion(bin)).resolves.toBeUndefined() + const quiet = await inspectHookDoctor(root, 'pre-commit') + expect(quiet.checks.filter((check) => check.message.includes('but this CLI is'))).toEqual([]) + + // Now the resolvable binary belongs to a different install. + await rename(bin, join(shadow, basename(bin))) + const shadowed = join(shadow, basename(bin)) + await expect(installedCliVersion(shadowed)).resolves.toBe('0.0.1-older') + const priorPath = process.env.PATH + process.env.PATH = `${shadow}${delimiter}${priorPath ?? ''}` + try { + const doctor = await inspectHookDoctor(root, 'pre-commit') + const shadowWarning = doctor.checks.find((check) => check.message.includes('0.0.1-older')) + expect(shadowWarning, JSON.stringify(doctor.checks)).toBeDefined() + expect(shadowWarning!.level).toBe('warning') + expect(shadowWarning!.message).toMatch(/first on PATH|remove the older/) + // A version skew is not a broken installation — it must not fail doctor. + expect(doctor.ok).toBe(true) + } finally { + if (priorPath === undefined) delete process.env.PATH + else process.env.PATH = priorPath + await rm(shadow, { recursive: true, force: true }) + } + }) + it('keeps pre-commit-only health independent from agent scope while reporting verification trust', async () => { const root = await repo() const bin = join(root, 'node_modules', '.bin', process.platform === 'win32' ? 'codetruss.cmd' : 'codetruss') @@ -883,19 +926,68 @@ describe('agent hook runtime', () => { }) }) - it.each(['claude', 'codex'] as const)('blocks invalid UserPromptSubmit input with the %s decision contract', async (surface) => { + // Variant 1 of the reported bug: a promptless UserPromptSubmit blocked the + // human's prompt outright. A promptless turn is a legitimate turn shape, so it + // must capture from tree state and never return a block decision. + it.each(['claude', 'codex'] as const)('captures a promptless %s turn instead of blocking the prompt', async (surface) => { const root = await repo() const output = await handleAgentHook(root, surface, { - session_id: `${surface}-invalid-prompt`, + session_id: `${surface}-promptless`, + turn_id: `${surface}-promptless-turn`, hook_event_name: 'UserPromptSubmit', cwd: root, }, config()) - expect(output).toEqual({ - decision: 'block', - reason: expect.stringContaining('missing the submitted prompt'), - }) - expect(output).not.toHaveProperty('continue') - expect(output).not.toHaveProperty('stopReason') + + expect(output).toBeUndefined() + const session = stateDir(root, surface, `${surface}-promptless`) + const current = JSON.parse(await readFile(join(session, 'current.json'), 'utf8')) as { turnKey: string } + const state = JSON.parse(await readFile(join(session, current.turnKey, 'state.json'), 'utf8')) as Record + expect(state).toMatchObject({ status: 'ready', task: '[no prompt] agent turn with no submitted prompt text' }) + expect(state.baselineCommit).toEqual(expect.any(String)) + }) + + // Variant 2 of the reported bug: because the promptless turn never captured a + // baseline, Stop reported "no exact baseline was captured for this agent turn" + // and the turn went unreviewed. Capturing promptless turns fixes it at the root. + it('reviews a promptless turn at Stop instead of reporting no baseline', async () => { + const root = await repo() + const receipt = join(root, '.codetruss', 'receipts', 'promptless.md') + await mkdir(dirname(receipt), { recursive: true }) + await writeFile(receipt, '# promptless turn\n') + const turn = { + session_id: 'promptless-stop-session', + turn_id: 'promptless-stop-turn', + cwd: root, + } + const runReview = vi.fn(async (request: HookReviewRequest) => hookReviewResponse(request, 'PASS', 0, receipt)) + + // No `prompt` field at all — a harness machine event, as delivered on resume. + await expect(handleAgentHook(root, 'claude', { + ...turn, + hook_event_name: 'UserPromptSubmit', + }, config(), { runReview })).resolves.toBeUndefined() + + const stop = await handleAgentHook(root, 'claude', { + ...turn, + hook_event_name: 'Stop', + background_tasks: [], + }, config(), { runReview }) + + expect(stop).toBeUndefined() + expect(runReview).toHaveBeenCalledTimes(1) + expect(runReview.mock.calls[0][0].task).toBe('[no prompt] agent turn with no submitted prompt text') + }) + + it.each(['claude', 'codex'] as const)('never blocks a %s prompt when capture cannot even start', async (surface) => { + const root = await repo() + const output = await handleAgentHook(root, surface, { + hook_event_name: 'UserPromptSubmit', + prompt: 'Change the value', + cwd: root, + }, config()) + + expect(output).toEqual({ systemMessage: expect.stringContaining('missing session_id') }) + expect(output).not.toHaveProperty('decision') }) it('fails closed on a cross-session current selector and leaves the targeted session untouched', async () => { @@ -988,6 +1080,33 @@ describe('agent hook runtime', () => { await expect(readFile(join(root, '.codetruss', 'receipts', 'latest'), 'utf8')).rejects.toMatchObject({ code: 'ENOENT' }) }) + it('stays quiet mid-turn on a path the turn evidence already places in scope', async () => { + // A repository with no approved allow roots would otherwise be told every + // single edit is drift — the exact wall of noise that gets an always-on + // hook uninstalled on day one. Inference covers it; the receipt discloses + // it. Surfaces inference may never reach still speak up. + const root = await repo() + const quiet = await handleAgentHook(root, 'claude', { + session_id: 'session-inferred', hook_event_name: 'PostToolUse', cwd: root, + tool_input: { file_path: join(root, 'server', 'auth', 'password-reset.ts') }, + }, config([])) + expect(quiet).toBeUndefined() + + const sensitive = await handleAgentHook(root, 'claude', { + session_id: 'session-inferred-sensitive', hook_event_name: 'PostToolUse', cwd: root, + tool_input: { file_path: join(root, 'infra', 'prod.tf') }, + }, config([])) + expect(JSON.stringify(sensitive)).toContain('outside the allowed task scope') + expect(JSON.stringify(sensitive)).toContain('sensitive iac surface') + + // An approved policy still narrows what a single tool call can establish. + const narrowed = await handleAgentHook(root, 'claude', { + session_id: 'session-inferred-narrow', hook_event_name: 'PostToolUse', cwd: root, + tool_input: { file_path: join(root, 'server', 'auth', 'password-reset.ts') }, + }, config(['src/**'])) + expect(JSON.stringify(narrowed)).toContain('outside the allowed task scope') + }) + it('normalizes outside paths and tolerates unknown tool schemas', async () => { const root = await repo() const outside = await handleAgentHook(root, 'claude', { @@ -1135,6 +1254,88 @@ describe('agent hook runtime', () => { expect(spawnSync('git', ['-C', root, 'cat-file', '-e', requests[0].baselineRef], { env: privateGitReadEnvironment(requests[0].objectDirectory) }).status).not.toBe(0) }) + it('retries transient working-tree drift while capturing Stop evidence', async () => { + const root = await repo() + const receipt = join(root, '.codetruss', 'receipts', 'stop-retry.md') + await mkdir(dirname(receipt), { recursive: true }) + await writeFile(receipt, '# stop retry\n') + const prompt = { + session_id: 'stop-retry-session', + turn_id: 'stop-retry-turn', + hook_event_name: 'UserPromptSubmit', + prompt: 'Retry the final snapshot once', + cwd: root, + } + let captureCalls = 0 + const captureBaseline = vi.fn(async (captureRoot: string, parent: string, store: PrivateGitObjectStore) => { + captureCalls++ + if (captureCalls === 2) throw new WorkingTreeChangedError('src/value.ts') + return createExactHookBaseline(captureRoot, parent, store) + }) + const runReview = vi.fn(async (request: HookReviewRequest) => hookReviewResponse(request, 'PASS', 0, receipt)) + const dependencies = { captureBaseline, runReview } + + await expect(handleAgentHook(root, 'claude', prompt, config(), dependencies)).resolves.toBeUndefined() + await expect(handleAgentHook(root, 'claude', { + ...prompt, + hook_event_name: 'Stop', + background_tasks: [], + }, config(), dependencies)).resolves.toBeUndefined() + + expect(captureBaseline).toHaveBeenCalledTimes(3) + expect(runReview).toHaveBeenCalledTimes(1) + }) + + it('preserves ready Stop state after retry exhaustion and recovers on the next Stop', async () => { + const root = await repo() + const receipt = join(root, '.codetruss', 'receipts', 'stop-recovery.md') + await mkdir(dirname(receipt), { recursive: true }) + await writeFile(receipt, '# stop recovery\n') + const prompt = { + session_id: 'stop-recovery-session', + turn_id: 'stop-recovery-turn', + hook_event_name: 'UserPromptSubmit', + prompt: 'Preserve the baseline if final capture is unstable', + cwd: root, + } + let captureCalls = 0 + let finalStable = false + const captureBaseline = vi.fn(async (captureRoot: string, parent: string, store: PrivateGitObjectStore) => { + captureCalls++ + if (captureCalls > 1 && !finalStable) throw new WorkingTreeChangedError('src/value.ts') + return createExactHookBaseline(captureRoot, parent, store) + }) + const runReview = vi.fn(async (request: HookReviewRequest) => hookReviewResponse(request, 'PASS', 0, receipt)) + const dependencies = { captureBaseline, runReview } + + await handleAgentHook(root, 'claude', prompt, config(), dependencies) + const firstStop = await handleAgentHook(root, 'claude', { + ...prompt, + hook_event_name: 'Stop', + background_tasks: [], + }, config(), dependencies) + + expect(captureBaseline).toHaveBeenCalledTimes(4) + expect(runReview).not.toHaveBeenCalled() + expect(firstStop).toEqual({ decision: 'block', reason: expect.stringContaining('working tree changed while snapshotting') }) + const session = stateDir(root, 'claude', prompt.session_id) + const current = JSON.parse(await readFile(join(session, 'current.json'), 'utf8')) as { turnKey: string } + const statePath = join(session, current.turnKey, 'state.json') + const readyState = JSON.parse(await readFile(statePath, 'utf8')) as Record + expect(readyState).toMatchObject({ status: 'ready' }) + expect(readyState).not.toHaveProperty('finalCommit') + expect(readyState).not.toHaveProperty('reviewAttemptId') + + finalStable = true + await expect(handleAgentHook(root, 'claude', { + ...prompt, + hook_event_name: 'Stop', + background_tasks: [], + }, config(), dependencies)).resolves.toBeUndefined() + expect(captureBaseline).toHaveBeenCalledTimes(5) + expect(runReview).toHaveBeenCalledTimes(1) + }) + it('resumes reviewing with the persisted final OID and deterministic attempt instead of recapturing a later tree', async () => { const root = await repo('codetruss-hook-review-resume-') await mkdir(join(root, 'src'), { recursive: true }) @@ -1493,7 +1694,7 @@ describe('agent hook runtime', () => { const first = handleAgentHook(root, 'codex', prompt, config(), { captureBaseline }) await captureStarted const duplicate = await handleAgentHook(root, 'codex', prompt, config(), { captureBaseline }) - expect(duplicate).toMatchObject({ decision: 'block', reason: expect.stringContaining('already running') }) + expect(duplicate).toEqual({ systemMessage: expect.stringContaining('already running') }) expect(duplicate).not.toHaveProperty('continue') expect(captureBaseline).toHaveBeenCalledTimes(1) releaseCapture() @@ -1544,7 +1745,72 @@ describe('agent hook runtime', () => { await expect(stat(join(liveTurnDir, 'state.json'))).resolves.toBeDefined() }, 30_000) - it('keeps private prompt state under Git metadata and blocks prompt processing if exact capture fails', async () => { + it('retries the whole exact capture after transient working-tree drift', async () => { + const root = await repo() + const prompt = { + session_id: 'capture-retry-session', + turn_id: 'capture-retry-turn', + hook_event_name: 'UserPromptSubmit', + prompt: 'Capture this task once the concurrent edit settles', + cwd: root, + } + let attempts = 0 + const captureBaseline = vi.fn(async (captureRoot: string, parent: string, store: PrivateGitObjectStore) => { + attempts++ + if (attempts === 1) throw new WorkingTreeChangedError('src/value.ts') + return createExactHookBaseline(captureRoot, parent, store) + }) + + await expect(handleAgentHook(root, 'claude', prompt, config(), { captureBaseline })).resolves.toBeUndefined() + + expect(captureBaseline).toHaveBeenCalledTimes(2) + const session = stateDir(root, 'claude', prompt.session_id) + const current = JSON.parse(await readFile(join(session, 'current.json'), 'utf8')) as { turnKey: string } + const state = JSON.parse(await readFile(join(session, current.turnKey, 'state.json'), 'utf8')) as Record + expect(state).toMatchObject({ status: 'ready', task: prompt.prompt }) + await expect(stat(join(session, current.turnKey, 'object-store', 'objects'))).resolves.toBeDefined() + }) + + it('lets the prompt through after bounded retries, and still holds the agent at Stop', async () => { + const root = await repo() + const prompt = { + session_id: 'capture-exhausted-session', + turn_id: 'capture-exhausted-turn', + hook_event_name: 'UserPromptSubmit', + prompt: 'Capture a stable baseline', + cwd: root, + } + const captureBaseline = vi.fn(async () => { + throw new WorkingTreeChangedError('src/value.ts') + }) + const failedAt = new Date('2026-08-06T20:00:00.000Z') + const dependencies = { captureBaseline, now: () => failedAt } + + const output = await handleAgentHook(root, 'claude', prompt, config(), dependencies) + + expect(captureBaseline).toHaveBeenCalledTimes(3) + expect(output).toEqual({ systemMessage: expect.stringContaining('working tree changed while snapshotting') }) + const session = stateDir(root, 'claude', prompt.session_id) + const current = JSON.parse(await readFile(join(session, 'current.json'), 'utf8')) as { turnKey: string } + await expect(stat(join(session, current.turnKey, 'object-store'))).rejects.toMatchObject({ code: 'ENOENT' }) + + // The turn is recorded as failed, so Stop remains the enforcement point: + // the human was not blocked, but the agent cannot finish unreviewed. + const state = JSON.parse(await readFile(join(session, current.turnKey, 'state.json'), 'utf8')) as Record + expect(state).toMatchObject({ status: 'failed' }) + + const stop = await handleAgentHook(root, 'claude', { + ...prompt, + hook_event_name: 'Stop', + background_tasks: [], + }, config(), dependencies) + expect(stop).toEqual({ + decision: 'block', + reason: expect.stringContaining('working tree changed while snapshotting'), + }) + }) + + it('keeps private prompt state under Git metadata and notifies without blocking if exact capture fails', async () => { const root = await repo() const prompt = { session_id: 'private-session', prompt_id: 'prompt-1', hook_event_name: 'UserPromptSubmit', prompt: 'private task text', cwd: root } await expect(handleAgentHook(root, 'claude', prompt, config())).resolves.toBeUndefined() @@ -1563,10 +1829,10 @@ describe('agent hook runtime', () => { expect(contextInfo.mode & 0o777).toBe(0o600) } expect(git(root, 'status', '--porcelain')).not.toContain('codetruss/hooks') - const failed = await handleAgentHook(root, 'claude', { ...prompt, session_id: 'failed-session', prompt_id: 'prompt-fail' }, config(), { - captureBaseline: async () => { throw new Error('unstable working tree') }, - }) - expect(failed).toEqual({ decision: 'block', reason: expect.stringContaining('unstable working tree') }) + const captureBaseline = vi.fn(async () => { throw new Error('unrecoverable capture failure') }) + const failed = await handleAgentHook(root, 'claude', { ...prompt, session_id: 'failed-session', prompt_id: 'prompt-fail' }, config(), { captureBaseline }) + expect(captureBaseline).toHaveBeenCalledTimes(1) + expect(failed).toEqual({ systemMessage: expect.stringContaining('unrecoverable capture failure') }) }) it('hashes maximum-length agent identifiers into path-budgeted hook state', async () => { @@ -2046,8 +2312,8 @@ describe('agent hook runtime', () => { cwd: root, } - const blocked = await handleAgentHook(root, 'codex', nextPrompt, config()) - expect(blocked).toEqual({ decision: 'block', reason: expect.stringContaining('active lease') }) + const notice = await handleAgentHook(root, 'codex', nextPrompt, config()) + expect(notice).toEqual({ systemMessage: expect.stringContaining('active lease') }) await expect(stat(legacy.objectStorePath)).resolves.toBeDefined() await expect(stat(hookStateRoot(root, 'v1', repositoryKey))).resolves.toBeDefined() expect((await readFile(legacy.statePath, 'utf8'))).toContain(`private live ${repositoryKey} task`) @@ -2083,8 +2349,8 @@ describe('agent hook runtime', () => { await writeFile(ownershipPath, '{"invalid":true}\n', { mode: 0o600 }) const nextPrompt = { ...prompt, prompt: 'Capture after validated cleanup' } - const blocked = await handleAgentHook(root, 'codex', nextPrompt, config()) - expect(blocked).toEqual({ decision: 'block', reason: expect.stringContaining('securely cleaned') }) + const notice = await handleAgentHook(root, 'codex', nextPrompt, config()) + expect(notice).toEqual({ systemMessage: expect.stringContaining('securely cleaned') }) expect(await readFile(legacy.statePath, 'utf8')).toContain('private ownership validation task') await expect(stat(hookStateRoot(root, 'v1', 'full'))).resolves.toBeDefined() await expect(stat(legacy.contextPath)).resolves.toBeDefined() @@ -2114,9 +2380,9 @@ describe('agent hook runtime', () => { state.sessionHash = createHash('sha256').update('different-session').digest('hex') await writeFile(legacy.statePath, `${JSON.stringify(state)}\n`, { mode: 0o600 }) - const blocked = await handleAgentHook(root, 'codex', prompt, config()) + const notice = await handleAgentHook(root, 'codex', prompt, config()) - expect(blocked).toEqual({ decision: 'block', reason: expect.stringContaining('full session hash') }) + expect(notice).toEqual({ systemMessage: expect.stringContaining('full session hash') }) expect(await readFile(legacy.statePath, 'utf8')).toContain('preserve exact session binding') await expect(stat(legacy.contextPath)).resolves.toBeDefined() await expect(stat(legacy.objectStorePath)).resolves.toBeDefined() @@ -2175,9 +2441,9 @@ describe('agent hook runtime', () => { const stateBefore = await readFile(statePath, 'utf8') const contextBefore = await readFile(join(turnDir, 'turn-context.json'), 'utf8') - const blocked = await handleAgentHook(root, 'codex', prompt, config()) + const notice = await handleAgentHook(root, 'codex', prompt, config()) - expect(blocked).toEqual({ decision: 'block', reason: expect.stringContaining('ownership cannot be proven') }) + expect(notice).toEqual({ systemMessage: expect.stringContaining('ownership cannot be proven') }) expect(await readFile(statePath, 'utf8')).toBe(stateBefore) expect(await readFile(join(turnDir, 'turn-context.json'), 'utf8')).toBe(contextBefore) await expect(stat(join(turnDir, 'object-store'))).resolves.toBeDefined() diff --git a/packages/cli/test/indexer-path-separators.test.ts b/packages/cli/test/indexer-path-separators.test.ts new file mode 100644 index 0000000..a2ec492 --- /dev/null +++ b/packages/cli/test/indexer-path-separators.test.ts @@ -0,0 +1,52 @@ +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' + +// Windows, simulated on any host: `relative` is the ONLY platform-shaped call in +// the repository walk, so swapping it for its win32 counterpart reproduces the +// separator a Windows run would actually produce. Everything else — `join`, the +// filesystem — stays real, which is what makes the assertion meaningful: an +// un-normalized `src\deep\users.ts` cannot even be stat'd back on this host, so +// the file drops out of the index entirely. +vi.mock('node:path', async (importOriginal) => { + const actual = await importOriginal() + return { ...actual, default: actual, relative: actual.win32.relative } +}) + +const { indexRepository } = await import('../src/indexer.js') + +const roots: string[] = [] + +afterEach(async () => { + await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true }))) +}) + +describe('repository index path separators', () => { + it('emits POSIX separators for nested files even when the platform yields backslashes', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-index-separators-')) + roots.push(root) + await mkdir(join(root, 'src', 'deep'), { recursive: true }) + await writeFile(join(root, 'src', 'deep', 'users.ts'), 'export const users = []\n') + await writeFile(join(root, 'root.ts'), 'export const root = 1\n') + + const index = await indexRepository(root) + const paths = index.files.map((file) => file.path).sort() + + expect(paths).toEqual(['root.ts', 'src/deep/users.ts']) + for (const path of paths) expect(path).not.toContain('\\') + }) + + it('still reads content for nested files after normalization', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-index-separators-')) + roots.push(root) + await mkdir(join(root, 'src', 'deep'), { recursive: true }) + await writeFile(join(root, 'src', 'deep', 'users.ts'), 'export const users = []\n') + + const index = await indexRepository(root) + + expect(index.files).toEqual([ + expect.objectContaining({ path: 'src/deep/users.ts', content: 'export const users = []\n' }), + ]) + }) +}) diff --git a/packages/cli/test/indexer.test.ts b/packages/cli/test/indexer.test.ts index 0160582..6e71068 100644 --- a/packages/cli/test/indexer.test.ts +++ b/packages/cli/test/indexer.test.ts @@ -1,4 +1,4 @@ -import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' @@ -32,4 +32,18 @@ describe('local binary-aware indexing profile', () => { }) }, ) + + // On a Windows host this runs unsimulated and is the real guard; elsewhere it + // pins the contract that indexed paths are the same bytes on every platform. + // See indexer-path-separators.test.ts for the host-independent version. + it('indexes nested files under POSIX separators on this platform', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-local-index-separators-')) + roots.push(root) + await mkdir(join(root, 'src', 'deep'), { recursive: true }) + await writeFile(join(root, 'src', 'deep', 'users.ts'), 'export const users = []\n') + + const index = await indexRepository(root) + + expect(index.files.map((file) => file.path)).toEqual(['src/deep/users.ts']) + }) }) diff --git a/packages/cli/test/local-sast.test.ts b/packages/cli/test/local-sast.test.ts new file mode 100644 index 0000000..a35dd4b --- /dev/null +++ b/packages/cli/test/local-sast.test.ts @@ -0,0 +1,153 @@ +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { analyzeRepository, computeVerdict } from '../src/analysis.js' +import { LOCAL_SAST_PASS_ID, localSastInputs } from '../src/local-sast.js' +import type { AnalyzerFinding, RepoIndex } from '@codetruss/analyzer-engine' +import type { ChangedFile } from '../src/types.js' + +const cleanup: string[] = [] +afterEach(async () => { + await Promise.all(cleanup.splice(0).map((path) => rm(path, { recursive: true, force: true }))) +}) + +function sastFinding(severity: AnalyzerFinding['severity'], ruleId: string): AnalyzerFinding { + return { + category: 'SECURITY_HYGIENE', + severity, + title: 'SQL injection', + description: 'x', + filePath: 'src/a.ts', + line: 1, + impactScore: 95, + analyzerId: LOCAL_SAST_PASS_ID, + metadata: { sast: true, ruleId }, + } +} + +const changed: ChangedFile[] = [ + { path: 'src/a.ts', change: 'modified', classification: 'allowed', dependency: false, additions: 1, deletions: 0 }, +] + +describe('local security findings report without blocking the turn', () => { + it('returns REVIEW_REQUIRED, not FAILED, for a critical local security finding', () => { + const outcome = computeVerdict({ + verifications: [], + files: changed, + startDirty: false, + findings: [sastFinding('CRITICAL', 'sql-injection')], + }) + + // A FAILED verdict at Stop halts the developer's agent. The local pass has + // not earned that yet, so it must surface as review on its first release. + expect(outcome.verdict).toBe('REVIEW_REQUIRED') + expect(outcome.reasons).toContain('1 local security finding(s) affect changed files (sql-injection)') + expect(outcome.reasons.join(' ')).not.toContain('high/critical security or dependency') + }) + + it('still fails on a high non-local security or dependency finding', () => { + const outcome = computeVerdict({ + verifications: [], + files: changed, + startDirty: false, + findings: [ + { + category: 'SECURITY_HYGIENE', + severity: 'HIGH', + title: 'Committed credential', + description: 'x', + filePath: 'src/a.ts', + impactScore: 90, + analyzerId: 'secrets', + }, + ], + }) + expect(outcome.verdict).toBe('FAILED') + }) + + it('counts a local finding once, under its own reason', () => { + const outcome = computeVerdict({ + verifications: [], + files: changed, + startDirty: false, + findings: [sastFinding('MEDIUM', 'db-call-in-loop')], + }) + expect(outcome.reasons.filter((reason) => reason.includes('finding(s)'))).toEqual([ + '1 local security finding(s) affect changed files (db-call-in-loop)', + ]) + }) +}) + +describe('local security pass file selection', () => { + const index = (files: Array<{ path: string; kind: string }>): RepoIndex => + ({ + root: '/tmp', + files: files.map((file) => ({ ...file, language: null, sizeBytes: 1, loc: 1, sha: null, content: 'const a = 1' })), + languages: {}, + frameworks: [], + packageManagers: [], + databases: [], + dependencies: new Set(), + totalLoc: 1, + primaryLanguage: 'TypeScript', + repoType: 'application', + vendoredDirs: {}, + }) as unknown as RepoIndex + + it('analyzes production JS-family source only', () => { + const inputs = localSastInputs( + index([ + { path: 'src/a.ts', kind: 'source' }, + { path: 'src/b.tsx', kind: 'source' }, + { path: 'src/c.js', kind: 'source' }, + { path: 'src/d.test.ts', kind: 'test' }, + { path: 'vendor/e.js', kind: 'vendored' }, + { path: 'src/f.generated.ts', kind: 'generated' }, + { path: 'src/g.py', kind: 'source' }, + { path: 'src/h.go', kind: 'source' }, + ]), + ) + expect(inputs.map((input) => input.filePath)).toEqual(['src/a.ts', 'src/b.tsx', 'src/c.js']) + }) + + it('skips declaration files, which hold no executable code to analyze', () => { + const inputs = localSastInputs( + index([ + { path: 'src/types.d.ts', kind: 'source' }, + { path: 'src/types.d.mts', kind: 'source' }, + { path: 'src/real.ts', kind: 'source' }, + ]), + ) + expect(inputs.map((input) => input.filePath)).toEqual(['src/real.ts']) + }) +}) + +describe('the local pass is a pass, not a registry analyzer', () => { + it('runs alongside the 13 registry analyzers without joining them', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-local-sast-pass-')) + cleanup.push(root) + await mkdir(join(root, 'src')) + await writeFile(join(root, 'src', 'a.ts'), 'export const a = 1\n') + + const analysis = await analyzeRepository(root) + const ids = analysis.passes.map((pass) => pass.id) + expect(ids).toHaveLength(14) + expect(ids.filter((id) => id === LOCAL_SAST_PASS_ID)).toHaveLength(1) + expect(ids.at(-1)).toBe(LOCAL_SAST_PASS_ID) + expect(analysis.passes.at(-1)?.result.complete).toBe(true) + }) + + it('reports a file it could not parse as lost coverage instead of staying silent', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-local-sast-degraded-')) + cleanup.push(root) + await mkdir(join(root, 'src')) + await writeFile(join(root, 'src', 'broken.ts'), 'export function ( { !!! not parseable\n') + + const analysis = await analyzeRepository(root) + const pass = analysis.passes.find((entry) => entry.id === LOCAL_SAST_PASS_ID) + expect(pass?.result.complete).toBe(false) + expect(pass?.result.truncated).toBe(true) + expect(pass?.result.detail).toMatch(/could not be parsed locally/) + }) +}) diff --git a/packages/cli/test/receipt.test.ts b/packages/cli/test/receipt.test.ts index 4a00974..d09177c 100644 --- a/packages/cli/test/receipt.test.ts +++ b/packages/cli/test/receipt.test.ts @@ -27,6 +27,20 @@ function fixture(root: string, patch = 'diff evidence'): Receipt { } } +/** A receipt as CLI <= 0.2.34 signed it, when no security pass ran locally. */ +function profileV1Fixture(root: string, patch = 'diff evidence'): Receipt { + const receipt = fixture(root, patch) + return { + ...receipt, + analyzers: { + passes: receipt.analyzers.passes, + findings: receipt.analyzers.findings, + index: receipt.analyzers.index, + analysisProfile: { id: 'local-registry-v1', omittedPasses: ['graph', 'sast'], scoreStatus: 'not-computed' }, + }, + } +} + function legacyFixture(root: string, patch = 'diff evidence'): Receipt { const receipt = fixture(root, patch) return { @@ -58,7 +72,7 @@ describe('signed receipts', () => { await expect(verifyReceipt(dir, receipt.sessionId)).resolves.toMatchObject({ verdict: 'PASS' }) const markdown = await readFile(paths.markdown, 'utf8') expect(markdown).toContain('Policy SHA-256') - expect(markdown).toContain('Profile: `local-registry-v1`') + expect(markdown).toContain('Profile: `local-registry-v2`') expect(markdown).not.toContain('Final scores:') await writeFile(paths.markdown, `${await readFile(paths.markdown, 'utf8')}tampered`) await expect(verifyReceipt(dir, receipt.sessionId)).rejects.toThrow('Markdown receipt does not match') @@ -85,23 +99,46 @@ describe('signed receipts', () => { expect(renderMarkdown(verified)).not.toContain('security 100') }) - it('names the omitted passes as detection gaps rather than a missing score', () => { + it('names both what the local security pass checked and what it still did not', () => { const markdown = renderMarkdown(fixture('/tmp/repo')) + const checked = markdown.slice( + markdown.indexOf('### What the local security pass checked'), + markdown.indexOf('### What did not run'), + ) const disclosure = markdown.slice(markdown.indexOf('### What did not run')) - expect(markdown).toContain('### What did not run') - // The gap a developer must not misread: injection was never examined. - expect(disclosure).toContain('**Security static analysis (SAST).**') - expect(disclosure).toMatch(/SQL injection.*command injection.*path traversal/) - expect(disclosure).toContain('says nothing either way about those classes') + // Claiming coverage is only honest next to its own boundary. + expect(checked).toContain('SQL injection') + expect(checked).toContain('Mass assignment') + expect(disclosure).toContain('**The rest of the security rule pack.**') + expect(disclosure).toMatch(/Command injection.*path traversal.*SSRF/) + expect(disclosure).toContain('means they were not analyzed, not that the code is clean') + expect(disclosure).toContain('**Non-JavaScript languages.**') expect(disclosure).toContain('**Hosted symbol graph.**') expect(disclosure).toContain('**Optional LLM review.**') expect(disclosure).toContain('force-disabled under agent hooks') + // The carve-out is stated where a reader will look for it. + expect(markdown).toContain('do not fail the verdict on their own') // Never an accusation: the receipt reports what ran, not a verdict on the code. expect(disclosure).not.toMatch(/vulnerab|insecure code|unsafe/i) expect(markdown).toContain('It is not a statement that this change is secure.') }) + it('reproduces the v1 wording for a receipt signed before SAST ran locally', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-profile-v1-receipt-')) + const dir = join(root, 'receipts') + process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') + const receipt = profileV1Fixture(root) + const paths = await writeReceipt(dir, receipt, 'diff evidence') + + const markdown = await readFile(paths.markdown, 'utf8') + expect(markdown).toContain('Profile: `local-registry-v1`') + // The claim that execution actually made, not the one the current CLI makes. + expect(markdown).toContain('**Security static analysis (SAST).**') + expect(markdown).not.toContain('### What the local security pass checked') + await expect(verifyReceipt(dir, receipt.sessionId)).resolves.toMatchObject({ verdict: 'PASS' }) + }) + it('does not list the LLM review as omitted when a model actually reviewed the diff', () => { const receipt = fixture('/tmp/repo') receipt.llm = { @@ -119,7 +156,7 @@ describe('signed receipts', () => { const root = await mkdtemp(join(tmpdir(), 'codetruss-prior-profile-receipt-')) const dir = join(root, 'receipts') process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') - const receipt = fixture(root) + const receipt = profileV1Fixture(root) const paths = await writeReceipt(dir, receipt, 'diff evidence') const priorMarkdown = renderPriorProfileMarkdown(receipt) receipt.evidence.markdownSha256 = sha256(priorMarkdown) @@ -134,6 +171,34 @@ describe('signed receipts', () => { await expect(verifyReceipt(dir, receipt.sessionId)).resolves.toMatchObject({ verdict: 'PASS' }) }) + it('still verifies a 0.2.30 receipt after the generated-exclusion disclosure is reworded', async () => { + // Receipt Markdown is signed, and verifyReceipt only accepts renderings it + // can reproduce. Analyzer wording is safe to change ONLY because it is + // carried in the signed JSON rather than re-derived at render time — this + // pins that. A receipt written by 0.2.30 with the superseded + // "e.g. " disclosure must keep verifying byte-for-byte. + const root = await mkdtemp(join(tmpdir(), 'codetruss-generated-disclosure-receipt-')) + const dir = join(root, 'receipts') + process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') + const receipt = fixture(root) + receipt.analyzers.findings = [{ + category: 'STRUCTURE', + severity: 'LOW', + analyzerId: 'structure', + title: 'Generated code excluded from analysis (2 files, 448 KB)', + description: 'CodeTruss detected 2 machine-generated or minified files (~22 LOC, 448 KB, e.g. `static/js/ace.js`) and excluded them from LOC totals, scores, and the architecture graph.', + filePath: 'static/js/ace.js', + impactScore: 20, + effort: 'low', + }] + + await writeReceipt(dir, receipt, 'diff evidence') + + await expect(verifyReceipt(dir, receipt.sessionId)).resolves.toMatchObject({ verdict: 'PASS' }) + const markdown = await readFile(join(dir, `${receipt.sessionId}.md`), 'utf8') + expect(markdown).toContain('Generated code excluded from analysis (2 files, 448 KB)') + }) + it('renders explicit optional LLM diff coverage', () => { const receipt = fixture('/tmp/repo') receipt.llm = { @@ -152,6 +217,75 @@ describe('signed receipts', () => { expect(renderMarkdown(current)).toBe(renderMarkdown(preProvenance)) }) + it('discloses inferred scope as inferred, names what it was read from, and still verifies', async () => { + const root = await mkdtemp(join(tmpdir(), 'codetruss-inferred-scope-')) + const dir = join(root, 'receipts') + process.env.CODETRUSS_SIGNING_KEY = join(root, 'signing.pem') + const receipt = fixture(root) + receipt.scope = { + allow: ['lib/**'], + deny: [], + inferred: [{ root: 'src/auth', basis: 'working-set', evidence: ['src/auth/reset.ts', 'src/auth/tokens.ts'] }], + } + receipt.files = [ + { path: 'src/auth/reset.ts', change: 'added', classification: 'inferred', dependency: false, additions: 9, deletions: 0 }, + { path: 'src/auth/tokens.ts', change: 'modified', classification: 'inferred', dependency: false, additions: 2, deletions: 1 }, + ] + await writeReceipt(dir, receipt, 'diff evidence') + const markdown = renderMarkdown(receipt) + + // The changed-file row must never read as plainly approved scope. + expect(markdown).toContain('| `src/auth/reset.ts` | added | allowed (inferred) |') + expect(markdown).toContain('## Inferred scope (1)') + expect(markdown).toContain('2 changed file(s) matched no approved allow root.') + expect(markdown).toContain('| `src/auth` | working set for this turn | `src/auth/reset.ts`, `src/auth/tokens.ts` |') + expect(markdown).toContain('applied them to this turn only') + expect(markdown).toContain('were not written to `.codetruss.yml`') + expect(markdown).toContain('Approved allow roots: `lib/**`.') + expect(markdown).toContain('Denied paths, sensitive surfaces, and dependency manifests are never inferable.') + await expect(verifyReceipt(dir, receipt.sessionId)).resolves.toMatchObject({ verdict: 'PASS' }) + }) + + it('says the scope was inferred entirely when the repository approved no roots', () => { + const receipt = fixture('/tmp/repo') + receipt.scope = { + allow: [], + deny: [], + inferred: [{ root: 'server/handlers', basis: 'task-reference', evidence: ['password reset'] }], + } + receipt.files = [{ + path: 'server/handlers/reset.ts', change: 'added', classification: 'inferred', dependency: false, additions: 9, deletions: 0, + }] + + const markdown = renderMarkdown(receipt) + + expect(markdown).toContain('This repository has no approved allow roots, so its scope for this turn was inferred entirely.') + expect(markdown).toContain('| `server/handlers` | named in the task | `password reset` |') + }) + + it('renders a receipt that inferred nothing exactly as before, so earlier signatures keep verifying', () => { + const receipt = fixture('/tmp/repo') + receipt.files = [ + { path: 'src/a.ts', change: 'modified', classification: 'allowed', dependency: false, additions: 1, deletions: 0 }, + { path: 'infra/prod.tf', change: 'modified', classification: 'unexpected', sensitive: 'iac', dependency: false, additions: 1, deletions: 0 }, + { path: 'secrets/key.pem', change: 'added', classification: 'denied', sensitive: 'secrets', dependency: false, additions: 1, deletions: 0 }, + ] + + const markdown = renderMarkdown(receipt) + + expect(markdown).not.toContain('Inferred scope') + expect(markdown).not.toContain('(inferred)') + // Byte-for-byte the pre-inference rows: the Scope cell is the raw value. + expect(markdown).toContain('| `src/a.ts` | modified | allowed | — | +1/−0 |') + expect(markdown).toContain('| `infra/prod.tf` | modified | unexpected | iac | +1/−0 |') + expect(markdown).toContain('| `secrets/key.pem` | added | denied | secrets | +1/−0 |') + // An inference-free receipt renders identically whether or not the signed + // JSON was written by a client that knew about the field at all. + const preInference = structuredClone(receipt) + preInference.scope = { allow: receipt.scope.allow, deny: receipt.scope.deny } + expect(renderMarkdown(receipt)).toBe(renderMarkdown(preInference)) + }) + it('rejects a forged receipt signed by a substituted embedded key', async () => { const root = await mkdtemp(join(tmpdir(), 'codetruss-receipt-forgery-')) const dir = join(root, 'receipts') diff --git a/packages/cli/test/scope-inference.test.ts b/packages/cli/test/scope-inference.test.ts new file mode 100644 index 0000000..69e36eb --- /dev/null +++ b/packages/cli/test/scope-inference.test.ts @@ -0,0 +1,236 @@ +import { describe, expect, it } from 'vitest' +import { applyInferredScope, inferTurnScope, type ScopeCandidate } from '../src/scope-inference.js' +import { computeVerdict } from '../src/analysis.js' +import { classifyPath, isDependencyFile, sensitiveCategory } from '../src/policy.js' +import type { ChangedFile } from '../src/types.js' + +/** + * Scope drift is the detection nobody else ships, and on a stranger's first + * session it fires on the first reasonable edit outside whatever directories + * `setup` found on disk. These cases pin the line between "in scope on weaker + * evidence, and said so" and real drift. + */ + +/** Classify exactly as a review does, so a case can never assume its own answer. */ +function turn(paths: Array, allow: string[], deny: string[] = []): ChangedFile[] { + return paths.map((entry) => { + const [path, oldPath] = Array.isArray(entry) ? entry : [entry, undefined] + return { + path, + ...(oldPath ? { oldPath } : {}), + change: oldPath ? 'renamed' : 'modified', + classification: classifyPath(path, oldPath, allow, deny), + ...(sensitiveCategory(path) ? { sensitive: sensitiveCategory(path)! } : {}), + dependency: isDependencyFile(path), + additions: 1, + deletions: 0, + } satisfies ChangedFile + }) +} + +function resolve(task: string, files: ChangedFile[], allow: string[], deny: string[] = []) { + return applyInferredScope(files, inferTurnScope({ task, files, allow, deny })) +} + +function scope(files: ScopeCandidate[]): Record { + return Object.fromEntries(files.map((file) => [file.path, file.classification])) +} + +describe('turn-scoped scope inference', () => { + it('reads a cohesive working set as in scope and an unrelated subsystem as drift', () => { + const allow = ['lib/**'] + const files = turn([ + 'src/auth/password-reset.ts', + 'src/auth/tokens.ts', + 'src/billing/webhooks.ts', + ], allow) + + const { files: classified, roots } = resolve('Add password reset', files, allow) + + expect(scope(classified)).toEqual({ + 'src/auth/password-reset.ts': 'inferred', + 'src/auth/tokens.ts': 'inferred', + 'src/billing/webhooks.ts': 'unexpected', + }) + expect(roots).toEqual([{ + root: 'src/auth', + basis: 'working-set', + evidence: ['src/auth/password-reset.ts', 'src/auth/tokens.ts'], + }]) + }) + + it('leaves a lone file in an unnamed subsystem outside scope', () => { + const allow = ['lib/**'] + const files = turn(['src/billing/webhooks.ts'], allow) + + const { files: classified, roots } = resolve('Add password reset', files, allow) + + expect(scope(classified)).toEqual({ 'src/billing/webhooks.ts': 'unexpected' }) + expect(roots).toEqual([]) + }) + + it('takes a single file in scope when the task names its path', () => { + const allow = ['lib/**'] + const files = turn(['src/auth/password-reset.ts'], allow) + + const { files: classified, roots } = resolve('Add password reset under src/auth', files, allow) + + expect(scope(classified)).toEqual({ 'src/auth/password-reset.ts': 'inferred' }) + expect(roots).toEqual([{ root: 'src/auth', basis: 'task-reference', evidence: ['src/auth'] }]) + }) + + it('takes a single file in scope when the task names its directory by feature name', () => { + const allow = ['lib/**'] + const files = turn(['src/password-reset/token.ts'], allow) + + const { files: classified, roots } = resolve('Add password reset', files, allow) + + expect(scope(classified)).toEqual({ 'src/password-reset/token.ts': 'inferred' }) + expect(roots).toEqual([{ root: 'src/password-reset', basis: 'task-reference', evidence: ['password reset'] }]) + }) + + it('never treats a generic directory name as the subject of the task', () => { + const allow = ['lib/**'] + const files = turn(['scripts/seed.ts'], allow) + + const { roots } = resolve('Add a script that seeds the database', files, allow) + + expect(roots).toEqual([]) + }) + + it('never lets inference reopen a denied path', () => { + const allow = ['lib/**'] + const deny = ['src/auth/secrets.ts'] + const files = turn(['src/auth/reset.ts', 'src/auth/tokens.ts', 'src/auth/secrets.ts'], allow, deny) + + const { files: classified, roots } = resolve('Add password reset', files, allow, deny) + + expect(scope(classified)).toEqual({ + 'src/auth/reset.ts': 'inferred', + 'src/auth/tokens.ts': 'inferred', + 'src/auth/secrets.ts': 'denied', + }) + expect(roots.map((root) => root.root)).toEqual(['src/auth']) + }) + + it('never infers scope for a secrets, config, or dependency surface inside an inferred root', () => { + const files = turn([ + 'src/auth/reset.ts', + 'src/auth/.env.local', + 'src/auth/package.json', + 'prisma/migrations/001_add_reset.sql', + ], []) + + const { files: classified } = resolve('Add password reset', files, []) + + expect(scope(classified)).toEqual({ + 'src/auth/reset.ts': 'inferred', + 'src/auth/.env.local': 'unexpected', + 'src/auth/package.json': 'unexpected', + 'prisma/migrations/001_add_reset.sql': 'unexpected', + }) + }) + + it('refuses to climb above an allow root the repository deliberately narrowed', () => { + const allow = ['src/auth/**'] + const files = turn(['src/billing/invoice.ts', 'src/billing/webhooks.ts'], allow) + + // `src` would swallow the sibling the repository chose not to approve. + // A named sibling working set may still be inferred; its parent may not. + const { roots } = resolve('Rework invoices', files, allow) + expect(roots.map((root) => root.root)).toEqual(['src/billing']) + + const climbing = resolve('Rework invoices', turn(['src/a.ts', 'src/b.ts'], allow), allow) + expect(climbing.roots).toEqual([]) + expect(scope(climbing.files)).toEqual({ 'src/a.ts': 'unexpected', 'src/b.ts': 'unexpected' }) + }) + + it('makes the turn its own scope when no allow root was ever approved, but never the repository root', () => { + const files = turn(['server/handlers/reset.ts', 'README.md'], []) + + const { files: classified, roots } = resolve('Add password reset', files, []) + + expect(scope(classified)).toEqual({ + 'server/handlers/reset.ts': 'inferred', + 'README.md': 'unexpected', + }) + expect(roots.map((root) => root.root)).toEqual(['server/handlers']) + }) + + it('admits a test file only when it mirrors a source file already in scope', () => { + const allow = ['src/**'] + const files = turn(['src/auth/reset.ts', 'tests/auth/reset.test.ts', 'tests/billing/refund.test.ts'], allow) + + const { files: classified, roots } = resolve('Add password reset', files, allow) + + expect(scope(classified)).toEqual({ + 'src/auth/reset.ts': 'allowed', + 'tests/auth/reset.test.ts': 'inferred', + 'tests/billing/refund.test.ts': 'unexpected', + }) + expect(roots).toEqual([{ + root: 'tests/auth/reset.test.ts', + basis: 'sibling-test', + evidence: ['src/auth/reset.ts'], + }]) + }) + + it('holds a rename in scope only when its origin is in scope too, and reports no allowance that covered nothing', () => { + const files = turn([['src/auth/reset.ts', 'infra/legacy-reset.ts']], []) + + const { files: classified, roots } = resolve('Add password reset', files, []) + + expect(scope(classified)).toEqual({ 'src/auth/reset.ts': 'unexpected' }) + expect(roots).toEqual([]) + }) + + it('is a pure function of the task, the changed files, and the policy, so a receipt can be replayed', () => { + const allow = ['lib/**'] + const files = turn(['src/auth/reset.ts', 'src/auth/tokens.ts'], allow) + + expect(inferTurnScope({ task: 'Add password reset', files, allow, deny: [] })) + .toEqual(inferTurnScope({ task: 'Add password reset', files, allow, deny: [] })) + }) +}) + +describe('inferred scope on the verdict', () => { + const verdictFor = (files: ChangedFile[]) => computeVerdict({ + agentExitCode: 0, + verifications: [{ command: 'test', exitCode: 0, durationMs: 1, output: '', truncated: false }], + files, + startDirty: false, + findings: [], + }) + + it('keeps an inferred path out of the drift reasons and never calls it approved scope', () => { + const allow = ['lib/**'] + const { files } = resolve('Add password reset', turn(['src/auth/reset.ts', 'src/auth/tokens.ts'], allow), allow) + + const outcome = verdictFor(files) + + expect(outcome.verdict).toBe('PASS') + expect(outcome.reasons.join('\n')).not.toContain('outside approved scope') + expect(outcome.reasons).toContain( + '0 changed file(s) are within approved scope; 2 more matched scope inferred from this turn and disclosed on the receipt', + ) + }) + + it('still requires review for the file inference refused to cover', () => { + const allow = ['lib/**'] + const { files } = resolve( + 'Add password reset', + turn(['src/auth/reset.ts', 'src/auth/tokens.ts', 'src/billing/webhooks.ts'], allow), + allow, + ) + + const outcome = verdictFor(files) + + expect(outcome.verdict).toBe('REVIEW_REQUIRED') + expect(outcome.reasons).toContain('1 file(s) changed outside approved scope: src/billing/webhooks.ts') + }) + + it('keeps the approved-scope sentence unchanged when nothing was inferred', () => { + expect(verdictFor(turn(['src/a.ts'], ['src/**'])).reasons) + .toContain('all 1 changed file(s) are within approved scope') + }) +}) diff --git a/packages/cli/test/verify-trust.test.ts b/packages/cli/test/verify-trust.test.ts index b89eca3..9c00603 100644 --- a/packages/cli/test/verify-trust.test.ts +++ b/packages/cli/test/verify-trust.test.ts @@ -1,5 +1,5 @@ -import { mkdtemp, readFile, writeFile } from 'node:fs/promises' -import { tmpdir } from 'node:os' +import { mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises' +import { homedir, tmpdir } from 'node:os' import { join } from 'node:path' import { describe, expect, it } from 'vitest' import { @@ -7,8 +7,29 @@ import { trustVerifyCommands, verifyCommandTrustHash, verifyCommandTrustStatus, + verifyTrustFilePath, } from '../src/verify-trust.js' +/** A real environment with XDG_CONFIG_HOME set, or explicitly unset. */ +function env(configHome?: string): NodeJS.ProcessEnv { + return { ...process.env, XDG_CONFIG_HOME: configHome } +} + +/** Node's homedir() reads $HOME on POSIX and %USERPROFILE% on Windows. */ +async function withHome(home: string, body: () => Promise): Promise { + const prior = { HOME: process.env.HOME, USERPROFILE: process.env.USERPROFILE } + process.env.HOME = home + process.env.USERPROFILE = home + try { + await body() + } finally { + for (const [name, value] of Object.entries(prior)) { + if (value === undefined) delete process.env[name] + else process.env[name] = value + } + } +} + describe('user-local verification command trust', () => { it('binds trust to the canonical repository and exact ordered commands without storing either', async () => { const parent = await mkdtemp(join(tmpdir(), 'codetruss-verify-trust-')) @@ -39,6 +60,51 @@ describe('user-local verification command trust', () => { await expect(verifyCommandTrustStatus(root, commands, trustFile)).resolves.toMatchObject({ trusted: false }) }) + it('honors XDG_CONFIG_HOME without orphaning an existing trust store', async () => { + // auth-storage.ts already honors XDG_CONFIG_HOME; this file honored only + // homedir(), so one user's CodeTruss state was split across two directories. + const parent = await mkdtemp(join(tmpdir(), 'codetruss-xdg-')) + const home = join(parent, 'home') + const configHome = join(parent, 'xdg') + await mkdir(home, { recursive: true }) + const legacy = join(home, '.config', 'codetruss', 'verify-command-trust.json') + const preferred = join(configHome, 'codetruss', 'verify-command-trust.json') + + await withHome(home, async () => { + expect(homedir()).toBe(home) + expect(verifyTrustFilePath(env())).toBe(legacy) + expect(verifyTrustFilePath(env(' '))).toBe(legacy) + expect(verifyTrustFilePath(env(configHome))).toBe(preferred) + expect(() => verifyTrustFilePath(env('relative/config'))) + .toThrow('XDG_CONFIG_HOME must be an absolute user config path') + }) + }) + + it('keeps reading an approval already stored at the legacy path', async () => { + // Migration: setting XDG_CONFIG_HOME must never silently revoke commands + // the user already inspected and trusted. + const parent = await mkdtemp(join(tmpdir(), 'codetruss-xdg-migration-')) + const home = join(parent, 'home') + const configHome = join(parent, 'xdg') + await mkdir(home, { recursive: true }) + const legacy = join(home, '.config', 'codetruss', 'verify-command-trust.json') + const preferred = join(configHome, 'codetruss', 'verify-command-trust.json') + + await withHome(home, async () => { + // An approval made before XDG_CONFIG_HOME was set keeps being honored. + await trustVerifyCommands(home, ['pnpm test'], legacy) + expect(verifyTrustFilePath(env(configHome))).toBe(legacy) + await expect( + verifyCommandTrustStatus(home, ['pnpm test'], verifyTrustFilePath(env(configHome))), + ).resolves.toMatchObject({ trusted: true }) + + // Once a store exists at the XDG path it wins outright. + await mkdir(join(configHome, 'codetruss'), { recursive: true }) + await writeFile(preferred, '{"version":1,"trusted":{}}\n') + expect(verifyTrustFilePath(env(configHome))).toBe(preferred) + }) + }) + it('fails closed on a corrupt user trust store', async () => { const parent = await mkdtemp(join(tmpdir(), 'codetruss-verify-trust-corrupt-')) const trustFile = join(parent, 'verify-command-trust.json') diff --git a/public/downloads/codetruss-cli-0.2.36.sbom.cdx.json b/public/downloads/codetruss-cli-0.2.36.sbom.cdx.json new file mode 100644 index 0000000..1cce3fc --- /dev/null +++ b/public/downloads/codetruss-cli-0.2.36.sbom.cdx.json @@ -0,0 +1,170 @@ +{ + "$schema": "https://cyclonedx.org/schema/bom-1.6.schema.json", + "bomFormat": "CycloneDX", + "serialNumber": "urn:uuid:ff01a909-707a-5338-90a8-39a81331538d", + "specVersion": "1.6", + "version": 1, + "metadata": { + "component": { + "type": "application", + "bom-ref": "pkg:npm/%40codetruss/cli@0.2.36", + "name": "@codetruss/cli", + "version": "0.2.36", + "description": "Local-first scope, quality, and verification receipts for coding agents", + "licenses": [ + { + "license": { + "name": "CodeTruss CLI Proprietary License" + } + } + ], + "purl": "pkg:npm/%40codetruss/cli@0.2.36" + }, + "properties": [ + { + "name": "codetruss:distribution", + "value": "single-file JavaScript bundle" + }, + { + "name": "codetruss:runtimeDependencies", + "value": "0" + } + ] + }, + "components": [ + { + "type": "library", + "bom-ref": "pkg:npm/%40codetruss/analyzer-engine@0.1.0", + "name": "@codetruss/analyzer-engine", + "version": "0.1.0", + "licenses": [ + { + "license": { + "name": "CodeTruss CLI Proprietary License" + } + } + ], + "purl": "pkg:npm/%40codetruss/analyzer-engine@0.1.0", + "properties": [ + { + "name": "codetruss:bundled", + "value": "true" + } + ] + }, + { + "type": "library", + "bom-ref": "pkg:npm/balanced-match@4.0.4", + "name": "balanced-match", + "version": "4.0.4", + "licenses": [ + { + "license": { + "id": "MIT" + } + } + ], + "purl": "pkg:npm/balanced-match@4.0.4", + "properties": [ + { + "name": "codetruss:bundled", + "value": "true" + } + ] + }, + { + "type": "library", + "bom-ref": "pkg:npm/brace-expansion@5.0.7", + "name": "brace-expansion", + "version": "5.0.7", + "licenses": [ + { + "license": { + "id": "MIT" + } + } + ], + "purl": "pkg:npm/brace-expansion@5.0.7", + "properties": [ + { + "name": "codetruss:bundled", + "value": "true" + } + ] + }, + { + "type": "library", + "bom-ref": "pkg:npm/minimatch@10.2.5", + "name": "minimatch", + "version": "10.2.5", + "licenses": [ + { + "license": { + "id": "BlueOak-1.0.0" + } + } + ], + "purl": "pkg:npm/minimatch@10.2.5", + "properties": [ + { + "name": "codetruss:bundled", + "value": "true" + } + ] + }, + { + "type": "library", + "bom-ref": "pkg:npm/yaml@2.9.0", + "name": "yaml", + "version": "2.9.0", + "licenses": [ + { + "license": { + "id": "ISC" + } + } + ], + "purl": "pkg:npm/yaml@2.9.0", + "properties": [ + { + "name": "codetruss:bundled", + "value": "true" + } + ] + } + ], + "dependencies": [ + { + "ref": "pkg:npm/%40codetruss/analyzer-engine@0.1.0", + "dependsOn": [] + }, + { + "ref": "pkg:npm/%40codetruss/cli@0.2.36", + "dependsOn": [ + "pkg:npm/%40codetruss/analyzer-engine@0.1.0", + "pkg:npm/minimatch@10.2.5", + "pkg:npm/yaml@2.9.0" + ] + }, + { + "ref": "pkg:npm/balanced-match@4.0.4", + "dependsOn": [] + }, + { + "ref": "pkg:npm/brace-expansion@5.0.7", + "dependsOn": [ + "pkg:npm/balanced-match@4.0.4" + ] + }, + { + "ref": "pkg:npm/minimatch@10.2.5", + "dependsOn": [ + "pkg:npm/brace-expansion@5.0.7" + ] + }, + { + "ref": "pkg:npm/yaml@2.9.0", + "dependsOn": [] + } + ] +} diff --git a/public/downloads/codetruss-cli-0.2.36.tgz b/public/downloads/codetruss-cli-0.2.36.tgz new file mode 100644 index 0000000..3d385f1 Binary files /dev/null and b/public/downloads/codetruss-cli-0.2.36.tgz differ diff --git a/public/downloads/codetruss-cli-0.2.36.tgz.sha256 b/public/downloads/codetruss-cli-0.2.36.tgz.sha256 new file mode 100644 index 0000000..f65d2dd --- /dev/null +++ b/public/downloads/codetruss-cli-0.2.36.tgz.sha256 @@ -0,0 +1 @@ +953e3f48725a7471043b48a55aedd33ef78854615a341dd95881d90eeae3e814 codetruss-cli-0.2.36.tgz diff --git a/public/downloads/codetruss-cli-latest.json b/public/downloads/codetruss-cli-latest.json index 818e0a0..9fdb214 100644 --- a/public/downloads/codetruss-cli-latest.json +++ b/public/downloads/codetruss-cli-latest.json @@ -1,13 +1,13 @@ { "name": "@codetruss/cli", - "version": "0.2.30", - "url": "/downloads/codetruss-cli-0.2.30.tgz", + "version": "0.2.36", + "url": "/downloads/codetruss-cli-0.2.36.tgz", "latestUrl": "/downloads/codetruss-cli-latest.tgz", - "sha256": "9c97f573aa7e7a052fe8d4c578efda6a8d43f2bcfec9a74ab7d2fdf6b53eccdc", - "sbomUrl": "/downloads/codetruss-cli-0.2.30.sbom.cdx.json", - "sbomSha256": "3a01c0977bfe3dc4407d39fd38af8e47521d06fe952570838bd4f011308a6b6d", + "sha256": "953e3f48725a7471043b48a55aedd33ef78854615a341dd95881d90eeae3e814", + "sbomUrl": "/downloads/codetruss-cli-0.2.36.sbom.cdx.json", + "sbomSha256": "41db21a7ada85d04be35aa89858ef486f2d1d74e1161d0235029c7a1b1f8f736", "node": ">=20.9.0", "repository": "https://github.com/DeliriumPulse/codetruss-cli", - "releaseUrl": "https://github.com/DeliriumPulse/codetruss-cli/releases/tag/v0.2.30", - "attestationCommand": "gh attestation verify codetruss-cli-0.2.30.tgz --repo DeliriumPulse/codetruss-cli" + "releaseUrl": "https://github.com/DeliriumPulse/codetruss-cli/releases/tag/v0.2.36", + "attestationCommand": "gh attestation verify codetruss-cli-0.2.36.tgz --repo DeliriumPulse/codetruss-cli" } diff --git a/public/downloads/codetruss-cli-latest.sbom.cdx.json b/public/downloads/codetruss-cli-latest.sbom.cdx.json index 4725dea..1cce3fc 100644 --- a/public/downloads/codetruss-cli-latest.sbom.cdx.json +++ b/public/downloads/codetruss-cli-latest.sbom.cdx.json @@ -1,15 +1,15 @@ { "$schema": "https://cyclonedx.org/schema/bom-1.6.schema.json", "bomFormat": "CycloneDX", - "serialNumber": "urn:uuid:b2765fa3-4817-59b8-8f08-7350aedbf1ca", + "serialNumber": "urn:uuid:ff01a909-707a-5338-90a8-39a81331538d", "specVersion": "1.6", "version": 1, "metadata": { "component": { "type": "application", - "bom-ref": "pkg:npm/%40codetruss/cli@0.2.30", + "bom-ref": "pkg:npm/%40codetruss/cli@0.2.36", "name": "@codetruss/cli", - "version": "0.2.30", + "version": "0.2.36", "description": "Local-first scope, quality, and verification receipts for coding agents", "licenses": [ { @@ -18,7 +18,7 @@ } } ], - "purl": "pkg:npm/%40codetruss/cli@0.2.30" + "purl": "pkg:npm/%40codetruss/cli@0.2.36" }, "properties": [ { @@ -139,7 +139,7 @@ "dependsOn": [] }, { - "ref": "pkg:npm/%40codetruss/cli@0.2.30", + "ref": "pkg:npm/%40codetruss/cli@0.2.36", "dependsOn": [ "pkg:npm/%40codetruss/analyzer-engine@0.1.0", "pkg:npm/minimatch@10.2.5", diff --git a/public/downloads/codetruss-cli-latest.tgz b/public/downloads/codetruss-cli-latest.tgz index 9ca20d8..3d385f1 100644 Binary files a/public/downloads/codetruss-cli-latest.tgz and b/public/downloads/codetruss-cli-latest.tgz differ diff --git a/public/downloads/codetruss-cli-latest.tgz.sha256 b/public/downloads/codetruss-cli-latest.tgz.sha256 index ffe8aa0..48eac5b 100644 --- a/public/downloads/codetruss-cli-latest.tgz.sha256 +++ b/public/downloads/codetruss-cli-latest.tgz.sha256 @@ -1 +1 @@ -9c97f573aa7e7a052fe8d4c578efda6a8d43f2bcfec9a74ab7d2fdf6b53eccdc codetruss-cli-latest.tgz +953e3f48725a7471043b48a55aedd33ef78854615a341dd95881d90eeae3e814 codetruss-cli-latest.tgz diff --git a/release-reference.json b/release-reference.json index 52f88b1..d32df5a 100644 --- a/release-reference.json +++ b/release-reference.json @@ -1,8 +1,8 @@ { "schemaVersion": 1, - "version": "0.2.30", - "websiteArchive": "https://codetruss.com/downloads/codetruss-cli-0.2.30.tgz", - "archiveSha256": "9c97f573aa7e7a052fe8d4c578efda6a8d43f2bcfec9a74ab7d2fdf6b53eccdc", - "sbomSha256": "3a01c0977bfe3dc4407d39fd38af8e47521d06fe952570838bd4f011308a6b6d", - "bundleSha256": "d3be1602d416b8c6b3e92107be8e2897bff46f43982698736f68a3e064088a09" + "version": "0.2.36", + "websiteArchive": "https://codetruss.com/downloads/codetruss-cli-0.2.36.tgz", + "archiveSha256": "953e3f48725a7471043b48a55aedd33ef78854615a341dd95881d90eeae3e814", + "sbomSha256": "41db21a7ada85d04be35aa89858ef486f2d1d74e1161d0235029c7a1b1f8f736", + "bundleSha256": "62d99ca35ac4d0e701e4169cb63d7115a814dc5f436fcb9a638ad119a3306381" }