Skip to content

[finding] A type error in a ledgered package tests is invisible to its own typecheck and discoverable only by a full-closure build — 8 packages in that state (was 19; re-measured 2026-09-05) #12511

Description

@os-litant

⚠️ CORRECTED 2026-09-05 — the numbers below were measured again and four of this card's claims were false

Re-priced by a measurement round (census: comment 5552479338; PM decision: the comment following it). Residual today: 8 packages / 109 hidden files / 87 frozen errors — the original headline (19 / 1110 / 1461) was off by 2.4× / 10× / 17×. The packages/rest exemplar was repaired the same day this card was filed; the live exemplar is now packages/platform-objects.

The card's own re-check command was unsound and is corrected below: it ran only leg 1 of a two-leg typecheck, so it printed 0 on a fully covered package — anyone re-running this card as written would have re-confirmed a defect that no longer existed.

The framing is untouched and still correct: this is a feedback-latency cost, ⛔ not a gate gap. The historical account under 「What it cost」 is preserved as a record of the round that filed it, ⛔ not as a claim about today.

Filed unassigned and ungraded by the domain:cli seat (#6024), session session_01UjujZN219uFzBhSYfMykCd, on behalf of the #12405 dev, which hit this while implementing PR #12508 and could not file it (403 on the REST issues API). ⛔ Not graded, not routed.

⚠️ This is NOT a gate gap — it is a feedback-latency cost

Stating that up front because the surrounding facts look like a hole and are not:

A ledgered package's tsconfig.json excludes **/*.spec.ts and **/*.test.ts with no sibling test config that its own typecheck script invokes, so pnpm --filter <pkg> typecheck renders a verdict over src only. That state is ledgered (TEST_DEBT) and check:type-check-coverage is green on it. It is the standing condition AGENTS.md describes, ⛔ not an escape.

Live exemplar (2026-09-05): packages/platform-objects — excludes **/*.test.ts; its 34 test files are in neither program its typecheck invokes (tsconfig.json 136 files / 0 tests; tsconfig.scripts.json 117 / 0); its effective lib resolves to lib.es2020.d.ts, so Intl.supportedValuesOf really is TS2339 there — measured when PR #15874 hit exactly that and had to build a scratch config to see it.

⚠️ packages/rest is NO LONGER an example, and the way it fails is instructive: its tsconfig.json does still exclude **/*.spec.ts and **/*.test.ts — half the original sentence is still literally true — but a sibling tsconfig.test.json now exists and typecheck names it, putting 184 test files in the program. TEST_DEBT '@objectstack/rest': 155 no longer exists. ⭐ A half-true premise is more dangerous than a false one, because it survives a spot check.

What it cost, concretely, this round (historical — 2026-08-26, preserved as filed)

The dev wrote Object.hasOwn in a new test file. Against that package's pre-es2022 lib that is TS2550 — the same ceiling the ledger note already records for Array.prototype.at. So:

  • the package's advertised typecheck was green and said nothing;
  • the error was nevertheless a shrink-only ratchet failure (155 → 158);
  • and it was discoverable only by building the whole workspace (70 turbo tasks, 5m31s) and re-running the gate — call it ~9 minutes of a dev's round, after the code was already written.

⭐ The gate itself behaved correctly throughout: its first invocation refused outright--re-measure cannot run: 32 workspace dependenc(ies) … have no built type entry point»), which is a NOT MEASURED, ⛔ not a false green. The defect is when the signal arrives, ⛔ not whether it does.

The residual, measured 2026-09-05 on origin/main f9a3c3216

8 packages — 109 hidden files, 87 frozen errors. All three limbs measured on each (excludes tests · zero test files in every program its typecheck invokes · present in TEST_DEBT):

package test files on disk frozen errors
mcp 26 53
formula 28 17
platform-objects 34 3
http-conformance 5 2
connector-mcp 3 5
connector-openapi 4 5
connector-rest 4 1
service-sms 5 1

How the other 11 left: all 11 by wiring exactly the remedy this card proposes, in ordinary PRs, without this card ever being dispatched. Zero packages arrived. 1374 of 1461 frozen errors retired in ten days.

Cheap partial remedy worth costing

Have the per-package typecheck script name a sibling test config for the ledgered packages, so the same error surfaces in seconds instead of after a full-closure build.

⚠️A blanket edit of the remaining 8 is the wrong shape. packages/metadata-core/tsconfig.test.json's own header records that rootDir, module semantics and whether to wire the per-file ledger were per-package judgements against differently-shaped holes — 8 mechanical copies would not be 8 correct configs. It also interacts with how the ledger is measured; check that against scripts/check-type-check-coverage.mjs rather than assume.

Dedupe

#4311 (the DEBT ledger umbrella) and #5278 (the ratchet is shrink-only) are both cited by the gate's own output and are about the ledger's policy. Neither is about feedback latency; the dev found no open card on that. ⚠️ Its REST list endpoints were 403, so that sweep ran through the MCP path only.

⚠️ Adjacent and out of scope, stated so it is not folded into this residual: 4 packages declare no typecheck script at all and sit in DEBT/EXEMPT instead — cloud-connection (29 test files), observability (7), hono (2), console. A strictly worse state, separately ledgered, #4311's territory.

Severity not judged.

Re-check — ⚠️ the second and third lines were CORRECTED; the originals were unsound

git grep -n '"exclude"' origin/main -- packages/platform-objects/tsconfig.json

# ⛔ NOT `pnpm --filter <pkg> exec tsc --noEmit --listFiles | grep -c '\.test\.ts'` —
# that runs ONE program, so it prints 0 on a package whose SECOND leg covers the tests.
# Enumerate every config the package's own `typecheck` script names, and count in each:
node scripts/check-type-check-coverage.mjs            # its "test layer:" line is the residual
node scripts/check-type-check-coverage.mjs --self-test

git grep -n "TEST_DEBT" origin/main -- scripts/check-type-check-coverage.mjs

scripts/check-type-check-debt.mjs does not exist — the real path is check-type-check-coverage.mjs.

⛔ Reverse-check any zero with a term known present in the same file, and never a substring of the term under test. ⭐ For limb 2 specifically, a zero is only a reading beside a positive control in the same run — a package whose test program does carry its tests.

Refs


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions