You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173
scripts/test-shard-timings.json is stale for @objectstack/cli: the partitioner predicts 672s for the shard that carries it, and that shard measured 28m46s against a timeout-minutes: 30 budget. ⇒ Shard 1/6 is one slow run away from being killed on any PR that touches the CLI.
Filed by the domain:engine PM dispatch seat from a measurement a dev round handed back for judgement, ⛔ not filed blind. Unassigned and bare: domain:*, type and priority are triage's.
The measurement
On PR #16154, head 011dfc57f, CI run 34009395649 (attempt 1, and exactly one run on that head, so supersession is excluded on the decisive reading rather than on "the head did not move" — ⚠️ci.yml's concurrency group is keyed on the PR number, not the head sha):
Test Core (1/6) ran 30m05s and was killed; its siblings finished in 9–12m.
⛔ Not a stall and not a concurrency cancel: the stall guard never fired, there was no output silence, there were zero##[error] annotations, and the job was still emitting at 04:06:24.
⭐ The work had actually finished. Turbo printed Tasks: 62 successful, 62 total at 04:06:24 — five seconds before the kill. All five packages on the shard were green (@objectstack/cli 265 files / 3159 passed + 6 expected fail, plus example-embed-objectql, plugin-dev, connector-mcp, hono).
The gap is not marginal noise: 672s predicted versus 1726s measured, a factor of ~2.6, on the shard that carries the repo's largest test package. That is a systematic mis-estimate, not a slow day. ⛔ "Flake" is not a root cause here and neither is "re-run it" — a re-run is a coin flip against a 30-minute wall that the shard is already inside by 74 seconds.
⚠️ And the failure mode it produces is the dangerous one, not a red: a killed shard yields no reading, while the rollup reads green. So the cost is not a lost cycle — it is a PR that can land with a whole shard unmeasured, and nobody looking.
Two shapes, ⛔ not a ruling — the owning lane decides
(a) Refresh the timings and let the partitioner rebalance. Cheapest, and it is the data the partitioner already expects to be current. ⚠️ Ask what makes it go stale and whether anything refreshes it on a schedule — a one-off refresh restores the imbalance the next time the CLI suite grows.
(b) Raise timeout-minutes for this leg. ⛔ This seat's non-binding view: weaker, because it treats the symptom and hides the imbalance until the next threshold. Worth doing as well if the true runtime is genuinely near the budget, but ⛔ not instead.
⭐ Whichever is taken, the durable half is a check that the predicted total per shard is within some factor of the measured one, so the file cannot silently rot again. Without that, this card recurs.
Dedup: the complete open-issue enumeration was read and proven complete at that instant (646 issues + 34 PRs = 680, matching open_issues_count 680 exactly). ⚠️ Stated honestly: an immediately preceding run read 681 vs 682 — a race with live merges, since PRs were closing out of the merge queue while it paged; the matched run is the one quoted. Probes: test-shard-timings0, shard timing0, timeout-minutes0, shard2 (both above). Firing controls on the same corpus: Test Core2, the336.
Refs: #16154 / #15779 (where it was measured) · #16157 (the rollup consequence) · #15208 (same script, different defect).
scripts/test-shard-timings.jsonis stale for@objectstack/cli: the partitioner predicts 672s for the shard that carries it, and that shard measured 28m46s against atimeout-minutes: 30budget. ⇒ Shard 1/6 is one slow run away from being killed on any PR that touches the CLI.Filed by the
domain:enginePM dispatch seat from a measurement a dev round handed back for judgement, ⛔ not filed blind. Unassigned and bare:domain:*, type and priority are triage's.The measurement
On PR #16154, head⚠️
011dfc57f, CI run34009395649(attempt 1, and exactly one run on that head, so supersession is excluded on the decisive reading rather than on "the head did not move" —ci.yml's concurrency group is keyed on the PR number, not the head sha):Test Core (1/6)ran 30m05s and was killed; its siblings finished in 9–12m.##[error]annotations, and the job was still emitting at04:06:24.Tasks: 62 successful, 62 totalat04:06:24— five seconds before the kill. All five packages on the shard were green (@objectstack/cli265 files / 3159 passed + 6 expected fail, plusexample-embed-objectql,plugin-dev,connector-mcp,hono).Test Corecheck green over untested packages — the attestation gate zeroes the whole roster oncancelled#16157.Why this is a card and not a re-run
The gap is not marginal noise: 672s predicted versus 1726s measured, a factor of ~2.6, on the shard that carries the repo's largest test package. That is a systematic mis-estimate, not a slow day. ⛔ "Flake" is not a root cause here and neither is "re-run it" — a re-run is a coin flip against a 30-minute wall that the shard is already inside by 74 seconds.
Two shapes, ⛔ not a ruling — the owning lane decides
timeout-minutesfor this leg. ⛔ This seat's non-binding view: weaker, because it treats the symptom and hides the imbalance until the next threshold. Worth doing as well if the true runtime is genuinely near the budget, but ⛔ not instead.⭐ Whichever is taken, the durable half is a check that the predicted total per shard is within some factor of the measured one, so the file cannot silently rot again. Without that, this card recurs.
Neighbours, checked — ⛔ neither is a duplicate
Test Corecheck green over untested packages — the attestation gate zeroes the whole roster oncancelled#16157 — a cancelled shard makes the requiredTest Corerollup read green over untested packages. ⭐ That is the consequence; this card is one cause. They should be read together and fixed independently: fixing the timings does not make the rollup honest, and fixing the rollup does not stop the shard being killed.partition-test-shards' balancing-pins banner says "these five" over six numbered pins. Same script family, a prose defect, unrelated.Dedup: the complete open-issue enumeration was read and proven complete at that instant (646 issues + 34 PRs = 680, matching⚠️ Stated honestly: an immediately preceding run read 681 vs 682 — a race with live merges, since PRs were closing out of the merge queue while it paged; the matched run is the one quoted. Probes:
open_issues_count680 exactly).test-shard-timings0,shard timing0,timeout-minutes0,shard2 (both above). Firing controls on the same corpus:Test Core2,the336.Refs: #16154 / #15779 (where it was measured) · #16157 (the rollup consequence) · #15208 (same script, different defect).
Generated by Claude Code