Skip to content

feat(spec): ExecutionStepMetrics gains an optional failures slot; FlowRunSummary.failed is declared as the fold INCLUDING what a delegating node rolled up from its child (#15617, spec half) - #16635

Draft
os-trump wants to merge 4 commits into
mainfrom
claude/issue-15617-execution-step-metrics-failure-slot
Draft

feat(spec): ExecutionStepMetrics gains an optional failures slot; FlowRunSummary.failed is declared as the fold INCLUDING what a delegating node rolled up from its child (#15617, spec half)#16635
os-trump wants to merge 4 commits into
mainfrom
claude/issue-15617-execution-step-metrics-failure-slot

Conversation

@os-trump

@os-trump os-trump commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Fixes #15617

Clause-②: yes

Spec half (lane 1) of the maintainer's ruling on #15617 — decision batch #55, director seat, 2026-09-06 13:51Z, the maintainer's reply verbatim 「同意」 (comment 5559667158). The services half is card #16314, pm:blocked on this card by design; #16314 is not addressed here and nothing under packages/services/** is touched — contract first, the engine follows.

What this declares

  • ExecutionStepMetricsSchema (packages/spec/src/automation/execution.zod.ts) gains an optional failures slot — integer ≥ 0 — declared as: node executions that failed inside a child run this execution delegated to and went on from (a subflow child, or a map item, whose run COMPLETED while containing failures) — the child's summary.failed, rolled up so the parent answers "what did this run cause". It folds into the delegating node's nodes[].failures and so into the run-level failed — the same fold SHAPE acted has, but not the same rule at the failed-child boundary (next bullet). Absent = delegated nothing, or the child tracked no count, or the producer did not track it (every subflow/map step the engine emits between this landing and service-automation: subflow and map roll a child's contained failures into the parent's failed through the ExecutionStepMetrics failure slot ruled on #15617 #16314) — never zero.
  • It is NOT the step's own outcome. A step that failed is status: 'failure' and counts once through nodes[].failures as before; a child that FAILED — whether or not it also contained failures before it failed — is exactly that step failure: its own failed, contained and fatal alike, stays on the child's run row and nothing rides up here, so one failure is never counted twice. This is where the rule parts from acted, which does carry a failed child's writes up (subflow-node.ts:144-147, map-node.ts:197-203). The control the card measured (a failing child ⇒ parent failed=1) keeps counting exactly as today.
  • The two FlowRunSummary paragraphs are reconciled: the roll-up paragraph now says which totals roll up and on what rule each — selected / acted as totals from a completed and a failed child alike, unmeasured as one per-execution flag (creditChildRun collapses N child effects to one), failed as a COMPLETED child's contained failures, skipped not at all — and names map beside subflow; the failed TSDoc says the fold INCLUDES what a delegating node rolled up from its child; and — the failure mode triage called out — the field's own .describe() says so at the point of use, not only the paragraph above the schema. FlowRunNodeSummary.failures's describe names the roll-up path (metrics.failures), and its status describe states that status is judged on the node's OWN executions, so a delegating node whose child contained failures reads success beside failures > 0 — the shape service-automation: subflow and map roll a child's contained failures into the parent's failed through the ExecutionStepMetrics failure slot ruled on #15617 #16314's engine has to produce, declared here so it cannot be misread.
  • ExecutionStepLog.metrics's describe mentions the delegating case.
  • Options 2 (prose carve-out) and 3 (a second total) are not taken, per the ruling.

Patch round R1.1 — contract review VERDICT: FAIL, four text fixes, no design change (commits aebacb8c, 7653f814)

The at-tier review upheld the conservative mixed-case rule and minor with no BREAKING banner; what failed was the text. All four items are in the contract text now:

  1. False header clause — "every total here answers it, not only the ones that count writes" is gone. The header paragraph now says, total by total, what rolls and on what rule: selected / acted as totals from a completed and a failed child alike; unmeasured as ONE per-execution flag (N child effects collapse to one, measured at creditChildRun); failed as a COMPLETED child's contained failures; skipped not at all (no metrics slot, no child.summary.skipped reader).
  2. False acted analogy — all six sites changed, plus the changeset: the metrics TSDoc (was :96-97) now says "the same fold shape acted has … NOT the same rule at the failed-child boundary"; the metrics.failures describe (:117), the nodes[].failures describe (:207), the header paragraph (:252), the failed TSDoc (:296-297) and the failed describe (:310) each drop "the way acted …" / "the rule acted follows" / "the same path the child's writes take into acted" and say instead "unlike acted, which carries a failed child's writes"; the changeset's title and both bullets likewise. Residual grep for the five old spellings: 0 hits.
  3. Mixed case in the contract text: every "a child that FAILED rather than contained" became "a child that FAILED — whether or not it also contained failures before it failed — …its own failed, contained and fatal alike, stays on its own run row" — in the metrics TSDoc, the metrics.failures describe, the header, the failed TSDoc, the failed describe, and the changeset.
  4. Third absence arm: metrics.failures describe and TSDoc, and the changeset, now read "Absent = delegated nothing, or the child tracked no count, or the producer did not track it; never zero".

Changeset "Consumers" paragraph now discloses that on a delegating node nodes[].failures may exceed runs (runs: 5, failures: 15 is a legal shape) and no longer equals the node's own failed executions, so a failure RATE derived from failures / runs must read a delegating node's number as failures it caused, child's contained ones included. The nodes[].failures describe says the same at the point of use.

Two pins added (+2 ⇒ 53 in the file): the mixed case, the departure from acted and the third arm are asserted on the metrics.failures describe and the failed describe; runs: 5, failures: 15 parses on FlowRunNodeSummarySchema and its describe says it may exceed runs. Ablation on committed 7653f814: deleting the mixed-case phrase from the metrics.failures describe (on-disk count 1 → 0, blob 2ca9708e ≠ HEAD 1e4824f1) reds exactly the new pin (1 failed | 52 passed); git checkout HEAD -- under a trap restores blob 1e4824f1, git diff HEAD empty, 53 passed.

R1.1 verification on 7653f814: spec build green (dts 34/34); check:generatedcheck:docs stale, regenerated (execution.mdx +6 −6, describe text only); pnpm --filter @objectstack/spec test 461 passed / 12813 passed; typecheck green incl. test layer; gates re-derived from the patched diff — the same 97 families — all run on a fresh ledger, --ran 97/97; 91 green first pass, the six non-zero are the same prerequisite misses as R1 (lint closure rebuilt → doc-formula-expressions, doc-security-posture, docs-transcript-drift re-run exit 0; skill-examples, dual-build-cjs-loads, type-check-debt still need the whole-repo build → CI); eslint narrowed run 2 files / 0 / 0; check:nul-bytes 0. content/docs/automation/flows.mdx:1030 untouched, per the coordinator.

Pins (execution.test.ts, +8 in R1, +2 in R1.1)

  • ExecutionStepMetricsSchema: keeps failures through the parse (a z.object STRIPS an undeclared key, so keeping the value is what proves the declaration); absent stays absent, not defaulted; negative / fractional refused; the describe names subflow, map, summary.failed and separates the slot from the step's own outcome.
  • ExecutionStepLogSchema: a success subflow step inside a loop iteration carries metrics.failures: 1.
  • FlowRunSummarySchema: the card's measured shape as ruled — parent loop { subflow(child) }, subflow node status: success, runs: 5, failures: 1, acted: 4, failed: 1 equal to the fold over nodes[].failures; the control — subflow node status: failure, failures: 1, failed: 1, nothing on top; and the point-of-use describes (failed, nodes[].failures, nodes[].status) carry the widened rule.

Premise check (rule 6) and the PM's mechanism assumptions

  • A2.1 anchors, re-taken on this branch's base 554a1603: ExecutionStepMetricsSchema at :87 (card said :81); node-level failures describe at :183 (card said :177); the subflow roll-up sentence at :224–227 (the ruling's ":218" is the "reads BESIDE that filter" line two paragraphs earlier, so that anchor was stale too); FlowRunSummarySchema at :229; the failed field describe at :270 (card said :264). Both paragraphs were there and did disagree — premise valid.
  • A2.2 — does map flow through ExecutionStepMetrics? Measured, yes: packages/services/service-automation/src/builtin/map-node.ts emits metrics: { selected, acted, unmeasuredEffect } accumulated from each item's child.summary on all three exits (lines 190, 199–202, 243 at 554a1603); subflow-node.ts does the same at 132–140. Both are real producers of this schema, so the declaration names both. The other gap in that comment — whether objectui reads summary.failed — could not be measured here: no ../objectui checkout exists in this container and packages/console/dist is absent. Left open below.
  • A2.3: the engine numbers quoted here (parent failed=0 beside child failed=[0,0,0,0,1]; control parent failed=1) are the card's / the services seat's, not re-run by me.

Verification (R1, on dfe92a7b; base 554a1603 — R1.1 numbers are in the patch-round section above)

  • pnpm --filter @objectstack/spec build — green under the verify lock (2m36s shared-box); check-dts-emitted 34/34.
  • Rebuilt .d.ts: every unmeasuredEffect: ZodOptional of ZodBoolean shape in packages/spec/dist is followed by failures: ZodOptional of ZodNumber — 7 of 7.
  • Consumer-side probe through the real automation entry (scratch tsconfig mapping @objectstack/spec/automation to dist/automation/index.d.ts): { selected: 5, acted: 4, failures: 1 } typechecks (exit 0); the misspelling failuers is refused with TS2561 (exit 2) — the consumer reads the rebuilt declaration, not a cache.
  • pnpm --filter @objectstack/spec check:generated: one artifact stale (check:docs), regenerated with --fix; authorable-surface/automation.json gained the one line automation/ExecutionStepMetrics:failures from the build's gen:schema; api-surface unchanged (it records existence, not shape). authorable-surface.base.json untouched.
  • pnpm --filter @objectstack/spec test: Test Files 461 passed (461) · Tests 12811 passed (12811). The pin file alone: 51 passed (51).
  • pnpm --filter @objectstack/spec typecheck: green, including check:scripts-typecheck and check:test-typecheck (54 files / 261 pinned); tsc -p tsconfig.test.json --listFiles lists src/automation/execution.test.ts — the test layer does compile the new pins.
  • Ablation (committed first, HEAD dfe92a7b; the pin file resolves ./execution.zod SOURCE, so no dist leg is involved): deleted the failures key + describe from ExecutionStepMetricsSchema — anchor count 1 → 0 on disk, blob bd23acc6 ≠ HEAD blob be78cd77; pin file: Tests 4 failed | 47 passed (51) — exactly the four pins that assert the declaration (keeps-through-parse, negative refused, point-of-use describe, step-log carrier) — direction: turned red, as predicted. Restore via git checkout HEAD -- PATH under an EXIT/INT/TERM trap on the absolute path: git diff HEAD empty, git status --porcelain empty, blob equals HEAD (be78cd77), anchor count 1; pin file 51 passed (51).
  • Gatesnode scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (no paths; derived from the merge base) printed 97 commands; every one was run with its exit code recorded to disk as it went; --ran reconciles 97 derived, 97 run, 0 UNRUN. 90 green on the first pass. The seven non-zero first runs, each read from the gate's own verdict line: check-plugin-teardown-shape --self-test exit 1 — "cannot read the positive control" because the shared .git was shallow when it ran; re-run after the clone was deepened: 47 cases pass, exit 0. lint check:doc-formula-expressions, lint check:doc-security-posture, check:docs-transcript-drift — exit 3 PREREQUISITE NOT MET (@objectstack/lint / @objectstack/formula unbuilt); the 4-package lint closure was built under the lock and all three re-run exit 0. spec check:skill-examples (exit 1: packages/client-react/dist unbuilt), check:dual-build-cjs-loads (exit 3: 85 packages without dist/), check:type-check-debt (exit 3: closure not built) — NOT MEASURED locally; they need the whole-repo build and are declared to CI.
  • pnpm check:nul-bytes exit 0; control-byte self-scan of the three hand-edited files: no hits.
  • Repo-wide pnpm lint is CI's run; delivered as a proven narrowing: ① population read from eslint.config.mjs**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs} minus NEVER_LINTED; of this diff's five files exactly two are in it (the .md, .mdx and .json are outside); ② eslint --no-inline-config --format json on those two: 2 files, 0 errors, 0 warnings, exit 0; ③ invariance: the config enables no parserOptions.project and no typed @typescript-eslint rules (its own comment at line 328), so every file's verdict is a function of that file alone and this diff cannot move an untouched file's verdict.
  • check:changeset family: check-empty-changeset, check-changeset-no-major, check-adr-0087-registration all exit 0 (additive minor, no BREAKING banner, so no ADR-0087 marker is owed).

验收备注

维护者速读(草稿)

改了什么:给流程运行的"每一步做了什么"记录(ExecutionStepMetrics)加了一个可选的 failures 槽,用来承载子流程(subflow / map 的每一项)跑完但内部包含了失败时的失败数;并把运行汇总里 failed 的两段互相矛盾的说明统一为:failed 是逐节点失败数的合计,包含委托节点从跑完的子流程卷上来的部分。合计的形状与 acted 相同,但在"子流程整体失败"这一处规则不同:acted 会把失败子流程已写的行卷上来,failures 什么都不卷——子流程整体失败(无论它之前有没有包含过失败)只按父节点自己那一次失败计数。字段自己的描述、节点级 failuresstatus 的描述都写明了这条规则;10 条 pin 钉住声明;@objectstack/spec minor、纯新增。

为什么改:services 席在真实引擎上量到:父流程 loop { subflow(child) } 五次里子流程有一次丢了行,父运行却显示 failed=0——因为 acted 会卷上来而 failed 没有通道。维护者 2026-09-06 裁定方案 1(契约先行、两车道)。本 PR 是契约半边:先把规则声明清楚并钉住;引擎在 #16314 落实卷积。

风险与代价(含回滚):本 PR 不改任何引擎行为、不改任何已发布数字——引擎在 #16314 落地前不会填这个槽,failed 数值与今天完全一致;文档页 flows.mdx#15609 收窄过的措辞按裁决保留到两半都落地。新增的是一个可选键与更宽的声明语义;回滚 = revert 本 PR(纯新增,无迁移、无 tombstone)。"先包含失败、后整体失败"的子流程只算一次、不卷子流程内部的那几次——这条规则契约复审判定正确,现已写进契约文本本身(R1.1)。

席位意见:(留空)

你要做的:确认"failed 包含委托节点从子流程卷上来的失败;整体失败的子流程只按父节点自己的那一次失败计数"这条规则与你的裁决意图一致,然后人工合并(或授权批准入队);合并后 #16314 自动解除阻塞。

🤖 Generated with Claude Code

https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf

…fold reconciliation (#15617 spec half)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
…ExecutionStepMetrics failure slot (#15617)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
@github-actions github-actions Bot added the size/m label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

4 anchor(s) derived from 1 changed package(s); no hand-written page names any of them. ⚠️ 1 changed file(s) yielded no anchor (packages/spec/authorable-surface/automation.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/spec/authorable-surface/automation.json) — pages documenting those are invisible to this run
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 130 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json bc0ac1d1f5775659e72ad35452c463443a71f1bepackageMentionDocs.

Which tree this was computed on

This run read content/docs from c8876d5549f8ce37195eb80cf19f29c3a14d65c5 — the merge of head 7653f814e2bc1e4e548fff9a161a9b7378564c6a into base bc0ac1d1f5775659e72ad35452c463443a71f1be, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin c8876d5549f8ce37195eb80cf19f29c3a14d65c5 && git checkout c8876d5549f8ce37195eb80cf19f29c3a14d65c5
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin bc0ac1d1f5775659e72ad35452c463443a71f1be 7653f814e2bc1e4e548fff9a161a9b7378564c6a && git checkout -B drift-repro bc0ac1d1f5775659e72ad35452c463443a71f1be && git merge --no-ff 7653f814e2bc1e4e548fff9a161a9b7378564c6a

node scripts/docs-audit/affected-docs.mjs --json bc0ac1d1f5775659e72ad35452c463443a71f1be

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

… clause to the totals that roll, drop the acted analogy at the failed-child boundary, state the mixed case and the third absence arm in the contract text

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8SRGcf2eKTK7RRpWCGxwf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants