Skip to content

[finding] service-automation: the durable run-history row folds cancelled (and timed_out) into failed — after a restart the Runs surface cannot tell a cancelled run from a failed one #15223

Description

@os-warren

Observed while delivering the services half of #13937 (branch claude/issue-13937-stranded-run-operator-verb); recorded, not fixed.

The reading, on origin/main 50d6c92

  • RunRecord.status (packages/services/service-automation/src/engine.ts) is declared 'completed' | 'failed'.
  • AutomationEngine.recordLog writes the durable row for every terminal entry (completed, failed, cancelled, timed_out) with status: entry.status === 'completed' ? 'completed' : 'failed'.
  • ObjectStoreSuspendedRunStore.deserializeTerminal reads it back as row.status === 'failed' ? 'failed' : 'completed'; the in-memory store round-trips what it was given (failed).
  • getRun prefers the in-memory ring entry, which still says cancelled; after a restart (or ring-buffer eviction) the durable row answers, and it says failed.

⇒ In-process a cancelled run reads cancelled; after a restart the same run reads failed on getRun, listRuns and the Runs surface. restoreConsumedSuspension's refusal ladder can name RUN_CANCELLED only for a run this process saw cancelled — the durable row gives another replica nothing to distinguish (measured in stranded-run-status.test.ts on the named branch: the stale-copy refusal after a cancellation elsewhere is NO_CONSUMED_SUSPENSION, honestly, because that is all the row says). The narrowing is not explained in either file; ExecutionStatus in @objectstack/spec declares all four terminal members.

Not measured: whether any consumer branches on a durable cancelled today (plugin-approvals' dead-run sweep treats every terminal status alike, so it is unaffected).

Related

#13937 · #13909 · #2585 (run-history retention, the row's origin) · #7533 (the trigger columns on the same row)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions