Fix #301: WhenAll wait-all + aggregate-exception semantics (root cause of rewind deadlock)

[YunchuWang]

Summary

WhenAllTask completed on the first child failure (fail-fast). This changes it to wait-all-then-aggregate — the WhenAll now completes only after all children reach a terminal state, and if any child failed it surfaces a native AggregateError carrying every child failure. This matches .NET Task.WhenAll/AggregateException and Java CompletableFuture.allOf/CompositeTaskFailedException, and is the root-cause fix for #301.

Fixes #301

Root cause

On Azure Storage, rewind is server-side and driven by TaskFailed history events. When a whenAll fan-out has 2+ failing siblings:

1. durabletask-js's WhenAllTask completed on the first child failure (fail-fast), so the orchestration went terminal after only the first TaskFailed was processed.
2. The later failing siblings' TaskFailed events then arrived at the now-terminal instance and were dropped as late messages — so they were never committed to history.
3. Those siblings kept a bare TaskScheduled with no completion.
4. On rewind, the worker's handleTaskScheduled pops the pending action (assuming a completion will arrive), but after rewind none ever does → the whenAll waits forever → 30s deadlock / timeout.

Worker signature: Waiting for 2 -> Returning 1.

The fix (wait-all-then-aggregate)

packages/durabletask-js/src/task/when-all-task.ts → onChildCompleted:

• Increment the completed counter on every child completion.
• Only when all children are terminal (_completedTasks == _tasks.length) mark the WhenAll complete.
• If any child failed, complete as failed with an aggregate of all child failures (see below). Otherwise build the result array.
• Notify the parent only at completion.

Completing only when all siblings are terminal is exactly what prevents later failing siblings' TaskFailed events from being dropped against an already-terminal instance.

Exception aggregation

When one or more children fail, the WhenAll now completes with a native JS AggregateError:

• .errors carries every failed child's exception, in task-array order (deterministic across replay — independent of the order failures arrive). Each entry is the child's TaskFailedError (with its .details).
• Always wraps, even when only a single child failed. This matches .NET AggregateException and Java CompositeTaskFailedException (durabletask-java's TaskOrchestrationExecutor.allOf unconditionally throws CompositeTaskFailedException even for a single failure).
• The top-level message inlines every child message ("{n} of {m} tasks in the whenAll failed: msg1; msg2; …"). Rationale: the serializer pb-helper.util.ts::newFailureDetails only reads e.name/e.message/e.stack — it does not walk .errors. So for an uncaught whenAll failure, only e.message survives into the orchestration FailureDetails. Inlining ensures an uncaught failure still records every child's message; .errors still carries the structured exceptions for the caught case.
• Native AggregateError (ES2021) is used directly — the repo targets ES2022 + node>=22, so it's available and typed. No custom exception class.

Implementation note (subtle but important)

_exception is intentionally not set until the task is complete. Task.isFailed is (_exception != undefined), and runtime-orchestration-context.ts::resume() checks isFailed before isComplete. Setting _exception on the first failure would make the WhenAll report isFailed === true while isComplete === false, and resume() would throw the failure into the generator before the other siblings finish — re-introducing fail-fast at the executor level and breaking the two-batch regression test. Instead the aggregate is built at completion, so isFailed and isComplete flip together. This keeps the entire fix in one source file with no new class field (which would also be wiped by useDefineForClassFields after super()).

Production-code safety

The WhenAll _exception is consumed only by resume() → generator.throw() → user code. A grep of src/ found no instanceof TaskFailedError and no code that assumes a whenAll surfaces a single exception. Retry logic (tryHandleRetry → RetryableTask/RetryHandlerTask) operates on the individual child CompletableTask in handleFailedTask before the whenAll aggregates; WhenAllTask is a CompositeTask and never enters retry. Blast radius is user catch blocks + tests only.

Cross-SDK parity evidence

SDK	WhenAll behavior	Immune to the #301 rewind deadlock?
.NET Task.WhenAll / AggregateException	wait-all + aggregate (always wraps)	✅ yes
Java CompletableFuture.allOf / CompositeTaskFailedException	wait-all + aggregate (always wraps, even single failure)	✅ yes
durabletask-js (before this PR)	fail-fast, single exception	❌ no
durabletask-python	fail-fast	❌ no (needs same fix — see follow-ups)

.NET and Java both wait for all siblings before surfacing a failure and aggregate all failures, so every sibling's completion is committed to history and rewind has no orphaned TaskScheduled. This PR aligns durabletask-js with that behavior.

Relationship to PR #300

This is the root-cause replacement for the now-closed band-aid PR #300. The fix stands alone in when-all-task.ts — no archive/revive/isRewindRevival/_handledTaskScheduledIds machinery is introduced.

Behavior change (user-visible)

⚠️ Two deliberate changes, both matching .NET/Java:

1. A whenAll failure now surfaces only after all siblings finish (previously on the first failure).
2. The surfaced exception is now an AggregateError wrapping all child failures (previously a single exception). Even a single failure is wrapped.

Existing tests updated

Test	Reason
when-all-task.spec.ts — "should fail fast when any child fails" → "should wait for all children before surfacing a failure"	Wait-all: one failed child doesn't complete the WhenAll while a sibling is pending; now asserts the exception is an AggregateError.
when-all-task.spec.ts — "should fail immediately when a pre-completed child is failed" → "should not surface a pre-failed child while a sibling is still pending"	Pre-failed child + pending sibling keeps waiting; asserts AggregateError.
when-all-task.spec.ts — "should not double-complete when child completes after fail-fast" → "should notify the parent exactly once, only after all children complete"	Parent notified once, only after all children finish; asserts AggregateError.
when-all-task.spec.ts — "...surfaced exception is the FIRST (object-identity)" → "should aggregate a single failure (always wrapped)..."	Single failure now wraps: AggregateError, .errors.length === 1, .errors[0] is the failed child's exception (object identity).
when-all-task.spec.ts — "should surface the first failure (not an aggregate)..." → "should aggregate ALL failures in task-array order when multiple children fail"	Inverted: AggregateError, .errors.length === 2, .errors[0]/[1] are the two failures in task-array order (object identity), message contains both.
orchestration_executor.spec.ts — "fan-in completes as failed only after all tasks finish when one fails"	whenAll failure errortype → AggregateError (message still contains the single failure).
orchestration_executor.spec.ts — "should fail whenAll correctly when the failing task is the last to complete"	errortype → AggregateError.
orchestration_executor.spec.ts — "should complete whenAll as failed after all tasks finish when one task fails"	errortype → AggregateError.
orchestration_executor.spec.ts — "should keep waiting until all whenAll siblings fail across separate event batches (#301)"	Message assertion flipped: errortype → AggregateError; the message must now contain both "first activity failed" and "second activity failed" (proving neither TaskFailed was dropped).
task.spec.ts — "should notify parent via onChildCompleted when parent is set"	Single-child whenAll is now a real behavior change (wrapped): asserts AggregateError with one TaskFailedError in .errors.

New tests added

• when-all-task.spec.ts: single-failure-always-wraps (.errors.length === 1); multi-failure aggregates all in task-array order; determinism test — children fail out of array order (child[2] then child[0]) and .errors still follows task-array order (child[0] before child[2]), proving order is deterministic w.r.t. the task array, not arrival/wall-clock.
• orchestration_executor.spec.ts — "should keep waiting until all whenAll siblings fail across separate event batches (Rewind after a multi-failed-sibling WhenAll deadlocks the UNSIGNALED sibling subtree (emit-once replay can't reconcile a re-driven orphan with no rewind signal) #301)": whenAll([a, b]) where both fail across two batches. After only the first TaskFailed: no complete action (still waiting). After the second: completes as failed with an AggregateError whose message contains both failures. This is the crux that prevents the dropped-sibling → rewind orphan.

Non-goals / follow-ups

• durabletask-python has the same fail-fast divergence and needs the same fix (wait-all + aggregate), tracked separately. This PR is JS-only.

Validation

• npm run build:core ✅
• npm run lint ✅
• npm test ✅ — 63 suites, 1124 tests pass

[Copilot]

Pull request overview

This PR changes the core WhenAllTask semantics in the durabletask-js orchestration task model from fail-fast (complete on first child failure) to wait-all-then-fail (complete only after all children are terminal, surfacing the first failure if any). This aligns behavior with .NET/Java WhenAll and addresses the rewind deadlock described in issue #301 by preventing later sibling failure events from being dropped against a terminal orchestration.

Changes:

• Update WhenAllTask.onChildCompleted to wait for all child tasks to reach a terminal state before completing, and to surface the first failure by task order (if any).
• Update and expand unit tests to validate wait-all semantics, parent notification behavior, and the multi-batch failure regression scenario for #301.
• Adjust test comments/documentation to reflect the new semantics (including the single-child WhenAll case).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
packages/durabletask-js/src/task/when-all-task.ts	Implements wait-all-then-fail completion semantics and surfaces first failure only at completion time.
packages/durabletask-js/test/when-all-task.spec.ts	Updates existing tests and adds new cases to assert wait-all behavior and first-failure surfacing.
packages/durabletask-js/test/orchestration_executor.spec.ts	Updates executor-level tests to validate orchestration behavior across event batches under wait-all semantics (issue #301 regression).
packages/durabletask-js/test/task.spec.ts	Updates commentary to correctly describe single-child WhenAll behavior under wait-all semantics.