Skip to content

[Test] Supervision is never stressed with failures inside preRestart, postRestart and the constructor at once #1384

Description

@pathosDev

Use case

tests/unit/Supervision.test.ts has 30 cases and tests/unit/RestartRegressions.test.ts has 4. They test failures raised from message handling — the actor throws while processing, and the strategy decides.

The failure sites that are not covered are the ones inside the restart machinery itself:

  • A throw in preRestart, i.e. the old instance failing while being torn down.
  • A throw in postRestart, i.e. the new instance failing while being brought up.
  • A throw in the constructor, i.e. the new instance never existing at all.

Each is a different branch in ActorCell's restart path, and each leaves the cell in a different intermediate state. The restart sequence is: cancel timers → dead-letter the stash → preRestart on the old instance → stop children (awaited) → construct the new instance → reset the behavior stack → postRestart → resume mailbox → resume children → publish ActorRestarted. A failure at any step leaves everything after it undone, with the mailbox suspended and children in a half-stopped state.

The constructor case is already a known bug: #914 reports that a throwing actor constructor leaves a permanently suspended zombie — never stopped, never removed from its parent, and its watchers never notified. That is one of the three sites, found by reading. The other two are unexamined.

The invariant that makes this class tractable is separate from the individual outcomes: an actor must never process a user message while it is suspended. Every one of these failure paths runs with the mailbox suspended, and every one of them has a window where a bug could resume it early. That invariant is checkable continuously, by the actor itself, rather than case by case.

Proposed shape

A randomised hierarchy stress test rather than a case matrix, because the interesting failures are combinations:

  • Build a tree of actors several levels deep.
  • Send a mix of ordinary messages and failure commands, where each failure command randomly selects the directive (restart / resume / stop / escalate), the depth at which it fires, and whether it additionally fails in preRestart, in postRestart, in the constructor, or stops some children on the way.
  • Have every actor assert its own invariants continuously: it is not processing a message while suspended, and its children are the ones it expects.
  • At the end, drain: every outstanding request is answered, the hierarchy has the expected shape, and no actor is left suspended, unstopped, or unregistered from its parent.

Seed the randomisation and print the seed, so a failure is reproducible — the same requirement as the other generative work in this catalogue.

Two things to settle at implementation time: the tree size and message volume that make this reliable rather than flaky, and whether it belongs on the per-commit path at all. Given its nature it probably belongs in the nightly slot with a small smoke variant per commit.

Acceptance

Verification status

Confirmed by reading. tests/unit/Supervision.test.ts (30 cases) and tests/unit/RestartRegressions.test.ts (4 cases) were surveyed for their failure sites; all raise from message handling. The restart sequence is transcribed from ActorCell's onRecreate / completeRecreate path (timer cancel at :1595, stash dead-letter at :1596, children at :1610-1620, stack reset at :1637, resume at :1639, escalation of an initialization failure at :1655). The constructor case is #914, already filed and open.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: highTop priority — high impact, plan next

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions