Skip to content

folded_health fixtures fail when the CI host itself is io-stalled — the real degradation probe leaks into tests that assume an undegraded node #567

Description

@emooreatx

On PR #566 (workflow/installer files only), clippy + test (ubuntu-latest) run 34229542137 failed four tests/folded_health.rs cases:

a_brain_warning_alone_degrades_the_folded_pair            fixture: this case needs an undegraded node
a_critical_warning_past_the_cap_still_degrades_the_pair    fixture: this case needs an undegraded node
a_malformed_or_flooding_brain_cannot_break_the_nodes_liveness_answer   left: Bool(true) right: false
one_enormous_warning_cannot_bloat_the_liveness_answer      … io stalled: EVERY task in this container was blocked on io 21.7% of the last 10s (some 22.1%) …

The last message is the tell: the node's real degradation probe (PSI io-stall) fired because the GitHub runner was io-stalled, so the "undegraded node" the fixtures start from was already degraded by a true warning about the host. The suite passes 3/3 locally on the same commit (7b06270 main).

This is the same shape as #542 (a test that reads a process-wide surface the code under test does not own): the fixture assumes the degradation registry holds only what the test raised. Options, in order of preference:

  1. the fixture clears/snapshots the degradation registry (or disables the host probes) for the duration of a test that asserts on the folded pair;
  2. the host-pressure probes carry a source: host marker and the folded-health assertions filter to source: test;
  3. skip-with-reason when a host warning is present at test start (weakest — hides the class).

Not fixing in #566 (out of scope); rerunning the job. Filed so the next sighting has somewhere to go.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions