Skip to content

perf(poll): adaptive back-off when the backend responds slowly - #119

Merged
tdody merged 2 commits into
mainfrom
perf/adaptive-poll-backoff
Jun 26, 2026
Merged

perf(poll): adaptive back-off when the backend responds slowly#119
tdody merged 2 commits into
mainfrom
perf/adaptive-poll-backoff

Conversation

@tdody

@tdody tdody commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Summary

A safety net for responsiveness under load: when /api/state is slow, the dashboard now automatically widens its poll cadence instead of stacking requests onto a struggling backend.

The dashboard polls /api/state every 500 ms while a terminal modal is open. If the backend is slow to answer (e.g. the host is swapping, or pre-#118 when collect_state forked a tmux subprocess per window), those polls pile up and contend with the keystroke round-trip that shares the backend — which is what makes typing lag. Backing off lets the backend recover.

How

  • usePolling now measures each poll's wall-clock latency (EWMA) and counts superseded polls — ones aborted by the next tick before they completed. The supersede count is the signal the EWMA alone can't see: when latency exceeds the poll interval, fetches never finish to be measured, so "we keep aborting in-flight polls" is what tells us we're outpacing the backend.
  • A pure nextDegraded(current, ewmaMs, supersedes) applies hysteresis — enter degraded when smoothed latency > 800 ms or ≥ 2 consecutive supersedes; exit only when latency is clearly low and polls are completing again (no flapping).
  • usePolling exposes a degraded flag; pickPollInterval(..., degraded) floors the cadence at DEGRADED_POLL_MS (5 s) when degraded — overriding the modal/active/input tiers, but never lowering an already-slower interval.
  • Latency state lives in refs so it survives the effect re-run when the interval changes (backing off mustn't reset the signal that triggered it).

With #118 (batched collect_state, ~30 ms /api/state) this never trips in the common case — it only engages when the backend is genuinely degraded.

Tests (TDD)

  • Pure policy (pollTier): updateLatencyEwma seeding/smoothing; nextDegraded hysteresis + supersede escalation + null-sample handling; pickPollInterval degraded floor (overrides modal/input, never lowers a slower interval, no-op when not degraded). 22 tests.
  • usePolling: flags degraded on high latency; stays clear on fast responses; flags degraded after repeated supersedes. Watched each fail (no degraded), then pass.
  • Full frontend suite: 611 passing; tsc + vite build clean.

Notes

🤖 Generated with Claude Code

Polling /api/state at the 500ms modal-open cadence against a backend
that takes seconds to answer (e.g. host under memory pressure) just
stacks requests it can't keep up with — starving the keystroke
round-trip that shares the backend. Add an adaptive floor: when
responses are slow, widen the poll cadence so the backend can recover.

usePolling now measures each poll's latency (EWMA) and counts superseded
polls (aborted before completing because the next tick fired — the
signal the EWMA can't see when latency exceeds the interval), exposing a
`degraded` flag via hysteresis (enter > 800ms or 2 supersedes; exit only
when clearly recovered). App threads it into pickPollInterval, which
floors the cadence at DEGRADED_POLL_MS (5s) when degraded — overriding
the modal/active/input tiers but never lowering an already-slower one.

With batched collect_state (~30ms /api/state) this never trips in the
common case; it's a safety net for degraded conditions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tdody
tdody merged commit e20809e into main Jun 26, 2026
3 checks passed
@tdody
tdody deleted the perf/adaptive-poll-backoff branch June 26, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant