fix(bridge): retake the transport when its owner dies later - #5
Merged
Conversation
Reclaiming a stale lock at startup was already right. The mistake was treating that one decision as final: a daemon that lost the race stayed degraded for its whole life, even once the winner exited. health went on reporting owns:false and naming a dead PID, and the conflict message told the operator to close a session that was already closed. Observed today by a second session: it removed the stale lock file and nothing changed, because the losing daemon had read it once at boot and kept the verdict in memory. It ended up killing its own daemon and losing the transport until a human reconnected it. status() now re-attempts the acquisition whenever it does not hold the lock. It does NOT test the recorded PID first: the acquisition is atomic and already refuses a live owner, and retrying blind also covers what a liveness test misses -- an owner that released the file cleanly while its process is still around. Two negative controls, both watched red. Never reclaiming leaves the late-death test failing. Reclaiming unconditionally turns three red, including the one asserting a live owner keeps the directory -- which is the two-daemon failure this lock exists to prevent, and the reason the blind retry has to go through acquireTransportLock rather than around it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reclaiming a stale lock at startup was already right. The mistake was treating that one decision as final.
A daemon that lost the race stayed degraded for its whole life, even once the winner exited.
healthwent on reportingowns: falseand naming a dead PID, and the conflict message told the operator to close a session that was already closed.How it surfaced
A second session hit it today: it removed the stale lock file and nothing changed, because the losing daemon had read that file once at boot and kept the verdict in memory. It ended up killing its own daemon and losing the transport until a human reconnected it — a self-inflicted outage caused entirely by the tool describing a competitor that no longer existed.
Same family as the three map-reading bugs fixed earlier today: not wrong about the bytes, wrong about what they designate.
The fix
status()re-attempts the acquisition whenever it does not hold the lock.It deliberately does not test the recorded PID first. The acquisition is atomic (
wx) and already refuses a live owner, so retrying blind is safe — and it covers what a liveness test misses: an owner that released the file cleanly while its process is still around. My first attempt did gate onisAlive, and the new test caught it.Two negative controls, both watched red
acquireTransportLockrather than around it.132 tests.