Skip to content

Don't call an unread feature-flag key list "the strap named none" (#761) - #913

Merged
ryanbr merged 2 commits into
ryanbr:mainfrom
vishk23:fix/feature-flag-probe-verdict-honesty
Jul 28, 2026
Merged

Don't call an unread feature-flag key list "the strap named none" (#761)#913
ryanbr merged 2 commits into
ryanbr:mainfrom
vishk23:fix/feature-flag-probe-verdict-honesty

Conversation

@vishk23

@vishk23 vishk23 commented Jul 28, 2026

Copy link
Copy Markdown

The defect

FeatureFlagProbeReport.verdict (Packages/WhoopProtocol/.../FeatureFlagProbe.swift) returned

strap announced N flag(s) but named none

whenever keys was empty and the strap had answered 117 — with no check that a single SEND_NEXT_FF(118) reply was ever decoded.

The path is reachable, and it is plausibly the first-contact result on a 5/MG:

  1. BLEManager.probeFeatureFlags() sends START_FF_KEY_EXCHANGE(117)
  2. the reply lands → noteStart(r) records reportedCount
  3. sendFeatureFlagStep(.sendNextFeatureFlag) puts 118 on the wire
  4. the 8s timer fires → noteTimeout(command: 118, seconds: 8) appends a trace line and sets stopReason
  5. finishFeatureFlagProbe() renders

Zero 118 replies were received — yet the headline asserts a fact about the strap's key list. It blames the strap for our timeout, and it's the sentence someone would paste into #103 as evidence that the enumerate verb answers but serves nothing. The run established no such thing.

It also contradicts the principle the file states two branches above and applies correctly to skipped > 0:

"named none" would blame the strap for OUR decode. […] never report our limitation as the strap's behaviour.

Same defect class as the ECG probe verdict fixed in #896: a verdict asserting a conclusion the run's own inputs could not support.

The fix

steps (the decoded-118-reply counter) already existed, so the gate is one condition:

if keys.isEmpty && steps == 0 {
    return "strap announced \(announcedFlags); no SEND_NEXT_FF(118) reply was decoded — "
        + "the key list was never read (inconclusive)"
}

Worded "no reply was decoded" rather than "no reply arrived" because steps == 0 also covers a 118 reply that failed to parse (noteFailure), where the strap did answer. stopReason renders directly beneath the verdict and names which of the two it was, so one branch covers both without over-claiming either.

A 118 reply that genuinely walked to the end marker carrying no name is the opposite case — a real fact about the strap — and still says "named none" plainly, unchanged.

Secondary (judgement call, flagging it explicitly): the verdict also restated the announced count as bare fact even when countIsPlausible was false. noteStart marks an implausible count in the trace (:277), but the verdict is the line that gets quoted, so it now carries the same doubt (an implausible 9999 flag(s)). A plausible count renders exactly as before — real reports are byte-for-byte unchanged. Happy to drop this half if you'd rather keep the change to the single gate.

Test change — the old test pinned the bug

testAnnouncedCountWithNoNamesIsSaidPlainly did noteStart + noteTimeout(118) and asserted the "named none" string: it pinned the buggy path as intended behaviour. Split into the two cases it was conflating:

test case verdict
…WithNo118ReplyIsInconclusiveNotBlamedOnTheStrap 117 answered, 118 timed out inconclusive; asserts !contains("named none")
…WithARealEmptyWalkIsSaidPlainly 118 replied with the 0xFF end marker "named none" — unchanged
…ImplausibleAnnouncedCountIsNotRestatedAsFactInTheVerdict count outside 1…128 count carries the doubt

Kotlin twins added for all three; rendered strings stay byte-identical across platforms per the parity contract.

Verification

Both new tests were confirmed to fail against the unfixed source before the fix was restored — Swift 4 assertions, Kotlin 2 tests — so they demonstrably catch the defect rather than merely describing it. The unchanged-behaviour test passed on both sides, as intended.

suite result
cd Packages/WhoopProtocol && swift test 427 tests, 0 failures
cd android && ./gradlew testFullDebugUnitTest (JDK 17) 3198 tests, 0 failures, 5 skipped

Android counts are read from app/build/test-results/testFullDebugUnitTest/*.xml (391 files) on a forced --rerun, not from the Gradle exit code — a plain BUILD SUCCESSFUL can hide a cached or skipped test task.

Pure Packages/ + Kotlin protocol change: no app-target Swift touched, no BLE behaviour changed, no new storage. The probe stays read-only — this only changes the sentence it prints.

Refs #761.

`FeatureFlagProbeReport.verdict` returned

    strap announced N flag(s) but named none

whenever `keys` was empty and the strap had answered 117 — with no check
that a single SEND_NEXT_FF(118) reply had ever been decoded. That case is
reachable and is the likely first-contact result on a 5/MG:
`probeFeatureFlags()` sends 117, `noteStart` records the count,
`sendFeatureFlagStep(.sendNextFeatureFlag)` goes out, the 8s timer fires,
`noteTimeout(command: 118)` writes a trace line, and the report renders.
Zero replies were read, yet the headline asserted a fact about the strap's
key list — blaming the strap for our own timeout.

That is the conclusion someone pastes into #103 as evidence the enumerate
verb answers but serves nothing, when the run established no such thing.
It also contradicts the discipline the branch two conditions above states
outright and applies correctly to `skipped > 0`: never report our
limitation as the strap's behaviour. Same defect class as the ECG probe
verdict fixed in #896 — a verdict asserting a conclusion the run's own
inputs could not support.

`steps` already counts decoded SEND_NEXT replies, so the gate is
`steps == 0`, and the walk having produced no reply is reported as
inconclusive. `stopReason` renders directly beneath and names which of
the two causes it was (timeout, or a 118 reply that failed to decode),
so the one branch covers both without over-claiming either.

Also stops the verdict restating an implausible announced count as bare
fact. `noteStart` already marks a count outside 1…128 in the trace, but
the verdict is the line that gets quoted; it now carries the same doubt.
A plausible count renders exactly as before, so real reports are
unchanged.

The Swift test that pinned the old string as intended behaviour asserted
it after `noteStart` + `noteTimeout(118)` — the buggy path itself. Split
into the two cases it was conflating: no 118 reply (inconclusive) versus
a 118 reply that walked to the end marker carrying no name, which is a
genuine fact about the strap and still says "named none" plainly.

Kotlin twin updated in step; the rendered strings stay byte-identical
across platforms.

Verified: Packages/WhoopProtocol `swift test` 427 tests, 0 failures.
`./gradlew testFullDebugUnitTest` (JDK 17) 3198 tests, 0 failures,
5 skipped, counted from app/build/test-results/*.xml on a forced
`--rerun`. Both new tests confirmed to fail against the unfixed source
(Swift 4 assertions, Kotlin 2 tests) before the fix was restored.

@ryanbr ryanbr left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The defect is real and I checked it on main rather than from the description. The keys.isEmpty branch is at :387 with no steps guard, and the comment stating the principle it breaks —

Same class as isSkippable: never report our limitation as the strap's behaviour.

— is at :377. Eight lines apart. The file states the rule and then breaks it in the next branch, which is the most convincing form this kind of report can take. steps already exists at :255, so the gate really is one condition.

Wording the guard as "no reply was decoded" rather than "no reply arrived" is the right call, and the reason given is the correct one: steps == 0 also covers a 118 reply that failed to parse, where the strap did answer. One branch, neither over-claim, with stopReason underneath to say which.

The thing that actually needs deciding: this collides with #917

Both rewrite FeatureFlagProbe on both platforms, and they conflict — I merge-tested the two heads against each other, not just against main.

Worse than a textual conflict: #917 touches this exact line and does not fix it.

-            return "strap announced \(reportedCount ?? 0) flag(s) but named none"
+            return "strap announced \(reportedCount ?? 0) \(namespace.entryNoun)(s) but named none"

It parameterises the noun and carries the defect forward, reworded. So the guard has to survive whichever order these land in, and the rebase is where it would quietly not:

  • #913 first (my preference — small, focused, one condition) — then #917 rebases onto it and must keep the steps == 0 gate. #917 rewrites both test files too (+405/-30 Swift, +446/-27 Kotlin), so the test that pins the gate can disappear in the same rebase as the gate. Worth checking explicitly after the rebase that the guard's test still exists and still fails without the guard.
  • #917 first — then this reapplies on top of entryNoun, which is a smaller edit but leaves the defect on main in the meantime, and #917 is the larger review.

Either way it is worth saying in #917 that this line is spoken for.

The secondary change

Keep it. An implausible count that the trace flags but the verdict restates as bare fact is the same defect in miniature — the verdict is the line that gets quoted, and :277 already knows the count is doubtful. Real reports are unchanged because a plausible count renders as before, which is the property that makes it safe.

Test change

Splitting testAnnouncedCountWithNoNamesIsSaidPlainly is right, and worth being explicit about what it was: the old test asserted the buggy string, so it pinned the defect as intended behaviour. A test can do that, and when it does it is the reason a defect survives review — the suite goes green and the wrong sentence looks deliberate.

The failing check is not yours

check is red on AppChangelog.kt:42, a hardcoded What's New title on a line this PR does not touch. Cutting 9.2.1 added it — appchangelog-gen.py still writes that title as a raw literal (#878, reopened). Fixed on main in #916; the audit is a whole-tree gate, so it fails every PR branched before that. #914 hit the same thing and cleared on a branch update.

@ryanbr
ryanbr merged commit be499bb into ryanbr:main Jul 28, 2026
11 checks passed
ryanbr added a commit that referenced this pull request Jul 28, 2026
…rdict (#690) (#918)

The bare-stub report said two things at once:

    Verdict: opcode 84 answered with a bare stub — ambiguous
    ...
    No payload beyond the command byte (bare stub) — no body-location data on this firmware

The verdict is right and the detail line settles what the verdict just called
unsettled. A reply carrying no payload is one reply: it does not distinguish a
firmware without body-location data from one that needs the strap worn, or in a
determinate position, or asked with an argument this probe does not send. This
file's own enum has UNKNOWN(0) and NOT_CONCLUSIVE(128), so the firmware plainly
models "cannot tell right now" — which is exactly the reading the detail line
ruled out.

Same class as #913 and #914: a probe asserting a conclusion the run's inputs
could not support, in the sentence a reader quotes. Found while sweeping the
probes after #913 rather than reported.

Both platforms carried the identical string — this is a shared defect, not a
divergence — so both are reworded to say what was observed and what it does not
establish, and both tests now assert the over-claim is ABSENT rather than only
that "bare stub" appears. The old assertions passed under either wording, which
is why the contradiction survived.

The golden parity lock is unaffected: it pins a payload-bearing frame, so the
bare-stub branch was never covered by it.
ryanbr added a commit that referenced this pull request Jul 28, 2026
WhoopBleClient logs every non-SUCCESS command response, which is how the MG
haptics rejection (#48) would surface in-app. But every WHOOP 4.0
GET_BATTERY_LEVEL reply on record carries a zeroed [seq][result] prefix, so a
battery read that returned a perfectly good percentage logs as:

    Command response: GET_BATTERY_LEVEL(26) → FAILURE(0)

next to a battery gauge showing 42%. A log line asserting a failure that did not
happen is exactly the artefact #900 exists to warn about — that issue began when
a FAILURE(0) was read as evidence about firmware, and this line manufactures the
same misreading on every battery read a 4.0 owner makes.

The line still prints. Suppressing it would hide the anomaly, and the anomaly is
the thing #900 wants a capture of. It is annotated instead, when the same frame
also decoded a value:

    Command response: GET_BATTERY_LEVEL(26) → FAILURE(0) (the reply still
    carried a value: battery 42.5% — see #900, the 4.0 result byte is not
    established)

A genuine failure that carried no value — the extended-battery case from #791,
the UNSUPPORTED rejections — is untouched, because the annotation is gated on a
decoded value being present.

This does not decide what the 4.0 result byte means; #900 still wants one real
capture of known provenance. It stops the log asserting an answer in the
meantime, the same discipline as #913, #914 and #918.

Android-only: the Swift side has no command-response result consumer, so there
is no parity twin to change. No test — the string is inside a class that needs
the Android framework, and the annotation is a log line rather than behaviour;
the rendered output was checked against both real 4.0 battery fixtures instead.
ryanbr added a commit that referenced this pull request Jul 28, 2026
WhoopBleClient logs every non-SUCCESS command response, which is how the MG
haptics rejection (#48) would surface in-app. But every WHOOP 4.0
GET_BATTERY_LEVEL reply on record carries a zeroed [seq][result] prefix, so a
battery read that returned a perfectly good percentage logs as:

    Command response: GET_BATTERY_LEVEL(26) → FAILURE(0)

next to a battery gauge showing 42%. A log line asserting a failure that did not
happen is exactly the artefact #900 exists to warn about — that issue began when
a FAILURE(0) was read as evidence about firmware, and this line manufactures the
same misreading on every battery read a 4.0 owner makes.

The line still prints. Suppressing it would hide the anomaly, and the anomaly is
the thing #900 wants a capture of. It is annotated instead, when the same frame
also decoded a value:

    Command response: GET_BATTERY_LEVEL(26) → FAILURE(0) (the reply still
    carried a value: battery 42.5% — see #900, the 4.0 result byte is not
    established)

A genuine failure that carried no value — the extended-battery case from #791,
the UNSUPPORTED rejections — is untouched, because the annotation is gated on a
decoded value being present.

This does not decide what the 4.0 result byte means; #900 still wants one real
capture of known provenance. It stops the log asserting an answer in the
meantime, the same discipline as #913, #914 and #918.

Android-only: the Swift side has no command-response result consumer, so there
is no parity twin to change. No test — the string is inside a class that needs
the Android framework, and the annotation is a log line rather than behaviour;
the rendered output was checked against both real 4.0 battery fixtures instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants