tools: multi-device SpO2 @82 candidate validation harness (#103)

[dnesdan]

What this PR does

Adds a multi-device SpO₂ @82 validation harness so owners can test whether spo2_candidate_82 tracks the official export — the bar required before any promote to spo2Pct (#103).

New: Tools/linux-capture/validate_spo2_candidate.py

• Nightly mean of in-band @82 (70–100) while sleep_state=asleep
• Paired vs CSV blood_oxygen_pct (English + DE/ES headers)
• Pearson r, MAE, bias
• Offset-specificity scan over bytes 74–92 (only @82 should win)
• Checklist gates: ≥5 nights, spread ≥1%, r≥0.7, MAE≤1.0, best offset=82
• --batch devices.json for multi-device
• --postable summary without raw SpO₂ values (safe for WHOOP 5.0 raw HCI capture — where to submit? (docs reference #174, which doesn't exist) #103)
• Does not change app metrics or promote spo2Pct

Also

• English export aliases in correlate_ground_truth.py (Blood oxygen % → blood_oxygen_pct) — previously English exports missed SpO₂ ground truth
• whoop_activity.decode_v18 exposes spo2_candidate_82 (tri-mode, mirrors Swift/Kotlin)
• Docs in WHOOP5_DEEP_DATA.md + linux-capture README
• Unit tests (planted multi-device perfect corr, incomplete nights, awake gating, specificity, privacy)

Type of change

• New feature (research tooling)
• Documentation

How it was tested

cd Tools/linux-capture
python3 -m unittest test_validate_spo2_candidate test_whoop_activity test_correlate_ground_truth -v
# 21 tests, OK (~0.2s)

No BLE hardware path changed. Synthetic planted captures only in CI; real multi-device evidence still comes from owner captures on #103.

Checklist

• Swift package tests N/A (Python tooling + docs)
• Android unit tests N/A
• python3 -m unittest for touched Tools/linux-capture tests pass
• No hardcoded hex frame bytes in app code; protocol facts stay in existing decoders
• Follows CONTRIBUTING (no proprietary material; facts only; does not invent SpO₂)
• No secrets / generated project files

Related issues

• Helps WHOOP 5.0 raw HCI capture — where to submit? (docs reference #174, which doesn't exist) #103 (SpO₂ / deep-data validation)
• Related WHOOP 5/MG deep-data roadmap: capability labels, validated decode and guarded probes #761 (capability honesty — still no silent SpO₂ card)
• Follow-up to docs PR docs(whoop5): SpO2 sleep path, SLEEPFLAG, skin-temp wire scale #807 (protocol facts)

[ryanbr]

Reviewed. Good PR — research tooling only, no app code, and the two claims that matter both hold.

Verified:

• Parity with the app decoders. Your in-band gate 70 <= frame[82] <= 100 matches Interpreter.swift:502 (if (70...100).contains(v)) exactly, inclusive on both ends. The tri-mode test covering 0x80/0xA0/0x20/69/101 is the right shape.
• --postable really is value-free. format_postable emits device,paired_nights,r,mae,bias,best_offset,checklist_pass — aggregates and derived deltas, no readings. Safe to paste on WHOOP 5.0 raw HCI capture — where to submit? (docs reference #174, which doesn't exist) #103.
• The shared-decoder change is additive. spo2_candidate_82 sits alongside aux_byte_82, which is preserved; the only consumer of whoop_activity in either tools tree is its own test, which you updated. No blast radius.
• Nothing promotes spo2Pct, which is the line that matters for WHOOP 5.0 raw HCI capture — where to submit? (docs reference #174, which doesn't exist) #103.

Why your checks are missing, and it isn't your doing: you're a first-time contributor from a fork, so GitHub is holding the workflow runs at action_required pending approval. Both #807 and #808 are sitting there — that's what the UNSTABLE state is. @ryanbr needs to click "Approve and run workflows"; nothing for you to change. Note swift-packages.yml would skip anyway (paths: Packages/**), but i18n-coverage.yml has no path filter and should run.

One repo hazard, pre-existing but worth naming. There are two linux-capture directories differing only in case: Tools/linux-capture/ (8 files) and tools/linux-capture/ (17). This PR adds validate_spo2_candidate.py to the capitalised one and documents it in the lowercase one's README, which deepens the split. On a case-insensitive filesystem — macOS default, and this is a macOS-primary project — those two collide into a single directory on checkout, which produces phantom modifications and confusing diffs. Not caused here and not yours to fix, but consolidating them is worth its own issue before the tree grows further.

Otherwise this is exactly the right shape for #103: an evidence harness with explicit gates rather than an early promote.

[ryanbr]

Ran your tests here rather than take the numbers on trust. The code is sound, but the test command in the description doesn't cover the part of this PR that changes behaviour.

Your new decoder test never runs under python3 -m unittest.

test_whoop_activity.py is pytest-style — 21 bare def test_* functions, zero unittest.TestCase — so unittest collects none of them, including test_decode_v18_spo2_candidate_82_tri_mode at line 49, which is the test guarding your whoop_activity.py change.

The reported total is a coincidence that hides it:

test_validate_spo2_candidate    8 collected
test_correlate_ground_truth    13 collected
test_whoop_activity             0 collected   ← NO TESTS RAN
                               ── 
                               21   ← the "21 tests, OK" in the description

8 + 13 = 21, so the number looks right while your decoder test silently doesn't run. Worth either converting that file to TestCase, or documenting pytest as the runner for it — otherwise nothing guards the decode.

I executed the bare functions by hand: 17 pass, and the 4 that don't are pytest-only (tmp_path fixtures, one import pytest), not real failures. Both test_decode_v18_spo2_candidate_82_tri_mode and test_decode_v18_roundtrips_fields pass. So the decoder change is correct — it just isn't covered by the documented command.

The case-split is not cosmetic — it breaks the tests on Linux. correlate_ground_truth.py imports whoop_frame, which lives in the lowercase tools/linux-capture/. On a case-insensitive filesystem the two trees fold together and the import resolves, which is why it works for you. On Linux it does not:

ModuleNotFoundError: No module named 'whoop_frame'

That fails on main too, so it predates this PR — but it means these tests could never pass in CI on an Ubuntu runner, and it upgrades the directory split from a tidiness issue to a real one. Everything passes once the lowercase tree is on PYTHONPATH.

Otherwise verified: the in-band gate matches Interpreter.swift:502 ((70...100)) exactly; format_postable emits only device,paired_nights,r,mae,bias,best_offset,checklist_pass with no readings; the spo2_candidate_82 addition is additive with aux_byte_82 preserved; nothing promotes spo2Pct.

Also worth knowing: your checks show as missing because you're a first-time fork contributor and the runs are held at action_required pending maintainer approval — nothing for you to fix.

[ryanbr]

Anchoring the findings from running the suite locally. The harness itself is well-built and correctly scoped — these are about coverage and about a pre-existing directory split that turns out to be load-bearing.

[ryanbr]

This test does not run under the command in the PR description.

This file is pytest-style — 21 bare def test_* and zero unittest.TestCase — so python3 -m unittest test_whoop_activity collects nothing from it (NO TESTS RAN). That includes this test, which is the only thing guarding the whoop_activity.py change below.

The totals hide it, which is why it is easy to miss:

test_validate_spo2_candidate    8 collected
test_correlate_ground_truth    13 collected
test_whoop_activity             0 collected
                               ──
                               21   ← the "21 tests, OK" in the description

I executed the bare functions by hand: 17 pass, and the 4 that don't are pytest-only (tmp_path fixtures, one import pytest) rather than real failures. This one passes. So the behaviour is right — it just isn't covered by the documented runner.

Either convert the file to TestCase, or document pytest as its runner. As it stands nothing guards the decode.

[ryanbr]

Verified against the app decoder: this matches Interpreter.swift:502 (if (70...100).contains(v)) exactly, inclusive at both ends, and aux_byte_82 is preserved alongside it so nothing downstream loses the raw byte. The only consumer of this module in either tools tree is its own test, so there is no blast radius.

Correct as written — see the coverage note on the test above.

[ryanbr]

Worth flagging the split this sits on, because it is not cosmetic. The new tool lives in Tools/linux-capture/ (capital T) and is documented here in tools/linux-capture/ (lowercase) — two directories differing only in case, 8 files and 17 respectively.

On a case-insensitive filesystem the two fold together, which is why everything works for you. On Linux they don't, and the suite fails before it starts:

correlate_ground_truth.py: import whoop_frame
ModuleNotFoundError: No module named 'whoop_frame'

whoop_frame.py exists only in the lowercase tree. This reproduces on main, so it predates the PR — but it means these tests cannot pass on an Ubuntu CI runner, which matters now that workflow approval is the only thing standing between this branch and CI. Everything passes once the lowercase tree is on PYTHONPATH.

Not yours to fix here. Consolidating the two trees deserves its own issue.

[ryanbr]

Landed as a446d0b (via #814), with you as co-author. Thanks — the harness design is yours and went in unchanged: the nightly in-band mean while asleep, the pairing against blood_oxygen_pct, the 74–92 offset-specificity scan, the checklist gates, --batch, and the --postable summary that omits raw values. That last one matters more than it looks; it's what makes multi-device evidence shareable on #103 at all.

I opened a separate PR only because this one was stuck at action_required and couldn't get CI on a fork. Two things needed fixing first, and neither was visible from macOS — which is almost certainly why you didn't hit them.

Tools/linux-capture/ and tools/linux-capture/ are two different directories on Linux. Zero overlapping filenames, 11 and 17 files. On a case-insensitive filesystem they're one directory and everything imports fine. On Linux they're separate, and validate_spo2_candidate.py imports whoop_activity from the uppercase one and whoop_frame from the lowercase one — so it couldn't run in either. correlate_ground_truth.py was already broken this way before your PR. Consolidated into the lowercase side; the suite went from 104 passing (plus a hard-failing directory) to 159.

The 21 tests, OK didn't include your new test. test_whoop_activity.py is pytest-style — bare def test_*, no unittest.TestCase — so python3 -m unittest collects nothing from it and exits 0, which looks exactly like passing. The 21 was 8 + 13 from the other two files. Your @82 tri-mode decoder test was in the file contributing zero and had never executed. Added unittest's load_tests hook, which then surfaced four genuine failures — all pytest-fixture dependencies (tmp_path, pytest.skip), now rewritten to stdlib so both runners work.

Worth saying plainly: that one wasn't really yours either. The file was already pytest-style before you added to it, and the README documents unittest. You followed the existing pattern; the pattern was broken.

Also folded in a correction to my own docs — I'd written up @82 in #812 as an unmapped byte when it's been decoded as spo2_candidate_82 all along. Your PR is what made me look properly.

Closing this in favour of the merged version. #103 stays open — your harness is the instrument for answering it, and the real blocker there is the cross-device contradiction (8 nights at corr +0.99 against two nights on the original device moving the wrong way), which is precisely what --batch is for. If you have captures from a second strap, that's the next useful thing.