Skip to content

test(e2e): assert on rendered values, not bare numbers, in the floql tests - #63

Merged
oreofeolurin merged 1 commit into
devfrom
fix/61-ts-assertions
Aug 30, 2026
Merged

test(e2e): assert on rendered values, not bare numbers, in the floql tests#63
oreofeolurin merged 1 commit into
devfrom
fix/61-ts-assertions

Conversation

@oreofeolurin

Copy link
Copy Markdown
Contributor

Addresses #61 — with an honest caveat about what is and isn't proven.

The defect

ts floql renders each point as <epoch_ms>: <value>:

--- nglob_test (value) [1 points] ---
  1788100284942: 10.0000

These tests write with wall-clock timestamps, so a bare-number substring assertion matches against a 13-digit number as well as the value:

try testing.expect(result.contains("10"));
try testing.expect(!result.contains("30"));

Sampling epoch-ms across 48 hours:

frequency effect
timestamp contains "30" 4.2% negative assertion fails a correct run
timestamp contains "10" 9.6% positive assertion passes on a wrong value

The worst was contains("2") in the modulo test — one digit appears in a 13-digit timestamp roughly three quarters of the time, so that assertion was mostly measuring nothing.

The fix

Values always render with four decimals and timestamps never contain a ., so the rendered form cannot collide. Nine tests updated. floql glob tag filter =~ already did exactly this with "20.0000" — the idiom was already in the file, just not applied consistently.

Verification

I checked the new assertions actually bite rather than assuming: mutating an expected value (2.00009.0000) makes the test fail. Full e2e/ts suite and test-unit pass.

What is not proven

I have not shown this caused the #61 flake. I ran the old assertion 16 times locally and it passed 16/16. That is weak evidence in both directions — consecutive local runs share nearly all timestamp digits, and 4.2% predicts roughly zero failures in 16 anyway — but I would rather say so than present a tidy story.

What is certain is that the assertions were unsound in both directions and are now sound. I'd suggest leaving #61 open until a few CI cycles pass without recurrence, rather than closing it on this PR.

…tests

`ts floql` renders each point as `<epoch_ms>: <value>`, and these tests write
with wall-clock timestamps, so a bare-number substring assertion is matching
against a 13-digit number as well as the value.

    try testing.expect(result.contains("10"));
    try testing.expect(!result.contains("30"));

Both directions are unsound. Sampling epoch-ms across 48h: the timestamp
contains "30" about 4.2% of the time, which makes the negative assertion fail a
run that is behaving correctly; it contains "10" about 9.6% of the time, which
makes the positive assertion pass even when the value is wrong. The worst case
was `contains("2")` in the modulo test — a single digit appears in a 13-digit
timestamp roughly three quarters of the time, so that assertion was mostly
measuring nothing.

Values always render with four decimals and timestamps never contain a '.', so
matching the rendered form cannot collide. `floql glob tag filter =~` already
did this with "20.0000"; the rest now match.

Verified the new assertions actually bite: mutating an expected value makes the
test fail. The whole `e2e/ts` suite passes, as does test-unit.

Whether this is the cause of the intermittent failure in #61 is not proven —
16 local runs of the old assertion did not reproduce it, though consecutive
local runs share nearly all timestamp digits so that is weak evidence, and 4.2%
predicts roughly zero failures in 16. The assertion is unsound regardless.
@oreofeolurin
oreofeolurin merged commit b61c22b into dev Aug 30, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant