Add Dataset mode for comparing algorithms on real series - #4
Merged
Conversation
Adds a third top-level mode to the web visualizer ("Dataset", alongside
"Single Pattern" and "Mix Patterns") for loading real performance data and
seeing where each Otava algorithm variant places change points on the same
series. The TigerBeetle dataset bundled in apache/otava's perf/perf_test.py
ships as the default preset; a "Custom (paste below)" option accepts
JSON-array or whitespace/comma-separated numbers.
The Otava analysis panel now offers a checkbox per algorithm variant.
In addition to split-edivisive and orig-edivisive, I have also added
deterministic-edivisive to facilitate review of
apache/otava#154.
Backend additions:
- otava_test_data.datasets package + TIGERBEETLE preset.
- GET /api/datasets — bundled-preset metadata.
- GET /api/datasets/{name} — one preset's series + metadata.
- GET /api/algorithms — which algorithm functions are available.
- POST /api/compare — run multiple algorithms on a series, return all results.
- /api/generate and /api/analyze accept otava_algorithm=split|orig|deterministic
to pick which variant runs in single/mix mode.
Algorithm names are validated as Literal["split","orig","deterministic"], so
unknown values fail with 422 at the request boundary instead of returning
200 with an embedded error string.
Dataset mode reuses the existing chart container with a single chart that
overlays each enabled algorithm's change points as colour-coded vertical
lines. The new "Show All Graphs" / "Analyse" buttons and the ground-truth
sections (accuracy table, comparison tables, chart legend) are hidden in
Dataset mode. Compare requests are guarded by an AbortController so a slow
earlier request can't race a newer one. The results table is built with
DOM nodes (not string concatenation) so server-supplied strings can't inject
HTML. Custom-paste input filters out non-numeric tokens and tells the user
how many were dropped.
Also fixes an existing latent bug: Starlette 1.x changed TemplateResponse's
positional argument order. The pre-existing `/` route was using the old
signature, which crashed with "unhashable type: dict" once Jinja2's cache
saw the context dict as the template name. Both routes updated.
henrikingo
approved these changes
May 31, 2026
Contributor
There was a problem hiding this comment.
+1
The existing caude is entirely vibe coded, I assume yours is too, so I did glance it over, but I'm trying to adapt to a fast changing world and especially since this is a demo/sandbox app, then my review here is more superficial than it would be for the actual apache/otava main project.
Thanks for adding this . This is becoming a really useful tool and starting to look more useful than just a demo.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a third top-level mode to the web visualizer ("Dataset", alongside "Single Pattern" and "Mix Patterns") for loading real performance data and seeing where each Otava algorithm variant places change points on the same series. The TigerBeetle dataset bundled in apache/otava's perf/perf_test.py ships as the default preset; a "Custom (paste below)" option accepts JSON-array or whitespace/comma-separated numbers.
The Otava analysis panel now offers a checkbox per algorithm variant. In addition to split-edivisive and orig-edivisive, I have also added deterministic-edivisive to facilitate review of
apache/otava#154.
Backend additions:
Algorithm names are validated as Literal["split","orig","deterministic"], so unknown values fail with 422 at the request boundary instead of returning 200 with an embedded error string.
Dataset mode reuses the existing chart container with a single chart that overlays each enabled algorithm's change points as colour-coded vertical lines. The new "Show All Graphs" / "Analyse" buttons and the ground-truth sections (accuracy table, comparison tables, chart legend) are hidden in Dataset mode. Compare requests are guarded by an AbortController so a slow earlier request can't race a newer one. The results table is built with DOM nodes (not string concatenation) so server-supplied strings can't inject HTML. Custom-paste input filters out non-numeric tokens and tells the user how many were dropped.
Also fixes an existing latent bug: Starlette 1.x changed TemplateResponse's positional argument order. The pre-existing
/route was using the old signature, which crashed with "unhashable type: dict" once Jinja2's cache saw the context dict as the template name. Both routes updated.