Skip to content

Add Dataset mode for comparing algorithms on real series - #4

Merged
Gerrrr merged 1 commit into
mainfrom
custom-datasets
May 31, 2026
Merged

Add Dataset mode for comparing algorithms on real series#4
Gerrrr merged 1 commit into
mainfrom
custom-datasets

Conversation

@Gerrrr

@Gerrrr Gerrrr commented May 31, 2026

Copy link
Copy Markdown
Contributor

Adds a third top-level mode to the web visualizer ("Dataset", alongside "Single Pattern" and "Mix Patterns") for loading real performance data and seeing where each Otava algorithm variant places change points on the same series. The TigerBeetle dataset bundled in apache/otava's perf/perf_test.py ships as the default preset; a "Custom (paste below)" option accepts JSON-array or whitespace/comma-separated numbers.

The Otava analysis panel now offers a checkbox per algorithm variant. In addition to split-edivisive and orig-edivisive, I have also added deterministic-edivisive to facilitate review of
apache/otava#154.

Backend additions:

  • otava_test_data.datasets package + TIGERBEETLE preset.
  • GET /api/datasets — bundled-preset metadata.
  • GET /api/datasets/{name} — one preset's series + metadata.
  • GET /api/algorithms — which algorithm functions are available.
  • POST /api/compare — run multiple algorithms on a series, return all results.
  • /api/generate and /api/analyze accept otava_algorithm=split|orig|deterministic to pick which variant runs in single/mix mode.

Algorithm names are validated as Literal["split","orig","deterministic"], so unknown values fail with 422 at the request boundary instead of returning 200 with an embedded error string.

Dataset mode reuses the existing chart container with a single chart that overlays each enabled algorithm's change points as colour-coded vertical lines. The new "Show All Graphs" / "Analyse" buttons and the ground-truth sections (accuracy table, comparison tables, chart legend) are hidden in Dataset mode. Compare requests are guarded by an AbortController so a slow earlier request can't race a newer one. The results table is built with DOM nodes (not string concatenation) so server-supplied strings can't inject HTML. Custom-paste input filters out non-numeric tokens and tells the user how many were dropped.

Also fixes an existing latent bug: Starlette 1.x changed TemplateResponse's positional argument order. The pre-existing / route was using the old signature, which crashed with "unhashable type: dict" once Jinja2's cache saw the context dict as the template name. Both routes updated.

Adds a third top-level mode to the web visualizer ("Dataset", alongside
"Single Pattern" and "Mix Patterns") for loading real performance data and
seeing where each Otava algorithm variant places change points on the same
series. The TigerBeetle dataset bundled in apache/otava's perf/perf_test.py
ships as the default preset; a "Custom (paste below)" option accepts
JSON-array or whitespace/comma-separated numbers.

The Otava analysis panel now offers a checkbox per algorithm variant.
In addition to split-edivisive and orig-edivisive, I have also added
deterministic-edivisive to facilitate review of
apache/otava#154.

Backend additions:
- otava_test_data.datasets package + TIGERBEETLE preset.
- GET /api/datasets — bundled-preset metadata.
- GET /api/datasets/{name} — one preset's series + metadata.
- GET /api/algorithms — which algorithm functions are available.
- POST /api/compare — run multiple algorithms on a series, return all results.
- /api/generate and /api/analyze accept otava_algorithm=split|orig|deterministic
  to pick which variant runs in single/mix mode.

Algorithm names are validated as Literal["split","orig","deterministic"], so
unknown values fail with 422 at the request boundary instead of returning
200 with an embedded error string.

Dataset mode reuses the existing chart container with a single chart that
overlays each enabled algorithm's change points as colour-coded vertical
lines. The new "Show All Graphs" / "Analyse" buttons and the ground-truth
sections (accuracy table, comparison tables, chart legend) are hidden in
Dataset mode. Compare requests are guarded by an AbortController so a slow
earlier request can't race a newer one. The results table is built with
DOM nodes (not string concatenation) so server-supplied strings can't inject
HTML. Custom-paste input filters out non-numeric tokens and tells the user
how many were dropped.

Also fixes an existing latent bug: Starlette 1.x changed TemplateResponse's
positional argument order. The pre-existing `/` route was using the old
signature, which crashed with "unhashable type: dict" once Jinja2's cache
saw the context dict as the template name. Both routes updated.
@Gerrrr
Gerrrr requested a review from henrikingo May 31, 2026 03:37

@henrikingo henrikingo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

The existing caude is entirely vibe coded, I assume yours is too, so I did glance it over, but I'm trying to adapt to a fast changing world and especially since this is a demo/sandbox app, then my review here is more superficial than it would be for the actual apache/otava main project.

Thanks for adding this . This is becoming a really useful tool and starting to look more useful than just a demo.

@Gerrrr
Gerrrr merged commit 65182fc into main May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants