Skip to content

fix: clean up maestro test reporter output#935

Merged
thymikee merged 16 commits into
mainfrom
codex/maestro-reporter-cleanup
Jun 29, 2026
Merged

fix: clean up maestro test reporter output#935
thymikee merged 16 commits into
mainfrom
codex/maestro-reporter-cleanup

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

Clean up Maestro/replay test reporter output by dropping quoted titles and redundant filenames from passing rows, surfacing filenames only in failure details, and rendering shard labels with readable device names.

Adds optional shard device names to replay progress/results so default and JUnit output can keep stable metadata while the console uses friendlier labels.

Touched files: 10. Scope stayed within replay test CLI reporting and sharded test metadata.

Validation

Focused reporter/progress tests passed for default output, daemon progress rendering, JUnit metadata, and sharded suite metadata.

Static validation passed with lint and typecheck via pnpm check:quick.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.4 MB 1.4 MB +1.1 kB
JS gzip 453.3 kB 453.6 kB +330 B
npm tarball 558.5 kB 558.9 kB +338 B
npm unpacked 2.0 MB 2.0 MB +1.2 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.8 ms 26.8 ms +0.0 ms
CLI --help 47.8 ms 48.0 ms +0.2 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/generic.js +8.8 kB +3.4 kB
dist/src/9919.js +529 B +165 B
dist/src/session.js +89 B +21 B
dist/src/9722.js -17 B -9 B
dist/src/cli-help.js -4 B -9 B

@thymikee

Copy link
Copy Markdown
Member Author

Current head bfe2f4a is blocked by CI. Unit Tests and Coverage both fail in src/utils/tests/daemon-client.test.ts:561 and :666 because the assertions still expect the old quoted-title plus filename progress line, while the new output is now e.g. checkmark Login flow (1.23s) and checkmark Payments flow (2.50s). Fallow also fails on src/cli-test-progress.ts:109 addReplayTestCaseDetailLines with cyclomatic 11 / cognitive 13 / CRAP 37.1. Please update the daemon-client progress expectations and split/simplify that detail-line helper enough for Fallow before re-review.

@thymikee

Copy link
Copy Markdown
Member Author

Update on current head e23eaab: Fallow is now green, so the complexity blocker is resolved. Unit Tests and Coverage still fail on the same two stale daemon-client progress-output assertions at src/utils/tests/daemon-client.test.ts:561 and :666. The expected regex still includes quoted title plus file path, while the actual new output is the cleaned form without the filename, e.g. checkmark Login flow (1.23s) and checkmark Payments flow (2.50s). Please update those two test expectations before re-review.

@thymikee

Copy link
Copy Markdown
Member Author

Current head 035dfe0 is still blocked by CI, and the failure surface expanded from the earlier note. Unit Tests and Coverage now fail in five progress-output assertions:

  • src/__tests__/daemon-client-progress.test.ts:126, :242, :309
  • src/utils/__tests__/daemon-client.test.ts:561, :666

The tests expect parenthesized durations such as ✓ Login flow (1.23s) / ✓ Tab View - Coverflow (17.8s) and, in the daemon-client tests, the old quoted title plus file path. The current output is now ✓ Login flow 1.23s, ✓ Tab View - Coverflow 17.8s, and ✓ Payments flow 2.50s.

Please align the progress formatter/tests around one intended human-readable format before re-review. If the cleaned output is intended, update all five expectations consistently; if parenthesized durations are intended, restore that formatting in the progress renderer.

@thymikee thymikee force-pushed the codex/maestro-reporter-cleanup branch from 6fee563 to d7f21dc Compare June 29, 2026 18:44
@thymikee thymikee merged commit 6ae0612 into main Jun 29, 2026
21 checks passed
@thymikee thymikee deleted the codex/maestro-reporter-cleanup branch June 29, 2026 19:02
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-06-29 19:02 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant