perf(app): coalesce streamed assistant chunks per flush

[BetterAndBetterII]

Summary

• add a root benchmark task registry, runner, shared result contract/parser, and percentile helper
• register the agent-stream reducer benchmark as the first task
• keep the first assistant chunk on the existing reducer path, then coalesce compatible continuations within the same scheduled flush
• preserve provider, message-id, epoch, sequence-gap, and non-assistant event boundaries

Benchmark runner

npm run benchmark                                      # run every registered task
npm run benchmark -- --list                            # list tasks
npm run benchmark -- agent-stream-reducer              # run one task
npm run benchmark -- --output /tmp/paseo-benchmark.json

The versioned JSON envelope records Git commit/dirty state, Node runtime identity, task and case IDs, dimensions, named metrics, and raw samples. New benchmarks implement BenchmarkTaskResult and register a command in scripts/benchmarks/tasks.ts.

Agent stream benchmark

Workload: 512-byte assistant chunks, 8 chunks per reducer flush, 2 warmups, 7 measured runs. The task validates final text, item count, and timeline cursor on every run.

Reply size	Baseline p50	Candidate p50	p50 reduction	Baseline p95	Candidate p95	p95 reduction
64 KiB	2.83 ms	1.09 ms	61.6%	5.12 ms	2.11 ms	58.9%
256 KiB	34.58 ms	9.80 ms	71.7%	37.20 ms	11.38 ms	69.4%
1 MiB	632.48 ms	164.31 ms	74.0%	651.52 ms	176.55 ms	72.9%

Validation

• npx vitest run scripts/benchmarks/types.test.ts packages/app/src/timeline/session-stream-reducers.test.ts --bail=1 (70 tests)
• root runner all-task, filtered-task, list, stdout JSON, and output-file paths
• npm run typecheck
• npm run lint
• npm run format

The existing browser terminal stress fixture is not used for this result: its coalesced mock mode combines assistant updates server-side into one WebSocket message, so it does not reproduce multiple client reducer events in one flush.

[greptile-apps]

Greptile Summary

This PR introduces chunk coalescing in the processAgentStreamEvents loop so that contiguous assistant-message events within the same scheduler flush are collapsed into one applyStreamEvent call instead of N individual calls, delivering 60–74% latency reduction depending on reply size. It also adds a root benchmark registry (scripts/benchmarks/) with types, a stats helper, and a task runner that orchestrates individual task scripts via spawnSync.

• Core optimization (session-stream-reducers.ts): the first assistant chunk in a flush is processed on the existing path (to establish identity/whitespace semantics); subsequent contiguous chunks with matching epoch, provider, and messageId are coalesced in processCoalescedAssistantEvents, which concatenates their text and calls applyStreamEvent once.
• Benchmark infrastructure (scripts/benchmarks/): a typed task contract (BenchmarkTaskResult), a percentile-stats helper, and a root runner that spawns each registered task, reads its JSON output from a temp file, and wraps results in a versioned BenchmarkRunResult envelope.
• New tests validate coalescing boundaries: cross-provider, cross-epoch, and gap-detection cases all fall back to per-event processing correctly.

Confidence Score: 4/5

Safe to merge. The coalescing path is correctly gated and all boundary conditions fall back gracefully to per-event processing.

The stream-reducer change is well-reasoned and the new tests cross the module's public interface directly. The two findings are both in the benchmark scripts and neither affects the production reducer.

packages/app/scripts/benchmark-agent-stream-reducer.ts (validation inside timing loop) and scripts/benchmarks/run.ts (unconditional stdout write with --output).

Important Files Changed

Filename	Overview
packages/app/src/timeline/session-stream-reducers.ts	Core coalescing logic added: four new private helpers plus a rewritten processAgentStreamEvents loop. Logic is correct — first chunk takes the existing path, continuations are batched; gap/epoch/provider boundaries all fall back to per-event processing.
packages/app/src/timeline/session-stream-reducers.test.ts	Four new tests added covering: first-chunk individual processing, cross-provider boundary, sequence-gap fallback, and cross-epoch boundary. Tests assert through the module's public interface.
packages/app/scripts/benchmark-agent-stream-reducer.ts	Standalone benchmark implementation. Validation logic (including "x".repeat(messageBytes)) runs inside the timed runWorkload function, slightly contaminating the timing measurement for large workloads.
scripts/benchmarks/run.ts	Root benchmark orchestrator: arg parsing, task selection, spawnSync per task, git metadata, and JSON envelope. stdout is written unconditionally even when --output is specified.
scripts/benchmarks/types.ts	Typed contract and hand-rolled parser for BenchmarkTaskResult and BenchmarkRunResult. Validation is thorough: finite-number checks on metric values and samples.
scripts/benchmarks/stats.ts	Percentile helper: sorts a copy of samples, applies ceil-based index formula. Correct for the 7-sample MEASURED_RUNS workload.
scripts/benchmarks/tasks.ts	Registry of benchmark task definitions; one task registered. Platform-aware npx command for Windows compatibility.
scripts/benchmarks/types.test.ts	Tests parseBenchmarkTaskResult: happy path and malformed metric values. Also tests summarizeSamples for non-mutation and correct p50/p95.

Comments Outside Diff (1)

1. scripts/benchmarks/run.ts, line 663-668 (link)

   stdout write is unconditional when --output is specified

   When --output /tmp/out.json is passed, the runner writes the full JSON to the file and then immediately writes the same JSON to stdout. Users who just want the file (e.g. in CI, storing a baseline) will have several KiB of JSON printed to the terminal as well. If this is intentional for piping (npm run benchmark -- --output ... | jq), a comment explaining that would help; if not, gating process.stdout.write on !command.outputPath would make the behaviour match the documented flag semantics.

_{Reviews (1): Last reviewed commit: "build: add benchmark task registry" | Re-trigger Greptile}

[greptile-apps]

Correctness validation is inside the timed region

runWorkload both runs the workload and validates the result, so the measured samples include the cost of [...tail, ...head].filter(...), the "x".repeat(messageBytes) allocation, and the string-equality check. For the 1 MiB case "x".repeat(1048576) creates a 1 MiB string on every measured run. The overhead is modest (~0.1–0.5 ms against ~164 ms p50), but it makes the absolute numbers slightly misleading and means any future increase in validation cost silently inflates the benchmark. Moving the validation call outside the timed loop (run it once after warmup, before the timing loop) keeps the measurement focused purely on processAgentStreamEvents.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!