Skip to content

Add Debug Report condition performance timing foundation - #1065

Draft
vibhor1102 wants to merge 1 commit into
Nain57:masterfrom
vibhor1102:feature/debug-report-condition-performance
Draft

Add Debug Report condition performance timing foundation#1065
vibhor1102 wants to merge 1 commit into
Nain57:masterfrom
vibhor1102:feature/debug-report-condition-performance

Conversation

@vibhor1102

@vibhor1102 vibhor1102 commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Status: design checkpoint, not merge-ready

This draft PR intentionally stops after the low-level measurement and Debug Report storage foundation. It is opened now to give the maintainer an opportunity to review the detection-path boundary, overhead trade-offs, report schema, and likely UI direction before the broader Debug Report work is built on top.

The later UI and product work is essential before this can become mergeable. If this foundation is accepted, I intend to continue with the wider Debug Report overhaul described in discussion #958, not ship condition timing as an isolated screen. The original feature context is in discussion #1064.

What this foundation does

  • Times every condition that is actually reached by ConditionsVerifier.
  • Aggregates check count, fulfilled count, total duration, minimum duration, and maximum duration per condition ID.
  • Uses fixed-size primitive arrays allocated once at report start.
  • Performs no clock read or aggregate update when Debug Report generation is disabled.
  • Writes one aggregate protobuf message through the existing serialized Debug Report writer when the session ends.
  • Adds active ScenarioProcessor.process time and actual Execution Limiter suspension time to the existing report overview.
  • Adds a repository read path for the condition aggregates; no UI consumes it yet.
  • Preserves compatibility with reports that predate the new optional protobuf fields.

The measurement contract is documented in docs/debug-report-performance-timing.md.

Measurement boundary

The timer starts immediately before verifyCondition(condition) and stops immediately after it returns. This includes the preparation and detector work attributable to that reached condition, including image bitmap retrieval where applicable.

AND/OR short-circuiting is preserved: skipped conditions are not counted as checks. Configured but unreached conditions remain in the aggregate with zero values, which makes the absence of checks explicit.

Nanoseconds are used as the integer storage unit and monotonic-clock interface; this is not a claim of nanosecond measurement accuracy. Presentation would normally select microseconds or milliseconds.

Prototype and real-device validation

Before converting the prototype into this permanent architecture, I tested all four combinations independently:

  • A — baseline: existing Debug Report off, condition timing off.
  • B — existing report only: existing Debug Report on, condition timing off.
  • C — combined intended use: existing Debug Report on, condition timing on.
  • D — timing isolated: existing Debug Report off, condition timing on.
Mode Existing report Condition timing Runs Median elapsed Median CPU ticks Median CPU ticks/s
A Off Off 5 54.461 s 5,049 92.68
B On Off 5 53.848 s 6,719 122.93
C On On 5 53.470 s 6,703 125.63
D Off On 5 53.105 s 4,893 92.22

The experiment used 20 valid real-world runs on an Android 14 device. Each run executed the same six-loop game automation with a 10/s Execution Limiter. Modes used the balanced rotation C A B D / A D C B / C B A D / B D A C / D B A C to distribute warm-up and heating bias.

The two incremental comparisons were:

  • B → C: median elapsed -0.378 s, median raw CPU ticks -16, normalized CPU ticks approximately +2.2%.
  • A → D: median elapsed -1.356 s, median raw CPU ticks -156, normalized CPU ticks approximately -0.5%.

The negative values are not interpreted as improvements. Their mixed directions and size relative to the natural run variance indicate that the added measurement cost did not stand out in this workload. The existing Debug Report path did produce a clearly visible CPU-tick increase, so the experiment was capable of exposing an effect larger than the noise.

All runs completed successfully. Device temperature rose from 41 °C to 45 °C and then remained stable; Android thermal status stayed at 0. The ten timing-enabled runs recorded 604,013 condition checks and 348.080 seconds of accumulated condition-processing time without malformed or missing profiles.

The data also demonstrated why the feature is useful: four conditions in one event consumed 65.94% of measured condition time, while two individually cheap conditions were important because each was reached more than 80,000 times.

The complete methodology, ranges, interpretation, and limitations are documented in docs/debug-report-performance-benchmark.md.

Tests and build validation

Focused regression tests cover the failure modes most likely to produce plausible but incorrect reports:

  • AND/OR short-circuit reach counting;
  • no timing interaction on the disabled path;
  • aggregate arithmetic and session reset;
  • separation of active processing from limiter waiting;
  • exclusion of the unlimited-mode safety delay from limiter time;
  • partial limiter waits interrupted by cancellation;
  • protobuf round trips; and
  • old overview messages with absent timing fields.

The upstream-based debug APK compiled and verified successfully, and the full unit-test suite passed in hosted run 32485154791. That run used the same permanent code; this PR branch additionally squashes the history and adds the benchmark documentation while deliberately omitting fork-only workflows.

UI direction for later phases

No UI is proposed in code here. My current preference is a third Debug Report tab dedicated to condition performance, because the data needs cumulative-time ranking, check counts, average cost, unreached conditions, sorting, and explanatory context.

Discussion #958 originally preferred keeping only Overview and Timeline for the smaller Event activity feature. A compact Overview entry opening a condition-performance screen remains a credible alternative. The scope now appears large enough that a dedicated tab may be clearer, but this is deliberately open for maintainer input.

The eventual implementation is expected to address #958 as a cohesive Debug Report overhaul: timeline readability and navigation, terminology, event activity, adaptive time formatting, and condition performance should be designed together rather than appended piecemeal.

Feedback requested

In particular, feedback would be useful on:

  1. Is verifyCondition the right attribution boundary, or should any bitmap/preparation work sit outside it?
  2. Is one aggregate message at session end consistent with the intended Debug Report format?
  3. Should active processing and Execution Limiter waiting live in the overview as proposed?
  4. Is a dedicated condition-performance tab preferable to an Overview entry opening a separate screen?
  5. Are there other low-level measurements worth collecting now, before a UI contract is fixed?

FAQ

Why use elapsed time rather than CPU time?

The user can act on end-to-end condition latency: both detector work and condition-specific preparation delay the next evaluation. Per-thread CPU time would miss native or other-thread work and would be harder to interpret. The report does not claim that elapsed time maps directly to battery consumption.

Why time every check instead of sampling?

The timer and fixed-array update were not detectable above real-world variance, while full coverage gives exact reach counts and does not risk missing rare expensive conditions. Sampling can be reconsidered if the detection architecture changes materially.

Why aggregate instead of storing individual samples?

Individual samples would make memory and report size grow with session length and would introduce hot-path allocation or buffering pressure. The aggregate uses a fixed amount of memory per configured condition and performs no per-check I/O.

Why store nanoseconds if the detector is not nanosecond-accurate?

Nanoseconds avoid early rounding when many short checks are accumulated and match Android's monotonic clock API. They are a storage unit, not a display promise; the UI should use readable adaptive units.

Why keep minimum and maximum but not percentiles?

Min/max cost only two primitive fields and can expose extreme behavior. Reliable percentiles require retaining samples or updating a histogram on every check. That additional complexity and overhead is deferred until there is evidence that the UI needs it.

Why identify entries by condition database ID rather than duplicate names and types?

The report already belongs to a specific scenario. IDs allow readers to join timing to the scenario's condition metadata without duplicating mutable names, priorities, operators, and event relationships in every aggregate entry.

Why is there no separate profiling toggle?

This is a Debug Report sub-feature, not an independent profiler. Enabling the report enables its complete data collection. Keeping separate state would allow internally inconsistent reports and add a user-facing setting for an overhead that was not detectable in validation.

Why add a synchronous timing listener instead of reusing the existing asynchronous callbacks?

The duration must be captured at the exact condition boundary, and aggregate ordering must match the processing path. The listener only updates primitive arrays/integers; file writing and domain-object creation still happen once, after processing stops, through the existing report writer.

Why record active loop time and limiter waiting separately?

session duration - active duration is not limiter time: it also includes actions, unavailable-frame delays, and other waits. Measuring at the limiter's own suspension site makes that value honest and actionable. Unlimited-mode safety waits are explicitly excluded.

Why open a draft before the UI exists?

The measurement boundary and hot-path architecture are difficult to change after UI and analytics depend on them. This draft is a review checkpoint for those foundations, especially because the later work will be a broader Debug Report redesign rather than a small isolated addition.

Out of scope for this draft

This PR should remain draft and should not be merged in its current form.

@vibhor1102

Copy link
Copy Markdown
Contributor Author

@Nain57 Im waiting for a few days in case you're able to check in till that with any feedback.

Your feedback will be appreciated on:

  1. The foundations built within this PR.
  2. The future placement of per-condition data within a Debug Report's UI. I suggest it being a separate tab.
  3. The other suggestions in Improve Debug Report timeline readability, navigation, filtering, and event statistics #958, I'll be working upon those next.

I'll work further on the Debug Report in a few days of time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant