Skip to content

feat: opencode logging for perfetto trace - #3

Open
youngeunkwon0405 wants to merge 10 commits into
sdevare-nv:sdd/devfrom
youngeunkwon0405:youngeunk/opencode-logging
Open

feat: opencode logging for perfetto trace#3
youngeunkwon0405 wants to merge 10 commits into
sdevare-nv:sdd/devfrom
youngeunkwon0405:youngeunk/opencode-logging

Conversation

@youngeunkwon0405

@youngeunkwon0405 youngeunkwon0405 commented Aug 7, 2026

Copy link
Copy Markdown

What does this PR do?

This PR adds precise observability for OpenCode SWE rollouts executed through NeMo Gym.

It records:

  • LLM request timing, token usage, request type, turns, and session relationships (for subagents)
  • Tool execution timing with input and output messages
  • OpenCode runtime initialization time

These records are aggregated into NeMo Gym’s per-turn metrics format, allowing swe_trace_converter.py to reconstruct the complete rollout timeline in Perfetto.

Implementation summary

  • bench/cli.ts: Collects completed tool-call events while OpenCode runs, then combines them with LLM completion metrics in the benchmark output. It also separates OpenCode initialization time from the actual agent runtime and records initialize_runtime_time in NeMo Gym metrics.

  • bench/metrics.ts: Adds a conversion layer for LLM completion dumps and tool events. It validates timestamps and token fields, preserves session/subagent relationships, orders overlapping requests by start time, and safely updates nemo_gym_metrics.json.

  • cli/cmd/run.ts: ~2 line changes here. This was required for capturing subagent tool call events.

  • provider/sdk/nemo-gym/language-model.ts: Measures each LLM request from submission through response completion and records latency, request kind, turn number, session hierarchy, and token usage. It handles both generate (doGenerate) and stream (doStream) paths while leaving model request and response behavior unchanged.

Issue for this PR

N/A

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds precise OpenCode metrics used to build the final Perfetto rollout trace:

  • LLM request timing, request kind, session relationships, turn IDs, and token usage
  • exact tool start/end timing with tool input/output details
  • subagent task and child-session relationships
  • aggregation into the NeMo Gym per-turn metrics format

Metrics that are not consumed by the final trace converter have been removed.

How did you verify your code works?

  • tsgo --noEmit passed.
  • Focused Bun metric tests passed: 3 passed, 0 failed.
  • A 4-node / 32-GPU, 3-step NeMo Gym run completed successfully and produced 9,493 trace events from 112 detailed rollouts.
  • Removing the unused fields from that run's 128 raw metric files produced a byte-identical Perfetto trace (SHA-256 4ae44814d24701cb46845576ec40abc95d937d9841bdb800e59a85092bdd9888).
  • Validated tool details and subagent span containment with no invalid timestamps or lane mismatches.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com>
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Thanks for updating your PR! It now meets our contributing guidelines. 👍

@youngeunkwon0405 youngeunkwon0405 changed the title feat: opencode logging for perfetto trace (WIP) feat: opencode logging for perfetto trace Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant