Add NeMo Gym V1 integration#2032
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
de9f35a to
9cea5a6
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e42f163ddc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9d9bc2c. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9d9bc2c714
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
9d9bc2c to
af98d46
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: af98d46f17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| NeMoGymConfigT = TypeVar("NeMoGymConfigT", bound=NeMoGymConfig, default=NeMoGymConfig) | ||
|
|
||
|
|
||
| class NeMoGymTaskset(Taskset[NeMoGymTask, NeMoGymConfigT]): |
There was a problem hiding this comment.
When users run the new generic nemo-gym taskset, or subclass it exactly as shown in the docs, the loader discovers config types by scanning concrete Taskset[...] generic arguments. Because this base class uses the NeMoGymConfigT TypeVar here, taskset_config_type('nemo-gym') falls back to plain TasksetConfig, so documented TOML containing dataset_path and task.resources_url is rejected as extra fields before load() can run. Make the exported base resolve to NeMoGymConfig (or otherwise override config resolution) so the documented custom resources-server workflow is usable.
Useful? React with 👍 / 👎.
af98d46 to
f18134d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f18134dd1e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| global_config_dict=OmegaConf.create({}), | ||
| ), | ||
| ) | ||
| app = server.setup_webserver() |
There was a problem hiding this comment.
When the managed resource server relies on NeMo Gym's expose_tools_over_mcp path, this custom launcher builds the FastAPI app directly and skips the run_webserver() step that calls maybe_auto_expose after setup_webserver(). In that context /seed_session will not be wrapped to return MCP metadata, so rows that depend on auto-harvested MCP tools expose no tools to the harness unless every schema is duplicated in the JSONL.
Useful? React with 👍 / 👎.
| { | ||
| "type": "function_call_output", | ||
| "call_id": message.tool_call_id, | ||
| "output": content, |
There was a problem hiding this comment.
When an MCP tool returns multimodal content, the trace stores ToolMessage.content as a list, and this forwards that list as function_call_output.output. NeMo Gym's Responses model expects the function-call output to be a string, so /verify rejects these rollouts before scoring; stringify or otherwise normalize non-string tool content instead of sending the list through.
Useful? React with 👍 / 👎.
1 participant
Overview
Adds a NeMo Gym resources-server integration for Verifiers V1. Existing Verifiers harnesses drive the rollout while NeMo Gym remains responsible for resource-server state, native tools, and scoring.
Details
/verifyrewards and numeric metrics.nemo-gymoptional dependency and reuses resource-server classes from the published package, with V1 owning startup and cleanup.environments/nemo_gym_weather_v1as a thin, self-starting example package; external resource-server URLs remain supported for custom deployments.Fixes RES-1090
Note
Medium Risk
New HTTP/MCP bridge, trace-to-response scoring, and subprocess server lifecycle affect eval correctness and startup; scope is optional and localized to the new taskset path.
Overview
Adds NeMo Gym V1 tasksets so verifiers harnesses run rollouts while Gym’s resources server handles per-rollout state, tools, and
/verifyscoring—not Gym’s native agent stack.NeMoGymTasksetloads JSONL rows (responses_create_params), callsPOST /seed_sessionwith the full row, and scores by converting a single-branch trace into a NeMo Responses object andPOST /verify._NeMoGymToolsetexposes Gym tools to MCP harnesses via HTTP MCP metadata from seed or direct function endpoints.NeMoGymEnvcan spawn a managed resources server subprocess (verifiers.v1.tasksets.nemo_gym.server) whenresource_serveris set; external URLs stay supported viaresources_url.Also adds optional
verifiers[nemo-gym](nemo-gym==0.4.0, Python ≥3.12) with uv dependency-metadata to avoid pulling Gym’s full Ray/agent stack, a reusablemcp_sessionhelper in MCP launch, docs, andenvironments/nemo_gym_weather_v1as a thin weather example.Reviewed by Cursor Bugbot for commit 9656d11. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add NeMo Gym V1 integration with
NeMoGymTasksetand managed resource server lifecycleNeMoGymTasksetin verifiers/v1/tasksets/nemo_gym/taskset.py that loads tasks from a JSONL dataset, seeds per-rollout sessions via/seed_session, and scores them via/verifyon a NeMo Gym resources server.NeMoGymEnvthat automatically spawns and tears down the resources server subprocess per worker when no externalresources_urlis configured._NeMoGymToolsetsupporting both MCP-over-HTTP and direct function endpoint tool calls, with cookie persistence across requests within a rollout.verifiers/v1/tasksets/nemo_gym/server.py) runnable aspython -m verifiers.v1.tasksets.nemo_gym.serverfor managed or manual startup.nemo-gym-weather-v1environment package in environments/nemo_gym_weather_v1/ and accompanying documentation.Macroscope summarized 9656d11.