chore: unify setup nemogym - #3360
Merged
Merged
Conversation
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
yuki-97
added
the
CI:Lfast
Contributor
Author
|
/ok to test 7830ff9 |
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Contributor
Author
|
/ok to test d4a06dd |
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Contributor
Author
|
/ok to test ca51938 |
RayenTian
reviewed
Jul 28, 2026
RayenTian
previously approved these changes
Jul 28, 2026
Contributor
|
LGTM! Approved! |
…p_nemo_gym_actor Signed-off-by: Yuki Huang <yukih@nvidia.com>
Contributor
Author
|
/ok to test 9bacae7 |
yuki-97
enabled auto-merge (squash)
July 28, 2026 08:17
RayenTian
approved these changes
Jul 28, 2026
6 tasks
terrykong
added a commit
that referenced
this pull request
Jul 29, 2026
vLLM 0.25.1 moves max(train/critic/grad_norm) from 110.54 to 700.22 and max(train/critic/explained_var) from -0.72 to -5.28, while every policy-side metric in the same run is unchanged (token_mult_prob_error 1.026 -> 1.020, probability ratios identical). That points at the critic's regression target rather than at generation. Measured, not assumed: 0.20 baseline from PR #3360 job 90219748938, which passes at 110.54; 0.25 from this PR's job 90199439799. Raise the bound to 1500 so the bump is not blocked, with the numbers and a pointer to #3412 recorded above the invocation. This is explicitly a placeholder -- the mechanism is being debugged in a follow-up, and the bound should end up set from measured behaviour or the cause fixed. Note this is the second time this ceiling has been raised. The first was reverted in 068cd6c because it claimed the metrics were healthy with no run to back it; the difference now is that the numbers exist and are written down. The comment sits above 'uv run tests/check_metrics.py' rather than inline: a comment between backslash-continued arguments terminates the command, so placing it inline would silently drop the grad_norm check and then try to execute the assertion string as a command. Signed-off-by: Terry Kong <terryk@nvidia.com>
terrykong
added a commit
that referenced
this pull request
Jul 30, 2026
vLLM 0.25.1 moves max(train/critic/grad_norm) from 110.54 to 700.22 and max(train/critic/explained_var) from -0.72 to -5.28, while every policy-side metric in the same run is unchanged (token_mult_prob_error 1.026 -> 1.020, probability ratios identical). That points at the critic's regression target rather than at generation. Measured, not assumed: 0.20 baseline from PR #3360 job 90219748938, which passes at 110.54; 0.25 from this PR's job 90199439799. Raise the bound to 1500 so the bump is not blocked, with the numbers and a pointer to #3412 recorded above the invocation. This is explicitly a placeholder -- the mechanism is being debugged in a follow-up, and the bound should end up set from measured behaviour or the cause fixed. Note this is the second time this ceiling has been raised. The first was reverted in 068cd6c because it claimed the metrics were healthy with no run to back it; the difference now is that the numbers exist and are written down. The comment sits above 'uv run tests/check_metrics.py' rather than inline: a comment between backslash-continued arguments terminates the command, so placing it inline would silently drop the grad_norm check and then try to execute the assertion string as a command. Signed-off-by: Terry Kong <terryk@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
grpo.pyalso usespinup_nemo_gym_actorinnemo_rl.environments.nemo_gym, which is missing in feat(sc): NeMo-Gym path #3267.routed_experts_dtypeanduse_fastokensto include new changes in main branch.