Add Newton inverse kinematics action#5756
Merged
Merged
Conversation
![isaaclab-review-bot[bot]](https://avatars.githubusercontent.com/u/157846462?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
maxkra15
force-pushed
the
max/newton-ik-manager
branch
2 times, most recently
from
0ccb62c to
bbc8e51
Compare
This comment was marked as outdated.
This comment was marked as outdated.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
maxkra15
force-pushed
the
max/newton-ik-manager
branch
from
bbc8e51 to
01d05fa
Compare
ooctipus
reviewed
Jun 4, 2026
ooctipus
reviewed
Jun 4, 2026
| """Set the Newton model builder.""" | ||
| NewtonManager._builder = builder | ||
|
|
||
| @classmethod |
Collaborator
There was a problem hiding this comment.
I see what you are trying to do here. I need to take some time to think about what the best design, the current way makes sense, but let me think through this tomorrow
ooctipus
reviewed
Jun 4, 2026
Collaborator
There was a problem hiding this comment.
Nice work @maxkra15 , how urgent is this work? I think what you have there for saving the source builder makes sense, but its a bit duplicated to what we aready have in clone plan, I am wondering if there are further redundancies could be removed. I need to think abit more tomorrow.
If this work is urgent, I am ok with just merge it first and I find a time to clean it up later
maxkra15
force-pushed
the
max/newton-ik-manager
branch
from
1626028 to
743ef3a
Compare
Collaborator
|
@isaaclab-review-bot please review again |
Collaborator
|
@greptile-apps please review again |
ooctipus
force-pushed
the
max/newton-ik-manager
branch
2 times, most recently
from
f061260 to
c686b58
Compare
Move the per-step IK compute off Torch: the pose objective exposes its command convention as Warp data (command code, relative flag, scale, offset), and the action term computes targets, assembles the seed, solves and gathers through Warp kernels. Torch now appears only as the policy action at the boundary, viewed zero-copy into Warp.
ooctipus
force-pushed
the
max/newton-ik-manager
branch
from
c686b58 to
a85a217
Compare
ooctipus
approved these changes
Jun 12, 2026
maxkra15
added a commit
to maxkra15/IsaacLab
that referenced
this pull request
Jun 12, 2026
Squashed replay of max/waterhose-coupled-experimental (64 commits, old history preserved on backup/waterhose-coupled-before-develop-rebase- 20260612) onto upstream/develop (d5f0455). Conflict-resolution policy: - Newton pin stays on the verified PR-2848 coupled-solver commit; warp floor stays >=1.14.0.dev (upstream's ==1.14.0 would downgrade below what that Newton build needs). - Upstream's new Newton IK (solver/objectives, isaac-sim#5756) is taken verbatim; the branch's interim IK manager is kept temporarily until the waterhose task is ported to the new action API. - newton_manager: upstream restructure + branch features (coupled reset protocol, FK articulation filter, Kit xform-op sync, pre-render callbacks). Builder creation runs both attribute-registration hooks (upstream's subclass hook and the branch's cfg-driven walk that covers coupled sub-solver entries). - Cloner (isaac-sim#5770 restructure): ported the branch's cfg-driven VBD graph coloring for hook-built entities (cables), the static-visual-shape import for Newton-side visualizers, and hide_collision_shapes parity into the new replicate/build_source_builders flow. The waterhose cfg sets simplify_meshes=False - convex-hull approximation would fill the socket bore. - Deformable contrib, visualizers, mjwarp manager and the manager abstraction test resolved as the union of both sides' features.
maxkra15
added a commit
to maxkra15/IsaacLab
that referenced
this pull request
Jun 12, 2026
Port the waterhose task from the interim NewtonIKManager to upstream's Newton IK action (isaac-sim#5756), used as intended: one objectives list. - Right end-effector: command-driven absolute pose objective (root-frame position + (x, y, z, w) quaternion). - Multi-body holds: left_gripper_base (weight 1) and torso_hip_yaw (weight 50) pose objectives replace the old fixed_body_names hack; the scripted state machine writes their captured targets each step. - Soft joint-limit objective replaces joint_limit_weight. - Teleop variant: thin subclass rotates the relative rotation-vector from the end-effector frame into the root frame and delegates; no hold objectives (the operator owns the posture). - The interim NewtonIKManager and the contrib reach duplicate (which registered the same gym ids upstream's core reach now owns) are removed. Validated: phase convergence timings match the pre-rebase demo step-for-step with the holds active.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add Newton-backed inverse kinematics support to Isaac Lab.
This PR introduces a
NewtonIKManagerwrapper around Newton's objective-list IK solver API. The manager builds Newton pose objectives, optional joint-limit regularization, sampler settings, and custom objective passthrough into a batched Isaac Lab interface. Targets are updated from Torch tensors and converted to Warp arrays before calling Newton'sIKSolver.The PR also adds a manager-based
NewtonInverseKinematicsActionthat plugs this solver into Isaac Lab MDP action terms. Because Newton IK needs a single-articulation model while Isaac Lab runs replicated scenes, the Newton cloner now registers prototype builders during replication.NewtonManager.get_prototype_model()lazily finalizes the matching prototype on the active device, and the action resolves Isaac Lab joint/body names into Newton prototype joint-coordinate and link indices before solving.Finally, this adds a Franka reach task variant using the new Newton IK action:
Isaac-Reach-Franka-Newton-IK-Rel-v0Isaac-Reach-Franka-Newton-IK-Rel-Play-v0The implementation includes tests for prototype-model lookup and cached prototype finalization behavior.
Fixes # (no issue filed)
Type of change
Screenshots
Not applicable.
Test plan
source/isaaclab_newton/test/physics/test_newton_prototype_models.py.