Skip to content

dynamic shape arg#4233

Open
apbose wants to merge 3 commits into
mainfrom
abose/dynamic-shapes-passthrough
Open

dynamic shape arg#4233
apbose wants to merge 3 commits into
mainfrom
abose/dynamic-shapes-passthrough

Conversation

@apbose
Copy link
Copy Markdown
Collaborator

@apbose apbose commented May 5, 2026

  • Add name_dims to torch_tensorrt.Input — tag a dynamic axis with a name (Input(..., name_dims={0: "B"})); the same name across inputs is exported as one shared torch.export.Dim, letting users share a dim (e.g. a batch axis on input_ids and attention_mask) without a separate dynamic_shapes argument or any torch.export knowledge at the call site.
  • Unblock kwarg-only nn.Module calls — relax the arg_inputs is None assertion when kwarg_inputs is given, normalize arg_inputs=None to [] (instead of [None]), and stop inputs or arg_inputs from overwriting an empty inputs tuple with None.

@meta-cla meta-cla Bot added the cla signed label May 5, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels May 5, 2026
@github-actions github-actions Bot requested a review from zewenli98 May 5, 2026 04:54
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/models/test_shared_dynamic_dim.py	2026-05-05 04:54:37.448873+00:00
+++ /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/models/test_shared_dynamic_dim.py	2026-05-05 04:55:00.216122+00:00
@@ -14,10 +14,11 @@
caller supply a shared ``Dim`` directly to ``torch_tensorrt.compile`` --
mirroring the ``torch.export.export(dynamic_shapes=...)`` signature -- so the
shared-batch case compiles end to end without the caller having to pre-export
the module themselves.
"""
+
import unittest

import pytest
import torch
import torch.nn as nn

@apbose apbose force-pushed the abose/dynamic-shapes-passthrough branch from 64bcdc4 to adffa87 Compare June 4, 2026 17:31
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/models/test_shared_dynamic_dim.py	2026-06-04 17:31:18.741538+00:00
+++ /home/runner/work/TensorRT/TensorRT/tests/py/dynamo/models/test_shared_dynamic_dim.py	2026-06-04 17:31:43.478755+00:00
@@ -13,10 +13,11 @@
name; the same name across inputs is exported as a single shared ``Dim``. All
the dynamic-shape intent lives on the ``Input`` objects -- no separate
``dynamic_shapes`` argument and no ``torch.export`` knowledge required at the
call site.
"""
+
import unittest

import pytest
import torch
import torch.nn as nn

@apbose apbose force-pushed the abose/dynamic-shapes-passthrough branch from adffa87 to ae1ce2f Compare June 4, 2026 17:43
@apbose apbose requested review from cehongwang and narendasan June 4, 2026 18:04
Copy link
Copy Markdown
Collaborator

@cehongwang cehongwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok

@apbose
Copy link
Copy Markdown
Collaborator Author

apbose commented Jun 4, 2026

@narendasan are you suggesting something like this? B and S are the named tuples
B = shared_dims("B", min=1, opt=4, max=8) # batch
S = shared_dims("S", min=16, opt=128, max=512) # sequence

torch_tensorrt.compile(
model, ir="dynamo",
kwarg_inputs={
"input_ids": Input(shape=(B, S), dtype=torch.int64),
"attention_mask": Input(shape=(B, S), dtype=torch.int64),
},
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants