user provided bound for torchtrt compile when export dimension is unb… by apbose · Pull Request #4213 · pytorch/TensorRT

apbose · 2026-04-27T19:27:22Z

This PR validates user bounds against any finite exporter bounds, with three branches:

Upper overflow (user_max > exp_max) → ValueError. TRT engine profile follows the exporter; shapes above exp_max would be rejected at runtime anyway.
Lower underflow (user_min < exp_min) → ValueError, except the (user_min=1, exp_min=2) case which is just PyTorch's 0/1-specialization artifact (Dim(min=1) silently becomes 2 in ShapeEnv), this particular case emits a warning.
Subset (user range strictly inside exporter range) → engine profile is narrowed to the user's bounds (logged at INFO). The user's Input(min_shape, max_shape) is the contract for the TRT profile; a subset of the exporter's valid range is still correct, and a tighter profile lets TRT specialize more aggressively.
Dim.DYNAMIC (unbounded exporter upper) → no check, user fills the gap (the intended use).

apbose · 2026-04-30T00:12:20Z

CI index put cases failing. Looking at it.

narendasan · 2026-05-06T21:18:00Z

This PR is only for externally exported programs?

apbose · 2026-05-06T21:26:25Z

@narendasan yeah it addresses (torch_tensorrt.dynamo.compile(exported_program, ...)). The flow for torch_tensorrt.compile(model, inputs=..) should be unaffected, since in that case export bounds and input bounds should be same

narendasan

You should use the user bounds in the narrow case

…ounded

…ympy.oo in extract_var_range_info, validate Input against finite export bounds

…specialization artifact, unify bound-mismatch message templates

…lders; sync assertions with refactored bound-mismatch messages

…orter bound

narendasan · 2026-06-02T23:07:36Z

+            if exp_max_unbounded:
+                # Dim.DYNAMIC: user fills the gap (intended use).
+                continue
+            try:
+                exp_min = int(exp_lower)
+                exp_max = int(exp_upper)
+            except (TypeError, ValueError):
+                continue
+            if user_min == exp_min and user_max == exp_max:
+                continue


Can you explain what the logic here is?

This block runs only when the exported program already bounds the symbol to a finite range (e.g. exported with Dim("batch", min=10, max=20)). The compiled engine's TRT optimization profile follows that exported range, so we compare the user's Input range against it here, at compile time, and handle four cases:

exported upper is unbounded (Dim.DYNAMIC) → user's Input fills the gap, nothing to validate (the continue just above);

Input range == exported range → nothing to do;

Input extends beyond the exported range (user_max > exp_max or user_min < exp_min) → those shapes can never run, so we raise now;

Input strictly inside the exported range → safe to honor, we narrow the engine profile to the Input.
The user_min == exp_min and user_max == exp_max check on this line is just the "exact match → no-op" early-out.

narendasan · 2026-06-02T23:08:14Z

+
+            if user_max > exp_max:
+                raise ValueError(
+                    f"{mismatch} Input.max_shape exceeds the exporter's max "


the exported program right?

yeah the exported program. will change the wording

narendasan · 2026-06-02T23:08:29Z

+            if user_max > exp_max:
+                raise ValueError(
+                    f"{mismatch} Input.max_shape exceeds the exporter's max "
+                    f"({user_max} > {exp_max}); TRT will reject shapes above "


TRT will reject or pytorch will?

TRT will. [10,20] + Input min_shape=2: shows
IExecutionContext::setInputShape: Error Code 3: API Usage Error
(Set dimension [2,8] for tensor x does not satisfy any optimization profiles.
Valid range for profile 0: [10,8]..[20,8])
through core/runtime/execute_engine.cpp:149.

narendasan · 2026-06-02T23:09:01Z

+                # 1->2 is the 0/1 specialization artifact, not a user error.
+                if user_min == 1 and exp_min == 2:
+                    logger.warning(
+                        "%s Input.min_shape=1 vs exporter min=2 is the "


I think we need to make sure these error messages are clear to follow from a user perspective

meta-cla Bot added the cla signed label Apr 27, 2026

github-actions Bot added component: tests Issues re: Tests component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 27, 2026

github-actions Bot requested a review from lanluo-nvidia April 27, 2026 19:27

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 4d1ff02 to eec2c04 Compare April 27, 2026 23:35

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 0fe114e to 2905ff6 Compare April 30, 2026 01:05

apbose requested a review from narendasan May 6, 2026 21:09

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from dee44b1 to 22728b3 Compare May 6, 2026 22:15

narendasan reviewed May 7, 2026

View reviewed changes

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 02d94e5 to 01798b0 Compare May 21, 2026 18:08

apbose added 7 commits May 28, 2026 13:56

user provided bound for torchtrt compile when export dimension is unb…

d1b9ebf

…ounded

user passing input, adding test case

4cb00ce

fix(dynamo): honor Input(max_shape) for Dim.DYNAMIC dims, crash-fix s…

487e868

…ympy.oo in extract_var_range_info, validate Input against finite export bounds

fix(dynamo): downgrade user_min<exp_min to warning, treat 1→2 as 0/1-…

457213a

…specialization artifact, unify bound-mismatch message templates

use _first_sym_placeholder helper to skip lifted param/buffer placeho…

75d9826

…lders; sync assertions with refactored bound-mismatch messages

cleaning up the comments

37726a6

changing the narrower bound in case of user bound being subset of exp…

fd52275

…orter bound

apbose force-pushed the abose/torchTRT_dynamic_user_shape_bounds branch from 01798b0 to fd52275 Compare May 28, 2026 20:56

narendasan reviewed Jun 2, 2026

View reviewed changes

addressed review comments making the messages clearer

dae27b3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

user provided bound for torchtrt compile when export dimension is unb…#4213

user provided bound for torchtrt compile when export dimension is unb…#4213
apbose wants to merge 8 commits into
mainfrom
abose/torchTRT_dynamic_user_shape_bounds

apbose commented Apr 27, 2026 •

edited

Loading

Uh oh!

apbose commented Apr 30, 2026 •

edited

Loading

Uh oh!

narendasan commented May 6, 2026

Uh oh!

apbose commented May 6, 2026

Uh oh!

narendasan left a comment

Uh oh!

narendasan Jun 2, 2026

Uh oh!

apbose Jun 4, 2026

Uh oh!

narendasan Jun 2, 2026

Uh oh!

apbose Jun 4, 2026

Uh oh!

narendasan Jun 2, 2026

Uh oh!

apbose Jun 4, 2026

Uh oh!

narendasan Jun 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

apbose commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apbose commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

narendasan commented May 6, 2026

Uh oh!

apbose commented May 6, 2026

Uh oh!

narendasan left a comment

Choose a reason for hiding this comment

Uh oh!

narendasan Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

apbose Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

narendasan Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

apbose Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

narendasan Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

apbose Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

narendasan Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

apbose commented Apr 27, 2026 •

edited

Loading

apbose commented Apr 30, 2026 •

edited

Loading