Skip to content

Polygraphy: fix plugin autotuner network creation on TensorRT 11 - #4870

Open
LinCanNerd wants to merge 1 commit into
NVIDIA:mainfrom
LinCanNerd:polygraphy-autotune-trt11-explicit-batch
Open

LinCanNerd wants to merge 1 commit into
NVIDIA:mainfrom
LinCanNerd:polygraphy-autotune-trt11-explicit-batch

Conversation

@LinCanNerd

Copy link
Copy Markdown

What does this PR do?

polygraphy plugin autotune and the plugin AutoTuner Python API fail on TensorRT 11 before doing any work:

File ".../polygraphy/tools/plugin/subtool/autotuner/optimization_strategy.py", line 81, in _create_network_from_onnx
    1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
AttributeError: type object 'tensorrt_bindings.tensorrt.NetworkDefinitionCreationFlag' has no attribute 'EXPLICIT_BATCH'

OptimizationStrategy._create_network_from_onnx always requests the EXPLICIT_BATCH network creation flag. TensorRT 10 made explicit batch the default (the flag became a no-op) and TensorRT 11 removed it (NetworkDefinitionCreationFlag now only has STRONGLY_TYPED, PREFER_AOT_PYTHON_PLUGINS, PREFER_JIT_PYTHON_PLUGINS). Every autotune run hits this when building the baseline network. CreateNetwork in backend/trt/loader.py already avoids the flag on TensorRT 10+, but the autotuner creates its network directly.

Changes:

  • optimization_strategy.py: only request EXPLICIT_BATCH when the installed TensorRT still defines it. Behavior on versions that have the flag is unchanged.
  • examples/api/plugin_autotuner/03_timing_cache_demo/pluginv3_timing_cache_demo.py: had the same unconditional flag; it now uses create_network(0), since the example relies on PluginV3 and therefore TensorRT 10+.
  • Added TestAutotune::test_create_network_from_onnx to tests/tools/test_plugin.py and a CHANGELOG entry.

Testing

On NVIDIA Jetson AGX Thor (SM110, JetPack 7.2 / L4T R39.2), TensorRT 11.3.0.99 (tensorrt-cu13 from PyPI), Polygraphy from main (v0.53.6), Python 3.12:

Before After
polygraphy plugin autotune relu.onnx --plugin-dir plugins -o out.onnx AttributeError Baseline engine built and timed
examples/api/plugin_autotuner/01_conv_relu_demo/conv_relu_demo.py AttributeError Matching, baseline, and both plugin-replaced candidates built and timed
examples/api/plugin_autotuner/03_timing_cache_demo/pluginv3_timing_cache_demo.py AttributeError Completes successfully
pytest tests/tools/test_plugin.py new test fails with the same AttributeError 4 passed

black --check passes on the changed files.

Note, not addressed here: on this device the Conv+ReLU plugin did not beat the native baseline, and AutoTuner.get_optimized_model_path then raises No optimization strategy found rather than returning the unmodified model, so the demo exits with an error after the comparison. That is independent of this fix and may be intended behavior.

@LinCanNerd
LinCanNerd requested a review from a team as a code owner October 6, 2026 08:30
The plugin autotuner always requested the EXPLICIT_BATCH network creation
flag. TensorRT 10 made explicit batch the default and TensorRT 11 removed
the flag, so `polygraphy plugin autotune` and the AutoTuner API raised
AttributeError while creating the baseline network. Only request the flag
when the installed TensorRT still defines it, and drop it from the PluginV3
timing cache example, which already requires TensorRT 10 or newer.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: LinCanNerd <lincanecdl@gmail.com>
@LinCanNerd
LinCanNerd force-pushed the polygraphy-autotune-trt11-explicit-batch branch from af5f3ba to 46a6632 Compare October 6, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant