Repository navigation
Polygraphy: fix plugin autotuner network creation on TensorRT 11 - #4870
Open
LinCanNerd wants to merge 1 commit into
Open
LinCanNerd wants to merge 1 commit into
LinCanNerd wants to merge 1 commit into
Conversation
The plugin autotuner always requested the EXPLICIT_BATCH network creation flag. TensorRT 10 made explicit batch the default and TensorRT 11 removed the flag, so `polygraphy plugin autotune` and the AutoTuner API raised AttributeError while creating the baseline network. Only request the flag when the installed TensorRT still defines it, and drop it from the PluginV3 timing cache example, which already requires TensorRT 10 or newer. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: LinCanNerd <lincanecdl@gmail.com>
LinCanNerd
force-pushed
the
polygraphy-autotune-trt11-explicit-batch
branch
from
October 6, 2026 08:31
af5f3ba to
46a6632
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
polygraphy plugin autotuneand the plugin AutoTuner Python API fail on TensorRT 11 before doing any work:OptimizationStrategy._create_network_from_onnxalways requests theEXPLICIT_BATCHnetwork creation flag. TensorRT 10 made explicit batch the default (the flag became a no-op) and TensorRT 11 removed it (NetworkDefinitionCreationFlagnow only hasSTRONGLY_TYPED,PREFER_AOT_PYTHON_PLUGINS,PREFER_JIT_PYTHON_PLUGINS). Every autotune run hits this when building the baseline network.CreateNetworkinbackend/trt/loader.pyalready avoids the flag on TensorRT 10+, but the autotuner creates its network directly.Changes:
optimization_strategy.py: only requestEXPLICIT_BATCHwhen the installed TensorRT still defines it. Behavior on versions that have the flag is unchanged.examples/api/plugin_autotuner/03_timing_cache_demo/pluginv3_timing_cache_demo.py: had the same unconditional flag; it now usescreate_network(0), since the example relies on PluginV3 and therefore TensorRT 10+.TestAutotune::test_create_network_from_onnxtotests/tools/test_plugin.pyand a CHANGELOG entry.Testing
On NVIDIA Jetson AGX Thor (SM110, JetPack 7.2 / L4T R39.2), TensorRT 11.3.0.99 (
tensorrt-cu13from PyPI), Polygraphy frommain(v0.53.6), Python 3.12:polygraphy plugin autotune relu.onnx --plugin-dir plugins -o out.onnxAttributeErrorexamples/api/plugin_autotuner/01_conv_relu_demo/conv_relu_demo.pyAttributeErrorexamples/api/plugin_autotuner/03_timing_cache_demo/pluginv3_timing_cache_demo.pyAttributeErrorpytest tests/tools/test_plugin.pyAttributeErrorblack --checkpasses on the changed files.Note, not addressed here: on this device the Conv+ReLU plugin did not beat the native baseline, and
AutoTuner.get_optimized_model_paththen raisesNo optimization strategy foundrather than returning the unmodified model, so the demo exits with an error after the comparison. That is independent of this fix and may be intended behavior.