TensorRT silently produces wrong results for batch > 1 on SegFormer (b0/b1/b2 verified, fp32, static and dynamic engines, TRT 10.11 & 11.1, sm89 + sm120) #4853

Summary

TensorRT silently produces wrong results for batch > 1 when compiling SegFormer models. This was verified with all three public checkpoints nvidia/segformer-b0/b1/b2-finetuned-ade-512-512 and a reduced single encoder block of the b2 variant. Batch=1 is bit-accurate against ONNX Runtime; batch=2 is wrong by orders of magnitude while trtexec reports PASSED. The bug is:

• precision-independent (fp32 AND fp16 engines affected),
• shape-mode-independent (fully static batch=2 engines min=opt=max=2 AND dynamic 1–2 engines affected),
• architecture-independent (reproduced on sm_120 Blackwell laptop and sm_89 L4),
• present in TensorRT 10.11 (10.11.0.x) and 11.1 with identical error values,
• not affected by fusion level (--builderOptimizationLevel=0 still fails, though less catastrophically).

ONNX Runtime (CPU and CUDA EP) executes the same ONNX with batch=2 correctly on the same GPUs.

Environment

• TensorRT 10.11 — trtexec [TensorRT v101100] (CUDA 12) and TensorRT 11.1 — trtexec [TensorRT v110100] (from nvcr.io/nvidia/tritonserver:26.07-py3)
• GPUs: NVIDIA RTX PRO 2000 Blackwell Generation Laptop (sm_120), driver 595.84; NVIDIA L4 (sm_89)
• ONNX: opset 19, exported with torch.onnx.export (legacy TorchScript path) + onnxsim, dynamic batch axis
• Model: nvidia/segformer-b2-finetuned-ade-512-512 (public), 512×512 NHWC input, fp32

Symptom (full public models)

Build a plain fp32 engine with dynamic batch 1–2 and run the same random input as batch=1 and as b2[0] of a batch=2 request. The batch=2 input's first sample is identical to the batch=1 input, so a correct engine must produce identical outputs:

ORT CUDA EP: b2[0] vs b1 max_abs_diff = 0.018, argmax match = 100 %
ORT CPU    : batch=2 bit-identical to batch=1

The segmentation output is essentially garbage for batch≥2 (with a fine-tuned SegFormer variant at 1024×1024, batch=2 collapsed to all-background). The b0/b1/b2 checkpoints above were tested via generate/export_public.py from the attached zip, which regenerates each ONNX and the input files from the HuggingFace hub.

Bisection

Prefix-bisection (cut the graph after every encoder block, compare TRT vs ORT at batch=2, fp32):

Node-level cuts inside block.1.0 show: everything up to the mlp/dense1 output is clean at batch=2; the first wrong values appear at the mlp/dwconv (3×3 depthwise conv, groups=512) output (diff 3.1 vs fp32 noise 0.004).

However, the dwconv alone does not reproduce the bug:

• single Conv (same weights, shapes, real activations): correct at batch=2
• dense1 → transpose → reshape → dwconv: correct
• LayerNorm → dense1 → (dynamic Shape/Gather/Concat reshape chain) → dwconv: correct

Only the complete block (LayerNorm → efficient attention → residual/LN → MLP with dwconv, 64 nodes, 1.9 MB) reproduces. Ending the same graph at dense1 (removing the dwconv) heals it. So the broken kernel/tactic is only selected in the full-block compilation context.

Minimal repro (attached, trt_batch_bug_repro.zip)

repro.onnx — one SegFormer-b2 encoder block, fully synthetic weights (seeded RNG), no third-party data. Input x: [B, 4096, 128], output y: [B, 512, 64, 64]. The batch=2 input file's first sample is identical to the batch=1 input, so a correct engine must produce identical outputs — no reference model needed:

trtexec --onnx=repro.onnx --saveEngine=repro.engine \
        --minShapes=x:1x4096x128 --optShapes=x:2x4096x128 --maxShapes=x:2x4096x128

trtexec --loadEngine=repro.engine --shapes=x:1x4096x128 \
        --loadInputs=x:repro_input_b1.bin --exportOutput=out_b1.json \
        --iterations=1 --warmUp=0 --duration=0

trtexec --loadEngine=repro.engine --shapes=x:2x4096x128 \
        --loadInputs=x:repro_input_b2.bin --exportOutput=out_b2.json \
        --iterations=1 --warmUp=0 --duration=0

python3 verify.py out_b1.json out_b2.json

Measured max_abs_diff(out_b2[0], out_b1[0]) — identical input, same engine:

For comparison: ONNX Runtime CPU executes batch=2 bit-identical to batch=1,
and a correct fp32 engine gives ~1e-6 for the same comparison. Every trtexec
run prints &&&& PASSED.

Additional observations

• Same failure through Triton Inference Server (26.07 / TRT backend), so it is not trtexec-specific.
• Two identical images in the batch produce two identical wrong outputs (deterministic miscompile, not input mixing).
• With the 1024×1024 fine-tuned SegFormer variant we additionally verified: fp16 AND fp32 fail; static batch=2 (min=opt=max=2) fails; --builderOptimizationLevel=0 still fails (max_abs_diff ~22 instead of ~38; batch=2 no longer fully degenerate).
• Current workaround: build engines for batch=1 only.

Related reports

• Incorrect outputs of TensorRT 10.7 when compiling F.scaled_dot_product_attention on GPU L4 #4333 (ViT attention miscompile on L4, internally tracked)
• Silently incorrect fp16/bf16 outputs from fused MHA on consumer Blackwell (sm_120) for ViT/DETR model (SAM 3 detector) — fp32 exact, persists in 10.16.1, breaking the MHA fusion restores accuracy #4837 (silently wrong fused MHA outputs on sm_120)
• https://forums.developer.nvidia.com/t/319274 (MaxViT: batch>1 wrong, fp32, TRT 10.7, unresolved)
• https://forums.developer.nvidia.com/t/361820 (SegFormer-b2: ONNX vs TRT huge differences — same symptom, closed by reporter without root cause)

Happy to provide engine files, layer info dumps or run additional experiments on request.

trt_batch_bug_repro.zip