Potential perf win: Inline `SemanticContext` in `ATNConfig` and unbox `ATNConfig` in `ATNConfigSet`

[jonhoo]

As discussed on Bluesky, here's one of the LLM-generated performance improvement suggestions from profiling https://github.com/jonhoo/avdl on (synthetic) bigger .avdl files with the ANTLR-generated parser. I'd be happy to test out changes if you don't have a benchmark handy!

Symptom

ATNConfig::cloned is the single biggest ANTLR leaf function in CPU profiles, at 5.0% of self-time (190/3785 samples). Dropping ATNConfig adds another 2.4% (91 samples) and dropping SemanticContext adds 0.7% (26 samples). Dropping Box<ATNConfig> contributes 0.6% (21 samples). Combined, these account for ~8.7% of total self-time — all driven by unnecessary heap allocations during config cloning and destruction.

Root cause

Two layers of unnecessary boxing:

1. ATNConfig.semantic_context: Box<SemanticContext> (atn_config.rs:24) — every ATNConfig clone heap-allocates a new Box. The overwhelmingly common variant is SemanticContext::NONE (a Predicate with three i32 fields = 12 bytes + enum discriminant). The AND/OR variants contain Vecs but are rare. The code already has a TODO comment on this: "todo remove box?" at atn_config_set.rs:24.

2. ATNConfigSet.configs: Vec<Box<ATNConfig>> (atn_config_set.rs:25) — adds a pointer indirection and a per-config heap allocation. Every call to add_cached or add receives a Box<ATNConfig>, and every iteration through configs dereferences a pointer.

The cloned() method (atn_config.rs:163) calls self.clone(), which triggers Box<SemanticContext>::clone (heap alloc + memcpy) and Option<Arc<PredictionContext>>::clone (Arc refcount bump). The SemanticContext box clone is the dominant cost since it always allocates, while the Arc clone is just an atomic increment.

Affected files

• runtime/Rust/src/atn_config.rs:24 — semantic_context field type, cloned() method (line 163), constructors (new at line 113, new_with_semantic at line 129)
• runtime/Rust/src/atn_config_set.rs:25 — configs field type, add_cached (line 136), add (line 183), get_items (line 187), local_hash_key (line 122, dereferences semantic_context)
• runtime/Rust/src/parser_atn_simulator.rs — call sites creating Box::new(ATNConfig::...) at lines 738, 747, 763, 997, 1044, 1075
• runtime/Rust/src/lexer_atn_simulator.rs — call sites at lines 455, 458, 495

Suggested fix

1. Change semantic_context: Box<SemanticContext> to semantic_context: SemanticContext (inline it). The NONE variant is 12 bytes + discriminant (~16 bytes with alignment), which is small enough to inline without bloating ATNConfig. The rare AND/OR variants contain a Vec (24 bytes on x86-64), so the enum is ~28 bytes total — still reasonable to inline given the clone savings.

2. Change configs: Vec<Box<ATNConfig>> to configs: Vec<ATNConfig>. Remove the Box::new(...) wrapping at all call sites in parser_atn_simulator.rs and lexer_atn_simulator.rs.

3. Update add_cached and add signatures from config: Box<ATNConfig> to config: ATNConfig.

4. Update Key::Partial construction in local_hash_key (line 132): config.semantic_context.deref().clone() becomes just config.semantic_context.clone().

This is a mechanical refactor with no behavioral changes. The PartialEq, Hash, and Debug impls on ATNConfig work the same with an inlined SemanticContext.

Size impact

Before: ATNConfig = 8 (pointer to SemanticContext) + 4 (state) + 4 (alt) + 16 (Option) + 4 (reaches_into_outer_context) + 1 (precedence_filter_suppressed) + ~24 (config_type enum) ≈ 64 bytes, plus a separate 16-byte heap allocation for the Box<SemanticContext>.

After: ATNConfig ≈ 88 bytes (SemanticContext inlined at ~28 bytes instead of 8-byte pointer), no separate heap allocation. Net: one fewer allocation per config, slightly larger struct but better cache locality.

[tinovyatkin]

@jonhoo great writeup — this lines up with what we found. In a clean-room Rust ANTLR runtime I maintain (ophi-dev/antlr-rust-runtime) both of these are already in place: semantic_context is an inlined SemanticContext rather than Box<…>, and the config set holds Vec<AtnConfig> by value instead of Vec<Box<ATNConfig>>. Config cloning in closure/reach was our single hottest path too, so dropping that per-config allocation was a clean win — and I can confirm it's purely mechanical: PartialEq/Hash/Debug all carry over unchanged. Happy to compare notes if the config layout is a useful reference.