[P1][Core reliability] Reproduce and eliminate invalid tokenizer lifetime crashes in `OgaCreateTokenizerStream`

[sheetalarkadam]

Description

Two shipped Android native runtime families have produced the same process-terminating tokenizer-stream construction stack:

Generators::TokenizerStream::TokenizerStream(Generators::Tokenizer const&)+40
Generators::Tokenizer::CreateStream() const+40
OgaCreateTokenizerStream+36

Known binary provenance:

Android runtime family	ORT GenAI	ORT	Core	libonnxruntime-genai.so BuildId
Foundry Local Android 0.1.5	0.13.1	1.24.3	earlier Core family	e9c42fda43224e3104f208390e3fd99ca1d30920
Foundry Local Android 0.1.6 / current Android origin/main	0.14.1	1.26.0	1.2.4 family	0871373716b5b764d24da70ccdb891d5f0497c66

For the 0.13.1 binary, the supplied ARM64 frames are at relative PCs 0x267a98, 0x267fa8, and 0x2bb0ac. For the 0.14.1 binary, the constructor fault at relative PC 0x286688 reads Tokenizer storage before shared_from_this() can establish a strong owner. This is consistent with a null/stale tokenizer handle, use-after-free, disposal/finalizer race, generation replacement race, unsupported concurrent stream creation, or earlier pointer corruption. It does not establish which hypothesis is correct.

The supplied tombstone evidence is not joined to a production Kusto session. Do not claim production prevalence for this exact signature or infer that it caused a specific Android Binder death. A matching BuildId, normalized frames, original exit PID/time, and session correlation are required before making that claim.

Why this belongs in microsoft/Foundry-Local

Android packages prebuilt Core/ORT GenAI binaries and invokes the shared command surface, but the tokenizer registry, managed/native wrapper ownership, C API handles, and model/tokenizer disposal contract are Core/native concerns. Android can provide containment and process evidence; the durable ownership fix must be implemented and validated in the shared Core/runtime path.

Investigation hypotheses

Test these independently:

1. Request-scoped ownership gap: model/tokenizer lookup returns a borrowed object and a concurrent request, registry update, or cleanup releases/replaces it before stream creation.
2. Managed wrapper lifetime gap: raw IntPtr ownership permits explicit Dispose or finalization to race a native call. A SafeHandle or equivalent locked add-reference/release lease may be required.
3. Generation/unload race: unload removes or disposes an old generation while an admitted request still uses it, or a late cleanup disposes a replacement generation.
4. Live-tokenizer thread-safety: concurrent CreateStream calls on one valid tokenizer are unsupported or mutate shared state.
5. C API validation gap: null/disposed handles are not rejected before dereference. A null check hardens the boundary but cannot make a stale non-null pointer safe.
6. Earlier heap corruption: another handle destroy/type mismatch corrupts ownership data before this call.

A runtime upgrade is a comparison cell, not proof of remediation. The critical ownership pattern exists across both observed runtime families.

Reproduction plan

Phase A: one model, concurrent requests, no unload

Start here because the customer uses one model and the crash may not require lifecycle overlap.

1. Initialize once and load exactly one production-representative model once.
2. Do not unload, reload, replace, or dispose the model/tokenizer during this phase.
3. Run streaming and non-streaming cases separately with minimal input and max_tokens=1.
4. Release 2, 4, 8, and 16 callers from a barrier at the same time.
5. Run with telemetry/Core side commands enabled and disabled as separate cells.
6. Record every service/worker PID, process generation, terminal callback, Core/ORT/GenAI version, and ELF BuildId. A hidden service restart is a failure.
7. Repeat against the exact 0.13.1/e9c42… and 0.14.1/087137… binaries.

Add narrower harnesses to separate ownership layers:

• direct OGA: parallel OgaCreateTokenizerStream/destroy-stream calls against one unquestionably live tokenizer;
• managed wrapper: parallel CreateStream with forced GC/finalizer activity and explicit dispose tests;
• Core command harness: parallel chat commands against one permanently loaded model without Android Binder orchestration.

Phase B: deterministic unload/reload overlap

Run separately after Phase A.

1. Add a test hook immediately after model/tokenizer lookup and immediately before OgaCreateTokenizerStream.
2. Pause an admitted request at that hook.
3. Start unload or replacement of the same model generation.
4. Release the request into stream creation.
5. Repeat with reload of the same alias while the old request remains admitted.
6. Force every ordering: request wins, unload wins, replacement wins, cancellation during wait, and late cleanup from an old generation.

Without the pre-call hook, unload overlap is only probabilistic and cannot qualify a lifetime fix.

Required ownership contract

• Model/tokenizer lookup must return a request-scoped strong lease tied to an immutable generation.
• Unload removes the generation from new lookup before disposal, then either waits for admitted leases to drain or returns a defined busy/cancelled outcome.
• Stream creation must acquire/retain the tokenizer ownership it needs before native dereference.
• Existing streams must either retain their required model/tokenizer state or terminate through a documented safe transition.
• Managed wrappers must use SafeHandle or equivalent synchronized add-reference/release semantics so native calls cannot race explicit dispose or finalization.
• Close/dispose/unload must be idempotent and generation-safe; cleanup for an old generation must never release a replacement.
• If live-tokenizer concurrent CreateStream is unsupported even with valid ownership, serialize only that operation and document the constraint.
• Cancellation must remain non-blocking and must not create lock inversion through callbacks.

Diagnostics

Emit bounded lifecycle evidence without pointers, prompts, responses, model contents, paths, or credentials:

• Core/ORT/ORT GenAI versions and exact ELF BuildIds;
• process role/PID/generation and request/model generation;
• tokenizer/model opaque handle ID plus generation, state, active lease count, and create/release phase;
• create-stream, unload, replacement, dispose, and finalizer begin/end ordering;
• typed null/disposed/stale/busy outcomes;
• sanitizer finding or normalized native fingerprint when a process exits.

Deterministic acceptance criteria

• Phase A completes at least 10,000 full requests across the 2/4/8/16 concurrency matrix with zero native exits, hidden process restarts, missing terminal callbacks, deadlocks, or unbounded object growth.
• A direct OGA sanitizer run completes at least 1,000,000 stream create/destroy operations against a live tokenizer with zero ASan/HWASan findings.
• A barrier test forces at least 1,000 request-vs-unload/replacement interleavings and 100 full reload cycles with zero UAF, double-free, stale-generation release, or deadlock.
• If creation wins admission, it holds a valid strong lease and returns a usable stream. If unload/removal wins, creation returns a typed error before native dereference.
• Null, disposed, and stale-generation handles return defined errors and never enter TokenizerStream construction.
• Explicit dispose and forced managed finalization cannot release a handle while SafeHandle/equivalent admission is active.
• Repeated close/dispose/unload is idempotent and cannot release a replacement generation.
• Existing stream behavior during unload is explicitly tested and matches the documented contract.
• Malformed tokenizer/config/model artifacts fail through typed parse/integrity errors and never reproduce the lifetime crash.
• The patched build passes with any temporary Android global/per-model serialization diagnostic disabled.
• Every test artifact records exact Core, ORT, ORT GenAI, wrapper, and BuildId provenance.
• If the defect is inside ONNX Runtime GenAI, a minimal reproducer, upstream fix/version, sanitizer regression, and proof that the shipping binary contains the fix are attached.

Non-goals

• Claiming the exact signature is prevalent in production telemetry.
• Treating model corruption as the established cause of this constructor frame.
• Solving Android AEI symbolication or Binder recovery here.
• Calling process-wide OgaShutdown() and attempting to reinitialize the runtime.

Ownership

• Foundry Local Core: registry generations, request leases, unload/replacement ordering, wrapper lifetime, and deterministic hooks/tests.
• Managed wrapper: SafeHandle or equivalent call-vs-dispose/finalizer safety.
• ONNX Runtime GenAI: C API lifetime/null contract and any defect reproducible with a valid live handle.
• Android: temporary containment experiments, exact process/build evidence, and integration qualification; not the canonical lifetime implementation.