[CI] DotNetWatch CoreCLR Hot Reload crashes with Bad IL range

Failure signature

Xamarin.Tests.DotNetWatchTest.DotNetWatch using CoreCLR fails after applying a Hot Reload edit:

dotnet watch 🔥 C# and Razor changes applied
Unhandled exception. System.BadImageFormatException: Bad IL range.
   at HotReloadTestApp.Program.DoSomething(Int32 i)

The app builds and launches successfully, receives the edit, then crashes before the watched variable changes. The test reports Timed out waiting for the variable to change.

Confirmed monitored occurrences

PR / head	Azure run / job	Tests	Worker
#26454 / `b956034e64eae11ea91afcec93b11996be9adbe0`	15086535 — dotnettests_ios	1	`AcesShared 30`
#26454 / same head	15086535 — dotnettests_maccatalyst	2 (sandbox off/on)	`AcesShared 21`
#26454 / `918e83b75ed8152cb415cc7627e51ce69bc3124a`	15088820 — dotnettests_ios	1	`AcesShared 27`
#26454 / same head	15088820 — dotnettests_maccatalyst	2 (sandbox off/on)	`AcesShared 9`
#26454 / `3adbceca8e2a7ceb62aaf29c8d03633fd7d3fea2`	15090225 — dotnettests_ios	1	`AcesShared 6`
#26454 / same head	15090225 — dotnettests_maccatalyst	2 (sandbox off/on)	`AcesShared 14`
#26356 / `ac84fef62c4a0babd9079876486d13e12f888133`	15090832 — dotnettests_ios	1	`AcesShared 67`

Total: 10 confirmed test occurrences across 2 distinct monitored PRs, 4 heads, and 4 Azure runs.

First terminal occurrence: 2026-08-25T09:11:52Z. Latest terminal occurrence: 2026-08-25T17:06:44Z.

Classification and impact

Deterministic test/source-runtime failure. The same post-edit exception reproduces on iOS and Mac Catalyst, with and without Mac Catalyst sandboxing, across multiple heads and two dependency-update PRs. Builds and app launches succeed; the failure begins after Hot Reload applies the update. Unchanged retries are not appropriate.

This blocks the affected dotnettests_ios / dotnettests_maccatalyst jobs and can keep dependency-update PR CI red.

Investigation performed

Inspected Azure build metadata, timelines, failed tasks, test-summary artifacts, and raw TRX output.
Verified the exact exception and post-edit phase on every occurrence.
Confirmed Build Analysis has no known-issue match.
Searched all open and closed dotnet/macios issues separately by exact exception, test name, Bad IL, and Hot Reload component terms, then inspected candidate bodies and non-ignored comments.
Existing [CI] Flaky: DotNetWatchTest.DotNetWatch on iOS — timed out waiting for the app to start #25935 is a different pre-launch timeout. Closed dotnet watch fails when targeting sandboxed maccatalyst apps #25700 is a different sandbox/startup-hook load failure. Neither represents this signature.

Workaround

No validated workaround is known. The CoreCLR Hot Reload edit/runtime behavior requires investigation. Do not retry unchanged heads merely to see whether the deterministic crash clears.

Azure logs and artifacts are retention-limited.

Two additional exact-signature occurrences were confirmed on PR #26356 @ ac84fef62c4a0babd9079876486d13e12f888133:

Build 15090832 attempt 1 — dotnettests_maccatalyst / Run tests on AcesShared 40.
Both DotNetWatch(MacCatalyst, useMonoRuntime:false, enableSandbox:false) and the sandbox-enabled variant built and launched successfully, applied the Hot Reload edit, then crashed with System.BadImageFormatException: Bad IL range.

The canonical total is now 12 confirmed test occurrences across 2 distinct monitored PRs, 4 heads, and 4 Azure runs.

Classification: deterministic source/runtime failure. No retry submitted: build 15090832 remains active and an unchanged rerun is not appropriate.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

External retry outcome for PR #26356 @ ac84fef62c4a0babd9079876486d13e12f888133:

Build 15090832 attempt 2 completed failed at 2026-08-26T00:02:12Z. Azure did not expose the triggering actor.
dotnettests_ios / Run tests on AcesShared 4 reproduced DotNetWatch(iOS, useMonoRuntime:false, enableSandbox:false).
dotnettests_maccatalyst / Run tests on AcesShared 2 reproduced both sandbox-disabled and sandbox-enabled variants.
In all three tests, Hot Reload applied the edit and the app then crashed with System.BadImageFormatException: Bad IL range.

The canonical total is now 15 confirmed test occurrences across 2 distinct monitored PRs, 4 heads, 4 Azure runs, and 5 run attempts.

Classification: deterministic source/runtime failure. The unchanged failed-jobs retry reproduced every Bad IL occurrence, so no further retry is appropriate. Retry accounting for #26356 is 1 PR-lifetime / 1 current-head request.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

Three additional exact-signature occurrences were confirmed on PR #26454 @ 43cd3fb1c41f4d28eee679804e3421cd3f10022a:

Build 15097859 attempt 1 — dotnettests_ios / Run tests on AcesShared 28 reproduced DotNetWatch(iOS, useMonoRuntime:false, enableSandbox:false).
dotnettests_maccatalyst / Run tests on AcesShared 57 reproduced both sandbox-disabled and sandbox-enabled variants.
In all three tests, Hot Reload applied the edit and the app then crashed with System.BadImageFormatException: Bad IL range.

The canonical total is now 18 confirmed test occurrences across 2 distinct monitored PRs, 5 heads, 5 Azure runs, and 6 run attempts.

Classification: deterministic source/runtime failure. No retry submitted: the unchanged retry on #26356 already reproduced this exact signature, and build 15097859 remains active for unrelated validation.

The Catalyst job reached test execution without the previous _mono_helper_box_nullable undefined-symbol failure, so that separate fix is holding. The iOS/tvOS linker jobs also passed; those outcomes are not part of this Bad IL signature.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

Two external failed-job retries completed and reproduced the full Bad IL signature again:

PR [net11.0] Update dependencies from dotnet/dotnet #26356 @ ac84fef62c4a0babd9079876486d13e12f888133: build 15090832 attempt 3 completed failed at 2026-08-26T05:55:03Z.
- dotnettests_ios / Run tests on AcesShared 26 reproduced the iOS test.
- dotnettests_maccatalyst / Run tests on AcesShared 25 reproduced both Catalyst sandbox variants.
PR [release/11.0.1xx-rc.1] Update dependencies from dotnet/dotnet #26454 @ 43cd3fb1c41f4d28eee679804e3421cd3f10022a: build 15097859 attempt 2 completed failed at 2026-08-26T05:29:52Z.
- dotnettests_ios on AcesShared 24 reproduced the iOS test.
- dotnettests_maccatalyst on AcesShared 23 reproduced both Catalyst sandbox variants.

In all six tests, Hot Reload applied the edit and the app then crashed with System.BadImageFormatException: Bad IL range.

The canonical total is now 24 confirmed test occurrences across 2 distinct monitored PRs, 5 heads, 5 Azure runs, and 8 run attempts.

Classification: deterministic source/runtime failure. Multiple unchanged retries have reproduced every variant, so no further retry is appropriate. Retry usage is 2 lifetime / 2 current-head requests for #26356 and 1 lifetime / 1 current-head request for #26454. Azure did not expose either retry initiator.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

A new exact-signature occurrence is confirmed on PR #26454 @ c9902ed89a6f2305ba25e00803588040ae58b9c2:

Build 15108836 attempt 1 — dotnettests_maccatalyst / Run tests on AcesShared 26.
DotNetWatch(MacCatalyst, useMonoRuntime:false, enableSandbox:false) built and launched successfully. Hot Reload applied the edit in 428 ms, then CoreCLR threw System.BadImageFormatException: Bad IL range from HotReloadTestApp.Program.DoSomething, and the process exited 134. The test subsequently reported Timed out waiting for the variable to change.
Evidence came from TestSummary-simulator_testsdotnettests_maccatalyst-1, the HTML/NUnit report, and the Azure task log. The remaining simulator run jobs were still active when observed.

The canonical total is now 25 confirmed test occurrences across 2 distinct monitored PRs, 6 heads, 6 Azure runs, and 9 run attempts.

Classification: deterministic source/runtime failure. No retry was submitted: unchanged retries have already reproduced this signature, and this current head has used 0 retries (PR lifetime usage remains 1, on the superseded head). Draft PR #26473 adds diagnostics but is not a validated source/runtime fix.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

Azure logs and artifacts are retention-limited.

Copilot's analysis (https://gist.github.com/rolfbjarne/fc3b58a978c8ab763fa4fd69e525d8eb):

The "Bad IL range" error comes from CoreCLR's `PEAssembly::GetIL`, which means
the runtime looked up the updated method's IL in the *baseline* PE image using
the RVA from the metadata delta (that RVA points into the delta IL stream, so
it's out of range in the baseline image). In other words the runtime never
picked up the delta IL when re-jitting the updated method.

This looks like fallout from https://github.com/dotnet/runtime/pull/130159
("Use CodeVersioning for EnC"), which switched the delta IL to be stored as an
`ILCodeVersion`: the mobile CoreCLR builds don't have tiered compilation
enabled (unlike the desktop macOS build), and only the code-versioned JIT path
knows about the EnC IL. That matches what we're seeing: the macOS test cases
still pass, only the mobile ones fail.

Root-caused this, and filed it upstream: dotnet/runtime#132804

Short version: it's a regression from dotnet/runtime#130159 ("Use CodeVersioning for EnC"). That PR made the EnC delta IL be stored as an ILCodeVersion, which is only reachable through VersionedPrepareCodeConfig::GetILHeader(). The plain PrepareCodeConfig::GetILHeader() falls back to MethodDesc::GetILHeader() → Module::GetIL(GetRVA()), and after a delta that RVA is delta-relative, so reading it against the baseline image is out of range — which is exactly the Bad IL range we see.

Worth flagging one thing, since it's the obvious guess and it's wrong: this is not because code versioning is unavailable on mobile. IsEligibleForEnC() doesn't depend on tiered compilation, and FEATURE_CODE_VERSIONING is enabled there anyway via FEATURE_METADATA_UPDATER. The real difference is that FEATURE_DYNAMIC_CODE_COMPILED is off for Apple mobile, so there's no JIT and these apps run the CoreCLR interpreter — some interpreter-side path prepares the updated method with a non-versioned config. That also explains why only iOS and Mac Catalyst fail while desktop macOS is fine.

Nothing we can fix on our side, so I've disabled the affected test cases for now (the Mono and desktop macOS cases still run) — that's in the PR referencing this issue. We should re-enable them once the runtime fix flows in.

[tests] Ignore the CoreCLR 'dotnet watch' tests on the mobile platforms.

Workaround validation outcome for PR #26454 @ 565dc421e66fc277e7bfb2c690276bb396b732fb:

Simulator build 15111445 completed succeeded at 2026-08-27T00:55:51Z; all 53 visible checks succeeded (1 unrelated check was skipped).
dotnettests_ios and dotnettests_maccatalyst both succeeded.
The NUnit reports confirm the remaining Mono cases passed: DotNetWatch(iOS, useMonoRuntime:true, enableSandbox:false) and DotNetWatch(MacCatalyst, useMonoRuntime:true, enableSandbox:false). The affected CoreCLR mobile cases are no longer discovered, consistent with the documented temporary test disablement.

This validates the test-disable workaround only; it does not demonstrate that the upstream CoreCLR interpreter regression is fixed. The canonical occurrence count remains 25 across 2 monitored PRs, 6 failing heads, 6 failing Azure runs, and 9 failing run attempts. No retry was submitted.

Issue-specific checklist: none; the issue body has no Additional info to collect section.

Azure logs and artifacts are retention-limited.

Fix submitted upstream: dotnet/runtime#132818

Root cause is that FEATURE_REJIT is off on the platforms without a JIT (Apple mobile), which made CodeVersionManager::GetILCodeVersion unconditionally return the default IL code version instead of the EnC one — so after a hot reload edit the runtime read the updated method's IL from the baseline image using a delta-relative RVA, throwing Bad IL range. Details in dotnet/runtime#132804.

Once the fix flows into our runtime pack, the Assert.Ignore added in tests/dotnet/UnitTests/DotNetWatchTest.cs can be removed.

Fix "Bad IL range" when applying a hot reload update without FEATURE_…

[tests] Re-enable CoreCLR DotNetWatch tests. Fixes #26470 (#26559)

Navigation Menu

Repository navigation

[CI] DotNetWatch CoreCLR Hot Reload crashes with Bad IL range #26470

Failure signature

Confirmed monitored occurrences

Classification and impact

Investigation performed

Workaround

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[CI] DotNetWatch CoreCLR Hot Reload crashes with Bad IL range #26470

Description

Failure signature

Confirmed monitored occurrences

Classification and impact

Investigation performed

Workaround

Activity

dalexsoto commented on Aug 25, 2026

dalexsoto commented on Aug 26, 2026

dalexsoto commented on Aug 26, 2026

dalexsoto commented on Aug 26, 2026

dalexsoto commented on Aug 26, 2026

rolfbjarne commented on Aug 26, 2026

rolfbjarne commented on Aug 26, 2026

dalexsoto commented on Aug 27, 2026

rolfbjarne commented on Aug 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions