[simonrozsival]

Context: https://learn.microsoft.com/dotnet/core/testing/unit-testing-with-dotnet-test
Context: https://github.com/microsoft/testfx/blob/88a31ec394abe9f5c5dcf596f0a150ff998d3a0e/docs/source-generator/design.md
Context: #13432
Context: #13433

The on-device runtime tests (Mono.Android.NET-Tests, its Java.Interop-Tests fixtures, and Xamarin.Android.JcwGen-Tests) run NUnit through TestRunner.Core. NUnit discovers and invokes tests via reflection, which fights trimming and NativeAOT. To keep it working, the test app roots entire assemblies (<TrimmerRootAssembly RootMode="All"/> for Mono.Android.NET-Tests and Java.Interop-Tests), sets IlcGenerateCompleteTypeMetadata=true, and ships a NativeAOT.rd.xml whose only purpose is to keep NUnit's own reflection alive:

• EquatablesComparer.GetEquatableImplementations() calls GetGenericArguments()[0] on closed IEquatable<T> instantiations whose metadata ILC otherwise trims, and
• TaskAwaitAdapter+GenericAdapter<VoidTaskResult> is built via Type.MakeGenericType() for every async Task test.

That is, we were testing a trimmed app whose trimming had been substantially turned off for the benefit of the test framework.

Migrate these suites to MSTest running natively on Microsoft.Testing.Platform (MTP), with MSTest.SourceGeneration emitting the test registry and invocation delegates at compile time.

Android runtime tests

• Add tests/TestRunner.MSTest.Android, an Android Instrumentation that hosts MTP in-process, registers the fixture assemblies via AddMSTest, applies IncludeCategories/ExcludeCategories from runtimeconfig.json (and -e include, -e exclude, -e noexclusions true extras), and streams results back through the existing dotnet test Android protocol. Test failures return native exit code 2; infrastructure failures cancel instrumentation instead of reporting an empty successful run.
• Convert test attributes/assertions to MSTest, preserving each case's canonical name, data rows, categories and multiplicity. Exception assertions keep their original exactness; where NUnit's Assert.Throws semantics differed, the test body was restructured rather than weakened.
• Remove NativeAOT.rd.xml, IlcGenerateCompleteTypeMetadata, and the whole-assembly roots for Mono.Android.NET-Tests and Java.Interop-Tests.
• Keep roots that protect product/JNI behavior rather than test discovery: Mono.Android-Test.Library.NET, StartupHook, converter/reflection roots, the JCW whole-app root, and new narrow TrimmerRoots.xml entries for RegisterMeOnNewNativeThreadOne (loaded only by Java name from native code; without it RegisterTypeOnNewNativeThread returns -104) and the three ExportField initializers on ExportStaticField/ExportInstanceField (called only from Java field initialization; without them the generated Java ANSWER field disappears).
• The generated test registry calls tests that intentionally exercise trim/AOT-unsafe APIs, so Mono.Android.NET-Tests keeps the resulting IL2026/IL2062/IL2111/IL3050 analyzer diagnostics as warnings, matching its existing ILLinkTreatWarningsAsErrors=false policy. All other CI warnings-as-errors diagnostics were fixed in the tests themselves.

The existing apk-instrumentation.yaml CI path is unchanged: it still runs dotnet test from the app directory and passes IncludeCategories=JniReferenceLeak for the dedicated JNI lanes.

SourceWriter host tests

Xamarin.SourceWriter-Tests moves to MSTest/MTP as the host-side pilot. A new opt-in runNativeMtpHostTests pipeline parameter (default false) adds an independent Linux stage that installs $(DotNetSdkVersion) and runs:

DOTNET_TEST_RUNNER=Microsoft.Testing.Platform dotnet test \
  --project tests/Xamarin.SourceWriter-Tests/Xamarin.SourceWriter-Tests.csproj \
  -c Debug --report-trx --results-directory $(Agent.TempDirectory)/TestResults

The TRX is published with PublishTestResults@2 and failTaskOnMissingResultsFile: true. MTP selection is scoped to that one step via DOTNET_TEST_RUNNER, so every other NUnit suite keeps using VSTest and all existing stages are unchanged; skipTests still skips the new stage.

An earlier revision drove this stage through a custom Python adapter (run-native-mtp-tests.py) that did UID-based slicing, retried failed tests and rewrote TRX files. For an 11-test suite that was all cost and no benefit, so it was removed in favor of the plain dotnet test invocation above: MTP's exit code already fails the step, and the standard Azure task publishes the results.

Package versions

All migrated suites take their versions from eng/Versions.props: the Maestro-flowed $(MSTestPackageVersion) (4.6.0-preview.26531.5) plus a new testfx dependency in eng/Version.Details.xml for $(MicrosoftTestingExtensionsTrxReportPackageVersion), so TrxReport flows in lockstep with MSTest. Microsoft.Testing.Platform.MSBuild comes transitively from MSTest.TestAdapter. The repo-wide $(MicrosoftTestingPlatformVersion) used by Microsoft.Android.Run is not touched.

Documentation

• tests/Xamarin.SourceWriter-Tests/README.md describes how to build, run and filter the MTP suite.
• Documentation/workflow/UnitTests.md and the tests skill catalog describe the MSTest suites and their dotnet test commands.

Results

Release, full trim, API-35 arm64 emulator, local SDK built from this branch (rebased on main aa91d328), built with -p:_AndroidTreatWarningsAsErrors=true like CI, MSTest 4.6.0-preview.26531.5:

Suite	Cases	Passed	Skipped	Failed
Mono CoreCLR	1063	1055	8	0
Mono NativeAOT	1000	992	8	0
JCW CoreCLR	47	46	1	0

Before rebasing (on 598780a6), the same suites matched a fresh run of the original NUnit suites exactly (1046/1038/8, 983/975/8, 47/46/1); the 17 additional cases come from tests added on main since.

Each was run from its test project directory after ./dotnet-local.sh build -t:Install -c Release [-p:PublishAot=true] <project>, via ../../../dotnet-local.sh test <project> --no-build -c Release [-p:PublishAot=true] --report-trx.

Identities, categories and multiplicities were also compared case by case earlier in development, along with the JniReferenceLeak (Mono 6, JCW 1) and NTLM opt-in lanes; those category lanes were not re-run on this head.

Xamarin.SourceWriter-Tests: 11/11 passed with SDK 11.0.100-rc.3.26522.148 (the pipeline's $(DotNetSdkVersion)) and SDK 12; a filter matching nothing exits 8, so failures propagate.

On-device execution time was measured but is noisy: median 16.9s (MSTest) vs. 19.2s (NUnit), while repeated runs of the unchanged NUnit app ranged 16.7s–43.4s. This PR does not claim a CI speedup.

Known issues

• JCW under NativeAOT aborts with SIGABRT (RhFailFast) once JavaAbstractMethodTest starts, under both the original NUnit and the migrated MSTest app. This is pre-existing and not addressed here.
• MSTest.SourceGeneration still reports incomplete descriptors for some fixtures (generic helpers, inherited members) and retains a bounded reflection fallback; this is not "reflection-free" test execution.
• JavaMarshalGCBridgeTests.ManagedPeerCycleSurvivesJavaRootAndIsCollectedAfterRelease is intermittent under NativeAOT (missing global-to-weak bridge events): 1 of 3 local runs on this head failed it, the others passed. It was also seen intermittently earlier in development.
• Only API-35 arm64 with CoreCLR and NativeAOT was exercised; the Azure stages have not been run against this revision.