[TrimmableTypeMap] Improve trimmable typemap build incrementality

[simonrozsival]

Summary

Makes the trimmable typemap build pipeline (_AndroidTypeMapImplementation=trimmable, used by CoreCLR and NativeAOT) fully incremental, skipped in design‑time builds, and self‑consistent on every rebuild. Extracted as a focused, well‑tested slice from the larger #11617, and implemented on top of main's current generator/task API rather than the runtime rewrite in that PR.

The bulk of this description is a deep dive into how incrementality is achieved, because the pipeline has several non‑obvious moving parts (two generator passes, content‑based writes, dynamic outputs vs. IncrementalClean, and a post‑link Java regeneration step).

---

Background: what the trimmable typemap generator produces

The legacy typemap implementations (llvm-ir, managed) embed the managed ↔ Java type mapping into native binaries. The trimmable implementation instead generates, from a scan of the app's assemblies:

• a set of small managed TypeMap assemblies — one _<Assembly>.TypeMap.dll per input assembly plus a _Microsoft.Android.TypeMaps.dll root — that are trimmer‑friendly (unused entries are removed by the IL linker), and
• the Java Callable Wrapper (*.java, "JCW") sources that Java needs to call back into managed code, plus acw-map.txt and a merged AndroidManifest.xml.

This is driven by the GenerateTrimmableTypeMap MSBuild task. The active runtime is selected by $(_AndroidRuntime) (CoreCLR or NativeAOT); runtime‑specific targets extend the shared pipeline (…TypeMap.Trimmable.CoreCLR.targets, …TypeMap.Trimmable.NativeAOT.targets).

---

The pipeline (target graph)

Shared / Debug CoreCLR & NativeAOT (no trimming)

CoreCompile
   └─► _GenerateTrimmableTypeMap            (AfterTargets="CoreCompile")
          • scans @(ReferencePath) + framework/SDK assemblies + the app .dll
          • writes typemap/_*.TypeMap.dll + _Microsoft.Android.TypeMaps.dll
          • writes typemap/java/**/*.java (JCWs), acw-map.txt, merged AndroidManifest.xml
          • prunes stale JCWs, then touches typemap/_GenerateTrimmableTypeMap.stamp
   …
_ReadGeneratedTrimmableTypeMapAssemblies    (reads typemap-assemblies.txt → items)
_PrepareTrimmableNativeConfigAssemblies     (feeds _GeneratePackageManagerJava)
_PrepareTrimmableTypeMapAssemblies          (feeds packaging / assembly store)
_GenerateJavaStubs                          (copies JCWs → android/src, manifest, acw-map)
   └─► _CompileJava ─► _CompileToDalvik ─► packaging

_GenerateJavaStubs overrides the legacy target of the same name from BuildOrder.targets; on the trimmable path the JCWs already exist, so it only copies them into $(IntermediateOutputPath)android/src and wires up the manifest/acw-map.txt/native config.

Release CoreCLR (PublishTrimmed=true) — two generator passes

CoreCompile ─► _GenerateTrimmableTypeMap            (pre-trim: full assembly set)
…
ILLink (trim) ─► CrossGen/R2R
   └─► _ComputePostTrimTrimmableTypeMapInputs        (collect existing linked .dll)
   └─► _GeneratePostTrimTrimmableTypeMapJavaSources   (post-trim)
          • regenerates JCWs from the *linked* assemblies only
          • writes typemap/linked-java/**/*.java (GenerateTypeMapAssemblies=false)
          • touches stamp/_GeneratePostTrimTrimmableTypeMapJavaSources.stamp
_GenerateJavaStubs                                    (copies linked-java → android/src)

The post‑trim pass exists because, after trimming, the set of types that still need JCWs is a subset of the pre‑trim set. _GenerateJavaStubs sources its JCWs from $(_TypeMapJavaStubsSourceDirectory), which is typemap/linked-java for CoreCLR + PublishTrimmed and typemap/java otherwise.

NativeAOT (PublishTrimmed=true as well) does not use the post‑trim Java pass — it feeds the pre‑trim JCWs to ILC and its own native steps — so for NativeAOT the sources stay typemap/java.

---

How incrementality is achieved (deep dive)

The pipeline follows the repo's MSBuild best practices: every expensive target declares Inputs/Outputs, re‑emits its dynamic FileWrites, and uses stamp files where a real output can't serve as a reliable timestamp sentinel. There are six principles at work.

1. Stamp files are the incremental sentinels — because the real outputs are content‑addressed

GenerateTrimmableTypeMap writes every output with Files.CopyIfStreamChanged (and the per‑assembly WriteAssembliesToDisk additionally compares timestamps): an output whose content did not change keeps its old timestamp. That's exactly what you want to avoid churning downstream Java/native compilation — but it makes those files unusable as MSBuild Outputs, because their timestamps can be older than the Inputs even on a successful run, which makes MSBuild consider the target perpetually out‑of‑date and re‑run it every build.

So _GenerateTrimmableTypeMap declares a dedicated stamp as its sole output and touches only that stamp:

Inputs="@(ReferencePath);@(PrivateSdkAssemblies);@(FrameworkAssemblies);$(IntermediateOutputPath)$(TargetFileName);$(_AndroidManifestAbs);$(_AndroidBuildPropertiesCache)"
Outputs="$(_TrimmableTypeMapOutputStamp)"
…
<Touch Files="$(_TrimmableTypeMapOutputStamp)" AlwaysCreate="true" />

The stamp is always touched on a run and never touched on a skip, so the target is correctly skipped iff none of its inputs changed. The generated DLLs and typemap-assemblies.txt are not touched and are not declared as outputs — they are content‑addressed, and no other target consumes them as timestamp Inputs (verified), so touching them would be pure churn. (This is the @jonathanpeppers review point.)

The Inputs deliberately include @(PrivateSdkAssemblies) and @(FrameworkAssemblies): they can contribute managed ↔ Java mappings, so a change in them must re‑run generation.

_GenerateJavaStubs uses the analogous idea, but its sentinel must reflect whichever pass produced the JCWs it copies:

Inputs="$(_TrimmableJavaSourceStamp);@(_EnvironmentFiles)"

where _TrimmableJavaSourceStamp resolves to the post‑trim stamp for CoreCLR + PublishTrimmed, and the pre‑trim generator stamp otherwise. This is what lets _GenerateJavaStubs react to post‑trim JCW regeneration while staying incremental on no‑op builds. (This is the @Copilot review point; the non‑trim default was previously the TypeMap DLL, whose timestamp is unreliable for the reason above.)

2. Inputs must reference files that actually exist

MSBuild treats a non‑existent Inputs file as "out of date" and runs the target. The post‑trim pass originally declared Inputs="@(ResolvedFileToPublish)", which also lists non‑assembly publish outputs — e.g. …/bin/Release/<rid>/UnnamedProject.runtimeconfig.json — whose paths do not exist when the target runs. That single phantom input made _GeneratePostTrimTrimmableTypeMapJavaSources run on every build, wiping and regenerating linked-java each time.

The fix factors input collection into _ComputePostTrimTrimmableTypeMapInputs, which keeps only .dll items that exist on disk:

<_PostTrimTrimmableTypeMapInputAssemblies Include="@(ResolvedFileToPublish)"
    Condition=" '%(Extension)' == '.dll' and Exists('%(FullPath)') and (…RID filter…) " />

so the post‑trim pass is now genuinely incremental and its stamp is a trustworthy sentinel for _GenerateJavaStubs.

3. Dynamic outputs must be re‑published to @(FileWrites) on skipped builds

The set of generated assemblies and JCWs is data‑dependent (it comes from scanning), so the ItemGroups that register those files into @(FileWrites) live inside the generating targets — and therefore don't execute on a no‑op build where those targets are skipped. If nothing re‑declares them, MSBuild's IncrementalClean sees the previously‑tracked files as orphaned and deletes them before packaging reads its inputs.

_RecordTrimmableTypeMapFileWrites runs unconditionally (gated only on the assemblies‑list file existing) and re‑emits the previous dynamic outputs back into @(FileWrites) before IncrementalClean:

• the generated TypeMap assemblies (read back from typemap-assemblies.txt),
• the pre‑trim JCWs (typemap/java) and their android/src copies,
• the post‑trim JCWs (typemap/linked-java) and their android/src copies, plus the post‑trim stamp, and
• the merged manifest, acw-map.txt, ApplicationRegistration.java, and the generator stamp.

The post‑trim entries are new in this PR: once _GeneratePostTrimTrimmableTypeMapJavaSources became skippable (principle 2), its linked-java outputs would otherwise be deleted by IncrementalClean on the builds where it's skipped. The globs are empty for configurations that have no post‑trim pass, so this is a no‑op there.

Two more targets — _PrepareTrimmableNativeConfigAssemblies and _PrepareTrimmableTypeMapAssemblies — exist for the same "skipped target leaves item groups empty" reason: they re‑populate the TypeMap‑assembly item groups (used by packaging, the assembly store, and native config) from the on‑disk list via BeforeTargets, so incremental builds that skip generation still package correctly.

4. Content‑based writes + explicit stale pruning keep android/src exact

Because generation is content‑addressed (CopyIfStreamChanged) and the android/src copy uses SkipUnchangedFiles="true", an unchanged JCW never updates a timestamp and never triggers Java recompilation.

The flip side is removal: when a JCW disappears between builds — a type deleted from source, or trimmed away on the PublishTrimmed path — its android/src copy must not linger, or it would be compiled and packaged. Both generator passes report the JCWs they no longer produce as DeletedJavaFiles; the owning target mirrors each deletion into the android/src copy and, if anything was deleted, deletes $(_AndroidCompileJavaStampFile) so _CompileJava re‑runs and drops the stale .class:

<Delete Files="@(_DeletedCopiedJavaFiles)" />
<Delete Files="$(_AndroidCompileJavaStampFile)" Condition=" '@(_DeletedCopiedJavaFiles->Count())' != '0' " />

The two passes compute the deleted set differently because of how each manages its output directory:

• Pre‑trim (_GenerateTrimmableTypeMap, writing typemap/java): the task scans the output dir and deletes any *.java the current pass didn't produce.
• Post‑trim (_GeneratePostTrimTrimmableTypeMapJavaSources, writing typemap/linked-java with CleanJavaSourceOutputDirectory=true): the dir is wiped before regeneration, so the task snapshots the previous *.java set before the wipe and reports previous − regenerated. This keeps the deletion precise — only files the generator itself previously produced are ever removed from android/src, never unrelated sources like ApplicationRegistration.java.

The net invariant is two‑directional: android/src contains exactly the JCWs the active pass produces (typemap/linked-java for PublishTrimmed, typemap/java otherwise) — no missing files (copied by _GenerateJavaStubs) and no stale files (pruned via DeletedJavaFiles). Exercised directly by the …KeepsAndroidSrcConsistentWithLinkedJava and …DeletesStaleAndroidSrcWhenLinkedJavaShrinks tests.

5. The generator is skipped in design‑time builds and runs exactly once

_GenerateTrimmableTypeMap is gated on '$(DesignTimeBuild)' != 'true': in a design‑time build, project references may resolve to target paths that aren't produced when SkipCompilerExecution=true, and the output isn't needed for IDE information. Combined with the existing '$(_OuterIntermediateOutputPath)' == '' guard (which skips inner per‑RID builds), the generator runs exactly once per outer build.

6. Two‑pass consistency under trimming

For CoreCLR + PublishTrimmed, correctness depends on the pre‑trim and post‑trim sentinels composing cleanly:

• No‑op rebuild — both passes are skipped (their inputs are unchanged); _GenerateJavaStubs keys off the stable post‑trim stamp and is also skipped. (Newly true; previously the post‑trim pass ran every build.)
• Managed source change — the pre‑trim generator re‑runs (its inputs changed) and re‑linking changes the linked assemblies, so the post‑trim pass re‑runs too; _GenerateJavaStubs re‑copies.
• Trim‑only change (e.g. a change that alters the linked output without touching the pre‑trim inputs) — the linked .dll change re‑runs the post‑trim pass, which touches the post‑trim stamp, so _GenerateJavaStubs re‑runs even though the pre‑trim generator did not. This is precisely the staleness the @Copilot review flagged, and is why _GenerateJavaStubs must key off _TrimmableJavaSourceStamp rather than the generator stamp alone.

---

What changed in this PR

Area	Change
…TypeMap.Trimmable.targets	Stamp sentinel for _GenerateTrimmableTypeMap (sole Output, sole Touch); expanded Inputs (PrivateSdkAssemblies/FrameworkAssemblies); design‑time‑build skip; SkipUnchangedFiles JCW copy; stale‑JCW deletion wiring; _GenerateJavaStubs keyed off _TrimmableJavaSourceStamp; post‑trim outputs re‑emitted in _RecordTrimmableTypeMapFileWrites.
…TypeMap.Trimmable.CoreCLR.targets	New _ComputePostTrimTrimmableTypeMapInputs (existing‑.dll inputs) so _GeneratePostTrimTrimmableTypeMapJavaSources is incremental; post‑trim pass now mirrors DeletedJavaFiles into android/src + busts the Java compile stamp.
Tasks/GenerateTrimmableTypeMap.cs	New DeletedJavaFiles output + DeleteStaleJavaSources(); in the CleanJavaSourceOutputDirectory (post‑trim) case it snapshots the prior JCW set before the wipe and reports previous − regenerated.
Microsoft.Android.Sdk.TrimmableTypeMap/README.md	Documents the pipeline and its incrementality design.
TrimmableTypeMapBuildTests.cs	Tests for stale‑JCW pruning (both directions), updated‑JCW copying, the android/src ↔ linked-java invariant, post‑trim no‑op incrementality, and stale android/src removal when linked-java shrinks.

Review feedback addressed

• @copilot (×2): _GenerateJavaStubs now keys off $(_TrimmableJavaSourceStamp) (post‑trim stamp for PublishTrimmed, generator stamp otherwise), and the non‑trim default of _TrimmableJavaSourceStamp was changed from the TypeMap DLL to the generator stamp. The underlying reason the bare generator stamp was insufficient — a non‑incremental post‑trim pass — is fixed too.
• @jonathanpeppers: _GenerateTrimmableTypeMap declares only the stamp as its Outputs and touches only the stamp; the DLLs/list are content‑addressed and not consumed as timestamp inputs elsewhere.

---

Scope notes (and a little beyond)

• All changes are gated on _AndroidTypeMapImplementation == 'trimmable'; non‑trimmable (llvm-ir/managed) builds are unaffected.
• Unlike [TrimmableTypeMap] Implement reflection-free TrimmableTypeMapValueManager and TrimmableTypeMapTypeManager #11617, this PR keeps main's CoreCLR post‑trim linked-java machinery intact and instead makes it incremental.
• Beyond this PR, the post‑trim pass still regenerates linked-java wholesale (CleanJavaSourceOutputDirectory=true) when it runs; it now also reports the JCWs it dropped so android/src stays exact. A further optimization would replace the wholesale clean with content‑based writes + in‑place stale pruning (as the pre‑trim pass does), letting unchanged linked-java JCWs keep stable timestamps across a post‑trim run. Left as a follow‑up.

---

Testing

Verified locally against a Debug local SDK.

Host (Xamarin.Android.Build.Tests, full TrimmableTypeMapBuildTests): 24 passed, 2 skipped, 0 failed, including:

• …IncrementalBuild (Debug CoreCLR, Release CoreCLR, Release NativeAOT)
• …DeletesStaleGeneratedJavaSourcesAndCopies, …CopiesUpdatedGeneratedJavaSources
• …PublishTrimmed_KeepsAndroidSrcConsistentWithLinkedJava (the android/src ↔ linked-java invariant)
• …PublishTrimmed_DeletesStaleAndroidSrcWhenLinkedJavaShrinks (a JCW dropped from linked-java is removed from android/src)
• …PublishTrimmed_PostTrimJavaGenerationIsIncremental (no‑op rebuild skips post‑trim and _GenerateJavaStubs)
• …Succeeds, …ArrayRankChangeRegeneratesTypeMap, …DoesNotHitCopyIfChangedMismatch, R2R/multi‑RID packaging tests

Emulator (MSBuildDeviceIntegration, API 36 / arm64): DotNetRun deploy+run for trimmable CoreCLR (Debug + Release) and NativeAOT (Release), and TrimmableTypeMapInheritedVirtualOverrideUsesCorrectUco.

[Copilot]

Pull request overview

This PR improves MSBuild incrementality for the trimmable typemap pipeline (_AndroidTypeMapImplementation=trimmable) by introducing a stable stamp-based incremental sentinel, expanding the generator’s declared inputs, and ensuring stale generated Java artifacts are removed so downstream Java compilation stays correct.

Changes:

• Add new build tests covering stale-Java deletion and updated-Java copying behavior for the trimmable typemap pipeline.
• Extend GenerateTrimmableTypeMap to report deleted stale generated *.java sources (DeletedJavaFiles) and delete them from the generator output directory.
• Update Microsoft.Android.Sdk.TypeMap.Trimmable.targets to:
  • Use a dedicated generator stamp as an incremental sentinel,
  • Include additional assembly inputs (@(PrivateSdkAssemblies), @(FrameworkAssemblies)),
  • Mirror deleted Java sources into android/src and force Java recompilation when needed,
  • Copy JCWs with SkipUnchangedFiles="true".
• Add a README describing the pipeline and incrementality design.

Show a summary per file

File	Description
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/TrimmableTypeMapBuildTests.cs	Adds tests validating incremental behavior, Java source copying, and stale Java/class cleanup.
src/Xamarin.Android.Build.Tasks/Tasks/GenerateTrimmableTypeMap.cs	Adds stale generated Java cleanup and exposes deletions via a new MSBuild output item group.
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.targets	Implements stamp-based incrementality, expands generator inputs, mirrors deletions into android/src, and improves copy incrementality.
src/Microsoft.Android.Sdk.TrimmableTypeMap/README.md	Documents the trimmable typemap pipeline and its incrementality approach.

Copilot's findings

• Files reviewed: 4/4 changed files
• Comments generated: 2

[jonathanpeppers]

Is it possible to only touch the $(_TrimmableTypeMapOutputStamp) file and make it the only Output?

Or are @(_GeneratedTypeMapAssemblies);$(_TypeMapAssembliesListFile) Inputs to another target? -- that would be the only reason to touch so many files.

[simonrozsival]

Good call — done in 8a70372. _GenerateTrimmableTypeMap now declares only $(_TrimmableTypeMapOutputStamp) as its Outputs and touches only that stamp. The generated TypeMap DLLs and typemap-assemblies.txt are written via CopyIfStreamChanged (stable timestamps when unchanged) and are not consumed as timestamp Inputs by any other target, so they no longer need to be touched.

[simonrozsival]

/review

[simonrozsival]

✅ LGTM

Reviewed the full PR (6 commits) against the repo's MSBuild incremental-build and task conventions, and cross-checked with an independent review pass. The incrementality design is sound and well-tested.

Verified correct:

• Stamp sentinels (_TrimmableTypeMapOutputStamp, _PostTrimTrimmableTypeMapJavaStamp) are touched only when their producing target runs, so content-addressed outputs (CopyIfStreamChanged) can't make targets perpetually out-of-date; _GenerateJavaStubs keys off the conditional _TrimmableJavaSourceStamp (post-trim stamp for CoreCLR+PublishTrimmed, generator stamp otherwise).
• IncrementalClean re-emission — _RecordTrimmableTypeMapFileWrites re-globs both typemap/java and post-trim linked-java (plus their android/src copies) so skipped builds don't orphan dynamic outputs.
• Two-directional android/src invariant — missing JCWs are copied; stale JCWs (incl. trimmed-away types on the post-trim path, via the pre-wipe snapshot diff) are pruned and the Java compile stamp is busted so _CompileJava drops the stale .class.
• PE file handles disposed in finally; the FD exhaustion seen locally was environmental, not this code.

Issue counts: ❌ 0 · ⚠️ 0 · 💡 1 (one latent safety hardening, inline).

CI is green (40/40 checks). Nice work on the TDD discipline — each fix landed with a red→green regression test.

[simonrozsival]

🤖 💡 Error handling — The Log.HasLoggedErrors early-out that protects the snapshot/clean branch above is missing from this directory-scan branch, which performs an irreversible File.Delete. If GeneratedJavaFiles were ever incomplete here — e.g. a future caller using CopyJavaSourcesFromInputDirectory (which logs XA4255 and continues past a missing source) without CleanJavaSourceOutputDirectory — a still-valid generated source would be physically deleted while the build is already failing. Today this is unreachable (JavaSourceInputDirectory is only ever set together with CleanJavaSourceOutputDirectory=true, so this branch always pairs with WriteJavaSourcesToDisk, which writes every source and logs nothing), so it's purely latent. Consider hoisting if (Log.HasLoggedErrors) return []; to the top of the method so both branches share the guard before any deletion.

Rule: Assert boundary invariants (Postmortem #44)