Dogfood --report-azdo-summary and --report-azdo-progress in CI

[Evangelink]

Summary

Wires the recently-merged --report-azdo-summary and --report-azdo-progress AzureDevOpsReport options into both testfx CI test paths so we dogfood them on every run.

Both options are gated on TF_BUILD=true (AzureDevOpsSummaryReporter.cs#L99, AzureDevOpsProgressReporter.cs#L97) so they are a no-op for local dotnet test runs and only emit ##vso[...] commands inside an AzDO pipeline. Smoke-tested locally with Microsoft.Testing.Platform.UnitTests to confirm the options are accepted and produce the expected …skipping… notice outside AzDO.

Changes

• test/Directory.Build.targets — extends the always-on --report-azdo injection (Debug -p:UsingDotNetTest=true path).
• azure-pipelines.yml — extends the Release --test-modules script (Windows).
• eng/pipelines/steps/test-non-windows.yml — same extension for the Linux/macOS template.

Deferred to follow-up PRs

Considered (and rejected for now) — happy to do these as separate PRs if there is interest:

Option	Why deferred
--report-html	API is [Experimental(""TPEXP"")] and the test apps disable GenerateTestingPlatformEntryPoint, so each of ~15 Program.cs would need an explicit AddHtmlReportProvider() call plus a TPEXP suppression. Worth doing, but a different change.
--report-azdo-upload-artifacts files	All modules share one --results-directory, so the uploader's GetFiles(..., AllDirectories) would emit N² uploads. Needs per-module --results-directory isolation or precise include patterns first.
--minimum-expected-tests	Redundant — MTP already returns ExitCode.ZeroTests when zero tests run.
--retry-failed-tests	Changes test semantics; should be rolled out deliberately.
--publish-azdo-* / --report-azdo-flaky-history / --report-azdo-quarantine-file	Larger Arcade-integration work.

[Copilot]

Pull request overview

This PR wires the Azure DevOps Report extension’s new --report-azdo-progress and --report-azdo-summary switches into the repo’s CI test invocations so these reporters are exercised on every pipeline run (Debug -p:UsingDotNetTest=true path and Release --test-modules path).

Changes:

• Inject --report-azdo-progress and --report-azdo-summary into the common TestingPlatformCommandLineArguments for test projects.
• Extend the Windows (azure-pipelines.yml) and non-Windows (eng template) Release test scripts to pass the new switches.
• Add inline documentation in the pipeline YAMLs describing the new switches.

Show a summary per file

File	Description
test/Directory.Build.targets	Adds --report-azdo-progress and --report-azdo-summary to the standard test-host command line args (alongside --report-azdo).
eng/pipelines/steps/test-non-windows.yml	Updates Release dotnet test --test-modules invocation to pass the new AzDO reporter switches and documents them.
azure-pipelines.yml	Updates Windows Release dotnet test --test-modules invocation to pass the new AzDO reporter switches and documents them.

Copilot's findings

• Files reviewed: 3/3 changed files
• Comments generated: 2

[Evangelink]

🧪 Test quality grade — PR #8934

No new or modified test methods were identified in the changed regions
of this PR. Nothing to grade.

_{Re-run with /grade-tests.}

Generated by Grade Tests on PR (on open / sync) for issue #8934 · sonnet46 659.1K · ◷

[Evangelink]

Note

🤖 This review was generated by GitHub Copilot (workflow run).

#	Dimension	Verdict
20	Build Infrastructure	🟡 1 MODERATE

✅ 21/22 dimensions clean (dimensions 1–14, 16, 18–19, 21–22 are N/A or clean for this purely CI-config change).

• Build Infrastructure — --report-azdo-progress and --report-azdo-summary emit output-device warnings when TF_BUILD is absent; injecting them unconditionally in test/Directory.Build.targets adds warning noise to every local dotnet test run.

---

Overall: The intent is clear and the implementation is correct for CI. The three-file change is tightly scoped, comments are accurate in their CI-only context, and parity between the Windows and Linux/macOS Release paths is maintained. The single concern is the local-developer experience regression described in the inline comment.

Generated by Expert Code Review (on open) for issue #8934 · sonnet46 3.5M

[Evangelink]

[MODERATE] Build Infrastructure

Unlike --report-azdo (which is a pure IDataConsumer with no session lifecycle hooks and is truly silent outside AzDO), both --report-azdo-progress and --report-azdo-summary implement ITestSessionLifetimeHandler. Their OnTestSessionStartingAsync emits a warning on the output device whenever the option is set but TF_BUILD is not "true":

// AzureDevOpsSummaryReporter.OnTestSessionStartingAsync — fires every local run:
await _outputDevice.DisplayAsync(this, new WarningMessageOutputDeviceData(AzureDevOpsResources.SummaryRequiresTfBuildWarning), ...);

// AzureDevOpsProgressReporter.OnTestSessionStartingAsync — same:
await _outputDevice.DisplayAsync(this, new WarningMessageOutputDeviceData(AzureDevOpsResources.ProgressRequiresTfBuildWarning), ...);

Because this PropertyGroup is unconditional, every test project will emit two extra warnings on every local dotnet test run. With the number of test projects in this repo, that adds up to significant console noise that developers may find confusing.

Recommendation: Consider gating the two new options on the TF_BUILD environment variable at the MSBuild level, the same way --crashdump is gated on the TFM:

<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --report-azdo</TestingPlatformCommandLineArguments>
<TestingPlatformCommandLineArguments Condition=" '$(TF_BUILD)' == 'true' ">$(TestingPlatformCommandLineArguments) --report-azdo-progress --report-azdo-summary</TestingPlatformCommandLineArguments>

MSBuild inherits environment variables as properties, so $(TF_BUILD) resolves correctly inside an AzDO build. This keeps --report-azdo (which is silent outside AzDO) on its own unconditional line and limits the two noisier options to CI runs only.