[michaldengusiak]

NOTE: Depends on

Nothing. Consumed by the SAM_UI draft PR on the same branch name (claude/tas-openstudio-comparison, SAM-BIM/SAM_UI#238 for #231). SAM_UI's CI selects this branch by name, so it builds against this library before merge. Keep this PR draft until the owner has run the Windows comparison test. Do not merge before that.

Issues addressed by this PR

Implements SAM-BIM/SAM#211 (sub-issue of SAM-BIM/SAM#209, tracker SAM-BIM/SAM#206). Issues are disabled in this repository.

• Library. SAM.Analytical.Benchmark.Compare is now a net8.0 class library, built to build\SAM.Analytical.Benchmark.Compare.dll.
  • It holds the existing Classes/Query/Modify sources: no logic change, no new dependency.
  • Format becomes public, so consumers show exactly the strings in the reports.
• CLI. benchmark-compare moves to SAM.Analytical.Benchmark.Compare.Cli, a thin wrapper (Program.cs moved unchanged). Its name, location in build\, options, outputs and exit codes are the same.
• Per-engine metric enumeration (70f1411), for SAM_UI's Run Simulation partial-completion view:
  • Query.ModelMetricValues(BenchmarkModelResult?) and Query.SpaceMetricValues(BenchmarkSpaceResult?) list one engine's metrics under the comparator's keys and order. An unreported metric has a null value; nothing is substituted.
  • The comparator now builds its rows from these lists, so the key list exists once. Goldens are unchanged.
• Public API used by SAM_UI and later SAM MCP:
  • Query.Compare, Query.IsInformationalForNumericalGate, Query.ModelMetricValues, Query.SpaceMetricValues;
  • Modify.WriteReports / WriteMarkdown / WriteCsv / WriteSummaryJson;
  • Format.*;
  • the result DTOs, ToleranceProfile and the enums.
• CI. The engine-reference check now covers both projects; the public-API tests run.

Test files

• SAM.Analytical.Benchmark.Compare.Tests/ExtractionRegressionTests.cs with the goldens in Fixtures/Regression/{fixtures,scenario,weather-mismatch}/.
  • The goldens were produced by the pre-extraction CLI at base c9a7f19, for all --fail-on-gate modes.
  • The library route and the wrapper CLI must reproduce the reports, exit codes and console output byte for byte.
• SAM.Analytical.Benchmark.Compare.ApiTests/PublicApiTests.cs sits outside InternalsVisibleTo, so it compiles only against the public contract. It includes the metric-enumeration keys and order matching the comparator rows.

Local results on Linux (.NET SDK 10.0.112, net8.0). Tested content SHA: 70f1411 (record-only commit after it).

Check	Result
Comparator tests before the change	PASS 87/87
Comparator tests after (87 existing + 6 regression), including after the enumeration refactor	PASS 93/93 (goldens byte-identical)
Public-API tests	PASS 2/2
benchmark-compare executable on the scenario case	Exit code and the 3 reports identical to golden (at b0f3a16)
Hosted build/test	See the checks on the current head

All test inputs are fabricated documents, not engine results.

Additional comments

• Target framework: stays net8.0. It is the existing target, so the move needs no source changes, and SAM_UI (net8.0-windows) consumes it directly.
• Golden Markdown files: they end with a blank line, which is the tool's verbatim output (pinned to LF by .gitattributes). git diff --check flags it, and that is expected.
• PR record: docs/benchmark/COMPARE-LIBRARY-PR.md. PROJECT_PROGRESS.md is updated only after merge.

---

Generated by Claude Code