Phase 1: Unified "Run Simulation" (Tas / OpenStudio / Both) on one frozen snapshot + Tas/OpenStudio comparison report and lightweight view (gate G2) #231

Parent: #227 · Implementation: draft PR #238 (stacked on #237), with SAM-BIM/SAM_Validation#14.

Goal

Give SAM_UI one "Run Simulation" workflow with three choices: Tas only, OpenStudio/EnergyPlus only, or Both: Tas + OpenStudio/EnergyPlus.

For Both:

• one immutable model and weather snapshot;
• the engines run one after the other;
• each engine keeps its own outputs;
• SAM_Validation's comparison runs automatically when both results are available;
• a lightweight WPF comparison window opens with the results, diagnostics, input provenance and reports.

The view only presents SAM_Validation's comparison; it adds no new comparison engine. This issue is part of the current end-to-end validation milestone (Phase 1), not Phase 2 parallel execution.

Context / reuse

• Snapshot hash: canonicalModelHash and sourceFileHash from SAM_Validation SAM.Analytical.Benchmark (canonical JSON v1). Each run's snapshot file also gets its own SHA-256.
• B1a documents: written by the validated producers, benchmark-tas (SAM_Tas) and benchmark-openstudio (SAM_OpenStudio). SAM_UI runs them as child processes; no second schema writer or results-to-B1a mapping. A later move to in-process builders goes through [SAM_Tas] Phase 1: Reusable B1a document builder and TBD input read-back for the SAM_UI engine comparison SAM#208 (Tas) and an equivalent SAM_OpenStudio change.
• Output comparison: SAM_Validation SAM.Analytical.Benchmark.Compare, the same logic as benchmark-compare:
  • Query.Compare → ComparisonResult (metric rows with availability, values, absolute/relative/signed differences, Match/Warn/Fail/NotApplicable bands, notes, informational; space alignment; provenance compatibility; gates);
  • Format.* renders every value;
  • Modify.WriteReports writes comparison.md/.csv/comparison-summary.json.
  • Decided: the reusable-library approach, [SAM_Validation] Extract the benchmark comparator into a reusable .NET library (CLI stays a thin wrapper) SAM#211 (prerequisite). SAM_UI references SAM_Validation\build\SAM.Analytical.Benchmark.Compare.dll. No difference, band, alignment or gate is computed in SAM_UI.
  • For a run where only one engine succeeded, the view lists that engine's B1a metrics through SAM_Validation's public metric enumeration (Query.ModelMetricValues / SpaceMetricValues). This is the same key order the comparator uses, and it involves no comparison.
• Input comparison: the input manifests from Input manifest: per-space record of what EnergyPlus actually received (for Tas comparison) SAM_OpenStudio#30 and Phase 1: Tas input manifest: per-space record of what Tas actually received #230. They are not implemented yet; the view states this.
• Run records: openstudio-run.json from Phase 1: Minimal OpenStudio/EnergyPlus run from SAM_UI without disrupting Tas (gate G1, first PR) #229 (OpenStudio only), and engine-comparison-run.json from this workflow (Both).

Scope

A. Unified "Run Simulation" command

A new Simulate > Run Simulation ribbon command opens a small dialog with three choices. Each choice says what it runs and needs, and whether the producers were found:

Both: rules

• Immutable snapshot. snapshot.json and weather.epw are written once and marked read-only. Their SHA-256 is checked before each engine starts and before comparing; a changed file stops the run with a diagnostic. Edits to the open model during the run never reach it.
• Independent outputs. Each engine writes only to its own working folder, document and log. One engine's failure never deletes or changes the other's outputs.
• Statuses:
  • Compared: both producers succeeded and SAM_Validation compared the documents.
  • Partially completed: exactly one engine produced a successful B1a document. Its results are kept and shown alone, the failed engine's error and log are shown, and no comparison is made.
  • Failed: neither succeeded.
  • Cancelled.
• Never manufacture results.
  • A failed engine's metrics are never filled in, estimated or shown as 0.
  • A document whose state is Failure, or from a producer that exited non-zero, is not compared.
  • Not-equivalent (informational) and unit-mismatch metrics are never coloured or counted as passing.
• Execution order. The engines run one after the other in this milestone. Running them in parallel stays in Phase 2 (Phase 2: SAM_Multitasker assessment and background job-runner design (Tas COM/STA + licence constraints) #234/Phase 2: Parallel Tas + OpenStudio runs from SAM_UI with jobs panel, progress and cancel (gate G4) #235).

Migration: no workflow is removed or broken.

• Simulate > Energy Simulation (Tas) stays as is. Run Simulation > Tas only calls exactly the same code path.
• Simulate > OpenStudio Simulation (Phase 1: Minimal OpenStudio/EnergyPlus run from SAM_UI without disrupting Tas (gate G1, first PR) #229) stays. Run Simulation > OpenStudio only calls the same path.
• Compare Tas / OpenStudio stays as the place to open existing comparison run folders or B1a documents.
• The tooltips of the existing commands point to Run Simulation. Retiring any of them is a separate, later owner decision.

B. Report

The report is Markdown, CSV and JSON written by SAM_Validation's WriteReports, plus the provenance and assumption sections in the view. It covers:

• Inputs side by side: SAM vs Tas vs OpenStudio, with units and the tolerance band for each.
• Results side by side: model level and per space (identified by SAM GUID and name), with units.
• Simulation assumptions: route, engine versions, design-day source, conditioning pairing, peak definition (OpenStudio coincident vs Tas building-profile maximum), the producers' workflow settings.
• Metrics missing on either side, each with a reason.
• Diagnostics: EnergyPlus warning/severe/fatal counts and the head of eplusout.err; Tas warnings; producer exit codes and logs.

C. Lightweight WPF comparison view (Phase 1)

A single window built from standard WPF controls. It opens automatically after Run Simulation > Both, and from Compare Tas / OpenStudio for existing runs.

1. Run selection and status.
   • Load a comparison run folder or two B1a documents; show each run's model, snapshot hash, weather source and hash, engine version and timestamp.
   • A banner gives the run status: Compared / Partially completed / Failed / Cancelled.
   • It also shows NOT THE SAME INPUTS when SAM_Validation's provenance gate finds mismatches.
2. Model-level results grid: metric, unit, Tas value, OpenStudio value, absolute difference, percentage difference, band. All values come from ComparisonResult.
3. Per-space results grid: a space filter (GUID + name, aligned by AlignSpaces) with the same columns. Unmatched spaces are listed with their alignment diagnostic.
4. Missing, non-equivalent and unavailable metrics are always visible, never hidden as zero:
   • Missing (TasOnly/OpenStudioOnly): Missing on Tas / Missing on OpenStudio, with no difference.
   • Not equivalent (informational metrics, unit mismatch): Not equivalent, with SAM_Validation's note, and excluded from pass/fail styling.
   • Unavailable (neither engine reports it): Not reported by either engine.
   • Not compared (partial completion): the successful engine's values only, labelled Not compared – <engine> did not complete.
   • Band colouring applies to compared rows only, with the profile shown. A band is not an engineering verdict; that belongs to Phase 1: Benchmark cases, input-equivalence tolerances and engineering acceptance criteria #232/Phase 1: Investigate and classify Tas vs OpenStudio differences on benchmark cases (gate G3) #233.
5. Input-provenance panel: the provenance gate status and mismatches, snapshot and weather hashes, design-day source, and the input-manifest status per engine (SAM_OpenStudio#30 / Phase 1: Tas input manifest: per-space record of what Tas actually received #230; not available until implemented).
6. Diagnostics panel:
   • gate statuses and counts; unmatched, duplicate and ambiguous spaces; model-total reconciliation;
   • B1a warnings/notes;
   • each producer's exit code, duration, error and log;
   • for OpenStudio, warning/severe/fatal counts.
7. Report actions: Open report (comparison.md), Export… (all three files through WriteReports), Open run folder.

The view is read-only: it never changes the open model or any run folder.

Not in Phase 1: charts, time-series plots, dashboards or charting/dashboard frameworks, editing tolerance profiles in the UI, multi-run trend views, parallel engine runs.

Out of scope

Parallel execution (#234/#235); deciding pass/fail for engineering acceptance (#232/#233); any comparison logic outside SAM_Validation; removing existing simulation commands.

Depends on

• Phase 1: Reproducible Tas baseline from SAM_UI with no behaviour change (gate G0) #228, Phase 1: Minimal OpenStudio/EnergyPlus run from SAM_UI without disrupting Tas (gate G1, first PR) #229 (Simulate > OpenStudio Simulation: minimal OpenStudio/EnergyPlus run from SAM_UI (#229) #237), Phase 1: Tas input manifest: per-space record of what Tas actually received #230, Input manifest: per-space record of what EnergyPlus actually received (for Tas comparison) SAM_OpenStudio#30.
• [SAM_Tas] Phase 1: Reusable B1a document builder and TBD input read-back for the SAM_UI engine comparison SAM#208 (Tas B1a builder; optional while the producers run out of process).
• [SAM_Validation] Extract the benchmark comparator into a reusable .NET library (CLI stays a thin wrapper) SAM#211, SAM_Validation#14 (comparator library): a prerequisite.

The view's provenance panel uses the tolerances from #232 / SAM-BIM/SAM#209 once they exist.

Acceptance criteria

• Run Simulation:
  • Tas only behaves exactly like Energy Simulation: same dialog, results, outputs and Part O handling.
  • OpenStudio only behaves exactly like Phase 1: Minimal OpenStudio/EnergyPlus run from SAM_UI without disrupting Tas (gate G1, first PR) #229.
  • Both runs both engines one after the other on one read-only, hash-checked snapshot and weather file, and opens the comparison window when it finishes.
• Both:
  • Both B1a documents carry the same canonicalModelHash and weather hash, and the provenance gate passes.
  • If one engine fails, the other engine's outputs and B1a document are kept and shown, the status is Partially completed, and no comparison or report pretends otherwise.
  • Running twice on an unchanged model reproduces identical inputs (snapshot and weather hashes) and comparable results.
• Every metric in the schema is either compared or listed as missing, not equivalent or not reported, with a reason.
• View:
  • For the same two B1a files, every value, difference, band and availability equals SAM_Validation's comparison.csv, which shows that SAM_UI computes nothing.
  • Missing, non-equivalent and not-compared metrics are never shown as 0 or styled as passing.
  • A provenance mismatch shows the banner.
  • Export writes the three report files.
  • The view never changes the open model or any run folder.
• The existing Energy Simulation, OpenStudio Simulation and Compare commands still work unchanged.

Validation gate G2