Repository navigation
Phase 1: Benchmark cases, input-equivalence tolerances and engineering acceptance criteria #232
Description
Parent: #227
Goal
Define reproducible benchmark models and engineering acceptance criteria for comparing Tas and OpenStudio, without demanding an arbitrary percentage agreement between two different engines.
Cases
- Case A, single zone (Tas-legal). SingleBox is illegal in Tas, so use a legal construction set. Simple geometry and one glazed façade. Fixed gains, a constant setpoint, constant infiltration.
- Independent reference calculation at the heating design condition, committed as a spreadsheet or script: steady state, Φ = (Σ U·A + ρ·c_p·V̇)·ΔT, with ΔT, film coefficients and ground treatment stated.
- Case B, representative multizone. Mixed orientations, internal partitions, a schedule-driven occupancy. Candidates: HungaryHouse (13 spaces, already run in Tas) with weather that matches its location, or a SAM_samples model.
- Each case records provenance, model SHA256, EPW and its hash, and the design-day source. Corpus home: align with SAM_Validation milestone B4.
Input-equivalence tolerances (SAM ↔ Tas ↔ OpenStudio manifests)
| Quantity | Tolerance |
|---|---|
| Floor area, volume, surface areas | ≤ 0.5 % |
| Construction U-value, g-value | ≤ 1 % (film assumptions stated) |
| Gains densities, setpoints | exact (≤ 0.1 %) |
| Schedule annual full-load hours | ≤ 0.1 % |
| Weather file / design days | identical hash and names |
Output acceptance (engineering, not a percentage target)
- Case A peak heating from each engine lies within a documented band around the steady-state reference. The band is justified by the dynamic effects excluded from the steady-state reference: solar, thermal mass, and the timestep.
- Signs and magnitudes are sane: no negative loads, annual energy consistent with peak × plausible equivalent full-load hours, unmet hours reported.
- Case B: the ranking of zones by peak heating and peak cooling load is consistent between engines, or each difference is explained.
- Every Warn/Fail band from
benchmark-compare(default profile: under 5 % Match, 5–15 % Warn, 15 % or more Fail) has a classified root cause (see the difference investigation issue). An unexplained Fail blocks G3. An explained methodology difference does not. - Known methodology differences are listed up front: coincident vs building-profile peak, ideal loads vs Tas plant-free sizing, solar distribution, ground model, convection algorithms.
Depends on
#231 (to run). The case definitions can start now.
Acceptance criteria
- Cases A and B are committed (or linked with hashes), with the reference calculation and the tolerance table.
- The tolerance values are agreed by the owner before G3.
Activity