[michaldengusiak]

S11 OpenStudio results round-trip validation: Revision 1 + review correction + F-S11-1/F-S11-2 integration (offline) + Revision 2 attempt (BLOCKED)

Owner-authorised work. No Ready, no merge, no S12, no Phase B acceptance, no SAM_UI run. Record: documentation/S11-OpenStudio-ResultsRoundTrip-PR.md.

Phase B remains subject to S10's declared validation limitation (SAM|FixB|Heating|designDayPeak.hourOfDay BLOCKED); an S11 PASS is not a Phase B PASS.

Revision 2 attempt (2026-10-26): BLOCKED before engine execution (local)

• Engine attempts used: 0 of 2. The run phase stopped in its preflight (exit 70) before any benchmark-openstudio, EnergyPlus or OpenStudio process started.
  • No SQL, ESO, B1a or sam.json was produced, and no readback, compare or negative ran.
  • So no acceptance criterion has been evaluated on real data: completeness, comparison rows, SAM-P/SAM-I/B1a agreement, closure, negatives and OD-7 are all unevaluated.
• Build (local, succeeded):
  • fresh github.com/SAM-BIM clones at SAM_OpenStudio 01344e65, SAM 457defbb, SAM_Tas 922044fa, SAM_Validation 2a3acfaf and SAM_SQLite ec2eb86d, mapped temporarily as R: for path length;
  • APPDATA, TEMP and the NuGet caches redirected; no Grasshopper build or deployment;
  • the pinned SAM.Core.SQLite (3d1d9033…) built first, then Build-S11-Receipt.ps1: exit 0, 0 errors, receipt c057cf47… (51 files, four clean HEADs).
• Inputs (verified): fixtures 45524172… / 20da9b58…; Budapest EPW 472ffb4e…; portable CLI 3.10.0+86d7e215a1 (--version checked; never started for a simulation).
• Stop: Run-S11-Results.ps1 -Phase run … -ConfirmEngineRuns reported STOPPED: R:\SAM is not clean at the receipt commit 457defbb…. The checkout was actually clean.
• Cause: a driver defect.
  • Run-S11-Results.ps1:103-106 defines function Git { … & git … }. PowerShell resolves git to the function itself, so the helper recurses until "The script failed due to call depth overflow". That error is swallowed by 2>$null, so HEAD comes back empty after about 190 s.
  • As a result Test-Receipt stops the run, readback and negative phases on every real receipt.
  • The offline S11DriverTests use receipts with "commits": [] (S11DriverTests.cs:102), so they never exercised this path.
• Bounded correction recommended, not implemented:
  • resolve git as an application (Get-Command git -CommandType Application) or rename the helper, and fail closed on a non-zero exit or a non-40-hex HEAD;
  • harden the Build-S11-Receipt.ps1 fallback the same way;
  • add driver tests with a real Git commit in the receipt;
  • no fixture, tolerance, gate, metric or production change;
  • then a new receipt and Revision 2 under a fresh owner instruction.
• Evidence:
  • committed: documentation/S11-OpenStudio-ResultsRoundTrip-evidence/revision-2-2026-10-26/ (stopped summary-run.json, receipt, SHA256SUMS.txt, README);
  • machine-local: C:\TasOut\s11\r2-20261026\ and SAM-BIM\.s11r2-20261026\ (scripts, logs, hashes, weather, clones).

Integration of F-S11-1 and F-S11-2 (2026-10-24)

• sow/2026-Q4 (194ee1e6, with SAM_OpenStudio#35 merge 4b020585) merged into this branch as 43e3541c (merge commit, no conflicts). SAM#219 (merged ac68ad0f, closeout 457defbb) is consumed as the SAM dependency.
• Tested content 76d61612ebe477f57cb44517f82e5353b2abe3b0: tests only. The two tests that pinned the defects are replaced by positive regressions:
  • SignedOffTwoZone_SharedGuidPrefix_EachSpaceKeepsItsOwnResults: the unchanged signed-off s08-fix-two (FixA/FixB share 5080c000) PASSes with 0 FAIL/BLOCKED. Native space/zone/Ideal Loads are distinct, each production key equals its own native key, and the zone-specific fractional unmet hours (14.5/14.75 h vs 7.25/5.25 h) are exact per space in the reference, SAM-P, SAM-I and B1a. SAM-I results are attached to their own spaces.
  • FractionalUnmetHours_SurviveSqlImport_SamSaveReload_AndB1a (s08-fix-box): 14.5 h heating / 14.75 h cooling, exact through SQL import, sam.json save/reload, an AnalyticalModel SAM JSON save/reopen after Modify.AddResults, and b1a.json.
• The re-GUIDed two-zone workaround is retired. The CLI, completeness, F3 and negative tests now run on the signed-off fixture. Fixtures (45524172…, 20da9b58…), GUIDs, tolerances, metric definitions, gates, scripts and workflow are unchanged.
• Fail-before controls (tested binaries with one pre-fix DLL swapped in):
  • pre-F-S11-1: associate OpenStudio zone results by name identity, not Guid suffix #35 importer → the two-zone regression FAILs (FixA gets FixB's not-met time);
  • pre-#219 SAM → both regressions FAIL (14.5 → 14, 14.75 → 15).
  • Before the test change, the merged code failed exactly the two Finding_ pins: "Expected FAIL, but was PASS".

Local results (isolated workspace, Release x64, APPDATA redirected, no Grasshopper build/deploy)

• Dependencies:
  • SAM built from merged Q4 457defbb (SAM.Analytical.dll 21bec17f…, loaded by all 8 test/CLI outputs);
  • SAM_Tas 922044fa, SAM_Validation 2a3acfaf (0dfdc8ec…), SAM_SQLite ec2eb86 (ba9400e9…).
  • Full hashes are in the record.
• Builds: affected projects build with 0 errors and 0 CS warnings.
• Tests:
  • read-back (S09 + S11) 242/242;
  • producer 58/58;
  • importer offline (incl. F-S11-1: associate OpenStudio zone results by name identity, not Guid suffix #35's 17 identity tests) 488 passed, 2 skipped (uncommitted real-SQL fixtures).
• S09 replays: both saved-run replays (box, two) are byte-identical to the committed acceptance reports, on both the merge and the tested content.
• Not run: Simulation-category tests, the S11 run phase, the receipt build, SAM_UI and Tas. There are no engine outputs or processes.

Separately tracked, not blocking on these fixtures

• Modify.AddResults resolves design-day and surface results by the 8-character suffix (FixB → FixA). S11 is unaffected here: the annual family references the space GUID, the fixtures have no design days, and S11 compares no surface results.
• Generated-name identity: same name + same prefix stays unmapped; pinned by S11IdentityTests.
• SAM#219 Revit compatibility: deferred by the owner.

Earlier revisions (unchanged)

• Revision 1 246c8fed: producer split, F3 note, --sam-results, the behaviour-neutral S09 identity extraction, and the ResultsReadBack library plus os-results-readback CLI.
• Review correction 059aed6f (R1 driver stop/collect rules, R2 frequency/environment rejection): independently accepted.

Head and checks

• Final head: 9d2af1afd7769baf310fc5583300cba572875ae5, a documentation and evidence commit after 01344e65. It contains no code, test, script, fixture or workflow change, so the tested content is still 76d61612.
• Hosted CI at this head (offline tests only; it does not run the engine or the main importer test project):
  • Build (Windows) 40371186694;
  • SPDX 40371186671.
  • Both SUCCESS: producer tests 58/58, S09 + S11 read-back tests 242/242.
• Previous head 01344e65: Build 40123367373 (58/58 producer, 242/242 read-back) and SPDX 40123367384, both SUCCESS. The hosted SAM dependency 457defbb is inferred from branch timing, not shown in the log.
• Local acceptance (above) is distinct from these hosted offline checks.
• Independent review of the offline integration (2026-10-26): ACCEPT, offline only. The review is machine-local.

Next:

1. Owner decision on the bounded Run-S11-Results.ps1 git-resolution correction (code and offline tests only, on this Draft PR).
2. Review of that correction.
3. Revision 2 repeated under a fresh owner instruction: new receipt, then 2 engine runs (allowance unused).
4. Independent review of the real-run evidence.

Generated by Michal Dengusiak and CodeClaude

🤖 Generated with Claude Code