From 28fd8e3ec3a08588479ce8ad4aa71254114415be Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 21:10:55 +0000 Subject: [PATCH 1/2] fix(runners): make 5.1 able to read the runners, and to refuse them truthfully The three runners were pwsh-only by ACCIDENT rather than by declaration. They carried non-ASCII characters and no BOM, so Windows PowerShell 5.1 decoded them in the system ANSI codepage and died on a syntax error inside whatever word happened to contain a byte above 127. Someone opening an old Windows console was told that `honouring` was a syntax crime. The contract this fixes is deliberately NOT "the runners work under 5.1" - that is a different and much larger task. It is: 5.1 can READ the file, and says the true thing about it. Both halves are load-bearing. ASCII-only is what makes the file readable under any single-byte codepage; `#Requires -Version 7` is what the engine then acts on, refusing before a single statement of the body runs. Without the first the second is unreachable, and without the second the first only moves the failure somewhere less honest. Run-Audit.ps1 67 non-ASCII bytes -> 0 Run-Infersharp.ps1 14 -> 0 Run-Roslyn.ps1 6 -> 0 All of it was prose: em dashes, one section sign, one ellipsis, one middle dot. The middle dot was quoting the character `report.py` prints, so replacing it would have changed what the comment says; it is named as U+00B7 instead. A BOM would also make 5.1 decode correctly and is deliberately not the fix - it leaves the file non-ASCII and makes correctness depend on three bytes any editor may drop. The suite rejects one. `scripts/tests/Test-RunnerShellContract.ps1` is new and runs on BOTH editions, because the two editions can check different halves. Everywhere: bytes are ASCII (read as BYTES - reading as text is the step that hides the defect, since any assumed encoding yields SOME string), no BOM, `#Requires -Version 7` present, and the file parses. Under 5.1 additionally: each runner is invoked with an -Out directory that does not exist, and the checks are that it exits non-zero, that the message names the version rather than a word in a comment, and that the directory STILL does not exist - which is the evidence the body never ran. Under pwsh the runners are only parsed, never invoked; they drive worktrees, WSL and MSBuild. What pwsh checks instead is that comment-based help still resolves, which is what would break if `#Requires` detached it. Test-LineTolPolicy.ps1 loses its Desktop skip. That skip existed only because of this defect - it announced that the Run-Audit.ps1 structural checks "did not run" under 5.1 - so leaving it would have made a workaround outlive the thing it was working around. CI gains two Windows jobs, pwsh and Windows PowerShell 5.1. Windows only: the defect is 5.1 decoding, and no Linux runner reproduces it. Split rather than matrixed for the same reason as linetol-policy-ps51 - `shell:` accepts no expression. Verified: 19 checks pass under pwsh 7.4.6, the LineTol suite reports 45 checks and 0 skipped, and 5 probes confirm each new check bites - reintroducing a non-ASCII byte, deleting the requirement, adding a BOM, breaking the syntax, and detaching the help. The 5.1 leg cannot run in this container and is covered by CI alone; that is stated rather than glossed. actionlint clean, Python sweep green. Closes #64 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016Lmcv3X9PoELp8CGDfNc9m --- .github/workflows/ci.yml | 31 +++++ AGENTS.md | 1 + Run-Audit.ps1 | 45 +++--- Run-Infersharp.ps1 | 13 +- Run-Roslyn.ps1 | 9 +- scripts/tests/Test-LineTolPolicy.ps1 | 35 ++--- scripts/tests/Test-RunnerShellContract.ps1 | 153 +++++++++++++++++++++ 7 files changed, 239 insertions(+), 48 deletions(-) create mode 100644 scripts/tests/Test-RunnerShellContract.ps1 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6237f3a..5a6f23e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -115,6 +115,37 @@ jobs: shell: powershell run: ./scripts/tests/Test-LineTolPolicy.ps1 + runner-shell-contract: + # OwnAudit#64. The runners are pwsh-only; the contract is that Windows + # PowerShell 5.1 can still READ them and refuse them for a true reason, + # instead of dying on a syntax error inside a word in a comment. Windows only: + # the defect is Windows 5.1 decoding a BOM-less file in the system ANSI + # codepage, and no Linux runner reproduces it. + name: runner shell contract (windows-latest / pwsh) + runs-on: windows-latest + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + - name: ASCII, #Requires, and parse under pwsh + shell: pwsh + run: ./scripts/tests/Test-RunnerShellContract.ps1 + + runner-shell-contract-ps51: + # The leg that actually covers the defect. Split out for the same reason as + # linetol-policy-ps51: `shell:` accepts no expression, so a matrix cannot + # carry it. Here the suite invokes each runner for real and asserts the + # refusal names the version and leaves no -Out directory behind. + name: runner shell contract (windows-latest / Windows PowerShell 5.1) + runs-on: windows-latest + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + - name: 5.1 reads the runners and refuses them by version + shell: powershell + run: ./scripts/tests/Test-RunnerShellContract.ps1 + audit-report: name: SARIF export + dashboard smoke runs-on: ubuntu-latest diff --git a/AGENTS.md b/AGENTS.md index 290df5b..b171b14 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,5 +15,6 @@ This file provides guidance to agents when working with code in this repository. - Use `Run-Audit.ps1 -Codeql` for CodeQL corroboration. The clustering window is raised from 3 to 8 **automatically** when Infer# or Roslyn are folded in, because those tools report shifted/generated locations; the raise is announced on stdout. Passing `-LineTol` explicitly always wins — even a value below 8, which is warned about and then honoured, never silently corrected. The decision is `Resolve-LineTol` in `scripts/LineTolPolicy.ps1`, and its matrix is pinned by `scripts/tests/Test-LineTolPolicy.ps1` on Windows and Linux. - `Run-Infersharp.ps1` consumes built STS `.dll` + `.pdb` files via WSL and filters third-party/generated SARIF before `Run-Audit.ps1` folds it. - `Run-Roslyn.ps1` requires VS2022 BuildTools and injects analyzers with `/p:OwnAudit=true`; analyzer packs must stay in per-pack cache subdirs to avoid `Gu.Roslyn.Extensions` conflicts. +- **The three runners are pwsh-only, and say so where the engine can act on it.** `Run-Audit.ps1`, `Run-Infersharp.ps1` and `Run-Roslyn.ps1` each open with `#Requires -Version 7` and are kept byte-level ASCII (OwnAudit#64). The two go together: Windows PowerShell 5.1 reads a BOM-less file in the system ANSI codepage, so a single non-ASCII byte used to surface as a syntax error inside whatever word contained it - the requirement is unreachable if the file cannot be parsed to find it. The contract is NOT that the runners work under 5.1; it is that 5.1 can read them and refuse them for a true reason. Do not "fix" this with a BOM (it leaves the file non-ASCII and makes correctness depend on three bytes an editor may drop), and do not reintroduce typographic dashes into these files. `scripts/tests/Test-RunnerShellContract.ps1` enforces it on both editions. - Generated/derived outputs live under `artifacts/`, `arch/out/`, `report/out/`, `runtime/out/`, and `leakmine-out/`; avoid treating them as source truth unless a test fixture explicitly depends on them. - `.editorconfig` has a project-specific C# gotcha: `*.cs` files use CRLF while JSON/YAML/Markdown/PowerShell use 2-space indentation. diff --git a/Run-Audit.ps1 b/Run-Audit.ps1 index 553ac26..5dd6e79 100644 --- a/Run-Audit.ps1 +++ b/Run-Audit.ps1 @@ -1,3 +1,8 @@ +#Requires -Version 7 +# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell +# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax +# error in whatever word happened to contain it. This file is ASCII so 5.1 can read +# it, and this line is what 5.1 then reports instead: a refusal about versions. <# .SYNOPSIS Reproduce the STS health report: Own.NET analyzes, OwnAudit aggregates and reports. @@ -13,7 +18,7 @@ aggregate/ (Own.NET#266 slice 1A) and produces byte-identical findings.json. Scoring/reporting still run from the worktree until they are ported too. Two build-free tools means cross-tool AGREEMENT: a site both flag becomes a - high-confidence cluster (audit/ §3.5). PYTHONUTF8=1 dodges the cp1251 console crash. + high-confidence cluster (audit/ section 3.5). PYTHONUTF8=1 dodges the cp1251 console crash. .EXAMPLE pwsh ./Run-Audit.ps1 # OwnSharp only (fast) @@ -39,7 +44,7 @@ param( # Passing it explicitly always wins, including a lower value. ) $ErrorActionPreference = "Stop" -$env:PYTHONUTF8 = "1" # report.py prints '>=' / '·' — crashes on a cp1251 console +$env:PYTHONUTF8 = "1" # report.py prints '>=' and U+00B7 MIDDLE DOT - both crash a cp1251 console New-Item -ItemType Directory -Force -Path $Out | Out-Null # Read HERE, in this script's own scope: $PSBoundParameters describes this @@ -49,7 +54,7 @@ New-Item -ItemType Directory -Force -Path $Out | Out-Null $lineTolExplicit = $PSBoundParameters.ContainsKey('LineTol') . (Join-Path (Join-Path $PSScriptRoot "scripts") "LineTolPolicy.ps1") -# 1. worktree of main — audit/ is on main; the dev checkout may sit on a feature branch. +# 1. worktree of main - audit/ is on main; the dev checkout may sit on a feature branch. git -C $OwnNet fetch origin main -q if (Test-Path (Join-Path $Worktree ".git")) { git -C $Worktree fetch origin main -q 2>$null @@ -59,7 +64,7 @@ if (Test-Path (Join-Path $Worktree ".git")) { git -C $OwnNet worktree add --detach $Worktree $Ref } if (-not (Test-Path (Join-Path $Worktree "audit\aggregate\report.py"))) { - throw "audit/ not found in $Worktree — is '$Ref' the branch that has audit/?" + throw "audit/ not found in $Worktree - is '$Ref' the branch that has audit/?" } # pyyaml is for the WORKTREE's report.py (it still reads the YAML taxonomy through # its own normalize.py). The local aggregate/normalize.py is stdlib-only. @@ -71,14 +76,14 @@ $sarifInputs = @() # "tool=path" for normalize # ---- provenance (producer-provenance/v1, Own.NET#266 slice 1B) --------------- # SARIF carries no run identity, so occurrence identity has to come from outside # it. The run id is stamped HERE, before any producer starts, so it names the -# ANALYSIS rather than the normalization — re-normalizing this same recorded run +# ANALYSIS rather than the normalization - re-normalizing this same recorded run # later reuses this manifest and reproduces the same occurrence ids. The # normalizer never invents one: no manifest entry means occurrence_id: null, and # the record says why. # # A GUID, not just a timestamp: two audits started in the same second would # otherwise share producer run ids, and "rare" is not a property an identity -# contract may have. Determinism survives — the id is written to the manifest and +# contract may have. Determinism survives - the id is written to the manifest and # read back from it, never recomputed. $auditRunId = "audit-{0}-{1}" -f [DateTime]::UtcNow.ToString("yyyyMMddTHHmmssZ"), [Guid]::NewGuid().ToString("N") @@ -87,7 +92,7 @@ if ([string]::IsNullOrWhiteSpace($sourceCommit)) { $sourceCommit = $null } # The analyzers read the WORKING TREE, not HEAD. On a dirty target `git rev-parse # HEAD` still answers, and recording that answer would attribute findings to a -# commit that does not contain the analyzed bytes — the same fabrication as +# commit that does not contain the analyzed bytes - the same fabrication as # stamping a reused CodeQL database with today's commit, just harder to notice. # Unknown is a value this schema carries, so it is recorded as unknown. # @@ -99,7 +104,7 @@ if ($sourceCommit -and $dirty.Count -gt 0) { # One string, one -f: the format operator binds tighter than '+', so a # concatenation of two format strings would silently format only the second one # and leave the first one's placeholders as literal text. - $msg = "Target tree is dirty ({0} changed/untracked path(s)) — recording source_commit as null: the analyzed bytes are not the ones in {1}." -f $dirty.Count, $sourceCommit.Substring(0, 8) + $msg = "Target tree is dirty ({0} changed/untracked path(s)) - recording source_commit as null: the analyzed bytes are not the ones in {1}." -f $dirty.Count, $sourceCommit.Substring(0, 8) Write-Host $msg $sourceCommit = $null } @@ -114,7 +119,7 @@ function Add-Provenance { runners (Run-Infersharp.ps1 / Run-Roslyn.ps1), possibly yesterday, possibly against another commit and another configuration. Stamping them with this run's id and this checkout's HEAD would be provenance about an analysis - nobody observed — precisely the fabricated identity the contract exists to + nobody observed - precisely the fabricated identity the contract exists to prevent. So they get producer_name and input_digest, which are facts about the bytes on disk, and nothing else. Their occurrence ids stay null until those runners emit their own provenance sidecars. @@ -124,12 +129,12 @@ function Add-Provenance { [Parameter(Mandatory)][string]$SarifPath, $RunId = $null, $SourceCommit = $null, - $Version = $null, # null, not "" — unknown must read as unknown + $Version = $null, # null, not "" - unknown must read as unknown $ConfigDigest = $null ) # Lower-cased on purpose: Get-FileHash returns upper-case hex and the normalizer # compares the digest string verbatim, rejecting a mismatch outright. A case - # difference would fail the whole run rather than degrade quietly — correct, but + # difference would fail the whole run rather than degrade quietly - correct, but # a needless way to find out. $sha = (Get-FileHash -Algorithm SHA256 -LiteralPath $SarifPath).Hash.ToLowerInvariant() $nz = { param($v) if ([string]::IsNullOrWhiteSpace([string]$v)) { $null } else { [string]$v } } @@ -152,7 +157,7 @@ try { 1> $ownsarif 2> (Join-Path $Out "own-check.err") } finally { Pop-Location } if (-not (Test-Path $ownsarif) -or (Get-Item $ownsarif).Length -lt 2) { - throw "OwnSharp produced no SARIF — see $Out\own-check.err" + throw "OwnSharp produced no SARIF - see $Out\own-check.err" } Write-Host "OwnSharp SARIF: $ownsarif ($((Get-Item $ownsarif).Length) bytes)" $sarifInputs += "own-check=$ownsarif" @@ -169,15 +174,15 @@ if ($Codeql) { $codeqlDbBuiltNow = $false if ($RebuildCodeqlDb -or -not (Test-Path (Join-Path $CodeqlDb "codeql-database.yml"))) { New-Item -ItemType Directory -Force -Path (Split-Path $CodeqlDb) | Out-Null - Write-Host "CodeQL: building DB (build-free) over $Target — this is the slow step…" + Write-Host "CodeQL: building DB (build-free) over $Target - this is the slow step..." & $CodeqlExe database create $CodeqlDb --language=csharp --build-mode=none --source-root=$Target --overwrite $codeqlDbBuiltNow = $true } else { - Write-Host "CodeQL: reusing DB at $CodeqlDb (-RebuildCodeqlDb to force) — its source commit is unknown to this run" + Write-Host "CodeQL: reusing DB at $CodeqlDb (-RebuildCodeqlDb to force) - its source commit is unknown to this run" } $cqsarif = Join-Path $Out "codeql.sarif" # security-and-quality is the practical max for a desktop app; -Strict adds the - # experimental suite (marginal here — mostly web-shaped queries — but complete). + # experimental suite (marginal here - mostly web-shaped queries - but complete). $suites = @("codeql/csharp-queries:codeql-suites/csharp-security-and-quality.qls") if ($Strict) { $suites += "codeql/csharp-queries:codeql-suites/csharp-security-experimental.qls" } & $CodeqlExe database analyze $CodeqlDb --format=sarifv2.1.0 --output=$cqsarif --threads=0 @suites @@ -193,7 +198,7 @@ if ($Codeql) { -SourceCommit $(if ($codeqlDbBuiltNow) { $sourceCommit } else { $null }) } -# Infer# (build-required) — fold in if a SARIF is present. Produce it first with +# Infer# (build-required) - fold in if a SARIF is present. Produce it first with # Run-Infersharp.ps1 (WSL). Infer# reports at the last-access line, so the clustering # window is raised to 8 below unless -LineTol was passed explicitly. if (Test-Path (Join-Path $Out "infersharp.sarif")) { @@ -203,7 +208,7 @@ if (Test-Path (Join-Path $Out "infersharp.sarif")) { Add-Provenance -Tool "infersharp" -SarifPath (Join-Path $Out 'infersharp.sarif') } -# Roslyn analyzer packs (build-required) — fold in if a SARIF is present. Produce it +# Roslyn analyzer packs (build-required) - fold in if a SARIF is present. Produce it # first with Run-Roslyn.ps1 (VS2022 build). High volume and shifted/generated locations, # so the clustering window is raised to 8 below unless -LineTol was passed explicitly. if (Test-Path (Join-Path $Out "roslyn.sarif")) { @@ -214,7 +219,7 @@ if (Test-Path (Join-Path $Out "roslyn.sarif")) { } # The folding above is automatic; the clustering window it needs used to be manual. -# Resolve it now that the input set is final — Infer# and Roslyn report shifted or +# Resolve it now that the input set is final - Infer# and Roslyn report shifted or # generated locations, so at the default window one defect splits into several # findings. An explicitly passed -LineTol is honoured exactly, including when it is # lower than those tools want: the mismatch is reported, never corrected behind the @@ -240,13 +245,13 @@ foreach ($s in $sarifInputs) { $nargs += @("--sarif", $s) } $absStrip = ($Target -replace '\\', '/').TrimEnd('/') # Persist the manifest next to the artifacts it describes. Keep it: re-normalizing # this recorded run WITHOUT it produces occurrence_id: null everywhere, because the -# normalizer will not mint a run id of its own — and it should not, since a run id +# normalizer will not mint a run id of its own - and it should not, since a run id # minted at normalization time would describe the normalization instead. $manifest = Join-Path $Out "provenance.json" @{ schema_version = "producer-provenance/v1"; inputs = $provenanceInputs } | ConvertTo-Json -Depth 6 | Set-Content -LiteralPath $manifest -Encoding utf8 $withRun = @($provenanceInputs.Values | Where-Object { $_.producer_run_id }).Count -Write-Host "Provenance manifest: $manifest ($auditRunId; $withRun of $($provenanceInputs.Count) producer(s) carry a run id — the rest are pre-existing SARIF and their occurrence ids stay null)" +Write-Host "Provenance manifest: $manifest ($auditRunId; $withRun of $($provenanceInputs.Count) producer(s) carry a run id - the rest are pre-existing SARIF and their occurrence ids stay null)" python "$PSScriptRoot\aggregate\normalize.py" @nargs --strip "$leaf" --strip $absStrip --strip "/$absStrip" --provenance $manifest --json $findings foreach ($fmt in @(@{f='markdown';e='md'}, @{f='html';e='html'}, @{f='json';e='json'})) { python "$Worktree\audit\aggregate\report.py" --findings $findings --format $fmt.f --target $leaf --commit $commit --line-tol $LineTol | diff --git a/Run-Infersharp.ps1 b/Run-Infersharp.ps1 index e17df38..6ec25c5 100644 --- a/Run-Infersharp.ps1 +++ b/Run-Infersharp.ps1 @@ -1,15 +1,20 @@ +#Requires -Version 7 +# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell +# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax +# error in whatever word happened to contain it. This file is ASCII so 5.1 can read +# it, and this line is what 5.1 then reports instead: a refusal about versions. <# .SYNOPSIS Run Infer# (build-required) over STS's own built binaries via the WSL2 infersharp distro, and write artifacts/infersharp.sarif for the audit to fold in. .DESCRIPTION - Infer# analyzes COMPILED binaries (.dll + .pdb) — build-required (audit/ §3.2). It - copies the source-mappable STS assemblies (those with a .pdb — which skips the + Infer# analyzes COMPILED binaries (.dll + .pdb) - build-required (audit/ section 3.2). It + copies the source-mappable STS assemblies (those with a .pdb - which skips the DevExpress / 3rd-party DLLs that ship no PDB and couldn't be source-mapped anyway) out of the build output, runs Infer# inside the WSL distro, and drops the SARIF where Run-Audit.ps1 folds it in. Infer#'s Pulse analysis is SLOW (minutes to hours, - dominated by the big assemblies — Broker, BrokerDataClasses). + dominated by the big assemblies - Broker, BrokerDataClasses). Infer# reports a leak at the LAST-ACCESS line (the alloc line is in the message), so fold with a wider Run-Audit.ps1 -LineTol (e.g. 8) to cluster it with own-check/codeql. @@ -46,7 +51,7 @@ $wslSarif = & $toWsl (Join-Path $Out "infersharp.sarif") wsl -d $Distro -- bash -lc "cd ~/infersharp && rm -rf infer-out && ./run_infersharp.sh $wslBin && cp infer-out/report.sarif $wslSarif && echo OK" if (-not (Test-Path "$Out\infersharp.sarif")) { throw "Infer# produced no SARIF (wsl exit $LASTEXITCODE)" } # Drop findings in third-party assemblies Infer# pulled in via their PDBs (paths like -# 'file:/_/BASE/src/...') and any generated files — keep STS code only. +# 'file:/_/BASE/src/...') and any generated files - keep STS code only. $env:PYTHONUTF8 = '1' python -c @" import json,re diff --git a/Run-Roslyn.ps1 b/Run-Roslyn.ps1 index 31ebced..e04d99e 100644 --- a/Run-Roslyn.ps1 +++ b/Run-Roslyn.ps1 @@ -1,3 +1,8 @@ +#Requires -Version 7 +# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell +# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax +# error in whatever word happened to contain it. This file is ASCII so 5.1 can read +# it, and this line is what 5.1 then reports instead: a refusal about versions. <# .SYNOPSIS Run the NuGet Roslyn analyzer packs (build-required tier) over STS via VS2022 and @@ -7,7 +12,7 @@ Roslyn analyzers run DURING compilation, so this rebuilds the target with the packs injected (canonical inject props, gated on /p:OwnAudit=true) plus an is_global config that downgrades every analyzer diagnostic to 'warning' (so an error-default rule like - Meziantou MA0037 is COLLECTED, not build-breaking — real CS#### errors still fail). + Meziantou MA0037 is COLLECTED, not build-breaking - real CS#### errors still fail). Needs VS2022 BuildTools (Roslyn 4.x): the modern packs no-load on VS2019/Roslyn 3.x (Gu.Roslyn.Extensions version conflict). Each pack is cached in its OWN subdir so it @@ -35,7 +40,7 @@ $ErrorActionPreference = 'Stop' $env:PYTHONUTF8 = '1' $vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" $msbuild = (& $vswhere -products * -version "[17.0,18.0)" -latest -find "MSBuild\**\Bin\MSBuild.exe" 2>$null | Select-Object -First 1) -if (-not $msbuild) { throw "VS2022 msbuild not found — BuildTools 17.x (Roslyn 4.x) is required for the modern packs." } +if (-not $msbuild) { throw "VS2022 msbuild not found - BuildTools 17.x (Roslyn 4.x) is required for the modern packs." } $nuget = 'C:\Repos\_ownaudit\tools\nuget.exe' # 1. per-pack analyzer cache (each pack in its OWN subdir -> its own Gu.Roslyn.Extensions) diff --git a/scripts/tests/Test-LineTolPolicy.ps1 b/scripts/tests/Test-LineTolPolicy.ps1 index ded5ab8..9b37b99 100644 --- a/scripts/tests/Test-LineTolPolicy.ps1 +++ b/scripts/tests/Test-LineTolPolicy.ps1 @@ -171,28 +171,19 @@ Check ((Resolve-LineTol -Requested 3 -Explicit $false -Tools $derived).Value -eq # the script's OWN scope, and the decision is made AFTER the folding blocks # (resolving earlier would consult a tool list that is not final yet). # -# NOT under Windows PowerShell 5.1. `Run-Audit.ps1` carries 23 non-ASCII -# characters (em dashes, a section sign, an ellipsis) in comments and in -# user-facing messages, all of which predate this change, and 5.1 reads a -# BOM-less file in the system ANSI codepage -- so the runner does not parse -# there at all. That is a real property of the repo, tracked separately; it is -# not something this suite gets to assert away, and asserting the opposite here -# would fail for a reason that has nothing to do with the tolerance policy. -# -# The policy file itself IS held to 5.1 by the ASCII check above, which is what -# actually matters: it is dot-sourced, so it must load wherever the runner does. -if ($PSVersionTable.PSEdition -eq 'Desktop') { - Skip ("Windows PowerShell 5.1: Run-Audit.ps1 structural checks did not run - the runner " + - "is pwsh-only (23 non-ASCII chars, pre-existing) and 5.1 cannot parse it") - $text = $null -} else { - $tokens = $null; $errors = $null - $null = [System.Management.Automation.Language.Parser]::ParseFile( - (Resolve-Path $runAudit), [ref]$tokens, [ref]$errors) - Check ($errors.Count -eq 0) ` - "Run-Audit.ps1 does not parse: $($errors | ForEach-Object { $_.Message })" - $text = Get-Content -LiteralPath $runAudit -Raw -} +# These run on EVERY edition now, 5.1 included. They used to be skipped there: +# `Run-Audit.ps1` carried 23 non-ASCII characters and 5.1 reads a BOM-less file +# in the system ANSI codepage, so the runner did not parse at all. #64 made the +# three runners ASCII and declared them pwsh-only with `#Requires -Version 7`, +# which is a different thing from unparseable: 5.1 can now READ the file and +# refuse it for a reason that is true. The shell contract itself is asserted in +# Test-RunnerShellContract.ps1; what stays here is the tolerance policy. +$tokens = $null; $errors = $null +$null = [System.Management.Automation.Language.Parser]::ParseFile( + (Resolve-Path $runAudit), [ref]$tokens, [ref]$errors) +Check ($errors.Count -eq 0) ` + "Run-Audit.ps1 does not parse: $($errors | ForEach-Object { $_.Message })" +$text = Get-Content -LiteralPath $runAudit -Raw if ($null -ne $text) { Check ($text -match [regex]::Escape("PSBoundParameters.ContainsKey('LineTol')")) ` diff --git a/scripts/tests/Test-RunnerShellContract.ps1 b/scripts/tests/Test-RunnerShellContract.ps1 new file mode 100644 index 0000000..709cfcf --- /dev/null +++ b/scripts/tests/Test-RunnerShellContract.ps1 @@ -0,0 +1,153 @@ +#Requires -Version 5.1 +<# +.SYNOPSIS + The shell contract of the three runners (OwnAudit#64). Runs on BOTH editions. + +.DESCRIPTION + Run-Audit.ps1, Run-Infersharp.ps1 and Run-Roslyn.ps1 are pwsh-only. That was + true before #64 as well - but it was true by ACCIDENT: the files carried + non-ASCII characters and no BOM, so Windows PowerShell 5.1 decoded them in the + system ANSI codepage and died on a syntax error inside whatever word happened + to contain a byte above 127. Someone opening an old Windows console was told + that `honouring` was a syntax crime. + + The contract this suite pins down is deliberately NOT "the runners work under + 5.1". It is: + + 5.1 can READ the file, and says the true thing about it. + + Two halves, and both are load-bearing. ASCII-only is what makes the file + readable under any single-byte codepage. `#Requires -Version 7` is what the + engine then acts on - refusing before a single statement of the body runs. + Without the first, the second is unreachable; without the second, the first + just moves the failure somewhere less honest. + + Under 5.1 the refusal is exercised for real: the runner is invoked with an + -Out directory that does not exist, and the check is that it STILL does not + exist afterwards. Under pwsh the runners are only parsed, never invoked - they + drive worktrees, WSL and MSBuild, which is not something a test may start. + +.NOTES + Bare invocation, no Pester (it is not guaranteed in the dev shell): + pwsh -File scripts/tests/Test-RunnerShellContract.ps1 + powershell -File scripts/tests/Test-RunnerShellContract.ps1 + ASCII-only by its own rule - this file is a runner-adjacent script and is held + to the standard it enforces. +#> +Set-StrictMode -Version Latest +$ErrorActionPreference = 'Stop' + +$script:Count = 0 +$script:Fails = @() + +function Check([bool]$ok, [string]$msg) { + $script:Count++ + if (-not $ok) { $script:Fails += $msg } +} + +$repo = Split-Path -Parent (Split-Path -Parent $PSScriptRoot) +$runners = @('Run-Audit.ps1', 'Run-Infersharp.ps1', 'Run-Roslyn.ps1') + +# ---- 1. Byte-level ASCII. ---------------------------------------------------- +# Read as BYTES, not as text: reading as text is exactly the step that hides the +# defect, because whatever encoding the reader assumes will produce SOME string. +foreach ($name in $runners) { + $path = Join-Path $repo $name + Check (Test-Path -LiteralPath $path) "$name is missing" + if (-not (Test-Path -LiteralPath $path)) { continue } + + if ($PSVersionTable.PSVersion.Major -ge 6) { + $bytes = [System.IO.File]::ReadAllBytes($path) + } else { + $bytes = Get-Content -LiteralPath $path -Encoding Byte -ReadCount 0 + } + $high = @($bytes | Where-Object { $_ -gt 127 }) + Check ($high.Count -eq 0) ` + ("{0} carries {1} byte(s) above 127. Windows PowerShell 5.1 reads a BOM-less file in the system ANSI codepage, so those bytes become a parse error in an unrelated word." -f $name, $high.Count) + + # A BOM would also make 5.1 decode correctly - and is deliberately NOT the + # fix. It would leave the file non-ASCII and make correctness depend on three + # bytes that any editor may drop. + $hasBom = $bytes.Count -ge 3 -and $bytes[0] -eq 0xEF -and $bytes[1] -eq 0xBB -and $bytes[2] -eq 0xBF + Check (-not $hasBom) "$name starts with a UTF-8 BOM; the contract is ASCII content, not a decoding hint" +} + +# ---- 2. The requirement is DECLARED, where the engine can act on it. --------- +foreach ($name in $runners) { + $path = Join-Path $repo $name + if (-not (Test-Path -LiteralPath $path)) { continue } + $lines = Get-Content -LiteralPath $path + $req = @($lines | Where-Object { $_ -match '^\s*#Requires\s+-Version\s+7' }) + Check ($req.Count -ge 1) ` + "$name must declare '#Requires -Version 7' - a comment saying 'use pwsh' is advice, and the engine cannot act on advice" +} + +# ---- 3. Every edition can PARSE them, 5.1 included. -------------------------- +# This is the check that would have failed before #64, and it is the whole point: +# a refusal you can read beats a syntax error you cannot. +foreach ($name in $runners) { + $path = Join-Path $repo $name + if (-not (Test-Path -LiteralPath $path)) { continue } + $tokens = $null; $errors = $null + $null = [System.Management.Automation.Language.Parser]::ParseFile( + (Resolve-Path $path).Path, [ref]$tokens, [ref]$errors) + Check ($errors.Count -eq 0) ` + ("{0} does not parse under {1} {2}: {3}" -f $name, $PSVersionTable.PSEdition, + $PSVersionTable.PSVersion, (($errors | ForEach-Object { $_.Message }) -join '; ')) +} + +# ---- 4. Under 5.1 the refusal is real, and stops before the body. ----------- +if ($PSVersionTable.PSEdition -eq 'Desktop') { + foreach ($name in $runners) { + $path = Join-Path $repo $name + if (-not (Test-Path -LiteralPath $path)) { continue } + + # A directory that does not exist. Every runner creates its -Out early; + # if the body runs at all, this appears. Its ABSENCE afterwards is the + # evidence that #Requires fired first. + $probe = Join-Path ([System.IO.Path]::GetTempPath()) ("shellcontract-" + [Guid]::NewGuid().ToString("N")) + $out = & powershell.exe -NoProfile -NonInteractive -File $path -Out $probe 2>&1 + $text = ($out | Out-String) + + Check ($LASTEXITCODE -ne 0) "$name ran to success under Windows PowerShell 5.1; it is declared pwsh-only" + Check ($text -match '(?i)version|requires') ` + ("{0} failed under 5.1 without naming the version requirement. That is the #64 defect returning in another costume: the message must be about the shell, not about a word in a comment. Got: {1}" -f $name, $text.Trim()) + Check (-not (Test-Path -LiteralPath $probe)) ` + "$name created its -Out directory before refusing; #Requires must stop the script before any statement of the body" + if (Test-Path -LiteralPath $probe) { Remove-Item -Recurse -Force -LiteralPath $probe } + } +} else { + # Under pwsh the requirement is SATISFIED, so invoking would really start a + # worktree/WSL/MSBuild run. The half that can be checked here is that the + # declaration does not disturb the file: help still resolves, which is what + # would break if `#Requires` had been placed somewhere it detaches the + # comment-based help from the script. + foreach ($name in $runners) { + $path = Join-Path $repo $name + if (-not (Test-Path -LiteralPath $path)) { continue } + $help = Get-Help $path -ErrorAction SilentlyContinue + Check ($null -ne $help -and -not [string]::IsNullOrWhiteSpace(($help.Synopsis | Out-String).Trim())) ` + "$name lost its comment-based help; '#Requires' must sit above the help block without detaching it" + } +} + +# ---- 5. This suite holds itself to the same rule. --------------------------- +$self = $PSCommandPath +if ($PSVersionTable.PSVersion.Major -ge 6) { + $selfBytes = [System.IO.File]::ReadAllBytes($self) +} else { + $selfBytes = Get-Content -LiteralPath $self -Encoding Byte -ReadCount 0 +} +Check (@($selfBytes | Where-Object { $_ -gt 127 }).Count -eq 0) ` + "this suite is not ASCII; a test for readability that cannot itself be read is theatre" + +# ---- verdict ---------------------------------------------------------------- +foreach ($f in $script:Fails) { Write-Host "FAIL: $f" } +if ($script:Fails.Count -gt 0) { + Write-Host ("runner shell contract: FAIL - {0} of {1} checks failed on {2} {3}" -f + $script:Fails.Count, $script:Count, $PSVersionTable.PSEdition, $PSVersionTable.PSVersion) + exit 1 +} +Write-Host ("runner shell contract: OK - {0} checks passed on {1} {2}" -f + $script:Count, $PSVersionTable.PSEdition, $PSVersionTable.PSVersion) +exit 0 From b58f78859deed07f41528a33c62f8caf7a72a2bd Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 21:13:39 +0000 Subject: [PATCH 2/2] fix(test): stop the 5.1 leg from dying on the evidence it came to read The contract worked on the first CI run. Windows PowerShell 5.1 read Run-Audit.ps1 and refused it for the true reason: The script 'Run-Audit.ps1' cannot be run because it contained a "#requires" statement for Windows PowerShell 7.0. The suite failed anyway, at the invocation itself. That refusal arrives on STDERR, and 5.1 turns any stderr write by a native executable into a terminating NativeCommandError while $ErrorActionPreference is 'Stop' - so the script died on line 109 without reaching a single assertion. The check was killed by its own subject at the exact moment that subject started behaving correctly. $ErrorActionPreference drops to Continue around the call and is restored in a finally, and the exit code is captured immediately rather than read later, where an intervening call could clobber $LASTEXITCODE. pwsh does not exhibit this, which is the point of running the leg at all: the only reason this was found is that the suite really executes under 5.1 instead of asserting about it from 7. Refs #64 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016Lmcv3X9PoELp8CGDfNc9m --- scripts/tests/Test-RunnerShellContract.ps1 | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/scripts/tests/Test-RunnerShellContract.ps1 b/scripts/tests/Test-RunnerShellContract.ps1 index 709cfcf..3eb15e0 100644 --- a/scripts/tests/Test-RunnerShellContract.ps1 +++ b/scripts/tests/Test-RunnerShellContract.ps1 @@ -106,10 +106,24 @@ if ($PSVersionTable.PSEdition -eq 'Desktop') { # if the body runs at all, this appears. Its ABSENCE afterwards is the # evidence that #Requires fired first. $probe = Join-Path ([System.IO.Path]::GetTempPath()) ("shellcontract-" + [Guid]::NewGuid().ToString("N")) - $out = & powershell.exe -NoProfile -NonInteractive -File $path -Out $probe 2>&1 + + # $ErrorActionPreference must drop to Continue around this call. The + # refusal arrives on STDERR, and Windows PowerShell 5.1 turns any stderr + # write by a native executable into a terminating NativeCommandError when + # the preference is Stop - so the suite would die ON the evidence it came + # to read, at the very moment the contract was working. pwsh does not do + # this, which is why only the 5.1 leg can catch it. + $prevEap = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + try { + $out = & powershell.exe -NoProfile -NonInteractive -File $path -Out $probe 2>&1 + $code = $LASTEXITCODE + } finally { + $ErrorActionPreference = $prevEap + } $text = ($out | Out-String) - Check ($LASTEXITCODE -ne 0) "$name ran to success under Windows PowerShell 5.1; it is declared pwsh-only" + Check ($code -ne 0) "$name ran to success under Windows PowerShell 5.1; it is declared pwsh-only" Check ($text -match '(?i)version|requires') ` ("{0} failed under 5.1 without naming the version requirement. That is the #64 defect returning in another costume: the message must be about the shell, not about a word in a comment. Got: {1}" -f $name, $text.Trim()) Check (-not (Test-Path -LiteralPath $probe)) `