Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,37 @@ jobs:
shell: powershell
run: ./scripts/tests/Test-LineTolPolicy.ps1

runner-shell-contract:
# OwnAudit#64. The runners are pwsh-only; the contract is that Windows
# PowerShell 5.1 can still READ them and refuse them for a true reason,
# instead of dying on a syntax error inside a word in a comment. Windows only:
# the defect is Windows 5.1 decoding a BOM-less file in the system ANSI
# codepage, and no Linux runner reproduces it.
name: runner shell contract (windows-latest / pwsh)
runs-on: windows-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: ASCII, #Requires, and parse under pwsh
shell: pwsh
run: ./scripts/tests/Test-RunnerShellContract.ps1

runner-shell-contract-ps51:
# The leg that actually covers the defect. Split out for the same reason as
# linetol-policy-ps51: `shell:` accepts no expression, so a matrix cannot
# carry it. Here the suite invokes each runner for real and asserts the
# refusal names the version and leaves no -Out directory behind.
name: runner shell contract (windows-latest / Windows PowerShell 5.1)
runs-on: windows-latest
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
persist-credentials: false
- name: 5.1 reads the runners and refuses them by version
shell: powershell
run: ./scripts/tests/Test-RunnerShellContract.ps1

audit-report:
name: SARIF export + dashboard smoke
runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ This file provides guidance to agents when working with code in this repository.
- Use `Run-Audit.ps1 -Codeql` for CodeQL corroboration. The clustering window is raised from 3 to 8 **automatically** when Infer# or Roslyn are folded in, because those tools report shifted/generated locations; the raise is announced on stdout. Passing `-LineTol` explicitly always wins — even a value below 8, which is warned about and then honoured, never silently corrected. The decision is `Resolve-LineTol` in `scripts/LineTolPolicy.ps1`, and its matrix is pinned by `scripts/tests/Test-LineTolPolicy.ps1` on Windows and Linux.
- `Run-Infersharp.ps1` consumes built STS `.dll` + `.pdb` files via WSL and filters third-party/generated SARIF before `Run-Audit.ps1` folds it.
- `Run-Roslyn.ps1` requires VS2022 BuildTools and injects analyzers with `/p:OwnAudit=true`; analyzer packs must stay in per-pack cache subdirs to avoid `Gu.Roslyn.Extensions` conflicts.
- **The three runners are pwsh-only, and say so where the engine can act on it.** `Run-Audit.ps1`, `Run-Infersharp.ps1` and `Run-Roslyn.ps1` each open with `#Requires -Version 7` and are kept byte-level ASCII (OwnAudit#64). The two go together: Windows PowerShell 5.1 reads a BOM-less file in the system ANSI codepage, so a single non-ASCII byte used to surface as a syntax error inside whatever word contained it - the requirement is unreachable if the file cannot be parsed to find it. The contract is NOT that the runners work under 5.1; it is that 5.1 can read them and refuse them for a true reason. Do not "fix" this with a BOM (it leaves the file non-ASCII and makes correctness depend on three bytes an editor may drop), and do not reintroduce typographic dashes into these files. `scripts/tests/Test-RunnerShellContract.ps1` enforces it on both editions.
- Generated/derived outputs live under `artifacts/`, `arch/out/`, `report/out/`, `runtime/out/`, and `leakmine-out/`; avoid treating them as source truth unless a test fixture explicitly depends on them.
- `.editorconfig` has a project-specific C# gotcha: `*.cs` files use CRLF while JSON/YAML/Markdown/PowerShell use 2-space indentation.
45 changes: 25 additions & 20 deletions Run-Audit.ps1
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
#Requires -Version 7
# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell
# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax
# error in whatever word happened to contain it. This file is ASCII so 5.1 can read
# it, and this line is what 5.1 then reports instead: a refusal about versions.
<#
.SYNOPSIS
Reproduce the STS health report: Own.NET analyzes, OwnAudit aggregates and reports.
Expand All @@ -13,7 +18,7 @@
aggregate/ (Own.NET#266 slice 1A) and produces byte-identical findings.json.
Scoring/reporting still run from the worktree until they are ported too.
Two build-free tools means cross-tool AGREEMENT: a site both flag becomes a
high-confidence cluster (audit/ §3.5). PYTHONUTF8=1 dodges the cp1251 console crash.
high-confidence cluster (audit/ section 3.5). PYTHONUTF8=1 dodges the cp1251 console crash.

.EXAMPLE
pwsh ./Run-Audit.ps1 # OwnSharp only (fast)
Expand All @@ -39,7 +44,7 @@ param(
# Passing it explicitly always wins, including a lower value.
)
$ErrorActionPreference = "Stop"
$env:PYTHONUTF8 = "1" # report.py prints '>=' / '·' — crashes on a cp1251 console
$env:PYTHONUTF8 = "1" # report.py prints '>=' and U+00B7 MIDDLE DOT - both crash a cp1251 console
New-Item -ItemType Directory -Force -Path $Out | Out-Null

# Read HERE, in this script's own scope: $PSBoundParameters describes this
Expand All @@ -49,7 +54,7 @@ New-Item -ItemType Directory -Force -Path $Out | Out-Null
$lineTolExplicit = $PSBoundParameters.ContainsKey('LineTol')
. (Join-Path (Join-Path $PSScriptRoot "scripts") "LineTolPolicy.ps1")

# 1. worktree of main — audit/ is on main; the dev checkout may sit on a feature branch.
# 1. worktree of main - audit/ is on main; the dev checkout may sit on a feature branch.
git -C $OwnNet fetch origin main -q
if (Test-Path (Join-Path $Worktree ".git")) {
git -C $Worktree fetch origin main -q 2>$null
Expand All @@ -59,7 +64,7 @@ if (Test-Path (Join-Path $Worktree ".git")) {
git -C $OwnNet worktree add --detach $Worktree $Ref
}
if (-not (Test-Path (Join-Path $Worktree "audit\aggregate\report.py"))) {
throw "audit/ not found in $Worktree — is '$Ref' the branch that has audit/?"
throw "audit/ not found in $Worktree - is '$Ref' the branch that has audit/?"
}
# pyyaml is for the WORKTREE's report.py (it still reads the YAML taxonomy through
# its own normalize.py). The local aggregate/normalize.py is stdlib-only.
Expand All @@ -71,14 +76,14 @@ $sarifInputs = @() # "tool=path" for normalize
# ---- provenance (producer-provenance/v1, Own.NET#266 slice 1B) ---------------
# SARIF carries no run identity, so occurrence identity has to come from outside
# it. The run id is stamped HERE, before any producer starts, so it names the
# ANALYSIS rather than the normalization — re-normalizing this same recorded run
# ANALYSIS rather than the normalization - re-normalizing this same recorded run
# later reuses this manifest and reproduces the same occurrence ids. The
# normalizer never invents one: no manifest entry means occurrence_id: null, and
# the record says why.
#
# A GUID, not just a timestamp: two audits started in the same second would
# otherwise share producer run ids, and "rare" is not a property an identity
# contract may have. Determinism survives — the id is written to the manifest and
# contract may have. Determinism survives - the id is written to the manifest and
# read back from it, never recomputed.
$auditRunId = "audit-{0}-{1}" -f [DateTime]::UtcNow.ToString("yyyyMMddTHHmmssZ"),
[Guid]::NewGuid().ToString("N")
Expand All @@ -87,7 +92,7 @@ if ([string]::IsNullOrWhiteSpace($sourceCommit)) { $sourceCommit = $null }

# The analyzers read the WORKING TREE, not HEAD. On a dirty target `git rev-parse
# HEAD` still answers, and recording that answer would attribute findings to a
# commit that does not contain the analyzed bytes — the same fabrication as
# commit that does not contain the analyzed bytes - the same fabrication as
# stamping a reused CodeQL database with today's commit, just harder to notice.
# Unknown is a value this schema carries, so it is recorded as unknown.
#
Expand All @@ -99,7 +104,7 @@ if ($sourceCommit -and $dirty.Count -gt 0) {
# One string, one -f: the format operator binds tighter than '+', so a
# concatenation of two format strings would silently format only the second one
# and leave the first one's placeholders as literal text.
$msg = "Target tree is dirty ({0} changed/untracked path(s)) — recording source_commit as null: the analyzed bytes are not the ones in {1}." -f $dirty.Count, $sourceCommit.Substring(0, 8)
$msg = "Target tree is dirty ({0} changed/untracked path(s)) - recording source_commit as null: the analyzed bytes are not the ones in {1}." -f $dirty.Count, $sourceCommit.Substring(0, 8)
Write-Host $msg
$sourceCommit = $null
}
Expand All @@ -114,7 +119,7 @@ function Add-Provenance {
runners (Run-Infersharp.ps1 / Run-Roslyn.ps1), possibly yesterday, possibly
against another commit and another configuration. Stamping them with this
run's id and this checkout's HEAD would be provenance about an analysis
nobody observed — precisely the fabricated identity the contract exists to
nobody observed - precisely the fabricated identity the contract exists to
prevent. So they get producer_name and input_digest, which are facts about
the bytes on disk, and nothing else. Their occurrence ids stay null until
those runners emit their own provenance sidecars.
Expand All @@ -124,12 +129,12 @@ function Add-Provenance {
[Parameter(Mandatory)][string]$SarifPath,
$RunId = $null,
$SourceCommit = $null,
$Version = $null, # null, not "" — unknown must read as unknown
$Version = $null, # null, not "" - unknown must read as unknown
$ConfigDigest = $null
)
# Lower-cased on purpose: Get-FileHash returns upper-case hex and the normalizer
# compares the digest string verbatim, rejecting a mismatch outright. A case
# difference would fail the whole run rather than degrade quietly — correct, but
# difference would fail the whole run rather than degrade quietly - correct, but
# a needless way to find out.
$sha = (Get-FileHash -Algorithm SHA256 -LiteralPath $SarifPath).Hash.ToLowerInvariant()
$nz = { param($v) if ([string]::IsNullOrWhiteSpace([string]$v)) { $null } else { [string]$v } }
Expand All @@ -152,7 +157,7 @@ try {
1> $ownsarif 2> (Join-Path $Out "own-check.err")
} finally { Pop-Location }
if (-not (Test-Path $ownsarif) -or (Get-Item $ownsarif).Length -lt 2) {
throw "OwnSharp produced no SARIF — see $Out\own-check.err"
throw "OwnSharp produced no SARIF - see $Out\own-check.err"
}
Write-Host "OwnSharp SARIF: $ownsarif ($((Get-Item $ownsarif).Length) bytes)"
$sarifInputs += "own-check=$ownsarif"
Expand All @@ -169,15 +174,15 @@ if ($Codeql) {
$codeqlDbBuiltNow = $false
if ($RebuildCodeqlDb -or -not (Test-Path (Join-Path $CodeqlDb "codeql-database.yml"))) {
New-Item -ItemType Directory -Force -Path (Split-Path $CodeqlDb) | Out-Null
Write-Host "CodeQL: building DB (build-free) over $Target — this is the slow step…"
Write-Host "CodeQL: building DB (build-free) over $Target - this is the slow step..."
& $CodeqlExe database create $CodeqlDb --language=csharp --build-mode=none --source-root=$Target --overwrite
$codeqlDbBuiltNow = $true
} else {
Write-Host "CodeQL: reusing DB at $CodeqlDb (-RebuildCodeqlDb to force) — its source commit is unknown to this run"
Write-Host "CodeQL: reusing DB at $CodeqlDb (-RebuildCodeqlDb to force) - its source commit is unknown to this run"
}
$cqsarif = Join-Path $Out "codeql.sarif"
# security-and-quality is the practical max for a desktop app; -Strict adds the
# experimental suite (marginal here — mostly web-shaped queries — but complete).
# experimental suite (marginal here - mostly web-shaped queries - but complete).
$suites = @("codeql/csharp-queries:codeql-suites/csharp-security-and-quality.qls")
if ($Strict) { $suites += "codeql/csharp-queries:codeql-suites/csharp-security-experimental.qls" }
& $CodeqlExe database analyze $CodeqlDb --format=sarifv2.1.0 --output=$cqsarif --threads=0 @suites
Expand All @@ -193,7 +198,7 @@ if ($Codeql) {
-SourceCommit $(if ($codeqlDbBuiltNow) { $sourceCommit } else { $null })
}

# Infer# (build-required) — fold in if a SARIF is present. Produce it first with
# Infer# (build-required) - fold in if a SARIF is present. Produce it first with
# Run-Infersharp.ps1 (WSL). Infer# reports at the last-access line, so the clustering
# window is raised to 8 below unless -LineTol was passed explicitly.
if (Test-Path (Join-Path $Out "infersharp.sarif")) {
Expand All @@ -203,7 +208,7 @@ if (Test-Path (Join-Path $Out "infersharp.sarif")) {
Add-Provenance -Tool "infersharp" -SarifPath (Join-Path $Out 'infersharp.sarif')
}

# Roslyn analyzer packs (build-required) — fold in if a SARIF is present. Produce it
# Roslyn analyzer packs (build-required) - fold in if a SARIF is present. Produce it
# first with Run-Roslyn.ps1 (VS2022 build). High volume and shifted/generated locations,
# so the clustering window is raised to 8 below unless -LineTol was passed explicitly.
if (Test-Path (Join-Path $Out "roslyn.sarif")) {
Expand All @@ -214,7 +219,7 @@ if (Test-Path (Join-Path $Out "roslyn.sarif")) {
}

# The folding above is automatic; the clustering window it needs used to be manual.
# Resolve it now that the input set is final — Infer# and Roslyn report shifted or
# Resolve it now that the input set is final - Infer# and Roslyn report shifted or
# generated locations, so at the default window one defect splits into several
# findings. An explicitly passed -LineTol is honoured exactly, including when it is
# lower than those tools want: the mismatch is reported, never corrected behind the
Expand All @@ -240,13 +245,13 @@ foreach ($s in $sarifInputs) { $nargs += @("--sarif", $s) }
$absStrip = ($Target -replace '\\', '/').TrimEnd('/')
# Persist the manifest next to the artifacts it describes. Keep it: re-normalizing
# this recorded run WITHOUT it produces occurrence_id: null everywhere, because the
# normalizer will not mint a run id of its own — and it should not, since a run id
# normalizer will not mint a run id of its own - and it should not, since a run id
# minted at normalization time would describe the normalization instead.
$manifest = Join-Path $Out "provenance.json"
@{ schema_version = "producer-provenance/v1"; inputs = $provenanceInputs } |
ConvertTo-Json -Depth 6 | Set-Content -LiteralPath $manifest -Encoding utf8
$withRun = @($provenanceInputs.Values | Where-Object { $_.producer_run_id }).Count
Write-Host "Provenance manifest: $manifest ($auditRunId; $withRun of $($provenanceInputs.Count) producer(s) carry a run id — the rest are pre-existing SARIF and their occurrence ids stay null)"
Write-Host "Provenance manifest: $manifest ($auditRunId; $withRun of $($provenanceInputs.Count) producer(s) carry a run id - the rest are pre-existing SARIF and their occurrence ids stay null)"
python "$PSScriptRoot\aggregate\normalize.py" @nargs --strip "$leaf" --strip $absStrip --strip "/$absStrip" --provenance $manifest --json $findings
foreach ($fmt in @(@{f='markdown';e='md'}, @{f='html';e='html'}, @{f='json';e='json'})) {
python "$Worktree\audit\aggregate\report.py" --findings $findings --format $fmt.f --target $leaf --commit $commit --line-tol $LineTol |
Expand Down
13 changes: 9 additions & 4 deletions Run-Infersharp.ps1
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
#Requires -Version 7
# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell
# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax
# error in whatever word happened to contain it. This file is ASCII so 5.1 can read
# it, and this line is what 5.1 then reports instead: a refusal about versions.
<#
.SYNOPSIS
Run Infer# (build-required) over STS's own built binaries via the WSL2 infersharp
distro, and write artifacts/infersharp.sarif for the audit to fold in.

.DESCRIPTION
Infer# analyzes COMPILED binaries (.dll + .pdb) — build-required (audit/ §3.2). It
copies the source-mappable STS assemblies (those with a .pdb — which skips the
Infer# analyzes COMPILED binaries (.dll + .pdb) - build-required (audit/ section 3.2). It
copies the source-mappable STS assemblies (those with a .pdb - which skips the
DevExpress / 3rd-party DLLs that ship no PDB and couldn't be source-mapped anyway)
out of the build output, runs Infer# inside the WSL distro, and drops the SARIF where
Run-Audit.ps1 folds it in. Infer#'s Pulse analysis is SLOW (minutes to hours,
dominated by the big assemblies — Broker, BrokerDataClasses).
dominated by the big assemblies - Broker, BrokerDataClasses).

Infer# reports a leak at the LAST-ACCESS line (the alloc line is in the message), so
fold with a wider Run-Audit.ps1 -LineTol (e.g. 8) to cluster it with own-check/codeql.
Expand Down Expand Up @@ -46,7 +51,7 @@ $wslSarif = & $toWsl (Join-Path $Out "infersharp.sarif")
wsl -d $Distro -- bash -lc "cd ~/infersharp && rm -rf infer-out && ./run_infersharp.sh $wslBin && cp infer-out/report.sarif $wslSarif && echo OK"
if (-not (Test-Path "$Out\infersharp.sarif")) { throw "Infer# produced no SARIF (wsl exit $LASTEXITCODE)" }
# Drop findings in third-party assemblies Infer# pulled in via their PDBs (paths like
# 'file:/_/BASE/src/...') and any generated files — keep STS code only.
# 'file:/_/BASE/src/...') and any generated files - keep STS code only.
$env:PYTHONUTF8 = '1'
python -c @"
import json,re
Expand Down
9 changes: 7 additions & 2 deletions Run-Roslyn.ps1
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
#Requires -Version 7
# PowerShell 7+ only, and stated where the ENGINE can act on it. Windows PowerShell
# 5.1 reads a BOM-less file as ANSI, so a non-ASCII byte used to surface as a syntax
# error in whatever word happened to contain it. This file is ASCII so 5.1 can read
# it, and this line is what 5.1 then reports instead: a refusal about versions.
<#
.SYNOPSIS
Run the NuGet Roslyn analyzer packs (build-required tier) over STS via VS2022 and
Expand All @@ -7,7 +12,7 @@
Roslyn analyzers run DURING compilation, so this rebuilds the target with the packs
injected (canonical inject props, gated on /p:OwnAudit=true) plus an is_global config
that downgrades every analyzer diagnostic to 'warning' (so an error-default rule like
Meziantou MA0037 is COLLECTED, not build-breaking — real CS#### errors still fail).
Meziantou MA0037 is COLLECTED, not build-breaking - real CS#### errors still fail).

Needs VS2022 BuildTools (Roslyn 4.x): the modern packs no-load on VS2019/Roslyn 3.x
(Gu.Roslyn.Extensions version conflict). Each pack is cached in its OWN subdir so it
Expand Down Expand Up @@ -35,7 +40,7 @@ $ErrorActionPreference = 'Stop'
$env:PYTHONUTF8 = '1'
$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$msbuild = (& $vswhere -products * -version "[17.0,18.0)" -latest -find "MSBuild\**\Bin\MSBuild.exe" 2>$null | Select-Object -First 1)
if (-not $msbuild) { throw "VS2022 msbuild not found — BuildTools 17.x (Roslyn 4.x) is required for the modern packs." }
if (-not $msbuild) { throw "VS2022 msbuild not found - BuildTools 17.x (Roslyn 4.x) is required for the modern packs." }
$nuget = 'C:\Repos\_ownaudit\tools\nuget.exe'

# 1. per-pack analyzer cache (each pack in its OWN subdir -> its own Gu.Roslyn.Extensions)
Expand Down
Loading
Loading