[Tristan-Stoltz-ERC]

Summary

Adds a standalone Rust evaluator for engine-grounded PCB copper-routing experiments. It validates every attempt against an explicit dataset manifest, derives route validity from reported connectivity/DRC evidence, retains timeouts and tool errors in the denominator, and produces exact-head receipts plus paired comparisons.

Current implementation

• Pinned dataset and run identities, board-input hashes, per-board coverage, router/engine version pins, DRC-policy digest and runtime/environment record.
• Native KiCad DRC report parsing and source/output artifact hashing for the external-evidence path. Plain normalized scoring commands reject external manifests; external comparisons must use the verified path.
• PCBWorld CSV import that preserves the source metric fields and adds final-runtime clean-pass rates at 30, 60, 180, 300 and 1800 seconds. Unknown runtimes stay in the denominator and are not credited as timed passes.
• A monotone, proof-carrying anytime incumbent archive: reads native board and raw DRC bytes from safe paths, verifies source board identity against manifest bytes, accepts only route-valid checkpoints, preserves failed attempts for audit, and returns a wirelength/via Pareto front at a declared deadline.
• CLI commands for archive creation, checkpoint verification, candidate recording and deadline selection. Before recording or selecting, the CLI reopens all archived board/DRC files and rederives every receipt under the current exact evaluator head; edited eligibility flags and changed bytes fail closed.
• Adversarial tests for missing/duplicate rows, crossings, timeouts, manifest/source mismatch, path traversal, Pareto trade-offs, later invalid checkpoints and tampered archive/artifact contents.

Competitive context

PCBWorld-Bench is the intended external baseline, with identical board bytes, split, engine, DRC policy, run budget and machine class required for comparison. The upstream source currently identifies PCBWorld v1.1.1 and KiCad 9.0.8; the environment is BSD-3-Clause while the separate engine is GPLv3. The protocol records the exact upstream engine/split pins and warns against rebuilding the board corpus silently.

Public, vendor-reported figures checked on 2026-10-22: ProtoFlow reports 98/99 D3-A and 30/30 D3-B attempts at a 30-minute per-attempt cap, and 0.929/0.933 at three minutes. DeepPCB reports 92/99 and 24/30 on its three-minute snapshot scoring, with wide intervals on the ten-board D3-B set. These are not independently audited by this PR. A comparable score needs the same board bytes, policy, engine, time snapshot and scorer.

Exact evidence status

Current head: 1cb2722.

• Focused workflow #39715484859: queued.
• Repository CI #39715484865: queued.
• Earlier source heads had formatter-only blocks and their newer runs were cancelled when the branch advanced. No pass is claimed for the current head.

The 30-board intake audit currently authorizes 0/30 candidate boards for execution. It pins 14 original-source license blobs at immutable commits, but exact source-board bytes, required-net counts and license applicability review are still outstanding. A further board-directory license lead has been documented in issue #2473, but is not yet reflected in the committed audit fixture. No actual external PCBWorld routing run or SOTA claim is made.

Follow-up issues

• PCB-ROUTE-BENCH-001 #2473: reproduce engine-grounded routing baselines.
• PCB-ANYTIME-001 #2608: preserve verified clean incumbents.
• PCB-SKELETON-001 #2609: prevent placement/net/rule mutations from counting as routing success.

Current validation status (2026-10-22)

Latest branch head: 88995561a93ee32b0b69abc207c6b62579d01bc1. An earlier exact-head run surfaced test-fixture and policy-pin defects; those corrections are committed. The latest native test/evaluator runs are still queued, so no passing result is claimed for this head.

Immutable license-audit gate

The manifest builder no longer accepts caller-provided license labels or license-source identities. Intake schema v0.2 requires a path and SHA-256 for the audit manifest. The builder checks the audit's source PCBWorld commit/repository and split-config Git blob, requires the external audit status approved_for_benchmark_execution, requires each selected row to be explicitly eligible, checks the audit's original source repository and PCBench metadata blob, verifies the exact prepared-board bytes against the audit-approved board SHA-256, and verifies the local license-evidence bytes against the audited source Git blob and immutable repository/commit/file URL.

Synthetic approval fixtures are explicitly labelled synthetic and can only produce a synthetic-prefixed dataset/benchmark ID. They cannot be promoted into an external benchmark manifest. Negative tests cover stale audit digest, unapproved row, unlisted board, source-repository mismatch, license-blob mismatch, prepared-board digest mismatch, path traversal, file aliasing, and post-manifest byte changes.

The real candidate audit still records 0/30 boards approved for benchmark execution and has status metadata_inventory_only_not_benchmark_manifest; therefore it must remain blocked. The builder is intended to preserve that fact, not manufacture approval. The gate binds an explicit reviewed audit decision, but legal applicability remains a human review responsibility.

Benchmark-validity expansion

Opened PCB-DRC-FIDELITY-001 #2626 and added a protocol section for source-rule fidelity. A route should be scored separately under the published compatibility policy and the strongest reconstructable source-authoritative policy; inferred/relaxed rule differences must be disclosed, not folded into a single success rate. The new protocol also records the difference between the PCBWorld paper's @5 best-potential selection and DeepPCB's vendor-reported fixed attempt budgets. Those headline percentages are not directly interchangeable.

Earlier exact-head runs validated the synthetic routing evaluator and PCBWorld metric/DRC contracts on earlier commits. Their evidence does not transfer to this changed head.

The protocol also now includes the pinned paper Table 3 CP/time baselines, separates exact KDD paper reproduction (D3-A 99 boards + D3-B 10-board subset, @5 protocol) from the custom 30-board screening pilot, and explicitly treats D3-C as a new challenge evaluation, not a published comparison. See docs/research/PCB_ROUTE_BENCHMARK_PROTOCOL_V0_1.md.

Benchmark intake boundary

The external benchmark pilot is not yet eligible to run. The current 30-board source-license audit records 0/30 boards approved for benchmark execution; source-license blobs have been pinned for a subset, but the original source-byte intake and license review are not complete. I opened PCB-LICENSE-GATE-001 #2613 because the manifest builder currently trusts a caller-supplied license label rather than requiring each board to match an immutable reviewed license-audit record. Do not interpret this evaluator PR or synthetic fixture score as a run on real PCBWorld boards.

The external comparison protocol now records the published Table 3 @5 clean-pass baseline and distinguishes a same-protocol KDD reproduction (D3-A 99-board subset plus D3-B 10-board subset) from the current-release/custom 30-board screening pilot and the paper's unreleased D3-C challenge results. These dataset/metric boundaries must be aligned before any numerical superiority claim.

Companion design-invariance PR (2026-10-22)

Draft PR #2630, stacked on this branch, adds a native KiCad S-expression design-skeleton fingerprint. It binds a source-derived fingerprint into external schema v0.2 manifests and requires output/source skeleton equality in the verified comparison path. Only recognized routing primitives and generated zone fills are excluded; placement, pads/nets, net classes, outline and zone settings remain part of the fingerprint. Its exact-head evaluator workflow is queued in run #39738233832; no pass is claimed.

The source/license audit remains deliberately blocked at 0/30 real candidate boards authorized. The rule-fidelity program is now tracked separately in issue #2626.