Skip to content

feat(shipyard): audit non-anticipative recovery policy replay - #2730

Draft
Tristan-Stoltz-ERC wants to merge 37 commits into
research/shipyard-robust-scenario-audit-v0from
research/shipyard-nonanticipative-policy-v0
Draft

Tristan-Stoltz-ERC wants to merge 37 commits into
research/shipyard-robust-scenario-audit-v0from
research/shipyard-nonanticipative-policy-v0

Conversation

@Tristan-Stoltz-ERC

@Tristan-Stoltz-ERC Tristan-Stoltz-ERC commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a standalone, dependency-free replay gate for a shipyard-specific robustness dimension: non-anticipativity across a declared scenario set. A policy must not choose different current actions when its source-observable histories are identical.

Relationship to existing work

This complements PR #2053 — Mine Adaptive Causal Replay, which already checks mine-specific event-delivery admissibility, QC/source qualification, evaluator-only truth, and paired counterfactual forks with a digest-bound receipt. This PR is not a replacement for that contract: it checks the broader multi-scenario information-set property needed for shipyard recovery, with source-owned event logs physically separate in the API from candidate-generated action traces. We should extract common receipt/canonicalization primitives rather than copy domain-specific rules as both contracts mature.

Included

  • tools/shipyard-policy-replay: Rust 1.96 crate auditing candidate actions against a separate, source-owned event-observation log for each scenario.
  • A separate expected scenario inventory must match source event logs and candidate action traces exactly; missing or unexpected source logs/traces fail closed.
  • Candidate traces contain actions only. They cannot hide, rewrite, or retime the source observations used to construct decision-time information sets.
  • Each source event has an observation-content key; identical information sets require matching visible event IDs, observation ticks and payload keys.
  • PolicyVisible events enter the observed history; EvaluatorOnly outcomes are excluded, and different actions based on hidden branch outcomes are rejected when policy-visible history is identical.
  • Structural failures include empty inventory/source/trace/epoch sets, duplicate IDs, blank content keys, missing/extra actions, unordered epochs and blank action keys.
  • Separate results for Consistent, AnticipativityViolation, and Incomplete; policy consistency is not schedule feasibility.
  • Twelve unit-test functions, including post-observation adaptation, future-branch clairvoyance, evaluator-only truth leakage, visible-content divergence, exact-tick observation, missing/unexpected source and candidate coverage, and deterministic report order. The exhaustive 4×4×4 two-branch reveal/divergence matrix covers 64 timing combinations.
  • POLICY-REPLAY-BENCHMARK.md: benchmark strata, comparators, published 2026 baseline anchors, proposed gates, holdouts and falsification plan.
  • Rust 1.96 formatting, locked/offline test/check and Clippy-with-warnings-denied workflow.
  • Root README index entry.

Paradigm shift

Evaluate a policy mapping source-observable history to the current committed action, not just a final schedule that may exploit hindsight. Identical information states at the same decision tick must have identical canonical current actions. Hindsight/oracle schedules remain an upper-bound diagnostic, not fair online-policy competitors.

Current literature anchors

The 2026 irregular-block paper reports ABC + ALNS + disruption recovery, with Case9_50_4 metrics of 0.775 utilization, 308.0 seconds and 2,339 vehicle-minutes against GA/DE/ABC comparators (15 repetitions). A separate 2025 adaptive yard-layout paper reports 56% fewer interfering-block moves across 50 actual-data sets. These are different KPIs and instances; this PR does not compare against them directly. The 2026 paper's data are available from its corresponding author upon request, so exact reproduction remains open.

Explicit limitations

This is only an information-consistency checker. It does not optimize, compute or authenticate action/content digests, validate event truth, verify polygon/swept-path feasibility, check resources/precedence/mandatory work, or establish operational/physical safety. It does not reproduce a published study and makes no SOTA or industrial-performance claim. The caller must bind the expected inventory and source event logs to a trusted, immutable manifest/revision; this crate checks consistency and coverage but does not itself authenticate that manifest. It is also an action-trace audit, not runtime information-flow enforcement: a policy that reads hidden inputs but emits identical actions across all matching histories cannot be proven not to have read them without an instrumented or sandboxed observation API.

Validation status

I have not run Rust compilation, tests, formatting or Clippy locally (no Rust toolchain is available in this session). The exact-head workflow is the intended authoritative check; queued/pending is not a pass. Keep this PR draft until checks pass on its current head and the input/action canonicalization contract receives review.

Stacked on PR #2725 — finite-scenario robust recovery auditor. Follow-up work is tracked in Issue #2731 — observed-history rolling-horizon baseline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant