Skip to content

feat(benchmark): add causal replay checks for adaptive mine policies - #2053

Draft
Tristan-Stoltz-ERC wants to merge 8 commits into
mainfrom
feat/mine-adaptive-causality-bench-v0-1
Draft

Tristan-Stoltz-ERC wants to merge 8 commits into
mainfrom
feat/mine-adaptive-causality-bench-v0-1

Conversation

@Tristan-Stoltz-ERC

@Tristan-Stoltz-ERC Tristan-Stoltz-ERC commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements the first increment of MINE-ADAPTIVE-001 #2051: a standalone Rust trace auditor for causal admissibility and non-anticipative counterfactual replay.

  • Verifies every declared policy input was delivered no later than its decision timestamp; collection/observation time alone is not enough.
  • Blocks declared use of evaluator-only truth, unqualified/rejected or missing results, missing provenance, QC failures, unknown observation references, duplicate references, duplicate IDs and out-of-order decisions.
  • Pairs two counterfactual futures that share the same pre-fork delivered-event history; demands identical pre-fork actions and evidence references while requiring a genuine post-fork outcome divergence.
  • Rejects a planted pair where policy actions diverge before outcomes become observable.
  • Emits exact-corpus SHA-256 + source-commit receipts and reason codes.
  • Contains 11 frozen authored synthetic traces and a focused Rust workflow; no third-party mine data or physical measurement actions.

Paradigm shift

A score from a policy that can see future assay results is not merely a slightly optimistic estimate—it is a causally invalid experiment. The counterfactual fork check makes that failure falsifiable by changing only the future outcome while holding earlier observable history fixed.

Research anchors

  • Armstrong et al. (2021), Adaptive open-pit mining planning under geological uncertainty: rolling-horizon planning updates the geological model as information arrives; it improved mean outcomes on studied cases but did not win on every realization. https://doi.org/10.1016/j.resourpol.2021.102086
  • Khalifi et al. (2026 preprint), Adaptive mine planning under geological uncertainty: A POMDP framework for sequential decision-making: formalizes sequential action and belief updates. Its reported benefits remain claims to reproduce, not fixture values. https://arxiv.org/abs/2605.13702
  • Jeuken & Forbes (2024), value of drilling under chance-constrained blending: https://doi.org/10.1007/s42461-024-01061-8

Independent semantic check / CI status

An independently implemented JavaScript checker matched 11/11 expected dispositions and exact reason-code sets, counted five causal violations, accepted the valid counterfactual pair, and rejected both branches of the planted pre-fork disagreement.

Exact head 22dd5e0a5b0bdd46cda177cbb2caee4e18b7295d passed the focused Mine Adaptive Causal Replay Benchmark workflow 39165916229: rustfmt passed, 8 Rust unit tests passed, and the receipt matched all 11 controls, five causal violations, two counterfactual groups, and one rejected pair. Exact-head repo CI run 39165916263 subsequently completed successfully, in addition to focused run 39165916229. This verifies the Rust trace-auditor contract and repository CI at head 22dd5e0a5b0bdd46cda177cbb2caee4e18b7295d; its claim ceiling remains synthetic trace semantics only.

Acceptance targets / limitations

The frozen contract targets zero future-event use, zero use of evaluator-only truth, and identical actions for identical pre-fork observable history. It audits the declared input-event list; by itself it cannot prove a policy executable did not read an undeclared field. A later evaluation harness must capture the complete delivered observation snapshot, hash it, bind policy binary/configuration, and independently verify runtime logs or enforce a sandboxed API boundary.

The public fixtures are not a blinded predictive test. This PR audits trace semantics only—it does not score mining utility and says nothing about real geology, reserve quality, worker safety, or industrial performance.

Next stage: a separate policy runner/evaluator that compares static, two-stage, rolling-horizon, and decision-aware measurement policies against the same held-out synthetic realizations, with clairvoyant plans labeled only as upper bounds. Reuse MINE-SYSTEM, ORE-VOI, and blend contracts rather than duplicating those owners.

Exact-head verification update

Exact head 22dd5e0a5b0bdd46cda177cbb2caee4e18b7295d passed focused causal replay run 39165916229 and repository CI 39165916263. Focused Rust validation passed formatting, 8 tests and the 11-scenario receipt gate; repository CI completed successfully. Claim ceiling remains synthetic trace semantics; no field-performance claim.

Copy link
Copy Markdown
Contributor Author

Cross-domain follow-up from the shipyard recovery work: draft PR #2730 checks multi-scenario information-set consistency at each decision tick, with the expected scenario inventory and source-owned event logs separated from candidate-only action traces. This complements rather than replaces this PR's stronger domain-specific event-delivery/QC/evaluator-visibility checks and paired counterfactual receipt. I have explicitly noted that shared canonicalization/receipt/non-anticipativity primitives should be extracted as the contracts mature, rather than merging two independent implementations without review. #2730's exact-head Rust workflow remains pending, so the new code is not yet validated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant