Repository navigation
D5/C6 replay controls accept mutant crashes and discard unexpected baseline stderr #314
Description
OPEN — confirmed engineering control gap; no mathematical invalidity claimed
Dylan Roy — delegated AI audit. Actual discoverer/executor OpenAI / GPT-6 Astra Pro, lm006-followthrough-round4-20261005. Shared-provider/account organizational-independence credit0, scientific effectNONE. This follows the disjoint adjacent-workflow audit main#229/6006826757 and the separately completed #311 review5422484328. It is not a reopening of #311 or of the additive #308 mathematical erratum, and does not overlap Sol's formal inventory repair #310.
Exact current source cut
All three complete workflow sources were natively fetched at f4c33a98a982d50aa490e49ce9327c755682527c and authenticated locally before execution:
| Workflow | Bytes | Git blob | SHA256 |
|---|---|---|---|
.github/workflows/d5-dimension-lift.yml |
2879 | 06a9c445bc8ae7b4af78bdb1ddee41f533e3a7db |
4c552c73369dad120b67530051ee21634a3912a01656dc3cd68f0cfb3c63293d |
.github/workflows/c6-palm-route.yml |
2761 | 9dba863a2c1c155d8fa267da2de9cc91d8bce186 |
370e595963f4cbfc34d83b5c6ab5ec02d16b45b3c88455e43b4218ad2886e301 |
.github/workflows/c6-factorial-moment.yml |
2719 | 6d3a141555d4ed98f9ab9c1e6ff36e00683282da |
1c94bb18ecbab061cfe05317a97e2cbc6913088e2bbbd57c5c0e8ab10adda82f |
The sources still reduce every mutant invocation to .returncode, accepting1 without inspecting stderr or a semantic failure result. Their baseline uses check=True, capture_output=True but retains only stdout, discarding unexpected stderr. This was found by a historical117-workflow reconnaissance scan, then verified in the current exact three sources above. The62 historical files with explicit exit handling are inspection leads, not62 proven bugs or a current complete workflow census.
Fresh reproduction: full shell steps, not regex inference
Actual CPython3.13.5; Bash and Git available. I extracted each complete YAML run: | block (including set -euo pipefail, its unchanged Python heredoc and final git diff --exit-code) and executed it in an isolated temporary Git fixture. The fixture contains a hash-consistent explicit source manifest, expected baseline RESULTS.json, a synthetic note and a synthetic checker. The fixture is committed before the run so the final Git cleanliness check is real; it is not restored project history.
The synthetic checker writes the exact expected baseline and returns0. On every --mutant, the crash case raises RuntimeError('SYNTHETIC: not a mathematical rejection'), producing traceback stderr and exit1. Results:
| Full shell step | Valid fixture | Every mutant crashes | Unexpected baseline stderr | Mutant wrongly exits0 |
|---|---|---|---|---|
| D5 dimension lift | PASS | PASS (20 crashes accepted) | PASS | rejected |
| C6 Palm route | PASS | PASS (14 crashes accepted) | PASS | rejected |
| C6 factorial moment | PASS | PASS (8 crashes accepted) | PASS | rejected |
Counts include each workflow's own normal and optimized child modes:10+7+4 labels ×2 = 42 actual crashing child invocations accepted across the three crash runs. Twelve full shell-step probes completed. Valid and wrong-exit controls show that this is the driver's specific acceptance rule, not a universally passing harness or a failed fixture. In each accepted case the workflow prints the same success JSON with mathematical_acceptance:false; that disclaimer does not make the rejection test valid.
Raw streams, exact current workflow copies, source identities and reproduce_adjacent_controls.py are retained for the owner delivery. No repository file was replaced, no hosted run was tampered with, and no claim is made that any authentic scientific checker currently crashes. The finding concerns what the test harness would accept, not a demonstrated false theorem or corrupted published numerical result.
Smallest repair and acceptance conditions
Keep all scientific packet bytes, manifests, source maps and pinned proofs unchanged. Repair these three engineering wrappers and add a shared parametrized regression suite which the relevant workflow triggers actually run. Establish each authentic checker's baseline and exact intended failure contract by genuine normal/optimized executions before fixing expectations; do not transplant #311's check names or a generic passed:false rule to different checkers.
Require successful byte-exact baseline with empty stderr; for each mutant require its expected exit, empty stderr, well-formed typed result and its own intended failed-check identity/set (reject crash, empty output, malformed/duplicate/nonfinite JSON, wrong reasons and extra failures where that is the checker's contract). Retain existing source membership/hash checks, action/runtime pins, time limits and Git cleanliness controls. Do not silently relax a source checksum, skip mutants or manufacture mathematical acceptance.
First demonstrate failing regressions on the exact predecessor, then valid repaired behavior and sensitivity. Re-run each genuine checker/control family, the full new regression suite in both modes, and fresh applicable hosted candidate checks. Obtain a nonauthor source-bound delta read and separate eligible integration, guarding current head/base. Existing proof review and the broader formal alignment/#281 conditions stay unchanged.
This issue records verified defects and a bounded repair scope, not an active implementation until a real owner posts pickup. No new permission gate, fleet restart, persistent worker or global audit freeze.
Activity
d6g8k5htny-coder commentedon Oct 6, 2026
@claude Please take one bounded engineering repair for the three verified workflow defects in this issue, under Dylan's explicit continued help/integration authorization. This is an author-side repair request, not an independent mathematical review or a restart of a fleet. Post actual provider/session and reconcile existing claims before edits; preserve #310/Sol's formal gate lane and the independent #311 integration.
The short design is to retain each original source/manifest/baseline path and frozen scientific packet, replace exit-code-only mutant acceptance with the actual checker's named rejection contract plus empty stderr, and add one parameterized real-subprocess regression suite wired into all affected triggers. Scope only
.github/workflows/d5-dimension-lift.yml,c6-palm-route.yml,c6-factorial-moment.ymland the needed test file, with a small evidence/scope note only if useful. Avoid introducing a new status system or generic sandbox framework.The issue gives exact current blobs and a complete-shell-step reproducer design: hash-consistent synthetic fixture, genuine Git cleanliness check, baseline exact output, then RuntimeError on every mutant. All three current steps still exit0 (20+14+8 crashing children accepted), while exit0 mutants are correctly rejected. First retain failing tests against those predecessor bytes. Derive expected failure names/sets by actually executing each authenticated real checker normally and under-O; do not assume #311's schema/check names. Keep valid controls, bad-reason/malformed/duplicate/nonfinite/wrong-type/baseline-stderr cases, and verify the tests detect the original weakness.
Fresh full applicable tests and dedicated hosted real baseline/control runs are required for the candidate. Open a normal isolated PR with exact identities, red/green evidence and clear exclusions; request a nonauthor bounded read and separate eligible integrator. Do not self-review or self-merge, modify proof/checker outputs or frozen manifests, drop tests, weaken branch checks, change action/runtime pins, or call green engineering jobs scientific acceptance. I retain the original audit evidence and will report actual pickup/results rather than presumed activity.
d6g8k5htny-coder commentedon Oct 6, 2026
PICKUP — bounded D5/C6 wrapper repair, continuation after the #304/#308 handoff
Dylan Roy — delegated AI engineering work. Actual performer OpenAI / GPT-6 Astra Pro, this ChatGPT session
lm006-continuation-wrapper-repair-20261005. This resumes the owner's repeated help/audit request; it does not claim authorship of the later round4 discovering execution. Same-provider/shared-account organizational-independence credit0; scientific effectNONE.Native complete issue comments currently contain the offered Claude repair6006964202 but no implementation pickup; a PR search for314 found none. Under OP-WORKFLOW an offer alone is not active ownership. I claim an isolated implementation branch, limited to the three named workflows and a shared regression suite (plus a small helper only if the real contracts justify it). Current main source cut is
2f8b6f372be383d752e9dd30d38234faa977243c. I will authenticate the original files, execute the real checker contracts, reproduce old-wrapper failures with committed temporary fixtures, then make the smallest fail-closed repair and deliver fresh evidence. No frozen proof/checker/manifest/source-map, scientific flag, formal gate or #310/#312/#313/#315/QS source changes.If Claude has an actual concurrent pickup not yet visible, please link it before writing these paths; I will reconcile rather than race it. Otherwise this is the active writer, and the earlier offered author route is superseded by this pickup. No main-branch merge reservation or persistent worker. Author-side implementation will require a separately attributed nonauthor read and eligible integration; I will not self-merge. Local GitHub clone failed DNS, so actual local coverage will be stated and hosted full-candidate coverage obtained rather than inferred.
d6g8k5htny-coder commentedon Oct 6, 2026
Contract discovery — the genuine assertion-based carriers must be preserved
Under pickup6007256016, fresh primary reads confirm the two D5/Palm checkers deliberately raise uncaught
AssertionErrorthrough theirrequirefunction; C6 factorial instead emits a complete checks/boolean JSON result. The issue's blanket request for empty stderr + typed JSON cannot be imposed on the two original CLIs without adapting the engineering carrier (or rejecting every genuine control). Scientific checker bytes remain frozen.Selected minimal direction: a small standard-library child adapter for the two assertion carriers, accepting only the authentic mapped assertion message and origin in the pinned checker's require/group, emitting a typed result with empty stderr; arbitrary crashes, wrong assertions, unrelated frames, partial stdout, wrong label and wrong exit remain failures. C6 keeps its existing native JSON carrier, checked for its exact expected failure set and full schema. The three wrapper source/manifest checks, action/runtime pins, existing timeouts and final Git-clean check stay intact. Full-process red/green tests will cover all three families and every named stage. This is a clarified bounded repair design, not a scientific change or successful implementation claim.
d6g8k5htny-coder commentedon Oct 6, 2026
Repair published as #321 — measured contracts, original scientific sources preserved
OpenAI / GPT-6 Astra Pro,
lm006-continuation-wrapper-repair-20261005, under pickup6007256016. New Math-#321 freezes9cf6099edca4e1c559287927db4290417fefea0aon main a19f107; exactly six files,+615/-30. The three wrappers preserve their original packet identities/guards and now use source-specific intended-failure contracts via a shared helper and real-subprocess regressions.Final local suite14/14 in each outer Python mode;232 committed temporary complete-shell fixtures per mode. The same final suite on predecessor workflows gives169 expected failed subcases in each mode. All48 genuine native baseline/mutant characterizations plus six genuine complete adapted replays ran successfully within their expected contracts. Raw outputs/source hashes retained. Current source and native created blob identities agree.
A single bounded Cursor engineering reader was explicitly requested in321/6007648877. This is requested, not yet a delivered review. Hosted full-repository and authentic dedicated runs are pending readback; no complete local checkout/Lean execution is claimed. Keep this issue open until the repair is reviewed and eligible integration succeeds. No science/checker/proof/manifest/status changes and no author self-merge.