Repository navigation
Preserve downstream validation timeout evidence without weakening gates - #263
Conversation
d6g8k5htny-coder
left a comment
d6g8k5htny-coder
left a comment
There was a problem hiding this comment.
Historical frozen-candidate engineering review: AMEND
Actual reviewer: OpenAI/Codex, a fresh nonauthor engineering reviewer separate from the implementation agent. Source-exposed; same provider and GitHub account, organizational-independence credit 0. Posted through the verified account d6g8k5htny-coder. This COMMENT records the original failed candidate while the author prepares a correction. It is not current-head clearance, a GitHub APPROVE review, theorem acceptance, analytic proof, or a claim of Lean/hosted-check execution.
Exact identity and readback
- Remote reviewed commit:
cdaaf62433430757e3899948b2d59bde6471df76 - Reviewed tree:
9e65c5551b4b3dc7a24493edf07ec24493ea72ca - Original parent/base:
efc88c3fcdbd1d1384d57e2191cbcbadd557c6c6 - Locally tested/reviewed commit:
fed2f166c50990c672da695857f0430fed3147f4
I independently read GitHub's Git commit object for the remote SHA and verified the tree and sole parent exactly match the local reviewed candidate. The two commit identities differ in metadata, not source. At publication readback this PR is draft/open; its current main base is bbe85e270f2c8b747f2d5d9477c86e86e323fe15, so this review does not establish integration against that newer base. The exact original four-file diff, byte lengths, SHA-256 digests, and Git blob identities were independently checked. The reviewed checkout was clean, and I did not modify it.
Required amendment
-
Important: optional diagnostics can replace the original TimeoutExpired or prevent normal execution.
In run_validation.py at this commit, lines 69 and 73 call
time.monotonic()outside evidence-failure protection; line 87 similarly callsPath(cwd).resolve()outside the write guard. An OSError on the second clock call replaces the original TimeoutExpired before either captured stream is written. An OSError during cwd resolution replaces it after the stream writes. A first-clock-read OSError prevents subprocess.run from being called at all, including an otherwise successful ordinary baseline. These are new diagnostic-induced failure modes that violate the intended original-exception/non-timeout-behavior invariants.Reproduced independently on Python 3.11.16 in normal and optimized modes with provider faults injected at each operation. Tests compare exception object identity, not only its type. Make optional clock/cwd collection best effort, retain the originally supplied invocation identity without filesystem-dependent re-resolution where possible, and preserve the timeout even when diagnostic preparation fails. Add regression coverage before repairing the handler.
-
Related diagnostic robustness: nonfinite elapsed values produce non-standard JSON.
Lines 89-93 emit NaN or Infinity if the timing provider produces nonfinite values. Both values were independently injected; the resulting timeout.json fails strict JSON parsing. This is a fault-injection finding, not an assertion that ordinary CPython monotonic clocks normally yield these values. Represent unavailable/unusable elapsed time explicitly, with a diagnostic explanation, rather than emitting a non-JSON numeric constant or inventing a duration.
The author reproduced the initial 14-test/four-error adversarial run. I then added the first-clock-read case. The final frozen-candidate result remains 15 tests, five error instances, exit 1, in each mode: four test methods fail, with NaN/Infinity contributing two subcase errors. The initial failures are retained here; no correction or green successor is being substituted into this verdict.
Verification actually performed
- Python 3.11.16: the existing 12-test focused suite passed in normal and optimized modes, exit 0 in each.
- Python 3.12.14: the same focused suite passed in both modes as supplementary evidence, not as a substitute for the pinned Python version.
- Reviewer-authored adversarial suite: 11 methods passed in each mode; the four failing methods and five error instances are described above.
- Supported output matrix: None, empty bytes, valid UTF-8, invalid/truncated UTF-8, and NUL bytes. Captured bytes remain byte-for-byte identical; absent and empty streams remain distinguishable.
- Synthetic str payloads preserve the original timeout and record TypeError persistence failures, but do not persist the text, and their metadata counts characters rather than encoded bytes. This is nonblocking for the pinned standard subprocess.run provider, whose TimeoutExpired streams are bytes or None even with text=True. Supporting an alternate str-emitting provider would require an explicit encoding/count contract.
- Both stream writes failing, all evidence writes failing, JSON serialization failing, and final REPORT.json writing failing preserve the exact original timeout. Other subprocess exceptions and ordinary non-timeout write errors still propagate.
- Timeout during the optimized baseline or first optimized mutant stops immediately, leaves report.passed=false, retains only completed normal mode, and preserves promotion_permission=false/scientific_effect=NONE. Original tests also cover first-mode baseline/mutant timeout and a real timed-out child.
- Production timeout=30 remains unchanged in the actual subprocess call. Timeout is not accepted as mutant rejection: the timeout branch rethrows before the semantic/returncode checks.
- AST comparison verifies the 24-mutant mapping and the full completed-command output/coverage/returncode/assertion tail of execute are unchanged. Test discovery independently counts 71 packet tests without re-running the full master.
- Exactly four changed paths: workflow, runner, runner pin manifest, and the new root focused-test file. RESULTS.json, hard_gate.py, GRAPH.json, other packet files, all C126 files, and formal files are unchanged relative to the original base.
- All 12 SOURCE_FILES.json entries match actual bytes and digests. Its only changed entry is run_validation.py. The new root test is outside packet discovery.
- Workflow lines 35-38 invoke focused tests using -B -S and -B -O -S; Python 3.11 is still pinned. Existing replay counts, artifact retention, formal workflow, and required aggregate are unchanged.
git diff --checkpassed.
Focused commands, run with the verified Python 3.11.16 interpreter (and separately Python 3.12.14), from the repository root:
"$PYTHON" -B -S -m unittest discover -s tests -p test_downstream_validation.py -v
"$PYTHON" -B -O -S -m unittest discover -s tests -p test_downstream_validation.py -vHere $PYTHON denotes the selected interpreter executable; private machine paths are intentionally omitted. All remaining arguments are the actual tested arguments. Source verification used git's exact commit objects, SHA-256 over file bytes, AST equality for unchanged sections, and unittest discovery count.
Existing master evidence inspected, not re-executed
I independently read the author's original master REPORT.json and verified all 13 recorded source identities against the reviewed tree, the exit value 0, both completed modes, passed=true, sources_unchanged=true, 71 distinct tests, 24 distinct mutants, promotion_permission=false and scientific_effect=NONE. All 50 stdout/stderr pairs were inspected: both baseline logs report 71 tests/OK; all 48 mutant logs report 71 tests and assertion-failure rejection without skips or syntax/import/program errors. Both output JSON files equal RESULTS.json byte-for-byte. The successful master has no timeout sidecars.
Master REPORT SHA-256: 09087310124ff66addf912c25f81239d85319b00194c0aa0501ac41e2da5f55c.
This checks the existing receipt's consistency and exact-source binding. It is not a second master execution or organizationally independent mathematical verification. I did not rerun the reported 51 root tests, 38 formal Python tests, or 177 regressions. I did not execute Lean or claim hosted checks passed. Current-head/current-base required checks remain mandatory after correction.
Reproducible adversarial test artifact
The following is the full 15-test reviewer artifact, with only private machine paths replaced by repository-relative lookup. Save it as reviewer_adversarial.py outside the checkout and invoke it from the reviewed repository root with the same interpreter arguments used in the review:
"$PYTHON" -B -S "$REVIEWER_TEST"
"$PYTHON" -B -O -S "$REVIEWER_TEST"$REVIEWER_TEST identifies the saved script. Both original Python 3.11.16 runs exited 1 with Ran 15 tests and FAILED (errors=5). The test uses a disposable temporary directory and does not edit repository files.
import importlib.util,json,math,subprocess,sys,tempfile,unittest
from pathlib import Path
from unittest import mock
REPO=Path.cwd()
SCRIPT=REPO/'frontiers/downstream_gate_20260925/run_validation.py'
spec=importlib.util.spec_from_file_location('review_timeout_runner',SCRIPT)
M=importlib.util.module_from_spec(spec); spec.loader.exec_module(M)
class Adversarial(unittest.TestCase):
def setUp(self):
self.tmp=tempfile.TemporaryDirectory(prefix='timeout-review-'); self.addCleanup(self.tmp.cleanup)
self.out=Path(self.tmp.name); self.command=[sys.executable,'-B','-S','-c','pass']
def invoke(self,exc,name='probe',**kwargs):
with mock.patch.object(M.subprocess,'run',side_effect=exc) as run:
with self.assertRaises(type(exc)) as got:
M.execute(self.command,self.out,self.out,name,mode='optimized',mutant=True,mutation='allow_green_ci',**kwargs)
self.assertIs(got.exception,exc); self.assertEqual(run.call_count,1)
self.assertEqual(run.call_args.kwargs['timeout'],30)
return exc
def test_binary_and_absent_matrix(self):
for i,(stdout,stderr) in enumerate([(None,None),(b'',b''),(b'valid UTF8\xc3\xa9',None),(None,b'\xff\xe2\x82\x00'),(b'\xff\x00',b'\xfe\xff')]):
with self.subTest(i=i):
name='matrix'+str(i); self.invoke(subprocess.TimeoutExpired(self.command,30,output=stdout,stderr=stderr),name)
data=json.loads((self.out/(name+'.timeout.json')).read_text())
for label,value in [('stdout',stdout),('stderr',stderr)]:
self.assertEqual((self.out/(name+'.'+label)).read_bytes(),value or b'')
self.assertEqual(data['streams'][label],{'present':value is not None,'bytes':len(value or b'')})
self.assertTrue(math.isfinite(data['elapsed_seconds'])); self.assertGreaterEqual(data['elapsed_seconds'],0)
def test_string_stream_does_not_mask_timeout(self):
exc=self.invoke(subprocess.TimeoutExpired(self.command,30,output='é',stderr='plain text'))
data=json.loads((self.out/'probe.timeout.json').read_text())
self.assertEqual(len(data['persistence_failures']),2)
self.assertTrue(all('TypeError' in x for x in data['persistence_failures']))
def test_both_streams_write_fail_and_metadata_succeeds(self):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(Path,'write_bytes',side_effect=OSError('disk write failure')): self.invoke(exc)
data=json.loads((self.out/'probe.timeout.json').read_text())
self.assertEqual(len(data['persistence_failures']),2); self.assertEqual(len(exc.__notes__),2)
def test_all_evidence_write_failures_keep_original(self):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(Path,'write_bytes',side_effect=OSError('disk write failure')), mock.patch.object(Path,'write_text',side_effect=OSError('disk write failure')): self.invoke(exc)
self.assertEqual(len(exc.__notes__),3)
def test_json_serializer_failure_keeps_original(self):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(M.json,'dumps',side_effect=TypeError('serialization failed')): self.invoke(exc)
self.assertEqual((self.out/'probe.stdout').read_bytes(),b'x')
self.assertEqual(len(exc.__notes__),1)
def test_initial_clock_provider_failure_does_not_prevent_subprocess(self):
completed=subprocess.CompletedProcess(self.command,0,'out','Ran 71 tests\nOK\n')
with mock.patch.object(M.time,'monotonic',side_effect=OSError('clock provider failed')),mock.patch.object(M.subprocess,'run',return_value=completed) as run:
M.execute(self.command,self.out,self.out,'probe',mode='normal')
self.assertEqual(run.call_count,1)
def test_elapsed_clock_provider_failure_keeps_original(self):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(M.time,'monotonic',side_effect=[100.0,OSError('clock provider failed')]): self.invoke(exc)
def test_resolve_provider_failure_keeps_original(self):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(Path,'resolve',side_effect=OSError('cwd resolution failed')): self.invoke(exc)
def test_nonfinite_clock_does_not_emit_non_json_number(self):
for label,value in [('nan',float('nan')),('infinity',float('inf'))]:
with self.subTest(label=label):
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(M.time,'monotonic',side_effect=[100.0,value]): self.invoke(exc,label)
def reject(value): raise ValueError('non-JSON constant '+value)
json.loads((self.out/(label+'.timeout.json')).read_text(),parse_constant=reject)
def test_non_timeout_subprocess_exception_is_original(self):
for exc in [OSError('spawn failed'),ValueError('bad argv'),subprocess.CalledProcessError(2,self.command)]:
with self.subTest(kind=type(exc).__name__): self.invoke(exc)
self.assertEqual(list(self.out.iterdir()),[])
def test_non_timeout_write_exception_is_not_swallowed(self):
exc=OSError('ordinary output write failed'); completed=subprocess.CompletedProcess(self.command,0,'out','Ran 71 tests\nOK\n')
with mock.patch.object(M.subprocess,'run',return_value=completed),mock.patch.object(Path,'write_text',side_effect=exc):
with self.assertRaises(OSError) as got: M.execute(self.command,self.out,self.out,'probe',mode='normal')
self.assertIs(got.exception,exc)
def test_main_optimized_baseline_timeout_keeps_normal_only(self): self.main_late_timeout(False)
def test_main_optimized_mutant_timeout_keeps_normal_only(self): self.main_late_timeout(True)
def main_late_timeout(self,mutant):
baseline=subprocess.CompletedProcess(self.command,0,'out','Ran 71 tests\nOK\n')
gate=subprocess.CompletedProcess(self.command,0,(M.ROOT/'RESULTS.json').read_bytes(),b'')
rejected=subprocess.CompletedProcess(self.command,1,'','Ran 71 tests\nAssertionError\nFAILED (failures=1)\n')
exc=subprocess.TimeoutExpired(self.command,30,output=b'stop',stderr=b'now')
responses=[baseline,gate]+[rejected]*24+([baseline,gate] if mutant else [])+[exc]
out=self.out/'run';
with mock.patch.object(sys,'argv',[str(SCRIPT),'--output',str(out)]),mock.patch.object(M.subprocess,'run',side_effect=responses) as run:
with self.assertRaises(subprocess.TimeoutExpired) as got: M.main()
self.assertIs(got.exception,exc); self.assertEqual(run.call_count,len(responses))
report=json.loads((out/'REPORT.json').read_text()); self.assertFalse(report['passed']); self.assertEqual(report['modes'],['normal'])
self.assertFalse(report['promotion_permission']); self.assertEqual(report['scientific_effect'],'NONE')
def test_main_timeout_report_write_failure_keeps_original(self):
out=self.out/'main'; exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
original=Path.write_text
def write(path,*args,**kwargs):
if path.name=='REPORT.json': raise OSError('REPORT failed')
return original(path,*args,**kwargs)
with mock.patch.object(sys,'argv',[str(SCRIPT),'--output',str(out)]),mock.patch.object(M.subprocess,'run',side_effect=exc) as run,mock.patch.object(Path,'write_text',write):
with self.assertRaises(subprocess.TimeoutExpired) as got: M.main()
self.assertIs(got.exception,exc); self.assertEqual(run.call_count,1); self.assertTrue(any('REPORT.json' in note for note in exc.__notes__))
def test_main_non_timeout_report_write_failure_propagates(self):
out=self.out/'main'; trigger=ValueError('non-timeout subprocess issue'); report_error=OSError('REPORT failed'); original=Path.write_text
def write(path,*args,**kwargs):
if path.name=='REPORT.json': raise report_error
return original(path,*args,**kwargs)
with mock.patch.object(sys,'argv',[str(SCRIPT),'--output',str(out)]),mock.patch.object(M.subprocess,'run',side_effect=trigger),mock.patch.object(Path,'write_text',write):
with self.assertRaises(OSError) as got: M.main()
self.assertIs(got.exception,report_error)
if __name__=='__main__': unittest.main(verbosity=2)Verdict
AMEND on the frozen original tree. Repair fail-safe diagnostic preparation and finite elapsed metadata, then obtain source-bound re-review on the successor. No semantic gate weakening was found. The successful original master does not erase the independently reproduced diagnostic failures, establish current-base integration, authorize a different tree, or promote a scientific claim. Scientific effect: NONE.
d6g8k5htny-coder
left a comment
d6g8k5htny-coder
left a comment
There was a problem hiding this comment.
Successor nonauthor engineering re-review: PASS
Reviewed 2026-10-03 through 2026-10-04 UTC by OpenAI/Codex, the fresh nonauthor reviewer that delivered the original AMEND. Same provider/account and source-exposed: organizational-independence credit 0. Scientific effect NONE. This is scoped engineering review, not scientific acceptance, a GitHub APPROVE action, Lean verification, or hosted-check clearance.
Exact source identity
- Repository / PR: d6g8k5htny-coder/Math- #263
- Remote successor commit: 20acca8
- Tree: 95d0d4f3958505a1849d6b8d79dc9282dbf1eedb
- Remote parents: cdaaf62 and bbe85e2
- Refreshed main base: bbe85e2
- Local commit containing the identical tree: 351649c56ccea7b0a982623c563e6138f058f006
I independently read GitHub's Git commit object and verified its tree and ordered parents, and verified the local working tree/index against the frozen tree. Source review began while the index was frozen and continued without source edits; the final local commit has the same tree. The checkout was clean at completion of the source review. A fresh pre-publication GitHub readback still shows this exact PR head and refreshed base. Exactly four files differ from the refreshed base, and the entire source diff matches successor.diff. The 18 C126 refresh paths equal the refreshed base and are not part of this change.
Historical AMEND remains preserved at #263 (review), anchored to cdaaf62 / tree 9e65c5551b4b3dc7a24493edf07ec24493ea72ca. The original 15-test/five-error results remain valid for that old candidate and were not edited away.
Findings resolved
-
Initial and elapsed clock errors no longer replace the actual operation or its TimeoutExpired.
run_validation.py:70-75 guards the initial clock read. The subprocess is still invoked once when that optional diagnostic fails, and an otherwise successful baseline keeps its existing output/returncode behavior. Lines 92-103 make the elapsed reading best effort; unavailability is reported as elapsed_seconds=null plus diagnostic_failures and an exception note. Original partial stdout/stderr are attempted first at lines 83-91. Exact object-identity checks confirm that the original TimeoutExpired survives initial/end-clock errors, ordinary persistence errors, serialization errors, and final REPORT.json failure, including combined faults.
-
cwd metadata no longer requires a fallible filesystem query.
Lines 104-106 retain str(cwd), the argument actually supplied. No Path.resolve call occurs during timeout diagnostics. Relative spelling is preserved, including a supplied string with repeated separators. The production main passes absolute Paths, so normal baseline/mutant invocation identity is retained.
-
Nonfinite/unusable elapsed metadata is explicit, strict JSON.
Lines 95-100 reject nonfinite and negative elapsed results. Line 113 serializes with allow_nan=False. Tests cover start/read exceptions, NaN at either boundary, infinity, negative duration, nonnumeric/None provider faults, zero elapsed, and the normal positive elapsed case. Unavailable duration is null with one diagnostic explanation; zero remains valid zero. There is no retry loop or fabricated elapsed value.
Independently executed tests
Actual interpreter: Python 3.11.16.
- Author focused suite: 15 tests normal, 15 optimized; both exit 0.
- Unchanged original reviewer adversarial artifact: 15 tests normal, 15 optimized; both exit 0. These exact tests failed on the original candidate.
- Additional reviewer metadata/liveness/combined-fault suite: six tests normal, six optimized; both exit 0.
Commands run from the repository checkout (private executable/script locations replaced with variables; all interpreter flags and test arguments are unchanged). $PYTHON is the verified Python 3.11.16 interpreter, $ORIGINAL_ADVERSARIAL is the complete 15-test script published in the historical AMEND linked above, and $ADDITIONAL_TESTS is the six-test script below:
"$PYTHON" -B -S -m unittest discover -s tests -p test_downstream_validation.py -v
"$PYTHON" -B -O -S -m unittest discover -s tests -p test_downstream_validation.py -v
"$PYTHON" -B -S "$ORIGINAL_ADVERSARIAL"
"$PYTHON" -B -O -S "$ORIGINAL_ADVERSARIAL"
"$PYTHON" -B -S "$ADDITIONAL_TESTS"
"$PYTHON" -B -O -S "$ADDITIONAL_TESTS"
Own artifact writes and test output are confined to separate temporary directories, never the checkout. No master replay was duplicated.
Coverage includes byte preservation for valid/invalid/truncated UTF-8, NUL, None and empty bytes; supported partial-output metadata; exact argv/cwd/mode/mutation/mutant/30-second timeout; normal successful baseline and assertion-detected mutant rejection; all persistence errors; serialization failure; both-mode stop behavior and failed REPORT semantics; unrelated subprocess/write exceptions still propagating; and KeyboardInterrupt remaining unswallowed. The additional combined-fault test fails both streams, metadata, final REPORT and initial clock at once and still receives the exact original timeout after only one subprocess invocation.
Source and scope checks
- All 12 SOURCE_FILES.json entries match actual bytes/SHA-256; only run_validation.py's entry differs from current base.
- Runner: 10,454 bytes, SHA-256 ec1d3315b0c21f4608ab7b824f45d7d6e1041555f62e77feeec2bacaa8e5a9f5, blob fcf6480529d6b521628e2635e7c1e271ecfa39a4.
- Test: 16,443 bytes, SHA-256 b68002af318696029fa14fe1f2f2bae89b6fa81ac4d482af12bd46cc51f340d4, blob b5d42a9e9913513dfa3a587e4c194d8ae25d1516.
- Manifest: 1,760 bytes, SHA-256 ec9b8b8ae90f651641a94c2b7abec4c0c878d00d14efa8728ec2ecee8ae85009.
- Workflow remains exactly the original candidate's 6,376 bytes, SHA-256 8c6979e34a71dacb380bdf1a3247769bb3a6dcb822b9afa3cfe0e834c0a5c959.
- AST-equal to pre-change base: 24-mutant mapping and entire completed-command output/coverage/returncode/assertion-check tail of execute. AST-equal to original candidate: the entire main function, including stop/completed-mode/failed-REPORT handling.
- Packet discovery still counts exactly 71 tests; root focused tests remain outside packet discovery.
- RESULTS.json, hard_gate.py, GRAPH.json, all other packet files and scientific/formal/C126 files are unchanged relative to refreshed base.
- Workflow still pins Python 3.11 and independently runs the focused tests in normal and -O modes. No changes to required checks, formal workflow, downstream semantic counts, timeout budget, or evidence-retention behavior.
- git diff --check succeeds; index/worktree agree with the reviewed tree.
Successor master evidence independently checked, not re-executed
Author successor-master-311/REPORT.json exists with exit 0, Python 3.11.16, passed=true, sources_unchanged=true, both modes, 71 tests and 24 mutants, promotion_permission=false and scientific_effect=NONE. All 13 source identities match the successor. All 50 stdout/stderr pairs were inspected: two baseline successes and 48 assertion-detected semantic-mutant rejections, all at 71 tests and without skips or program-error substitutions. Both output JSON files equal preserved RESULTS.json. No timeout sidecar is present in the successful master.
Successor master REPORT SHA-256: dda3e1286eac3d304ec3bfa18e87c5e89eaff44b74d9c067c32c2b19a145666a.
These checks establish consistency/source-binding of the author's receipt, not a second replay or independent mathematics. Author-reported root/formal/regression runs were not rerun in this review. Lean and hosted workflows were not executed or claimed successful by this reviewer.
Residual contract notes
The retained standard subprocess provider contract is bytes/None for timeout streams. Synthetic string payloads still generate explicit persistence failures while retaining the timeout; str-stream support is not newly promised. Timing is diagnostic, best effort, and may be null. elapsed_seconds is sampled after partial-stream persistence, so it measures wrapper elapsed time including that work rather than an exact child-only runtime. None of these observations warrants another amendment for the requested scope.
Verdict
PASS for the bounded engineering change at exact tree 95d0d4f3958505a1849d6b8d79dc9282dbf1eedb, remote commit 20acca8, relative to refreshed base bbe85e2. Both original AMEND findings are resolved with independent reproducer reruns and additional failure-path checks. Required hosted/current-head checks and any merge authorization remain separate. No theorem-wide acceptance, scientific promotion, registry change, or organizational-independence credit is implied.
Additional reviewer artifact
The original 15-test reviewer script is preserved in the historical AMEND. The complete additional six-test artifact follows, with only private paths replaced by repository-relative lookup and the default temporary directory. Save outside the checkout and run from the reviewed repository root using the commands above. Both Python 3.11.16 modes reported Ran 6 tests / OK, exit 0.
import importlib.util,json,subprocess,sys,tempfile,unittest
from pathlib import Path
from unittest import mock
SCRIPT=Path.cwd()/'frontiers/downstream_gate_20260925/run_validation.py'
spec=importlib.util.spec_from_file_location('successor',SCRIPT); M=importlib.util.module_from_spec(spec); spec.loader.exec_module(M)
class SuccessorFallbackTests(unittest.TestCase):
def setUp(self):
self.tmp=tempfile.TemporaryDirectory(prefix='timeout-successor-'); self.addCleanup(self.tmp.cleanup); self.out=Path(self.tmp.name)
self.command=[sys.executable,'-B','-S','-c','pass']
def timed(self,readings,relative='relative//cwd'):
exc=subprocess.TimeoutExpired(self.command,30,output=b'\xffx\x00',stderr=b'y\xe2\x82')
with mock.patch.object(M.time,'monotonic',side_effect=readings) as clock,mock.patch.object(M.subprocess,'run',side_effect=exc) as run,mock.patch.object(Path,'resolve',side_effect=AssertionError('resolve must not be called')):
with self.assertRaises(subprocess.TimeoutExpired) as caught: M.execute(self.command,relative,self.out,'probe',mode='optimized',mutant=True,mutation='allow_green_ci')
self.assertIs(caught.exception,exc); self.assertEqual(run.call_count,1); self.assertEqual(run.call_args.kwargs['cwd'],relative); self.assertEqual(run.call_args.kwargs['timeout'],30)
def reject(x): raise ValueError(x)
data=json.loads((self.out/'probe.timeout.json').read_text(),parse_constant=reject)
self.assertEqual(data['cwd'],relative); self.assertEqual(data['argv'],self.command); self.assertEqual(data['mode'],'optimized'); self.assertTrue(data['mutant']); self.assertEqual(data['mutation'],'allow_green_ci')
self.assertEqual((self.out/'probe.stdout').read_bytes(),b'\xffx\x00'); self.assertEqual((self.out/'probe.stderr').read_bytes(),b'y\xe2\x82')
return data,exc,clock.call_count
def test_unavailable_timing_is_null_and_explained_without_retry_loop(self):
for readings in [[OSError('start')],[100.0,OSError('end')],[float('nan'),100.0],[float('inf'),float('inf')],[100.0,99.0],[None,1.0],[100.0,'bad']]:
with self.subTest(readings=readings):
data,exc,calls=self.timed(readings); self.assertIsNone(data['elapsed_seconds']); self.assertEqual(len(data['diagnostic_failures']),1); self.assertEqual(data['persistence_failures'],[])
self.assertEqual(calls,len(readings)); self.assertTrue(any('diagnostic failed' in note for note in exc.__notes__))
def test_zero_elapsed_is_valid_not_missing(self):
data,exc,calls=self.timed([100.0,100.0]); self.assertEqual(data['elapsed_seconds'],0.0); self.assertEqual(data['diagnostic_failures'],[]); self.assertFalse(hasattr(exc,'__notes__'))
def test_streams_exist_before_second_clock_read(self):
calls=[]
def reading():
calls.append(1)
if len(calls)==1: return 100.0
self.assertEqual((self.out/'probe.stdout').read_bytes(),b'x'); self.assertEqual((self.out/'probe.stderr').read_bytes(),b'y'); raise OSError('late clock failure')
exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(M.time,'monotonic',side_effect=reading),mock.patch.object(M.subprocess,'run',side_effect=exc):
with self.assertRaises(subprocess.TimeoutExpired) as caught: M.execute(self.command,self.out,self.out,'probe',mode='normal')
self.assertIs(caught.exception,exc); self.assertEqual(len(calls),2)
def test_combined_all_persistence_and_clock_failures_keep_original(self):
out=self.out/'run'; exc=subprocess.TimeoutExpired(self.command,30,output=b'x',stderr=b'y')
with mock.patch.object(sys,'argv',[str(SCRIPT),'--output',str(out)]),mock.patch.object(M.time,'monotonic',side_effect=OSError('clock')),mock.patch.object(M.subprocess,'run',side_effect=exc) as run,mock.patch.object(Path,'write_bytes',side_effect=OSError('bytes')),mock.patch.object(Path,'write_text',side_effect=OSError('text')):
with self.assertRaises(subprocess.TimeoutExpired) as caught: M.main()
self.assertIs(caught.exception,exc); self.assertEqual(run.call_count,1); self.assertEqual(len(exc.__notes__),5)
self.assertEqual(list(out.iterdir()),[])
def test_initial_clock_fault_never_converts_other_subprocess_errors(self):
for error in [OSError('spawn'),ValueError('argv'),subprocess.CalledProcessError(2,self.command)]:
with self.subTest(error=type(error).__name__):
with mock.patch.object(M.time,'monotonic',side_effect=OSError('clock')),mock.patch.object(M.subprocess,'run',side_effect=error) as run:
with self.assertRaises(type(error)) as caught: M.execute(self.command,self.out,self.out,'probe',mode='normal')
self.assertIs(caught.exception,error); self.assertEqual(run.call_count,1)
self.assertEqual(list(self.out.iterdir()),[])
def test_interrupt_is_not_swallowed_by_optional_diagnostics(self):
error=KeyboardInterrupt()
with mock.patch.object(M.time,'monotonic',side_effect=error),mock.patch.object(M.subprocess,'run') as run:
with self.assertRaises(KeyboardInterrupt) as caught: M.execute(self.command,self.out,self.out,'probe',mode='normal')
self.assertIs(caught.exception,error); self.assertEqual(run.call_count,0)
if __name__=='__main__': unittest.main(verbosity=2)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Current disposition
Merged normally with the expected-head guard as
af43e809184699007c230db1da94e94147776634; landed tree is the exact reviewed95d0d4f3958505a1849d6b8d79dc9282dbf1eedb. Landed push run 37164160146 is SUCCESS. Its original artifact bytes, logs, source identities and formal/aggregate receipt were verified against the landed commit. Scoped maintenance is complete; scientific effect NONE.Dylan Roy — delegated AI work. Actual implementation performer: OpenAI/Codex preserve_validation_timeout_evidence. Bounded maintenance claim, lease through 2026-10-04T01:35Z.
20acca8030672a474f19ae9cd3de72736ac30ff1; tree:95d0d4f3958505a1849d6b8d79dc9282dbf1eedbbbe85e270f2c8b747f2d5d9477c86e86e323fe15cdaaf624; exact successor nonauthor PASS resolves both findings on20acca8030672a474f19ae9cd3de72736ac30ff1. Actual reviewer: separate OpenAI/Codex engineering worker, same-provider/account independence credit 0d55f96f5fcf5b5888ebc887000f428351c14111bto the current base and exact runner identity. Both new 15-test modes, 71-test/24-mutant replay, 177 regressions/mode, 13 existing scalar Lean targets, five formal negative controls and the required aggregate passed. Landing check is also SUCCESSDefect and bounded fix
execute()previously raisedTimeoutExpiredbefore writing either captured stream. A timed-out command therefore lost its partial diagnostic output. The new handler writes raw stdout/stderr bytes, including incomplete UTF-8, and a per-invocation.timeout.jsoncontaining exact argv, supplied cwd, run name, mode, mutation identity, mutant flag, configured 30-second timeout, monotonic elapsed time, timeout status and stream presence/length.The same timeout exception is re-raised. A timeout is never accepted as mutant rejection, never appends a completed mode, and never advances to later commands. Independent evidence-write failures are recorded in the metadata where possible and attached to the original exception; a failing final REPORT write likewise cannot mask an active timeout. Initial/end clock failures and nonfinite or negative elapsed readings now produce null elapsed time plus diagnostic failures; captured streams and the original timeout survive. Cwd is recorded exactly as supplied without a filesystem-resolution step during failure handling. Other subprocess failures and completed-process checks retain their existing behavior.
Scope
Exactly four paths: runner, only its bytes/SHA entry in SOURCE_FILES.json, new tests/test_downstream_validation.py outside the packet's test discovery, and one focused normal/optimized CI step. The 30-second timeout, 71-test/24-mutant contract, hard_gate.py, GRAPH.json, RESULTS.json, scientific flags, C126 files/branch/handoff and formal workflows are unchanged. No new scientific or publication-tracking structure. Scientific effect: NONE.
Validation actually executed
On the frozen candidate tree using Python 3.11.16:
Local full Lean execution is not claimed. Current hosted formal execution and the required aggregate remain mandatory. This is engineering evidence only, not a mathematical acceptance declaration. Same-provider/account review earns zero organizational-independence credit.