Skip to content

fix(core): enable Windows support with msvcrt file locking and POSIX tree keys (fixes #151) - #159

Open
Cyberpradeep wants to merge 1 commit into
google-research:mainfrom
Cyberpradeep:fix/windows-compat-issue-151
Open

Cyberpradeep wants to merge 1 commit into
google-research:mainfrom
Cyberpradeep:fix/windows-compat-issue-151

Conversation

@Cyberpradeep

Copy link
Copy Markdown

Summary

This PR enables full Windows compatibility for Harness², resolving the startup fcntl import failure and the silent tree-key rejection that causes StopIteration during multi-candidate selection at $k \ge 2$ (#151).

All changes are strictly isolated to Windows runtime paths; POSIX/Linux behavior is 100% unaffected.


Root Causes Addressed

  1. POSIX fcntl on Windows (harness2/core/schedule.py):
    _write_summary imported POSIX-only fcntl.flock to atomically merge mode results into summary.json. On Windows, this immediately crashed the test suite and CLI execution with ModuleNotFoundError: No module named 'fcntl'.

    • Fix: Added _lock_summary(lf) and _unlock_summary(lf) branching on os.name == "nt" to use Python's standard library msvcrt.locking(lf.fileno(), ...) with an identical file descriptor locking contract.
  2. Backslash Tree Keys & False Stub Mismatch (harness2/core/tree.py):
    On Windows, Path.relative_to() produces backslash separators (e.g. 'plugin\README.md', 'scripts\summarize.py').

    • Because stubs and component regexes expect forward slashes (/), rel in stubs evaluated to False.
    • As a result, harmless README stubs were categorized as unauthorized editable files (rejected), causing the engine to fail every proposed harness and fall back to the unmodified base script.
    • At $k=1$, selection was skipped as a trivial pool, masking the defect. At $k \ge 2$, selection evaluated candidate deliverables, found no signed totals, and raised StopIteration.
    • Fix: Standardized relative path key extraction in read_tree and write_tree using the existing normalize_rel() function, guaranteeing POSIX-compliant keys on all operating systems.

Verification & Test Results

Tested on Windows 11 (Python 3.12.10, x64):

  1. Unit Test Suite (python -m unittest discover -v tests):

    • Before fix: FAILED (failures=4, errors=3) due to ModuleNotFoundError: No module named 'fcntl'.
    • After fix: Ran 5 tests in 54.71s — OK (Failures: 0, Errors: 0).
  2. Offline Deterministic Demo (harness2-demo --k 2):

    • Before fix: 0 tasks completed; 2 failed with agg: StopIteration: ; seq: StopIteration:.
    • After fix: 2 tasks completed; 0 failed — all candidate harnesses successfully evolved, executed, and produced expected signed totals (Total: 85.15).

Execution Logs & Tracebacks

Click to expand: Unpatched Unit Test Failures (fcntl crash)
======================================================================
ERROR: test_failed_selection_is_never_judged (test_workflow.WorkflowTests.test_failed_selection_is_never_judged)
----------------------------------------------------------------------
  [accounting] 1/2 exception    ledger-1  :: ModuleNotFoundError: No module named 'fcntl'
  [accounting] 2/2 exception    ledger-2  :: ModuleNotFoundError: No module named 'fcntl'

======================================================================
FAIL: test_recursion_budget_and_resume (test_workflow.WorkflowTests.test_recursion_budget_and_resume) (k=2)
----------------------------------------------------------------------
AssertionError: Tuples differ: (0, 2) != (2, 0)
- (0, 2)
+ (2, 0)

----------------------------------------------------------------------
Ran 5 tests in 19.226s
FAILED (failures=4, errors=3)
Click to expand: Unpatched k=2 Root Cause & StopIteration Log
EXPERIMENT: RUNTIME FAILURE AT k=2 (Backslash Tree Key Rejection)
Inspecting Path handling on Windows:
Path('scripts/summarize.py'): str='scripts\\summarize.py', as_posix='scripts/summarize.py'
Keys produced by unpatched read_tree: ['scripts\\summarize.py']
'scripts/summarize.py' in tree_read? False
'scripts\\summarize.py' in tree_read? True

Running demo CLI with k=2:
  [accounting] 1/2 partial      ledger-1  :: agg: StopIteration: ; seq: StopIteration: 
  [accounting] 2/2 partial      ledger-2  :: agg: StopIteration: ; seq: StopIteration: 
Local demo: 0 tasks completed; 2 failed.

Found summary at runs\unpatched_k2_demo\runs\demo-oc-agg2\ledger-1\summary.json:
Status: partial
Errors: ['agg: StopIteration: ', 'seq: StopIteration: ']
Click to expand: Post-Fix Test Suite Output (5/5 Passing)
test_complete_evidence_and_library_prefix (test_workflow.WorkflowTests.test_complete_evidence_and_library_prefix) ... ok
test_failed_selection_is_never_judged (test_workflow.WorkflowTests.test_failed_selection_is_never_judged) ... ok
test_invalid_proposal_retries_with_feedback_and_accumulates_cost (test_workflow.WorkflowTests.test_invalid_proposal_retries_with_feedback_and_accumulates_cost) ... ok
test_no_entry_stays_out_of_library_on_resume (test_workflow.WorkflowTests.test_no_entry_stays_out_of_library_on_resume) ... ok
test_recursion_budget_and_resume (test_workflow.WorkflowTests.test_recursion_budget_and_resume) ... ok

----------------------------------------------------------------------
Ran 5 tests in 51.284s

OK
Click to expand: Post-Fix Demo Output (k=2 Passes 2/2 Tasks)
$ python -m harness2.cli.demo --output runs/final_demo_verify --k 2
  [accounting] 1/2 ok           ledger-1
  [accounting] 2/2 ok           ledger-2
Local demo: 2 tasks completed; 0 failed. D:\my-research\harness2\runs\final_demo_verify\demo.json

Contributor License Agreement (CLA)

  • Contributor: Pradeep S (pradeepnaveen930@gmail.com)
  • Google Individual Contributor License Agreement (CLA) is on file and active.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant