Skip to content

Refuse an output folder that cannot be listed, and test two publication guards - #521

Merged
Deicyde merged 2 commits into
mainfrom
cleanup/managed-output-guards
Oct 31, 2026
Merged

Deicyde merged 2 commits into
mainfrom
cleanup/managed-output-guards

Conversation

@Deicyde

@Deicyde Deicyde commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Follow-ups to #501 from jackmccarthy-da's review notes there: one fail-open check closed, and tests for two guards nothing exercised.

Change

  • output_identity refuses a folder it cannot list. It digested the names and bytes that sorted(path.rglob("*")) lists, and rglob silently skips a directory the process cannot list. replace_outputs compares that digest before and after it moves the old output aside, so an edit under such a directory went unseen and was published over. A new _tree(path) walks with os.walk(onerror=...) and raises the listing error, which the existing except OSError turns into the usual "cannot inspect" SkeletonError. On any tree it can list, _tree returns the same paths in the same order as sorted(path.rglob("*")), so digests do not change; it does not follow symlinked directories, and the loop still refuses a symlink when it reaches one. The docstring, which described the skip, now says it is refused.
  • New tests/test_managed_output.py, for guards the publishing commands' tests do not reach:
    • test_output_identity_refuses_a_directory_it_cannot_list (POSIX, not as root): a subfolder with mode 0 is refused.
    • test_stage_output_removes_a_stage_it_cannot_finish, for a failed write (a lone surrogate the UTF-8 encoder rejects), fsync and chmod: the stage is gone and the old file is untouched.
    • test_stage_output_leaves_a_registered_stage_to_its_owner: with stages, the failed stage stays listed for the caller to remove.
    • test_rollback_preserves_output_that_changed_after_its_last_check: a second install fails, and another writer adds a file to the published tree after rollback checked it and before it moved the tree aside. The tree is kept under its .autoform-rollback- name with that file, and the error says so.
  • CHANGELOG: an "Output checks." entry under Fixed.

Verification

  • The locked ruff 0.15.17 check passes on the changed files.
  • On Python 3.14, _tree and sorted(rglob("*")) give the same 11 paths for a tree with dot files, nested folders, a non-ASCII name and symlinks to a folder and a file.
  • Called directly on macOS: the new file 6 passed; tests/test_skeleton.py 175 passed, 33 skipped (real Lean); tests/test_library_generate.py 89 passed, 13 skipped; tests/test_review_bundle.py 63 passed.
  • On main's code the new tests pass except the unlistable-folder test (DID NOT RAISE). With stage_output's cleanup removed, the three stage tests fail; with the quarantine identity check removed, the rollback test fails.

Overlap

Review notes on #501. output_identity digested what Path.rglob lists, and
rglob skips a directory the process cannot list, so the check that an
output did not change during publication missed edits under it. It now
walks with os.walk and raises that directory's error, which becomes the
usual "cannot inspect" refusal. The list it digests is otherwise the
same, in the same order.

New tests/test_managed_output.py covers that refusal and two guards no
test reached: stage_output removes a stage whose write, fsync or chmod
fails (and leaves a registered stage to its owner), and rollback keeps a
published tree that changed after its last check instead of deleting it.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 31, 2026
@Deicyde

Deicyde commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Review at c54367e: merge.

Nothing blocks merge, and there are no notes.

  • _tree lists what rglob listed, and refuses what it skipped. On a tree with dot files, nested folders, a non-ASCII name and symlinks to a folder and a file, _tree(d) == sorted(d.rglob("*")) on Python 3.14, so digests don't change. os.walk doesn't descend into symlinked folders, and the loop refuses the first symlink it reaches anyway, so nothing behind a symlink can differ. An unlistable folder now raises through onerror, and the existing except OSError turns that into the "cannot inspect" refusal, where rglob digested around it.
  • The new tests cover guards nothing reached before. The unlistable-folder case fails on main, as the body says. The stage and rollback tests pin stage_output's cleanup and the quarantine check.

Checks

  • Locally at this head, tests/test_managed_output.py, tests/test_library_generate.py and tests/test_review_bundle.py give 158 passed and 13 skipped (real Lean). The output, publish, packet, stage and rollback cases of tests/test_skeleton.py give 57 passed and 17 skipped.
  • The PR merges cleanly with main a87797f, and CI passes 9 of 9.
  • As the body says, whichever of this PR and Stream large skeleton packets and reports, and name each skeleton phase on stderr #516 lands second redoes output_identity's loop header.

Posted by Claude Code (session jackmccarthy-eb).

@Deicyde
Deicyde merged commit 703d095 into main Oct 31, 2026
9 checks passed
@Deicyde
Deicyde deleted the cleanup/managed-output-guards branch October 31, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant