[lahma]

A MoveFile, MoveDirectory or ReplaceFile whose paths are all in the same sub-directory can deadlock with any concurrent lookup in that directory, such as FileExists, OpenFile or CreateDirectory.

Problem

When the paths share a parent, MoveFileOrDirectory and ReplaceFileImpl take the filesystem lock shared. They then look each path up with EnterFindNode(..., KeepParentNodeExclusive | ...) and pass the results of the previous lookups as existingNodes. Take MoveFile("/d/a.txt", "/d/b.txt"):

1. The source lookup locks /, then /d (exclusive), releases / and locks /d/a.txt. It keeps /d.
2. The destination lookup starts again from the root. / is not in existingNodes, so it locks / again while it holds /d.

Every other lookup takes the directory locks hand-over-hand from the root down. It holds / until it has /d. So:

• thread A (move) holds /d and waits for /;
• thread B (FileExists("/d/other.txt")) holds / and waits for /d;

and both block forever. Moves within / are not affected (/ is then in existingNodes), and neither are moves across directories (they take the filesystem lock exclusively).

A move loop in /d with a FileExists("/d/other.txt") loop on another thread deadlocks on main within the first few hundred lookups, before the first move completes.

Fix

The second (and third) lookup of a same-directory rename no longer walks from the root. The new EnterFindChild looks the name up directly in the parent that the first lookup holds, and locks only the node it finds. The result never releases the parent; the first lookup's result still does. This is how the hierarchical locking scheme of the Unix kernel that MemoryFileSystem follows handles same-directory renames: the parent is locked once, and both names are looked up in it.

• MoveFileOrDirectory: when the source and destination are in the same folder, the destination is looked up with EnterFindChild in the source's parent.
• ReplaceFileImpl: when source, destination and backup are in the same folder, the first path in sorted order is looked up as before, and the others with EnterFindChild in its parent.

The locks taken, and the order they are taken in, are otherwise unchanged. The async counterpart (MemoryFileSystemAsync.gen.cs) is regenerated with Zio.AsyncCodeGen.

A coarser alternative would be to take the filesystem lock exclusively for every move and replace, as is already done when the directories differ. That would make same-directory renames exclude every other operation on the filesystem, so I went with the fine-grained fix.

Tests

The first commit adds TestMemoryFileSystemRenameConcurrency on its own, so it can be checked against main. Each test runs 2000 renames in /d against a concurrent FileExists("/d/other.txt") loop, and fails if the renames don't complete within 30 s. The tests cover MoveFile, MoveDirectory, ReplaceFile with a backup, and MoveFileAsync on MemoryFileSystemAsync.

• On main, all four tests fail on the timeout (deadlocked).
• With the fix: dotnet test -c Release passes on net10.0 (524 passed, 3 skipped) and net472 (500 passed, 3 skipped). The ZIO_NO_ASYNC build succeeds, and Zio.AsyncCodeGen -- --check is clean.
• The branch merges cleanly with Improve Perf of MemoryFileSystem node locks with a lock-free fast path #109 and Improve Perf of MemoryFileSystem.EnumerateItems #110, and the tests also pass with all three merged.

🤖 Generated with Claude Code