doc(fuse): Add backend split design by karlrado · Pull Request #352 · samsoir/xearthlayer

karlrado · 2026-11-03T18:17:44Z

Summary

Design for backend split

Changes

Add backend split design doc for review.

Related Issues

Related: #346
This does not include the actual Windows support work. It is just for the prerequisite backend changes.

Test Plan

make pre-commit passes (fmt + clippy + tests)

Checklist

Code follows the project's SOLID principles and patterns
Tests added or updated for new/changed behavior
Documentation updated if applicable (CLAUDE.md, docs/, config reference)
No new warnings from cargo clippy

samsoir · 2026-11-05T06:26:51Z

Hi Karl, thanks for sending this over. I will take a look when I get home this weekend. Appreciate your ongoing work to add Windows support to XEarthLayer.

samsoir

This is good. Only one minor change requested around naming of the common functionality naming

samsoir · 2026-11-09T18:26:43Z

+|  |── ortho_union_fs.rs
+|  |── shared.rs
+|  |── types.rs
+|── support/


nit: prefer common/ as the namespace for these components

Agreed. Fixed.

samsoir · 2026-11-09T18:27:21Z

+
+This operation is an initial straightforward renaming. However, further
+refactoring is required to move code that is backend neutral to the common
+`support/` area. This is to avoid this code being duplicated by a copy-and-paste


see above, lets use common for this

samsoir · 2026-11-09T18:42:13Z

+choose to use or ignore them.
+
+```rust
+pub struct MountRequest {


Minor: max_background and congestion_threshold are FUSE settings, and WinFSP will want its own (for example IrpTimeout). If they all go into MountRequest, every backend sees every other backend's settings.

Could these go into the FUSE backends' constructors instead, with select_backend taking the config it needs? MountRequest would then hold only what every backend uses, which today is the mount point.

OK, I think I see how to do that. Fixed.

samsoir · 2026-11-09T18:42:13Z

+stay plain.
+
+```rust
+pub trait UnionFilesystem: Send + Sync {


Could the doc define the types this trait uses? FsNode, FsError, FileAttributes, DirEntry and StatFsInfo are the contract itself, and at the moment the doc only names them. I'd like to see what each one contains, and how FsError maps to errno on FUSE.

I should have done that. Added to doc.

samsoir · 2026-11-09T18:42:13Z

+        size: usize,
+    ) -> Pin<Box<dyn Future<Output = Result<Bytes, FsError>> + Send + 'a>>;
+
+    fn list_directory<'a>(


If DirEntry doesn't carry attributes, readdirplus becomes one boxed attrs future per entry. That adds up on the large terrain/ and textures/ directories during scene load. I'd suggest including attributes in DirEntry, or a variant that returns them.

Question: What do you mean by attributes? File vs Directory, as a filter? I read the fuse3 readdirplus and see that it specifies kind: FileType::Directory. Is it something like this?

Not a filter, sorry, I should have been more specific. I mean the full FileAttributes you've now defined (size, times, kind, perm and so on), one per entry.

readdirplus is FUSE's "list and stat in one call": each DirectoryEntryPlus carries a complete attr, and the kernel caches it so X-Plane doesn't follow up with a getattr per file. Today readdirplus builds that attr itself, from virtual_dir_attr for directories and fs::metadata on the source file for real files.

With the current DirEntry (name, is_dir, size, mtime) the adapter can't fill attr, so it has to call attrs() per entry, and the attributes end up being worked out in two places. The real risk there is the two disagreeing, which has happened before: the test at ortho_union_fs.rs:1565 exists because readdirplus once reported the wrong size for DSF files. WinFSP's ReadDirectory also wants size, times and attributes per entry, so every backend needs them.

So I'd suggest:

pub struct DirEntry { pub name: OsString, pub node: FsNode, pub attrs: FileAttributes, }

built by the core with the same function attrs() uses.

One more thing: the DirEntry in the doc is currently the ortho_union index type, serde attributes included. That's the index cache's on-disk format, so I'd keep it out of the contract and give the contract its own type as above.

I see. This is even clearer after seeing the same thing in lookup below.

And just so I'm clear:

and the kernel caches it so X-Plane doesn't follow up with a getattr per file.

The kernel cache just prevents the calls from reaching our process again; X-Plane is still making the calls. Right?

Changed as suggested.

karlrado · 2026-11-10T16:04:52Z

Updated doc pushed.

samsoir

Hey Karl. Sorry more changes needed as I look at this more in context with all of the existing macOS and Linux fuse3 implementation. Appreciate you iterating on this with me. This should be it.

samsoir · 2026-11-11T05:01:59Z

+        &'a self,
+        node: &'a FsNode,
+        name: &'a str,
+    ) -> Pin<Box<dyn Future<Output = Result<FsNode, FsError>> + Send + 'a>>;


lookup has the same gap as DirEntry. Every backend's lookup reply carries attributes: fuse3's ReplyEntry.attr, and WinFSP's GetSecurityByName and Open both return a FILE_INFO. Today lookup computes them inline, including the async lit size probe (virtual_attr_for(...).await).

With lookup returning only FsNode, the adapter has to call lookup then attrs: two boxed futures and two resolutions on the hottest operation in the mount. One type returned by both lookup and list_directory would close both threads at once:

pub struct Entry { pub node: FsNode, pub attrs: FileAttributes, }

Got it. Changed as described.

samsoir · 2026-11-11T05:01:59Z

+/// mapper fills them from the file metadata (real files) or the process
+/// (virtual files), so `libc`/`MetadataExt` calls stay out of core logic.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct FileAttributes {


Two things on FileAttributes.

The comment says the FUSE mapper fills uid and gid, yet they are fields of the neutral struct. Either the core fills them (it has the Metadata in hand when it stats a real file, and getuid for virtual ones) or they do not belong here. I'd have the core fill everything and drop the comment, so getattr, lookup and readdirplus cannot disagree.

It also misses two times: ctime, which fuse3's FileAttr has on both platforms and we fill today, and a creation time (crtime on macFUSE, CreationTime on WinFSP). Both should be here so the adapters do not have to invent them.

Understood and changed as described.

samsoir · 2026-11-11T05:01:59Z

+xearthlayer/src/vfs
+|── fuse3/
+|  |── adapters.rs              # New adapters for LinuxFuse and MacFuse translations
+|  |── fuse_platform.rs         # Per-platform FUSE behavior/policy within fuse3 backend


The FusePlatform policy is what Non-Goal 2 rests on, and the doc never says what it decides. The cfg(target_os) sites inside fuse/ today are exactly four:

VIRTUAL_DDS_OPEN_FLAGS: direct I/O on Linux, page cache on macOS (ortho_union_fs.rs:109)

mount_with_unprivileged on Linux, mount elsewhere (:527, :561)

the volname mount option on macOS (:595)

crtime and flags on FileAttr (shared.rs:144 onwards)

Could the doc list these as the policy's contents? Two things follow from it. The acceptance can check that no cfg(target_os) remains in fuse3/ outside fuse_platform.rs. And since the contract's open returns a bare u64, the doc should say that open flags are the adapter's decision, keyed on the FsNode variant (virtual DDS gets the platform flags, real files get none).

I added a table to enumerate the policies. Acceptance adjusted and the note on open is added.

samsoir · 2026-11-11T05:02:00Z

+/// This is the one place in the crate that names a concrete backend. Callers
+/// ask for a backend here and then work only with [`MountBackend`] and
+/// [`MountSession`].
+pub fn select_backend() -> Box<dyn MountBackend> {


Two points on select_backend.

The project convention is cfg!, not #[cfg], so that both implementations compile and are unit tested on both CI platforms (paths/ and preflight/ do this). It matters more than usual here: both adapters wrap fuse3, so both can be built everywhere. With #[cfg], MacFuse is never compiled on the Linux job and LinuxFuse never on the macOS job.

Line 255 says select_backend passes max_background and congestion_threshold to the constructors, but the signature here takes no arguments. It needs a parameter for the FUSE limits, or whatever config type carries them.

cfg usage fixed. Added FuseLimits type and added it to MountRequest.

samsoir · 2026-11-11T05:02:00Z

+|── common/
+|  |── attrs.rs                 # Neutral file attribute types (refactor)
+|  |── conformance.rs           # Neutral one contract test suite
+|  |── core.rs                  # Neutral core of union filesystem (refactor)


The core's state has no home in the doc. ortho_union_fs.rs holds the DdsHandle memo, the PinnedTileBudget, the metrics client, the PrefetchStateObserver, the geo filter (resolve_lazy_geo) and the PNG alpha mask rule in lookup. All of it is backend neutral and all of it is the read path most likely to change quietly in a move.

Could the doc state that these move to core.rs unchanged, and say which metrics the core emits versus the adapter? Per read latency and fuse_handle_budget_exhausted are the two I'd want pinned down, since the dashboard reads them.

Doc now contains that info (Core state and metrics).

samsoir · 2026-11-11T05:02:00Z

+        node: &'a FsNode,
+    ) -> Pin<Box<dyn Future<Output = Result<Vec<DirEntry>, FsError>> + Send + 'a>>;
+
+    /// open can return a fh of 0 when the pinned-tile budget is exhausted.


This covers one of the four cases. Today open returns 0 for real files, lit textures and unknown virtual inodes as well as the exhausted budget. read treats 0 as "resolve per call", and release(0) must be a no op (today dds_handles.remove(&0) simply finds nothing). All four belong in the contract, so a new backend neither treats 0 as a real handle nor errors on release(0).

Added to contract. I guess 0 file handles freak me out a little :-)

samsoir · 2026-11-11T05:02:00Z

+    fn force_unmount(&self, path: &Path);
+}
+```
+The calls in `MountManager` that handle crash and stale mount recovery need to be migrated to


The crash path is panic.rs:192, not the MountManager. The hook calls unmount_fuse(path, false) and then unmount_fuse(path, true), and logs which one worked.

Could the doc say how the hook gets a backend (a select_backend() call in the hook is cheap enough), that force_unmount returns bool like today so the hook can still report the outcome, and that both methods are synchronous, need no runtime and must not panic? A panic inside a panic hook aborts the process with the mount still up.

samsoir · 2026-11-11T05:02:00Z

+real mount to exercise filesystem operations (lookup, attributes, virtual DDS read,
+directory listing).  These tests run for each backend.
+
+For implementation, `tests/macfuse_smoke.rs` can be turned into a live mount test


tests/macfuse_smoke.rs is #![cfg(target_os = "macos")] and #[ignore], so today there is no live Linux mount test at all. mount_with_unprivileged works on ubuntu-latest once fuse3 is installed.

If the parameterised test runs unignored on Linux and ignored on macOS (via make verify-macos), then "green CI" actually exercises the LinuxFuse adapter. As written, the Linux adapter's only check is the regression flight.

samsoir · 2026-11-11T05:02:00Z

+    }
+}
+```
+Backend specific parameters are passed in a `MountRequest` where each backend can


Stale sentence: the struct below now holds only mountpoint, and line 255 says the backend specific parameters go to the constructors. This one can go.

Also MountError (line 201) is used but never defined.

samsoir · 2026-11-11T05:02:00Z

+- structural checks
+  - `manager/mounts.rs` contains no `cfg` and names no backend
+  - `common` module contains no `fuse3::`, inode or kernal-flag symbols
+  - nothing outside of `vfs/` refers to FUSE


This one cannot pass as written. [fuse] config keys are user facing and stay, preflight/system.rs has the macfuse-available check, the metrics names are fuse_*, the dashboard has fuse_latency.rs, and system/unmount.rs is fusermount3 specific.

I'd replace it with two checkable items:

no crate::vfs::fuse3:: import outside vfs/ (today the only two are manager/mounts.rs:15 and tests/macfuse_smoke.rs:26)

system/unmount.rs moves into the fuse3 backend, so panic.rs and mounts.rs reach it only through MountBackend

Expect night/* and runtime/request.rs to churn on the rename: they import DdsFilename, TextureKind and DdsResponse from crate::fuse.

fixed and the churn noted.

karlrado · 2026-11-11T23:05:05Z

Hey Karl. Sorry more changes needed as I look at this more in context with all of the existing macOS and Linux fuse3 implementation. Appreciate you iterating on this with me. This should be it.

Thanks for the additional review and findings. Iterating is fine and I was totally expecting some as it is a pretty big code base. I am learning a lot.

Please find the latest update I just pushed. (oops, pound sign 2 not a good idea in my commit msg. I knew better.)

doc(fuse): Add backend split design

79a2f91

karlrado mentioned this pull request Nov 3, 2026

[Feature]: Add Windows Support (WinFSP backend) #346

Open

2 tasks done

samsoir requested changes Nov 9, 2026

View reviewed changes

docs(fuse) Address review comments

f037628

samsoir requested changes Nov 11, 2026

View reviewed changes

docs(fuse) Address review comments samsoir#2

5f2bc47

docs(fuse) Fix typos in round 2 changes

ac0580c

Conversation

karlrado commented Nov 3, 2026

Summary

Changes

Related Issues

Test Plan

Checklist

Uh oh!

samsoir commented Nov 5, 2026

Uh oh!

samsoir left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

karlrado commented Nov 10, 2026

Uh oh!

samsoir left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment