Repository navigation
Faster folder loading in COSMIC Files, no full download for content sniffing - #4
Merged
Merged
Conversation
15 of 19 tasks
…remote COSMIC Files only treats a fixed list of mount types as remote and read every file and sub-folder of a "local" folder one after the other, which downloaded the files. fuse.s3fs is on that list and has no special handling in GLib, KIO, Solid or libmount. The source stays "pdrive". Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
COSMIC Files guesses the type of every file by reading up to 18 730 bytes, which went past the 16 KiB head and downloaded the whole file. - HEAD_LENGTH is 32 KiB (older 16 KiB heads still serve the reads they cover). - The furthest range read is tracked per transfer: when the last reader closes the file without going beyond the head, the download is cancelled as soon as the head is written instead of after 20 s. - The kernel read-ahead is limited to the head length, so that a small read is not turned into a bigger request. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Opening a folder lists its not yet loaded sub-folders once, one level deep, four requests at a time, so that item counts and navigation do not wait for the network. Requests for a folder being listed wait for that listing instead of starting another. Thumbnails of folders listed ahead of time are fetched when the folder is opened. No polling: later changes still come from the Drive events. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
bpresles
force-pushed
the
cosmic-files-perf
branch
from
November 13, 2026 21:06
7fdfe49 to
ef49623
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bpresles
added a commit
that referenced
this pull request
Nov 14, 2026
…niffing (#4) * pdrived: mount as fuse.s3fs so that COSMIC Files treats the drive as remote COSMIC Files only treats a fixed list of mount types as remote and read every file and sub-folder of a "local" folder one after the other, which downloaded the files. fuse.s3fs is on that list and has no special handling in GLib, KIO, Solid or libmount. The source stays "pdrive". Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * pdrived: keep a 32 KiB head and stop downloads that only sniff a file COSMIC Files guesses the type of every file by reading up to 18 730 bytes, which went past the 16 KiB head and downloaded the whole file. - HEAD_LENGTH is 32 KiB (older 16 KiB heads still serve the reads they cover). - The furthest range read is tracked per transfer: when the last reader closes the file without going beyond the head, the download is cancelled as soon as the head is written instead of after 20 s. - The kernel read-ahead is limited to the head length, so that a small read is not turned into a bigger request. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * pdrived: list the sub-folders of a listed folder in the background Opening a folder lists its not yet loaded sub-folders once, one level deep, four requests at a time, so that item counts and navigation do not wait for the network. Requests for a folder being listed wait for that listing instead of starting another. Thumbnails of folders listed ahead of time are fetched when the folder is opened. No polling: later changes still come from the Drive events. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * PLAN.md: what was tested of the Dolphin and COSMIC Files integration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
COSMIC Files (1.11) treats
fuse.pdriveas a local file system. For every entry of a folder, it lists each sub-folder to count its items and reads the beginning of each file to guess its type (xdg-mime-rs, plainopen, up to 18 730 bytes). Measured on a real folder with 72 sub-folders and 24 files: 35.6 s to count the sub-folders on the first visit, and 0.8 to 1.7 s per uncached file. Uncached files were downloaded completely: the 16 KiB cached head was smaller than the read, and a download without readers ran for 20 more seconds.fuse.s3fs(FSNamestayspdrive): COSMIC Files only treats listed types as remote, andfuse.s3fsis one of them. The type has no special handling in GLib (is_remote_fs_type), KIO (KMountPoint::isOnRemoteStoragealready treats anyfuse.*not from/dev/as remote), Solid (fuse.sshfs,fuse.rclone,fuse.cephonly) or libmount (mnt_fstype_is_netfs), so Nautilus and Dolphin are unchanged. To be removed if COSMIC Files detects such mounts another way.set_max_readahead(HEAD_LENGTH)): with the 128 KiB default, a short read becomes a ~64 KiB FUSE request that exceeds the head. Serving only the available part of a readahead request was rejected: a short reply to a readahead request leaves zero-filled pages in the page cache.load_childrencalls for the same folder wait for the request in flight instead of sending another one. Each folder is listed once, then kept up to date by Drive events (no polling). Thumbnails of prefetched folders are fetched when the folder is first opened.Rebased on master after #3 (only
PLAN.mdconflicted: both sections kept).Test plan
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningscargo test --workspace(pdrived: 101, including the new tests for the head, early cancellation, in-flight listing deduplication andunloaded_subfolders)make -C extensionfindmnt: typefuse.s3fs, sourcepdrive;read_ahead_kbof the mount = 32🤖 Generated with Claude Code