Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,16 @@ Plan : ajouter Dolphin (KDE) et COSMIC Files à Nautilus, sans changer le daemon
- [x] Plugins Dolphin C++ / KF6 (`integrations/dolphin/`) : `pdriveoverlay` (`KOverlayIconPlugin`, emblèmes, cache borné, appels D-Bus asynchrones) et `pdriveactions` (`KAbstractFileItemActionPlugin`, menus limités aux fichiers du drive). Textes du domaine gettext `pdrive` par `i18nd` ; `scripts/update-po.sh` extrait `i18nd`.
- [x] Paquets : spec RPM (`__requires_exclude_from`), `debian/` (`dh_shlibdeps -X`, plus de `libnautilus-extension4` en `Depends`), PKGBUILD (`optdepends` `nautilus` et `dolphin`), `scripts/install-system.sh` (construit ce qui est détecté) et `uninstall.sh`, scripts de conteneur, job CI `dolphin-plugin` (Debian 13), README, `packaging/README.md`, pages de manuel.
- Limites : pas d'emblèmes dans COSMIC Files (aucune API d'extension) et ses deux actions apparaissent sur tous les fichiers ; Nemo, Thunar et Caja non traités ; les paquets ne nettoient pas les fichiers de configuration de l'utilisateur à la désinstallation.
- [ ] À tester à la main : COSMIC (favori, actions, notification hors drive, interrupteur), session KDE réelle (emblèmes, menus, entrée dans Emplacements, pas de second volume dans « Périphériques », icônes `emblem-pdrive-*` avec le thème Breeze), changement d'environnement sans réinstaller, installation du paquet sur une machine GNOME seule (ni KF6 ni Qt6 tirés).
- [x] Vérifié le 13/11/2026 (RPM de test, Fedora 45, session COSMIC) : COSMIC Files (favori unique, actions « PDrive : … », notification hors drive, interrupteur qui retire puis remet les entrées) ; Dolphin 26.08 hors Plasma (emblèmes, actions limitées au drive, entrée « Proton Drive » dans Emplacements, pas de second volume dans « Périphériques ») ; paquets RPM (rpmlint 0 erreur), DEB (lintian sans remarque hors `-dbgsym`, `Depends` sans KF6, Qt6 ni `libnautilus-extension`) et Arch (namcap 0 erreur) ; `install-system.sh` puis `uninstall.sh` dans un conteneur Fedora 45. Le plugin d'emblèmes était d'abord ignoré par Dolphin, qui exige `Q_PLUGIN_METADATA` et non une `KPluginFactory`.
- [ ] Reste à tester : emblèmes avec le thème Breeze dans une vraie session Plasma, passage d'une session COSMIC à une session KDE sans réinstaller, `uninstall.sh` après `install.sh` (retrait des entrées des gestionnaires) et avec des plugins appartenant à un paquet, AppImage.

### Performances avec COSMIC Files (13/11/2026)

- [x] Type de montage `fuse.s3fs` (source toujours `pdrive`) : COSMIC Files ne traite comme distants que certains types de montage et lisait donc chaque fichier (détection du type, téléchargement complet) et chaque sous-dossier (comptage des éléments) en séquence. Nautilus, Dolphin, GIO, KIO et Solid ne traitent pas ce type autrement. À retirer si COSMIC Files détecte un jour ces montages autrement.
- [x] Tête de fichier gardée en cache portée de 16 à 32 Kio (`xdg-mime` lit jusqu'à 18 730 octets, GIO 16 Kio) ; les anciennes têtes de 16 Kio restent servies pour ce qu'elles couvrent, une lecture plus longue lance le téléchargement comme avant.
- [x] Un lecteur qui ferme le fichier sans avoir lu au-delà de la tête n'attend plus les 20 s : le téléchargement est annulé dès que la tête est écrite (le premier bloc chiffré, 4 Mio au plus, reste téléchargé). Les lectures réelles gardent le délai de 20 s.
- [x] Lecture anticipée du noyau limitée à la longueur de la tête (`max_readahead`) : avec la fenêtre par défaut, une lecture de 18 730 octets devenait une requête de 64 Kio, au-delà de la tête, donc un téléchargement complet. Raisonnement d'après le code du noyau, non mesuré sur la machine.
- [x] Préchargement des sous-dossiers : quand un dossier est listé, ses sous-dossiers non encore chargés sont listés en arrière-plan (un seul niveau, 4 requêtes simultanées au plus). Un seul listage par dossier, déclenché par l'utilisateur, jamais répété (la suite vient des événements Drive) ; une demande pendant un listage en cours l'attend. Les miniatures des dossiers préchargés ne sont récupérées qu'à l'ouverture du dossier, pour n'ajouter ni requête ni écriture dans le cache de miniatures.

## 3. Autres pistes (backlog)

Expand Down
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,9 @@ flowchart LR
- **`pdrived`** mounts the drive with FUSE in a hidden directory
(`~/.local/share/pdrive/.mount`, ignored by GLib and gvfs so that the sidebar shows a single
entry), and `~/ProtonDrive` is a symbolic link to it. It owns the content cache, the upload
journal, the thumbnails and the D-Bus service.
journal, the thumbnails and the D-Bus service. The file system type is `fuse.s3fs` (the source
stays `pdrive`): COSMIC Files only treats a fixed list of mount types as remote, and otherwise
reads every file to guess its type, which downloads it.
- **D-Bus** (session bus, name `io.github.bpresles.PDrive`): interface
`io.github.bpresles.PDrive1` (per-file state, offline pinning, account, quota, recent
activity, mount / unmount) and the CloudProviders objects declared in
Expand Down Expand Up @@ -277,6 +279,9 @@ Without a journal, the log is `$XDG_STATE_HOME/pdrive/pdrive.log` (by default
- File permissions and a modification date alone are not kept (Proton Drive does not store them).
- Files "Shared with me" and Photos are not handled.
- `O_NOATIME` is refused on files that are not in the cache.
- Opening a folder also lists its sub-folders in the background (one level, a few requests at a
time), and the first 32 KiB of a file read by a file manager to guess its type are kept in the
cache: the rest of a file is only downloaded if something reads further.
- Deleting a file moves it directly to the Proton trash, not to the Nautilus trash.
- Rendering on KDE and GNOME, and the translated Nautilus and Dolphin menus, still need more real-world testing.
- COSMIC Files shows no state emblems (no extension API), and its two PDrive actions appear on every
Expand Down
162 changes: 148 additions & 14 deletions crates/pdrived/src/content.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,17 @@
//! reads are served as soon as the requested range has been written. The SDK delivers the
//! content in order, so a read waits until the download has reached the end of its range.
//!
//! A download nobody reads any more is cancelled after a grace period, so that a program reading
//! only the beginning of a big file does not fetch all of it. What was received is kept as the
//! "head" of the file, which serves later reads of that beginning (e.g. content type sniffing)
//! without downloading again.
//! A download nobody reads any more is cancelled, so that a program reading only the beginning
//! of a big file does not fetch all of it: at once (as soon as the head is written) if the last
//! reader never went beyond the head, after a grace period otherwise. What was received is kept
//! as the "head" of the file, which serves later reads of that beginning (e.g. content type
//! sniffing) without downloading again.

use std::collections::HashMap;
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;

Expand All @@ -25,8 +26,12 @@ use crate::activity::{ActivityLog, Kind, State};
use tokio::sync::watch;

const IDLE_CANCEL_DELAY: Duration = Duration::from_secs(20);
/// Smallest beginning worth keeping: GIO sniffs at most 16 KiB.
pub const HEAD_LENGTH: u64 = 16 * 1024;
/// Smallest beginning worth keeping: GIO sniffs at most 16 KiB and `xdg-mime` (COSMIC Files)
/// 18 730 bytes, the longest magic number range of shared-mime-info. It is also the largest
/// read-ahead the kernel is allowed (see `fs.rs`), so that sniffing a file never goes beyond it.
/// Heads cached with a shorter length (older versions kept 16 KiB) are still served for the reads
/// they cover; longer reads download the content, which then replaces them.
pub const HEAD_LENGTH: u64 = 32 * 1024;

#[derive(Clone, Copy, PartialEq)]
pub enum Progress {
Expand All @@ -39,19 +44,37 @@ pub enum Progress {
pub struct Transfer {
progress: watch::Receiver<Progress>,
readers: AtomicUsize,
/// End of the furthest range any reader asked for.
read_end: AtomicU64,
cancelled: AtomicBool,
cancellation: Option<CancellationToken>,
runtime: tokio::runtime::Handle,
}

impl Transfer {
/// Waits until `end` bytes are available (or the download ended). Returns false on failure.
/// Records the range as read.
pub async fn wait_for(&self, end: u64) -> bool {
self.read_end.fetch_max(end, Ordering::SeqCst);
self.wait_written(end).await
}

async fn wait_written(&self, end: u64) -> bool {
let mut progress = self.progress.clone();
let result = progress.wait_for(|p| !matches!(p, Progress::Running(written) if *written < end)).await;
matches!(result.as_deref(), Ok(Progress::Running(_) | Progress::Done))
}

/// Whether nobody reads this download and nobody read further than the head: the rest is
/// not wanted (type sniffing, thumbnails of what is already known…).
fn only_head_wanted(&self) -> bool {
self.readers.load(Ordering::SeqCst) == 0 && self.read_end.load(Ordering::SeqCst) <= HEAD_LENGTH
}

fn running(&self) -> bool {
matches!(*self.progress.borrow(), Progress::Running(_))
}

/// Cancels the download (once). Callable from any thread.
fn cancel(self: &Arc<Self>) {
if self.cancelled.swap(true, Ordering::SeqCst) {
Expand Down Expand Up @@ -199,6 +222,7 @@ impl ContentCache {
let transfer = Arc::new(Transfer {
progress: receiver,
readers: AtomicUsize::new(1),
read_end: AtomicU64::new(0),
cancelled: AtomicBool::new(false),
cancellation: Some(cancellation),
runtime: self.runtime.clone(),
Expand Down Expand Up @@ -302,13 +326,27 @@ impl ContentCache {
if transfer.readers.fetch_sub(1, Ordering::SeqCst) != 1 {
return;
}
self.runtime.spawn(async move {
tokio::time::sleep(IDLE_CANCEL_DELAY).await;
let running = matches!(*transfer.progress.borrow(), Progress::Running(_));
if running && transfer.readers.load(Ordering::SeqCst) == 0 {
transfer.cancel();
}
});
self.runtime.spawn(cancel_when_unwanted(transfer, IDLE_CANCEL_DELAY));
}
}

/// Cancels a download nobody reads any more: once its head is written if no reader went beyond
/// the head, otherwise after `idle_delay` (a reader may come back, e.g. a copy resuming).
async fn cancel_when_unwanted(transfer: Arc<Transfer>, idle_delay: Duration) {
if transfer.only_head_wanted() {
transfer.wait_written(HEAD_LENGTH).await;
if transfer.running() && transfer.only_head_wanted() {
transfer.cancel();
return;
}
// Finished meanwhile, or someone else reads it now (its own release decides).
if !transfer.running() || transfer.readers.load(Ordering::SeqCst) > 0 {
return;
}
}
tokio::time::sleep(idle_delay).await;
if transfer.running() && transfer.readers.load(Ordering::SeqCst) == 0 {
transfer.cancel();
}
}

Expand Down Expand Up @@ -495,12 +533,108 @@ mod tests {
Transfer {
progress: receiver,
readers: AtomicUsize::new(1),
read_end: AtomicU64::new(0),
cancelled: AtomicBool::new(false),
cancellation: None,
runtime: tokio::runtime::Handle::current(),
}
}

#[test]
fn the_head_covers_what_sniffers_read() {
// xdg-mime reads up to 18 730 bytes, GIO 16 KiB.
const { assert!(HEAD_LENGTH >= 18_730 && HEAD_LENGTH >= 16 * 1024) };
assert_eq!(HEAD_LENGTH, 32 * 1024);
}

#[test]
fn keep_head_keeps_what_is_longer_than_the_head() {
let dir = tempfile::tempdir().unwrap();
let (partial, head) = (dir.path().join("x.part"), dir.path().join("x.head"));
std::fs::write(&partial, vec![1u8; 4 * 1024 * 1024]).unwrap();
keep_head(&partial, &head, 4 * 1024 * 1024);
assert_eq!(std::fs::metadata(&head).unwrap().len(), 4 * 1024 * 1024);
// An old 16 KiB head is now too short to be kept from a new download.
std::fs::write(&partial, vec![1u8; 16 * 1024]).unwrap();
std::fs::remove_file(&head).unwrap();
keep_head(&partial, &head, 16 * 1024);
assert!(!head.exists());
}

#[tokio::test]
async fn a_reader_stopping_inside_the_head_cancels_once_the_head_is_written() {
let (sender, receiver) = watch::channel(Progress::Running(0));
let transfer = Arc::new(transfer(receiver));
// Reads what a sniffer reads, then closes.
assert!(!transfer.only_head_wanted());
transfer.readers.fetch_sub(1, Ordering::SeqCst);
assert!(transfer.only_head_wanted());
let task = tokio::spawn(cancel_when_unwanted(transfer.clone(), Duration::from_secs(3600)));
tokio::task::yield_now().await;
sender.send_replace(Progress::Running(HEAD_LENGTH - 1));
tokio::task::yield_now().await;
assert!(!task.is_finished() && !transfer.cancelled.load(Ordering::SeqCst));
sender.send_replace(Progress::Running(HEAD_LENGTH));
task.await.unwrap();
assert!(transfer.cancelled.load(Ordering::SeqCst));
}

#[tokio::test]
async fn a_reader_going_beyond_the_head_keeps_the_grace_period() {
let (sender, receiver) = watch::channel(Progress::Running(0));
let transfer = Arc::new(transfer(receiver));
sender.send_replace(Progress::Running(HEAD_LENGTH + 1));
assert!(transfer.wait_for(HEAD_LENGTH + 1).await);
transfer.readers.fetch_sub(1, Ordering::SeqCst);
assert!(!transfer.only_head_wanted());
cancel_when_unwanted(transfer.clone(), Duration::from_millis(50)).await;
assert!(transfer.cancelled.load(Ordering::SeqCst));
// Within the grace period, a new reader prevents the cancellation.
let (_sender, receiver) = watch::channel(Progress::Running(10));
let transfer = Arc::new(self::transfer(receiver));
transfer.read_end.store(HEAD_LENGTH + 1, Ordering::SeqCst);
transfer.readers.store(0, Ordering::SeqCst);
let task = tokio::spawn(cancel_when_unwanted(transfer.clone(), Duration::from_millis(100)));
transfer.readers.fetch_add(1, Ordering::SeqCst);
task.await.unwrap();
assert!(!transfer.cancelled.load(Ordering::SeqCst));
}

#[tokio::test]
async fn a_second_reader_prevents_the_early_cancellation() {
let (sender, receiver) = watch::channel(Progress::Running(0));
let transfer = Arc::new(transfer(receiver));
transfer.readers.store(0, Ordering::SeqCst);
let task = tokio::spawn(cancel_when_unwanted(transfer.clone(), Duration::from_secs(3600)));
tokio::task::yield_now().await;
// Another handle joins the download while the head is being written.
transfer.readers.fetch_add(1, Ordering::SeqCst);
sender.send_replace(Progress::Running(HEAD_LENGTH));
task.await.unwrap();
assert!(!transfer.cancelled.load(Ordering::SeqCst));
}

#[tokio::test]
async fn a_finished_download_is_not_cancelled() {
let (sender, receiver) = watch::channel(Progress::Running(0));
let transfer = Arc::new(transfer(receiver));
transfer.readers.store(0, Ordering::SeqCst);
let task = tokio::spawn(cancel_when_unwanted(transfer.clone(), Duration::from_secs(3600)));
tokio::task::yield_now().await;
sender.send_replace(Progress::Done);
task.await.unwrap();
assert!(!transfer.cancelled.load(Ordering::SeqCst));
}

#[tokio::test]
async fn wait_for_records_the_furthest_range() {
let (_sender, receiver) = watch::channel(Progress::Done);
let transfer = transfer(receiver);
transfer.wait_for(100).await;
transfer.wait_for(50).await;
assert_eq!(transfer.read_end.load(Ordering::SeqCst), 100);
}

#[tokio::test]
async fn wait_for_returns_when_the_range_is_written() {
let (sender, receiver) = watch::channel(Progress::Running(0));
Expand Down
Loading