From b09c2dde52b1fc93fc7db4866f462365e08dcfd6 Mon Sep 17 00:00:00 2001 From: ymote <151983+ymote@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:01:04 +0800 Subject: [PATCH 1/3] SERVICE.md: playback (deck.media) The maintainer approved playback on 13 Oct 2026: Quick Deck plays a deck's video and audio with the runtime's Video widget, which hands the file to the operating system's own player. The service adds one method, deck.media {doc, slide, id}, which writes the clip's bytes from the open document's model, as they are, to .cache/media/, and never decodes a clip; media.info, media.posterFrame, insert.audio and insert.video stay held (#448). The section is appended verbatim as approved. Co-Authored-By: Claude Opus 5.5 --- apps/quickdeck/design/SERVICE.md | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/apps/quickdeck/design/SERVICE.md b/apps/quickdeck/design/SERVICE.md index b3038795..0430d3ed 100644 --- a/apps/quickdeck/design/SERVICE.md +++ b/apps/quickdeck/design/SERVICE.md @@ -464,3 +464,40 @@ document's model (read-only, no engine command): banded_columns}`, the keys `deck.table {op: "options"}` takes. - An Office document that isn't a presentation (a `.docx` or `.xlsx`) is `unsupported:`. + +## Playback (maintainer approved, 13 Oct 2026) + +Quick Deck plays a deck's video and audio with the runtime's `Video` widget, +which hands the file to the operating system's own player (AVFoundation on +macOS and iOS, the platform player on Android). The deck engine's decoders +stay unused: the service never decodes, probes or transcodes a clip, and +`media.info`, `media.posterFrame`, `insert.audio` and `insert.video` stay +held (#448). + +| Method | Arguments | Answer | Engine | +| --- | --- | --- | --- | +| `deck.media` | `{doc, slide, id}` | `{path, kind, ext, bytes, autoplay, loop, poster}` | none: the clip's bytes from the open document's model, written as they are | + +- `id` is the shape id of a video or audio object; `deck.slide` reports such + a shape with `media: {id, video, autoplay, loop}`. Any other shape is + `unknown_id:`. +- `path` is a storage-relative file under `.cache/media/`, named by a hash of + the clip's bytes, holding exactly the bytes the deck stores. `ext` is the + deck's own extension for the clip, else one inferred from its declared + content type (`mp4`, `m4v`, `mov`, `mp3`, `m4a`, `aac`, `wav`); a clip + with neither answers `unsupported:`. `kind` is `video` or `audio`; + `autoplay` and `loop` are the clip's own settings; `poster` says whether + the clip has a poster image. +- A clip already written is reused (the same hash). The media cache holds at + most 32 MiB and 8 files per app, oldest first; a document's clips are + removed when it closes. If a clip doesn't fit, the render cache and then the + media cache are cleared before the write, and a clip that still doesn't fit + is `storage_full:` ("There isn't room in Quick Deck's storage to play this + clip."). +- Writes are staged and never replace a file the call didn't create. +- **Where it plays:** the app overlays a `Video` widget on the clip's box, + with the file as its storage source, in the editor (a play control on a + chosen clip) and in Present (autoplay clips start when their slide shows; + a tap plays or pauses; moving on or Exit stops). Where the `Video` widget + has no player (today: Linux and Windows desktops), the app keeps the poster + and says "Playback isn't available on this system". From 98358fe89a02d8f315b1f6747e48e4aa89ff0cf2 Mon Sep 17 00:00:00 2001 From: ymote <151983+ymote@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:23:39 +0800 Subject: [PATCH 2/3] deck: deck.media hands the app a clip's bytes to play deck.media {doc, slide, id} answers {path, kind, ext, bytes, autoplay, loop, poster}, as SERVICE.md "Playback" says. The clip's bytes come from the open document's model, the item the shape's MediaClip names, and are written as they are to .cache/media/., staged and placed by a link that refuses anything already there. No engine command runs and no engine function is called, so nothing decodes a clip; media.info, media.posterFrame, insert.audio and insert.video stay held (#448). - ext: the deck's own extension for the clip when it is one of mp4, m4v, mov, mp3, m4a, aac or wav, else the one its declared content type names; neither is unsupported:, and so is a clip whose bytes the deck doesn't hold (linked). Any shape but a clip is unknown_id:. - The media cache holds 32 MiB and 8 clips per storage; the clip used longest ago goes first, and a clip already there with the same bytes (compared in full) is handed out again and counts as just used. - A document's clips go when it closes or is released with its renders, unless another document open in the same storage uses them; clips no open document uses go when a document next opens. - Before a clip would not fit the storage, the render cache and then the media cache are cleared; one that still doesn't fit, or one larger than the whole media cache, is storage_full: in the contract's words. - deck.slide now reports a clip's media {id, video, autoplay, loop}: the engine's slide.inspect gave it, but shape_json didn't copy it. The deck paragraph in apps/AGENTS.md describes it. sha2 is the workspace's own 0.10, already in Cargo.lock. Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 1 + apps/AGENTS.md | 16 +- apps/deck/host-service/Cargo.toml | 3 + apps/deck/host-service/src/doc_tests.rs | 412 ++++++++++++++++++++++++ apps/deck/host-service/src/docs.rs | 40 ++- apps/deck/host-service/src/lib.rs | 14 +- apps/deck/host-service/src/media.rs | 325 +++++++++++++++++++ apps/deck/host-service/src/reading.rs | 2 +- 8 files changed, 796 insertions(+), 17 deletions(-) create mode 100644 apps/deck/host-service/src/media.rs diff --git a/Cargo.lock b/Cargo.lock index 54aa11ea..2c515319 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -11578,6 +11578,7 @@ dependencies = [ "octosense-engine-area", "octosense-skill-gen", "serde_json", + "sha2 0.10.9", "tempfile", "zip 3.0.0", ] diff --git a/apps/AGENTS.md b/apps/AGENTS.md index e7923134..667350fc 100644 --- a/apps/AGENTS.md +++ b/apps/AGENTS.md @@ -133,9 +133,19 @@ app's own agent and `tools.json`, the system toolbox, `glance.publish` and most 32, kept across closes and named by the file's name, size and modification time (`src/covers.rs`). Slide drawings go before covers, each oldest first, and the whole cache is cleared before a write would - fail with `storage_full:`. What the inspector shows beyond the - commands' answers (a box's anchor and autofit, a chart's and a table's - options, the deck's colour and font schemes and its footer, the engine's + fail with `storage_full:`. Playback (`deck.media`, `src/media.rs`) runs + no engine command: it writes a video or audio clip's bytes from the open + document's model, as the deck stores them and never decoded, to + `.cache/media/.` for the app's `Video` widget (`ext` from + the deck's own name for the clip or its declared type, from SERVICE.md's + list, else `unsupported:`), staged and create-only, reused by hash, in a + media cache of 32 MiB and 8 clips (the one used longest ago goes first); + a document's clips go when it closes or is released, unless another open + document uses them, and before a clip would not fit, the render cache and + then the media cache are cleared (`storage_full:`). What the inspector + shows beyond the commands' answers (a box's anchor and autofit, a + chart's and a table's options, the deck's colour and font schemes and + its footer, the engine's colour schemes with their colours) the service reads from the open document's model, read-only (`src/reading.rs`, `docs::design_state`). Errors start with a stable code diff --git a/apps/deck/host-service/Cargo.toml b/apps/deck/host-service/Cargo.toml index 63ac4478..dd181857 100644 --- a/apps/deck/host-service/Cargo.toml +++ b/apps/deck/host-service/Cargo.toml @@ -48,6 +48,9 @@ zip = { version = "3", default-features = false, features = ["deflate"] } # of the engine's `open_presentation`), since the engine's own sniff misses # a package whose content-type list comes late, as LibreOffice writes them. deckcraft-pptx = { workspace = true } +# Names a clip in the media cache by a hash of its bytes (src/media.rs); +# the workspace's sha2, as Mail's and the shell's. +sha2 = "0.10" [dev-dependencies] # Generates and checks the skill's references (tests/skill.rs). diff --git a/apps/deck/host-service/src/doc_tests.rs b/apps/deck/host-service/src/doc_tests.rs index a7c4e22b..fd6d6d51 100644 --- a/apps/deck/host-service/src/doc_tests.rs +++ b/apps/deck/host-service/src/doc_tests.rs @@ -340,6 +340,7 @@ fn an_unknown_or_foreign_handle_is_unknown() { ("redo", json!({ "doc": doc })), ("show", json!({ "doc": doc })), ("export", json!({ "doc": doc, "kind": "outline", "out": "x.txt" })), + ("media", json!({ "doc": doc, "slide": s, "id": 1 })), ] { coded(&other.err(method, args.clone()), "unknown_doc"); coded(&w.call_full("os.notes", method, args, true, 0).unwrap_err(), "unknown_doc"); @@ -1004,6 +1005,416 @@ fn a_deck_past_the_media_ceiling_is_too_large_before_the_engine_reads() { assert_eq!(media_ceiling(&deck(over - 1)), Ok(())); } +// ------------------------------------------------------------ playback (deck.media) + +/// A deck whose first slide holds one clip per `(name, content type, bytes, +/// video)`, side by side, after the sample deck's title slide's own shapes. +fn with_items(items: &[(&str, &str, Vec, bool)]) -> Presentation { + use deckcraft_model::{MediaClip, MediaId, MediaItem, Shape, ShapeKind, Xfrm}; + let mut p = sample(1); + for (k, (name, content_type, data, video)) in items.iter().enumerate() { + let media = MediaId(p.alloc_id()); + p.media.push(MediaItem { id: media, name: (*name).into(), content_type: (*content_type).into(), data: Arc::new(data.clone()), link: None }); + let clip = MediaClip { media, video: *video, volume: 1.0, ..Default::default() }; + let shape = Shape { + id: ShapeId(p.alloc_id()), + name: format!("Clip {}", k + 1), + xfrm: Some(Xfrm::new(20.0 + 60.0 * k as f64, 20.0, 50.0, 40.0)), + kind: ShapeKind::Media(clip), + ..Default::default() + }; + Arc::make_mut(&mut p.slides[0]).shapes.push(shape); + } + p +} + +/// Change the first clip on slide `index` of `p`. +fn set_clip(p: &mut Presentation, index: usize, f: impl FnOnce(&mut deckcraft_model::MediaClip)) { + let slide = Arc::make_mut(&mut p.slides[index]); + let clip = slide.shapes.iter_mut().find_map(|sh| match &mut sh.kind { + deckcraft_model::ShapeKind::Media(m) => Some(m), + _ => None, + }); + f(clip.expect("a clip on the slide")); +} + +/// `len` bytes that differ from clip to clip (`k`), and that the native +/// format stores as they are for a `video/` type. +fn clip_of(k: usize, len: usize) -> Vec { + let mut b = vec![k as u8; len]; + b[..8].copy_from_slice(&(k as u64).to_le_bytes()); + b +} + +/// The file's identity on its file system: equal before and after, it was +/// not written again. +fn inode(path: &Path) -> u64 { + #[cfg(unix)] + { + use std::os::unix::fs::MetadataExt; + std::fs::metadata(path).unwrap().ino() + } + #[cfg(not(unix))] + { + let _ = path; + 0 + } +} + +impl World { + /// The ids of slide `slide`'s video and audio shapes, in order: the + /// shapes `deck.slide` reports with `media`. + fn clip_ids(&self, doc: &str, slide: u64) -> Vec { + self.shapes(doc, slide).iter().filter(|s| s["media"].is_object()).map(|s| s["id"].as_u64().unwrap()).collect() + } + + /// `deck.media` for shape `id` on slide `slide`: the path it answered. + #[track_caller] + fn play(&self, doc: &str, slide: u64, id: u64) -> String { + self.ok("media", json!({ "doc": doc, "slide": slide, "id": id }))["path"].as_str().unwrap().to_string() + } +} + +/// `deck.media` writes a clip's bytes from the open document's model, as the +/// deck stores them, to `.cache/media/.`, for a video (`.mp4`) +/// and a sound (`.m4a`), from a package and from a native deck. `kind`, +/// `autoplay`, `loop` and `poster` are the model's own, and `deck.slide` +/// reports each clip's `media`. The engine reads no file for it. +#[test] +fn deck_media_writes_the_clips_bytes_as_the_deck_stores_them() { + use sha2::Digest as _; + let w = World::new(); + let video = b"\0\0\0\x18ftypisom\0\0\x02\0isomiso2mp41 a small movie".to_vec(); + let sound = b"\0\0\0\x18ftypM4A \0\0\0\0M4A isom a small sound".to_vec(); + let mut deck = with_clips(&video, &sound, true); + // The sound plays by itself, over and over. + set_clip(&mut deck, 2, |m| { + m.autoplay = true; + m.loop_play = true; + }); + w.put("clips.deckcraft", &native(&deck)); + w.put("clips.pptx", &pptx(&deck)); + for path in ["clips.deckcraft", "clips.pptx"] { + let doc = w.open(path); + let ids = w.slide_ids(&doc); + let film = w.shape_where(&doc, ids[0], "kind", "video"); + let tone = w.shape_where(&doc, ids[2], "kind", "audio"); + let reported = |slide: u64, id: u64| w.shapes(&doc, slide).into_iter().find(|s| s["id"] == id).unwrap()["media"].clone(); + let (film_media, tone_media) = (reported(ids[0], film), reported(ids[2], tone)); + assert_eq!((film_media["video"].clone(), film_media["autoplay"].clone(), film_media["loop"].clone()), (json!(true), json!(false), json!(false)), "{path}: {film_media}"); + assert_eq!(tone_media["video"], false, "{path}: {tone_media}"); + assert!(film_media["id"].is_u64() && tone_media["id"].is_u64(), "{path}: the clip's media item"); + // Whether the model holds a poster picture for the clip: a package's + // writer gives every clip a picture, so the sound has one once it + // went through a .pptx. + let poster = |id: u64| { + let deck = held_deck(&doc); + let clip = deck.slides.iter().find_map(|s| s.shape(ShapeId(id as u32))).and_then(|sh| match &sh.kind { + deckcraft_model::ShapeKind::Media(m) => m.poster, + _ => None, + }); + clip.and_then(|p| deck.media(p)).is_some_and(|item| !item.data.is_empty()) + }; + let before = engine_reads(); + let a = w.ok("media", json!({ "doc": doc, "slide": ids[0], "id": film })); + let want = format!(".cache/media/{:x}.mp4", sha2::Sha256::digest(&video)); + assert!(poster(film), "{path}: the video's red poster"); + assert_eq!(a, json!({ "path": want, "kind": "video", "ext": "mp4", "bytes": video.len(), "autoplay": false, "loop": false, "poster": true }), "{path}"); + assert_eq!(std::fs::read(w.root.join(&want)).unwrap(), video, "{path}: the deck's bytes, as they are"); + let b = w.ok("media", json!({ "doc": doc, "slide": ids[2], "id": tone })); + let want = format!(".cache/media/{:x}.m4a", sha2::Sha256::digest(&sound)); + let (autoplay, looped) = (tone_media["autoplay"].clone(), tone_media["loop"].clone()); + assert_eq!(b, json!({ "path": want, "kind": "audio", "ext": "m4a", "bytes": sound.len(), "autoplay": autoplay, "loop": looped, "poster": poster(tone) }), "{path}"); + if path.ends_with(".deckcraft") { + assert_eq!((autoplay, looped, poster(tone)), (json!(true), json!(true), false), "the clip's own settings, and no poster"); + } + assert_eq!(std::fs::read(w.root.join(&want)).unwrap(), sound, "{path}"); + assert_eq!(engine_reads(), before, "{path}: the engine read no file for it"); + w.ok("close", json!({ "doc": doc })); + } +} + +/// A clip already in the media cache is handed out again, not written twice: +/// the same answer, one file, the same file, for the same document and for +/// another one open on the same deck. Something else at the clip's name (a +/// file cut short) is a stale entry: the clip is written anew. +#[test] +fn deck_media_hands_out_a_clip_already_written_again() { + let w = World::new(); + let video = b"\0\0\0\x18ftypisom\0\0\x02\0isomiso2mp41 written once".to_vec(); + w.put("clips.deckcraft", &native(&with_clips(&video, b"a sound", false))); + let (one, two) = (w.open("clips.deckcraft"), w.open("clips.deckcraft")); + let ask = |doc: &str| { + let s = w.slide_ids(doc)[0]; + let id = w.shape_where(doc, s, "kind", "video"); + w.ok("media", json!({ "doc": doc, "slide": s, "id": id })) + }; + let first = ask(&one); + let rel = first["path"].as_str().unwrap().to_string(); + let file = w.root.join(&rel); + let written = inode(&file); + for doc in [&one, &one, &two] { + assert_eq!(ask(doc), first); + assert_eq!(w.files(".cache/media"), vec![rel.clone()], "one file"); + assert_eq!(inode(&file), written, "the same file, not written again"); + } + std::fs::write(&file, &video[..4]).unwrap(); + assert_eq!(ask(&two), first); + assert_eq!(std::fs::read(&file).unwrap(), video, "written anew, whole"); + assert_eq!(w.files(".cache/media"), vec![rel]); +} + +/// Any shape but a video or audio clip is `unknown_id:`, as a shape or slide +/// the deck doesn't have; a clip neither whose name nor whose declared type +/// is one the contract lists, or whose bytes the deck doesn't hold (a link), +/// is `unsupported:`. The deck's own extension comes first, then the +/// declared type's. +#[test] +fn deck_media_answers_unknown_id_for_another_shape_and_unsupported_for_an_unknown_kind() { + use deckcraft_model::{MediaClip, MediaId, MediaItem, Shape, ShapeKind, Xfrm}; + let w = World::new(); + let items = [ + ("clip.bin", "application/octet-stream", b"neither its name nor its type".to_vec(), true), + ("clip", "video/quicktime", b"a movie by its type".to_vec(), true), + ("Film.M4V", "video/mp4", b"its own name first".to_vec(), true), + ("theme.wma", "audio/mpeg", b"a name it doesn't play, a type it does".to_vec(), false), + ("ring", "Audio/X-WAV; rate=44100", b"a sound by its type".to_vec(), false), + ("talk.aac", "", b"by its name alone".to_vec(), false), + ]; + let mut deck = with_items(&items); + // A clip linked to a file elsewhere: no bytes in the deck. + let media = MediaId(deck.alloc_id()); + deck.media.push(MediaItem { id: media, name: "talk.mp4".into(), content_type: "video/mp4".into(), data: Arc::new(vec![]), link: Some("https://example.com/talk.mp4".into()) }); + let shape = Shape { id: ShapeId(deck.alloc_id()), name: "Linked".into(), xfrm: Some(Xfrm::new(600.0, 300.0, 80.0, 60.0)), kind: ShapeKind::Media(MediaClip { media, video: true, volume: 1.0, ..Default::default() }), ..Default::default() }; + Arc::make_mut(&mut deck.slides[0]).shapes.push(shape); + w.put("kinds.deckcraft", &native(&deck)); + let doc = w.open("kinds.deckcraft"); + let s = w.slide_ids(&doc)[0]; + let clips = w.clip_ids(&doc, s); + assert_eq!(clips.len(), 7); + let e = w.err("media", json!({ "doc": doc, "slide": s, "id": clips[0] })); + coded(&e, "unsupported"); + assert!(e.contains("mp4, m4v, mov, mp3, m4a, aac, wav"), "{e}"); + for (k, ext) in [(1, "mov"), (2, "m4v"), (3, "mp3"), (4, "wav"), (5, "aac")] { + let a = w.ok("media", json!({ "doc": doc, "slide": s, "id": clips[k] })); + assert_eq!(a["ext"], ext, "{a}"); + let path = a["path"].as_str().unwrap(); + assert!(path.ends_with(&format!(".{ext}")), "{a}"); + assert_eq!(std::fs::read(w.root.join(path)).unwrap(), items[k].2); + } + let e = w.err("media", json!({ "doc": doc, "slide": s, "id": clips[6] })); + coded(&e, "unsupported"); + assert!(e.contains("linked"), "{e}"); + let other = w.shapes(&doc, s).into_iter().find(|sh| sh["media"].is_null()).unwrap()["id"].clone(); + coded(&w.err("media", json!({ "doc": doc, "slide": s, "id": other })), "unknown_id"); + coded(&w.err("media", json!({ "doc": doc, "slide": s, "id": 999_999 })), "unknown_id"); + coded(&w.err("media", json!({ "doc": doc, "slide": 999_999, "id": clips[1] })), "unknown_id"); + assert_eq!(w.files(".cache/media").len(), 5, "nothing written for a refusal"); +} + +/// The media cache holds at most 32 MiB and 8 clips: the clip used longest +/// ago goes first, and a clip handed out again counts as just used. +#[test] +fn the_media_cache_holds_32_mib_and_8_clips_the_oldest_going_first() { + let w = World::new(); + let names: Vec = (0..9).map(|k| format!("clip{k}.mp4")).collect(); + let items: Vec<(&str, &str, Vec, bool)> = names.iter().enumerate().map(|(k, n)| (n.as_str(), "video/mp4", clip_of(k, 4096), true)).collect(); + w.put("nine.deckcraft", &native(&with_items(&items))); + let doc = w.open("nine.deckcraft"); + let s = w.slide_ids(&doc)[0]; + let ids = w.clip_ids(&doc, s); + // Apart in time, so "oldest" is plain whatever the file system's clock. + let play = |k: usize| { + std::thread::sleep(Duration::from_millis(25)); + w.play(&doc, s, ids[k]) + }; + let paths: Vec = (0..9).map(play).collect(); + let cached = || w.files(".cache/media"); + assert_eq!(cached().len(), media::MAX_FILES); + assert!(!cached().contains(&paths[0]), "the oldest went"); + assert!(paths[1..].iter().all(|p| cached().contains(p))); + // Clip 1 handed out again is just used: written again, clip 0 puts out + // clip 2, the oldest of the rest, and not clip 1. + assert_eq!(play(1), paths[1]); + assert_eq!(play(0), paths[0]); + let now = cached(); + assert!(now.contains(&paths[0]) && now.contains(&paths[1]) && !now.contains(&paths[2]), "{now:?}"); + assert_eq!(now.len(), media::MAX_FILES); + // 32 MiB: of three clips of 12 MiB, the last two stay. + let big = World::new(); + let names: Vec = (0..3).map(|k| format!("big{k}.mp4")).collect(); + let items: Vec<(&str, &str, Vec, bool)> = names.iter().enumerate().map(|(k, n)| (n.as_str(), "video/mp4", clip_of(100 + k, 12 << 20), true)).collect(); + big.put("big.deckcraft", &native(&with_items(&items))); + let doc = big.open("big.deckcraft"); + let s = big.slide_ids(&doc)[0]; + let ids = big.clip_ids(&doc, s); + let paths: Vec = ids + .iter() + .map(|id| { + std::thread::sleep(Duration::from_millis(25)); + big.play(&doc, s, *id) + }) + .collect(); + assert_eq!(big.files(".cache/media"), { + let mut kept = paths[1..].to_vec(); + kept.sort(); + kept + }); + let held: u64 = big.files(".cache/media").iter().map(|f| std::fs::metadata(big.root.join(f)).unwrap().len()).sum(); + assert!(held <= media::MAX_BYTES, "{held} bytes"); +} + +/// A document's clips go when it closes, except one another document open in +/// the same storage still uses; a document released while idle takes its +/// clips too; and a clip no open document uses (an earlier run's) goes when +/// a document next opens. +#[test] +fn a_documents_clips_go_when_it_closes() { + let w = World::new(); + w.put("clips.deckcraft", &native(&with_clips(b"\0\0\0\x18ftypisom movie", b"\0\0\0\x18ftypM4A sound", false))); + let clip = |doc: &str, slide: usize, kind: &str| { + let s = w.slide_ids(doc)[slide]; + let id = w.shape_where(doc, s, "kind", kind); + w.play(doc, s, id) + }; + let (one, two) = (w.open("clips.deckcraft"), w.open("clips.deckcraft")); + let film = clip(&one, 0, "video"); + let tone = clip(&one, 2, "audio"); + assert_eq!(clip(&two, 0, "video"), film); + assert_eq!(w.files(".cache/media").len(), 2); + w.ok("close", json!({ "doc": one })); + assert_eq!(w.files(".cache/media"), vec![film.clone()], "the sound went with its document; the video stays for the other"); + assert!(!w.root.join(&tone).exists()); + w.ok("close", json!({ "doc": two })); + assert!(w.files(".cache/media").is_empty(), "the last document took the video"); + let three = w.open("clips.deckcraft"); + clip(&three, 0, "video"); + docs::sweep_at(Instant::now() + docs::IDLE + Duration::from_secs(1)); + assert!(w.files(".cache/media").is_empty(), "released while idle, with its clips"); + w.put(".cache/media/left-behind.mp4", b"an earlier run's clip"); + w.open("clips.deckcraft"); + assert!(w.files(".cache/media").is_empty(), "pruned when a document opened"); +} + +/// An isolate closing takes its documents' clips with their renders. +#[test] +fn a_closed_isolates_clips_go_with_its_renders() { + use makepad_widgets::*; + crate::register(); + let w = World::new(); + w.put("clips.deckcraft", &native(&with_clips(b"\0\0\0\x18ftypisom movie", b"\0\0\0\x18ftypM4A sound", false))); + let mut cx = Cx::new(Box::new(|_, _| {})); + let mut card = cx.with_vm(|vm| { + makepad_widgets::script_mod(vm); + let value = vm.eval(script! {use mod.widgets.* Splash{}}); + Splash::script_from_value(vm, value) + }); + card.set_host_tag(&mut cx, Some(APP.into())); + card.set_sandbox_dir(&mut cx, Some(w.root.clone())); + card.set_text(&mut cx, "View {}"); + let heap = card.isolate_heap_key(&mut cx).unwrap(); + let from = |method: &str, args: Json| w.call_full(APP, method, args, true, heap).unwrap(); + let doc = from("open", json!({ "path": "clips.deckcraft" }))["doc"].clone(); + let s = from("slides", json!({ "doc": doc }))["slides"][0]["id"].clone(); + let shapes = from("slide", json!({ "doc": doc, "slide": s }))["shapes"].clone(); + let id = shapes.as_array().unwrap().iter().find(|sh| sh["kind"] == "video").unwrap()["id"].clone(); + let clip = from("media", json!({ "doc": doc, "slide": s, "id": id }))["path"].as_str().unwrap().to_string(); + from("view", json!({ "doc": doc, "slide": s, "width": 96 })); + let renders = w.root.join(format!(".cache/slides/{}", doc.as_str().unwrap())); + assert!(w.root.join(&clip).is_file() && renders.is_dir()); + octosense_appstore::services::cancel_heap(heap); + assert!(!w.root.join(&clip).exists(), "the clip went with its document"); + assert!(!renders.exists(), "and so did its renders"); +} + +/// Before a clip would not fit the storage, the render cache and then the +/// media cache are cleared; a clip that still doesn't fit is `storage_full:` +/// in the contract's words, with nothing of it written. A clip larger than +/// the whole media cache is `storage_full:` at once, with nothing cleared. +#[test] +fn the_render_cache_then_the_media_cache_are_cleared_before_storage_full() { + let roomy = World::new(); + let items = [ + ("a.mp4", "video/mp4", clip_of(1, 20_000), true), + ("b.mp4", "video/mp4", clip_of(2, 30_000), true), + ("c.mp4", "video/mp4", clip_of(3, 40_000), true), + ("d.mp4", "video/mp4", clip_of(4, 90_000), true), + ("huge.mp4", "video/mp4", clip_of(5, media::MAX_BYTES as usize + 1), true), + ]; + roomy.put("clips.deckcraft", &native(&with_items(&items))); + let doc = roomy.open("clips.deckcraft"); + let s = roomy.slide_ids(&doc)[0]; + let ids = roomy.clip_ids(&doc, s); + roomy.ok("view", json!({ "doc": doc, "slide": s, "width": 1920 })); + let a = roomy.play(&doc, s, ids[0]); + let renders = roomy.files(".cache/slides"); + assert!(!renders.is_empty()); + let full = format!("storage_full: {}", media::FULL); + // Larger than the whole media cache: refused at once, nothing cleared. + assert_eq!(roomy.err("media", json!({ "doc": doc, "slide": s, "id": ids[4] })), full); + assert_eq!(roomy.files(".cache/slides"), renders); + assert_eq!(roomy.files(".cache/media"), vec![a.clone()]); + let tight = |quota: u64| World { _dir: tempfile::tempdir().unwrap(), root: roomy.root.clone(), areas: roomy.tight(quota) }; + let sorted = |mut v: Vec| { + v.sort(); + v + }; + // B fits once the render cache is cleared; A stays. + let t = tight(30_000 - 1); + let b = t.play(&doc, s, ids[1]); + assert!(t.files(".cache/slides").is_empty(), "the render cache was cleared: {:?}", t.files(".cache/slides")); + assert_eq!(t.files(".cache/media"), sorted(vec![a.clone(), b.clone()])); + // C fits only once the media cache is cleared too. + let t = tight(40_000 - 1); + let c = t.play(&doc, s, ids[2]); + assert_eq!(t.files(".cache/media"), vec![c.clone()], "the media cache was cleared"); + // D doesn't fit even then. + let t = tight(0); + assert_eq!(t.err("media", json!({ "doc": doc, "slide": s, "id": ids[3] })), full); + assert!(t.files(".cache/media").is_empty(), "cleared trying"); + let staging = std::fs::read_dir(&roomy.root).unwrap().flatten().filter(|e| e.file_name().to_string_lossy().starts_with(octosense_engine_area::STAGING_PREFIX)).count(); + assert_eq!(staging, 0, "nothing staged is left"); +} + +/// A hostile "video" (bytes that would trip a decoder, `hostile_movie`) and +/// a hostile sound are written byte for byte, and nothing decodes them: the +/// engine reads no file for `deck.media`, the session holds the same media +/// before and after (the media ceiling counts the same), the method's module +/// calls nothing of the engine's (its code names neither the engine nor its +/// decoders), and the four commands that would decode stay held. +#[test] +fn a_hostile_clip_is_written_byte_for_byte_and_never_decoded() { + let w = World::new(); + let (movie, noise) = (hostile_movie(), hostile_sound()); + let deck = with_clips(&movie, &noise, false); + w.put("hostile.pptx", &pptx(&deck)); + w.put("hostile.deckcraft", &native(&deck)); + for path in ["hostile.pptx", "hostile.deckcraft"] { + let doc = w.open(path); + let ids = w.slide_ids(&doc); + let film = w.shape_where(&doc, ids[0], "kind", "video"); + let tone = w.shape_where(&doc, ids[2], "kind", "audio"); + let held = || docs::with_open(&doc, |d| Held::of(&d.session).media).unwrap(); + let (reads, media_held) = (engine_reads(), held()); + let a = w.ok("media", json!({ "doc": doc, "slide": ids[0], "id": film })); + let b = w.ok("media", json!({ "doc": doc, "slide": ids[2], "id": tone })); + assert_eq!(std::fs::read(w.root.join(a["path"].as_str().unwrap())).unwrap(), movie, "{path}: the movie, byte for byte"); + assert_eq!(std::fs::read(w.root.join(b["path"].as_str().unwrap())).unwrap(), noise, "{path}: the sound, byte for byte"); + assert_eq!((a["bytes"].clone(), b["bytes"].clone()), (json!(movie.len()), json!(noise.len()))); + assert_eq!(engine_reads(), reads, "{path}: the engine read no file"); + assert_eq!(held(), media_held, "{path}: the session's media is unchanged"); + assert!(media_over(media_held).is_none()); + w.ok("close", json!({ "doc": doc })); + } + let code: String = include_str!("media.rs").lines().filter(|l| !l.trim_start().starts_with("//")).collect::>().join("\n"); + for name in ["deckcraft_engine", "deckcraft_media", "deckcraft_render", "deckcraft_pptx", "gate::", "Session", "execute("] { + assert!(!code.contains(name), "media.rs calls into the engine: {name}"); + } + for id in ["media.info", "media.posterFrame", "insert.audio", "insert.video"] { + assert!(HELD_BACK.iter().any(|h| h.id == id), "{id} stays held"); + } +} + /// The import guard passes what the engine itself writes, at the sizes the /// service allows: a native deck's `presentation.json` repeats itself from /// slide to slide, so its compression ratio is the one to watch. @@ -2155,6 +2566,7 @@ fn every_v2_method_refuses_arguments_it_does_not_take() { ("redo", json!({ "doc": doc, "steps": 2 })), ("show", json!({ "doc": doc, "presenter": true })), ("export", json!({ "doc": doc, "kind": "pdf", "out": "x.pdf", "password": "x" })), + ("media", json!({ "doc": doc, "slide": s, "id": 1, "out": "x.mp4" })), ] { let e = w.err(method, args.clone()); coded(&e, "invalid"); diff --git a/apps/deck/host-service/src/docs.rs b/apps/deck/host-service/src/docs.rs index 78389a11..2aefdfdb 100644 --- a/apps/deck/host-service/src/docs.rs +++ b/apps/deck/host-service/src/docs.rs @@ -32,9 +32,11 @@ //! address, can be reused); when its storage scope changes or goes (makepad's //! `splash_storage::storage_for_heap` for that key, checked by every call's //! [`sweep`]); and, for a caller with no isolate (a component's host call, a -//! test), after [`IDLE`] without a call. Its renders go with it, and any a -//! released document left behind go when the caller next opens a document -//! ([`cache::prune`]). +//! test), after [`IDLE`] without a call. Its renders go with it, and so do +//! the clips it handed out for playback that no other document open in the +//! same storage uses ([`media::forget`]); any a released document left +//! behind go when the caller next opens a document ([`cache::prune`], +//! [`media::prune`]). use std::cell::{Cell, RefCell}; use std::collections::HashMap; @@ -56,6 +58,7 @@ use crate::cache; use crate::codes::{fail, invalid, too_large, Code}; use crate::gate::{self, HeldCache, Plan}; use crate::guard::Format; +use crate::media; use crate::Ctx; /// The most documents one caller (app and storage scope) keeps open. @@ -120,6 +123,9 @@ pub(crate) struct OpenDoc { /// What the session holds, as the door's ceilings count it, kept per /// slide between calls ([`gate`]). pub held: HeldCache, + /// The clips `deck.media` handed out for this document: storage-relative + /// files in the media cache ([`media`]). + pub clips: Vec, } impl OpenDoc { @@ -184,10 +190,17 @@ impl OpenDoc { matches!(&self.binding, Binding::Isolate { heap, .. } if closed.contains(heap)) } - /// The document goes: its engine session, and its renders in the - /// service's cache. + /// The document goes: its engine session, its renders in the service's + /// cache, and the clips it handed out that no document still open in the + /// same storage uses. fn release(self) { - cache::forget(&Area::new(self.owner.clone(), None, false), &self.handle); + let area = Area::new(self.owner.clone(), None, false); + cache::forget(&area, &self.handle); + if !self.clips.is_empty() { + if let Some(used) = clips_in_use(&self.owner) { + media::forget(&area, &self.clips, &used); + } + } } } @@ -246,6 +259,13 @@ pub(crate) fn isolate_closed(heap: usize) { } } +/// The clips the documents open in the storage `owner` use, whichever app +/// opened them; `None` while a call holds the table (the clips then stay, +/// within the media cache's bounds, until a later open prunes them). +fn clips_in_use(owner: &Path) -> Option> { + DOCS.with(|docs| docs.try_borrow().ok().map(|docs| docs.open.iter().filter(|d| d.owner == owner).flat_map(|d| d.clips.iter().cloned()).collect())) +} + /// Whose documents a call reaches: its area's root as the file system /// spells it (two spellings of one folder are one scope). pub(crate) fn owner(area: &Area) -> PathBuf { @@ -365,8 +385,8 @@ fn room_for_one(cx: &Ctx, owner: &Path) -> Result<(), String> { }) } -/// Add `doc` to the table, and clear the renders of documents no longer -/// open in its storage. +/// Add `doc` to the table, and clear the renders and clips of documents no +/// longer open in its storage. fn admit(doc: OpenDoc, area: &Area) { let (app, owner) = (doc.app.clone(), doc.owner.clone()); let open: Vec = DOCS.with(|docs| { @@ -375,6 +395,9 @@ fn admit(doc: OpenDoc, area: &Area) { docs.open.iter().filter(|d| d.app == app && d.owner == owner).map(|d| d.handle.clone()).collect() }); cache::prune(area, &open); + if let Some(used) = clips_in_use(&owner) { + media::prune(area, &used); + } } /// A fresh open document around `session` (which holds its deck). @@ -392,6 +415,7 @@ fn new_doc(cx: &Ctx, owner: PathBuf, session: Session, file: Option) -> deck_rev: 0, renders: HashMap::new(), held: HeldCache::default(), + clips: Vec::new(), }; // The service keeps the undo steps itself (crate::gate). doc.st_mut()?.history.limit = usize::MAX; diff --git a/apps/deck/host-service/src/lib.rs b/apps/deck/host-service/src/lib.rs index 3267cdcc..fe7d3bd6 100644 --- a/apps/deck/host-service/src/lib.rs +++ b/apps/deck/host-service/src/lib.rs @@ -11,11 +11,13 @@ //! name as `doc`; `close`, `state`, `save`, `slides`, `slide`, `hit`, //! `view`, `layouts`, `text`, `format_state`, `slides_op`, `notes`, //! `transition`, `insert`, `set_text`, `format`, `shape`, `picture`, -//! `table`, `chart`, `design`, `undo`, `redo`, `show` and `export` work on -//! it, and `themes`, `fonts`, `shapes` and `cover` need none. Who holds an -//! open document, on which thread, and what releases it: [`docs`]. How a -//! method's engine commands reach it, through the same door as `deck.run`, -//! as one undo step: [`gate`]. +//! `table`, `chart`, `design`, `undo`, `redo`, `show`, `export` and +//! `media` work on it, and `themes`, `fonts`, `shapes` and `cover` need +//! none. Who holds an open document, on which thread, and what releases +//! it: [`docs`]. How a method's engine commands reach it, through the same +//! door as `deck.run`, as one undo step: [`gate`]. `media` runs no engine +//! command: it writes a clip's bytes, as the deck stores them, for the +//! app's player ([`media`]). //! //! The service does all file I/O itself and feeds the engine bytes, so the //! engine never touches a path, and a link written inside a presentation @@ -70,6 +72,7 @@ mod covers; mod docs; mod gate; mod guard; +mod media; mod objects; mod pictures; mod reading; @@ -515,6 +518,7 @@ fn dispatch_in(method: &str, args: &Json, cx: &Ctx) -> Result { "redo" => slides::history(args, cx, true), "show" => reading::show(args, cx), "export" => writing::export(args, cx), + "media" => media::media(args, cx), other => Err(format!("deck.{other} is not a method of the deck service")), } } diff --git a/apps/deck/host-service/src/media.rs b/apps/deck/host-service/src/media.rs new file mode 100644 index 00000000..2c222d73 --- /dev/null +++ b/apps/deck/host-service/src/media.rs @@ -0,0 +1,325 @@ +//! Playback (SERVICE.md "Playback", maintainer approved 13 Oct 2026): +//! `deck.media {doc, slide, id}` hands Quick Deck a video or audio clip of +//! an open document as a file in its storage, which the app plays with the +//! runtime's `Video` widget (the operating system's own player). +//! +//! **Bytes only.** The clip's bytes come from the open document's model, +//! the item the shape's `MediaClip` names (`Presentation::media`), exactly +//! as the deck stores them, and are written as they are. No engine command +//! runs and no engine function is called: nothing here decodes, probes or +//! transcodes a clip. deckcraft's decoders (`deckcraft-media`) are reached +//! only through `media.info`, `media.posterFrame`, `insert.audio` and +//! `insert.video`, which stay held (#448). `kind`, `autoplay`, `loop` and +//! `poster` are the model's own fields, and `ext` comes from the deck's own +//! name for the clip or its declared content type ([`ext_of`]), never from +//! the bytes. +//! +//! **The media cache**, beside the render cache ([`crate::cache`]): each +//! clip at `.cache/media/.`, at most +//! [`MAX_BYTES`] (32 MiB) and [`MAX_FILES`] clips per storage. When it needs +//! room, the clip used longest ago goes first (written, or handed out again +//! by a later call). A clip already there with the same bytes is handed out +//! again, not written twice. A document's clips go when it closes or is +//! released, with its renders ([`forget`]), unless another document open in +//! the same storage uses them, and any a released document left behind go +//! when a document next opens there ([`prune`]). If a clip doesn't fit what +//! is left of the storage, the render cache and then the media cache are +//! cleared before the write; a clip that still doesn't fit is +//! `storage_full:`, and so is one larger than the whole media cache, at +//! once, with nothing cleared, since even an empty cache can't hold it. +//! +//! **Writes** are staged inside the storage and moved into place by a link +//! that refuses anything already there (`octosense_engine_area::Stage`), so +//! a write only ever creates a file. The folders on the way must be plain: a +//! link there is refused, and only regular files directly in `.cache/media` +//! are counted, handed out or removed. Whatever else sits at a clip's name +//! (a link, or a file that doesn't hold the clip's bytes) is a stale cache +//! entry and goes as an evicted clip would, before the clip is written anew. + +use std::io::Read; +use std::path::{Path, PathBuf}; +use std::time::SystemTime; + +use deckcraft_model::{MediaClip, MediaItem, Presentation, ShapeId, ShapeKind}; +use octosense_engine_area::Area; +use serde_json::{json, Value as Json}; +use sha2::{Digest, Sha256}; + +use crate::args; +use crate::cache; +use crate::codes::{self, fail, invalid, Code}; +use crate::docs; +use crate::Ctx; + +/// The media cache's folder, relative to the storage. +pub(crate) const DIR: &str = ".cache/media"; +/// The most bytes of clips one storage keeps. +pub(crate) const MAX_BYTES: u64 = 32 << 20; +/// The most clips one storage keeps. +pub(crate) const MAX_FILES: usize = 8; +/// The extensions a clip is written with (SERVICE.md "Playback"). +pub(crate) const EXTS: [&str; 7] = ["mp4", "m4v", "mov", "mp3", "m4a", "aac", "wav"]; +/// `storage_full:`'s words for a clip (SERVICE.md "Playback"). +pub(crate) const FULL: &str = "There isn't room in Quick Deck's storage to play this clip."; + +/// `deck.media {doc, slide, id}`: `{path, kind, ext, bytes, autoplay, loop, +/// poster}`. +pub(crate) fn media(a: &Json, cx: &Ctx) -> Result { + const M: &str = "deck.media"; + args::only(a, &["doc", "slide", "id"], M)?; + let slide = args::id(a, "slide", M)?; + let id = args::id(a, "id", M)?; + docs::with_doc(a, cx, M, |doc| { + let i = doc.slide_index(slide)?; + let deck = doc.deck()?; + let clip = match deck.slides[i].shape(ShapeId(id)).map(|sh| &sh.kind) { + Some(ShapeKind::Media(clip)) => clip, + Some(_) => return Err(fail(Code::UnknownId, format!("shape {id} on slide {slide} is not a video or audio clip"))), + None => return Err(fail(Code::UnknownId, format!("slide {slide} has no shape {id} (an undo may have removed it)"))), + }; + let item = clip_bytes(&deck, clip)?; + let ext = ext_of(item).ok_or_else(|| { + fail(Code::Unsupported, format!("this clip is not a kind Quick Deck plays ({}), by its name or by its declared type", EXTS.join(", "))) + })?; + let rel = write(cx.area, &item.data, ext, M)?; + if !doc.clips.contains(&rel) { + doc.clips.push(rel.clone()); + } + Ok(json!({ + "path": rel, + "kind": if clip.video { "video" } else { "audio" }, + "ext": ext, + "bytes": item.data.len(), + "autoplay": clip.autoplay, + "loop": clip.loop_play, + "poster": clip.poster.and_then(|p| deck.media(p)).is_some_and(|p| !p.data.is_empty()), + })) + }) +} + +/// The deck's item holding `clip`'s bytes, or `unsupported:` when the deck +/// holds none: a clip linked to a file elsewhere, which the service never +/// follows, or one whose item is missing or empty. +fn clip_bytes<'a>(deck: &'a Presentation, clip: &MediaClip) -> Result<&'a MediaItem, String> { + match deck.media(clip.media) { + Some(item) if !item.data.is_empty() => Ok(item), + Some(item) if item.link.is_some() => Err(fail(Code::Unsupported, "this clip is linked to a file outside the deck, which Quick Deck doesn't open")), + _ => Err(fail(Code::Unsupported, "the deck holds no bytes for this clip")), + } +} + +/// The extension a clip is written with: the deck's own for it (its item's +/// name's), when that is one of [`EXTS`], else the one its declared content +/// type names; `None` for neither. Never read from the bytes. +pub(crate) fn ext_of(item: &MediaItem) -> Option<&'static str> { + let own = Path::new(&item.name).extension().and_then(|e| e.to_str()).unwrap_or("").to_ascii_lowercase(); + if let Some(ext) = EXTS.iter().copied().find(|e| *e == own) { + return Some(ext); + } + let declared = item.content_type.split(';').next().unwrap_or("").trim().to_ascii_lowercase(); + Some(match declared.as_str() { + "video/mp4" => "mp4", + "video/x-m4v" => "m4v", + "video/quicktime" => "mov", + "audio/mpeg" | "audio/mp3" | "audio/x-mp3" | "audio/mpeg3" | "audio/x-mpeg-3" => "mp3", + "audio/mp4" | "audio/m4a" | "audio/x-m4a" => "m4a", + "audio/aac" | "audio/x-aac" | "audio/aacp" => "aac", + "audio/wav" | "audio/wave" | "audio/x-wav" | "audio/vnd.wave" => "wav", + _ => return None, + }) +} + +/// `storage_full:`, in the contract's words. +fn full() -> String { + fail(Code::StorageFull, FULL) +} + +/// Write `bytes` to the media cache as `.`, or find them there +/// already: the storage-relative path. +fn write(area: &Area, bytes: &[u8], ext: &str, m: &str) -> Result { + let rel = format!("{DIR}/{:x}.{ext}", Sha256::digest(bytes)); + let target = crate::inside(area, &rel, "path", m)?; + let dir = root(area)?; + if dir.is_some() && reuse(&target, bytes) { + return Ok(rel); + } + let len = bytes.len() as u64; + if len > MAX_BYTES { + return Err(full()); + } + if let Some(dir) = &dir { + match std::fs::symlink_metadata(&target) { + Ok(meta) if meta.is_dir() => return Err(invalid(format!("{m}: `{rel}` is a folder in the media cache"))), + Ok(meta) => { + std::fs::remove_file(&target).map_err(|e| codes::io(m, &e))?; + if meta.is_file() { + area.credit(meta.len()); + } + } + Err(_) => {} + } + area.credit(evict(dir, &target, len)); + } + // Not room enough in the storage: the render cache goes first, then the + // media cache. + cache::make_room(area, len); + if area.room().is_some_and(|room| len > room) { + area.credit(clear(area)); + } + let create = Area::new(area.root.clone(), area.room(), false); + let stage = create.stage().map_err(codes::area)?; + let staged = stage.path("clip"); + std::fs::write(&staged, bytes).map_err(|e| match e.kind() { + std::io::ErrorKind::StorageFull | std::io::ErrorKind::QuotaExceeded => full(), + _ => codes::io(m, &e), + })?; + stage.commit(&[(staged, target)]).map_err(|e| if octosense_engine_area::no_room(&e) || e.contains("No space left") { full() } else { codes::area(e) })?; + Ok(rel) +} + +/// Whether `path` is a plain folder (`Ok(false)`: not there); a link or a +/// file in the way is refused. +fn plain_dir(path: &Path, shown: &str) -> Result { + match std::fs::symlink_metadata(path) { + Ok(meta) if meta.is_dir() => Ok(true), + Ok(_) => Err(invalid(format!("the media cache folder `{shown}` is not a plain folder"))), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(false), + Err(e) => Err(invalid(format!("the media cache folder `{shown}`: {e}"))), + } +} + +/// The media cache's folder in `area`, when it is there and plain all the +/// way. +fn root(area: &Area) -> Result, String> { + let dot = area.root.join(".cache"); + if !plain_dir(&dot, ".cache")? { + return Ok(None); + } + let dir = dot.join("media"); + Ok(plain_dir(&dir, DIR)?.then_some(dir)) +} + +/// One clip in the media cache. +struct Kept { + path: PathBuf, + len: u64, + /// When it was written, or last handed out again ([`reuse`]). + used: SystemTime, +} + +/// The regular files directly in the media cache's folder `dir`. Links and +/// anything else are neither counted nor followed. +fn kept(dir: &Path) -> Vec { + let mut out = Vec::new(); + for file in std::fs::read_dir(dir).into_iter().flatten().flatten() { + let Ok(meta) = std::fs::symlink_metadata(file.path()) else { continue }; + if meta.is_file() { + out.push(Kept { path: file.path(), len: meta.len(), used: meta.modified().unwrap_or(SystemTime::UNIX_EPOCH) }); + } + } + out +} + +/// Whether the entry at `path` is a regular file, never a link, holding +/// exactly `bytes`. If it is, it counts as just used, so it goes last when +/// the cache needs room. +fn reuse(path: &Path, bytes: &[u8]) -> bool { + let Ok(meta) = std::fs::symlink_metadata(path) else { return false }; + if !meta.is_file() || meta.len() != bytes.len() as u64 { + return false; + } + let Ok(mut file) = std::fs::File::open(path) else { return false }; + // The file opened is the one looked at, not a link raced into its place. + if !same_file(&meta, &file) { + return false; + } + let mut buf = vec![0u8; 1 << 16]; + let mut at = 0usize; + loop { + match file.read(&mut buf) { + Ok(0) => break, + Ok(n) => { + if bytes.get(at..at + n) != Some(&buf[..n]) { + return false; + } + at += n; + } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} + Err(_) => return false, + } + } + if at != bytes.len() { + return false; + } + let _ = file.set_modified(SystemTime::now()); + true +} + +#[cfg(unix)] +fn same_file(looked: &std::fs::Metadata, file: &std::fs::File) -> bool { + use std::os::unix::fs::MetadataExt; + file.metadata().is_ok_and(|m| m.dev() == looked.dev() && m.ino() == looked.ino()) +} + +#[cfg(not(unix))] +fn same_file(_: &std::fs::Metadata, file: &std::fs::File) -> bool { + file.metadata().is_ok_and(|m| m.is_file()) +} + +/// Make room in the media cache's folder `dir` for a clip of `incoming` +/// bytes at `keep` (which does not count): the clips used longest ago go +/// until the cache with it holds at most [`MAX_BYTES`] and [`MAX_FILES`]. +/// The bytes it freed. +fn evict(dir: &Path, keep: &Path, incoming: u64) -> u64 { + let mut all: Vec = kept(dir).into_iter().filter(|k| k.path != keep).collect(); + all.sort_by_key(|k| k.used); + let mut bytes: u64 = all.iter().map(|k| k.len).sum(); + let mut files = all.len(); + let mut freed = 0; + for old in &all { + if bytes.saturating_add(incoming) <= MAX_BYTES && files < MAX_FILES { + break; + } + if std::fs::remove_file(&old.path).is_ok() { + bytes = bytes.saturating_sub(old.len); + files -= 1; + freed += old.len; + } + } + freed +} + +/// Clear the media cache in `area`: the bytes it held. +fn clear(area: &Area) -> u64 { + let Ok(Some(dir)) = root(area) else { return 0 }; + kept(&dir).iter().filter(|k| std::fs::remove_file(&k.path).is_ok()).map(|k| k.len).sum() +} + +/// A clip's storage-relative path, as [`media`] answers it and documents +/// keep it. +fn rel_of(path: &Path) -> Option { + path.file_name().and_then(|n| n.to_str()).map(|n| format!("{DIR}/{n}")) +} + +/// Remove the clips a document that went used (`gone`), but not those +/// another document open in the same storage uses (`used`). +pub(crate) fn forget(area: &Area, gone: &[String], used: &[String]) { + let Ok(Some(dir)) = root(area) else { return }; + for k in kept(&dir) { + if rel_of(&k.path).is_some_and(|rel| gone.contains(&rel) && !used.contains(&rel)) { + let _ = std::fs::remove_file(&k.path); + } + } +} + +/// Remove every clip in `area`'s media cache that no document open there +/// uses (`used`): clips of documents released without a close, by an +/// earlier run. +pub(crate) fn prune(area: &Area, used: &[String]) { + let Ok(Some(dir)) = root(area) else { return }; + for k in kept(&dir) { + if !rel_of(&k.path).is_some_and(|rel| used.contains(&rel)) { + let _ = std::fs::remove_file(&k.path); + } + } +} diff --git a/apps/deck/host-service/src/reading.rs b/apps/deck/host-service/src/reading.rs index 21b95929..7bec7cc7 100644 --- a/apps/deck/host-service/src/reading.rs +++ b/apps/deck/host-service/src/reading.rs @@ -177,7 +177,7 @@ fn shape_json(ctx: Option<&Ctx>, summary: &Json, sh: &Shape) -> Json { "box": summary["box"], "rotation": summary["rotation"], }); - for (from, to) in [("placeholder", "placeholder"), ("preset", "preset"), ("text", "text"), ("fill", "fill"), ("table", "table"), ("chart", "chart"), ("altText", "alt_text")] { + for (from, to) in [("placeholder", "placeholder"), ("preset", "preset"), ("text", "text"), ("fill", "fill"), ("table", "table"), ("chart", "chart"), ("altText", "alt_text"), ("media", "media")] { if !summary[from].is_null() { out[to] = summary[from].clone(); } From d50cd4231b0b6ca077fbc5aa512ab9f2305e0c36 Mon Sep 17 00:00:00 2001 From: ymote <151983+ymote@users.noreply.github.com> Date: Tue, 13 Oct 2026 11:29:57 +0800 Subject: [PATCH 3/3] SERVICE.md: playback as built (#515) The coordinator accepted the six choices the Playback section left open, as #515 built them, and asked for them here with three facts the app needs: - deck.slide's media.id is the clip's media item id; deck.media takes the shape's own id. - A clip larger than the whole 32 MiB media cache is storage_full: at once, with nothing cleared; one the deck holds no bytes for (linked, missing or empty) is unsupported:. - Oldest first means used longest ago (a reused clip's modification time is set to now); a stale entry at a clip's name goes and the clip is written anew (a folder there is invalid:); clips no open document uses go when a document next opens; a clip inside a group is found too. - deckcraft at d0e57d7e reads no autoplay or loop from a .pptx, and its .pptx writer gives a clip without a poster a placeholder picture. Co-Authored-By: Claude Opus 5.5 --- apps/quickdeck/design/SERVICE.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/apps/quickdeck/design/SERVICE.md b/apps/quickdeck/design/SERVICE.md index 0430d3ed..b1d63c94 100644 --- a/apps/quickdeck/design/SERVICE.md +++ b/apps/quickdeck/design/SERVICE.md @@ -501,3 +501,30 @@ held (#448). a tap plays or pauses; moving on or Exit stops). Where the `Video` widget has no player (today: Linux and Windows desktops), the app keeps the poster and says "Playback isn't available on this system". + +As built in #515 (coordinator, 13 Oct 2026); where these differ from the +bullets above, they win: + +- `deck.slide`'s `media.id` is the clip's media item id in the deck (two + shapes that play one item share it); `deck.media` takes the shape's own + `id`. +- A clip larger than the whole media cache (32 MiB) is `storage_full:` at + once, with nothing cleared. +- A clip the deck holds no bytes for (linked to a file elsewhere, which the + service never follows, or with a missing or empty item) is + `unsupported:`. +- "Oldest first" means used longest ago: a clip handed out again counts as + just used (its file's modification time is set to now), so a slide's + other clip can't push it out before it plays. +- Anything else at a clip's file name (a file cut short, or a link) is a + stale entry: it goes, and the clip is written anew, create-only. A folder + there is `invalid:`. +- Clips no open document uses (an earlier run's, or a released document's) + go when a document next opens in that storage. +- `deck.media` finds a clip inside a group too; `deck.slide` lists only + top-level shapes. +- From a `.pptx`: deckcraft at `d0e57d7e` doesn't read a clip's autoplay or + loop from a package, so such a clip answers `autoplay: false` and + `loop: false` (a native deck keeps both). And deckcraft's `.pptx` writer + gives a clip without a poster a placeholder picture, so after a round + trip through a `.pptx` every clip answers `poster: true`.