Repository navigation
Quick Deck: play a deck's video and audio - #518
Merged
Merged
Conversation
A chosen video or audio object shows a play control on the canvas (and Play, Pause and Stop in the inspector's new clip section): Play calls deck.media for the clip's file in the storage, then overlays the runtime's Video widget on the clip's box at the canvas zoom, with that file as its storage source (VideoDataSource.Filesystem). Pause, play on, stop and mute; while it plays the clip's frame has no handles and changes to it wait; another slide, choice, tab or view stops it. A sound gets a small speaker control. In Present, a clip set to play by itself starts when its slide shows (looping if set), a tap on a clip plays or pauses it, and another slide or Exit stops it. The widget calls no script, so the app asks it four times a second (state(), error(), the position and the duration) and moves the clip on with clip_step, a pure function. It is made with autoplay off and started with begin_playback() once it exists (one made with autoplay on stayed "preparing"), hidden until it plays so the poster shows, and its end found by a stalled position (the Mac's player reports none). States: preparing; storage_full: and unsupported: in plain words; "Playback isn't available on this system" when the widget can't play, or "This clip couldn't be played." once a clip has played here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dev fixture answers deck.media: make_fixture.py --media adds Launch Film (a video set to play by itself with the repository's synthetic sample clip, now dev-fixture/media/clip.mp4; a linked sound; a clip whose file isn't there; one too big for the storage). tests/ui.py gains a playback run in card-host (the clip plays over its box: two grabs a second apart must differ there; pause, stop, a slide change, the refusals, Present's autoplay and taps, and a fresh session's "Playback isn't available") and one in the shell with the real engine on tests/media's decks, made by tests/make_media_decks.py with deckcraft-cli at d0e57d7e (a Mac must play each video, Present's taps must pause and play it on the same slide, Linux must say it can't; deck.media's cache is checked against the clip's own bytes; after every step the shell's log must show no client removed and no module fault). Model tests cover which shapes are clips, the refusals' and failures' words, clip_step, the time and the control's place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #516 put the shell's bar after the scene, the shell's self.ui.widget(cx, ids!(bar)) found Quick Deck's toolbar, also called `bar`, first, and panicked applying its chrome to it with the shell's script VM (#519 fixes the shell). The toolbar region is now `qd_toolbar`, and a model test reads the names the shell looks its own widgets up by in its window (lib.rs, desktop_app.rs, mobile_app.rs), plus `bar`, and fails if Quick Deck names a widget after any of them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
How a clip plays and where (the runtime's backends: AVFoundation on macOS, GStreamer on a Linux desktop when its plugins are there, Media Foundation on Windows), the states and words, what closing the app does, the runs and the test decks, and what is verified: card-host on a Mac and the real engine in hidden shells on macOS and Linux, light and dark, on main ffd5e48, with tests/evidence/playback-20261013. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ymote
force-pushed
the
feat/quickdeck-playback
branch
from
October 13, 2026 06:04
d953aa3 to
d46fccb
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ymote
marked this pull request as ready for review
October 16, 2026 15:59
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Local CI passed on 2e9f7aa ( GitHub would run for this PR: desktop.yml, phone.yml, apps.yml. Merged on this local pass (macOS runner queue); GitHub CI runs on the merge commit on main.
Skipped (these did NOT pass; (!) marks a skip that blocks tools/ci-local-merge.sh):
Notes:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The maintainer approved playback on 13 Oct 2026 ("Add playback"). #511 lets decks with audio and video open (posters only), and #515 added
deck.media, which hands the app a clip's bytes as a file in its storage. This PR plays those clips with the runtime'sVideowidget, which uses the system's own player. The contract is main's SERVICE.md "Playback", including its "As built in #515" bullets; this PR doesn't change SERVICE.md. The branch is rebased on main ffd5e48 (#515, #516, #517).What
Editor
deck.media(with the shape's ownid, nevermedia.id), then overlays aVideoon the clip's box at the canvas zoom and plays it.Present
.pptxnever autoplay: deckcraft d0e57d7e doesn't read autoplay from a.pptx.States
storage_full:in the contract's words.unsupported:in plain words: the clip is linked to a file outside the deck, is empty, or isn't MP4, MOV, M4V, MP3, M4A, AAC or WAV.No widget called
barself.ui.widget(cx, ids!(bar))found Quick Deck's toolbar first and panicked; shell: find the bar's strip as the body's own child, not by name (fixes Quick Deck crash after #516) #519 fixes the shell.qd_toolbar.bar, and fails if Quick Deck uses any of them.Dev fixture and tests
make_fixture.py --mediaadds Launch Film, and the stand-in engine answersdeck.mediafor it. The sample clip isdev-fixture/media/clip.mp4: the repository's synthetic H.264 clip with no sound, fromcrates/video-api-smoke, nothing installed.tests/ui.pygets aplaybackrun incard-hostand one in the shell with the real engine. The shell run usestests/media/launch-film.{deckcraft,pptx}, made bytests/make_media_decks.pywith deckcraft-cli at d0e57d7e. After every step it checks that the shell removed no client and contained no module fault.clip_step, the time display, where the control sits, and the widget names.READMEs (both languages) and
tests/evidence/playback-20261013/.How the
Videowidget is usedCitations are to
.sources/makepad(makepad d653bc44 with the runtime patches on main):VideoDataSource.Filesystem {path}(widgets/src/video.rs:2573-2579, registered at:71-74), resolved inside the app's storage for a policed isolate (video.rs:1582-1590→splash_policy.rs:193-201→splash_storage.rs:246,:290). App Hub gives every admitted app its storage jail and a policy (crates/app-policy/src/splash_adapter.rs:26-37at 38635915).autoplay,is_looping,mute(video.rs:582-588),show_controls(:525).state()(:1047),error()(:1059),current_position_ms()(:1066),total_duration_ms()(:1071),seek_to()(:1077), andbegin_playback,pause_playback,stop_and_cleanup_resources,mute_playback,unmute_playbackthroughscript_control(:1888-1956). These come fromtools/runtime-patches/makepad-video-controls-lifecycle.patch.VideoActions for prepared, began, completed and reset (:1021-1036, sent at:1199-1347), but no script callbacks. The app therefore asksstate()anderror()four times a second.Event::VideoDecodingError. The widget keeps it aslast_error, drops its native player and returns to "unprepared" (:1871-1883,:1961-1981).error():platform/src/os/apple/apple_video_player.rs:106-135). With no media plugin installed, that fallback reports "playback session unavailable: no media plugin installed" (video_decode/software_video.rs:37; nothing registers one,media_plugin.rs:307).os/linux/gstreamer_sys.rs:309,linux_video_player.rs:119). On our host it fails with "Your GStreamer installation is missing a plug-in.".linux_wayland.rs:1063).windows_video_playback.rs:1), or "D3D11 device unavailable…" without D3D11 (windows.rs:1242).VideoPlaybackCompleted, so a clip whose position stops at its end counts as ended.view.rs:989,event.rs:684). The widget is made hidden and shown once it plays, so the poster stays until then.Videomade withautoplay: truestayed "preparing" incard-host(three tries, 5 to 9 s each). The likely cause: the apply asks for the player beforeon_after_newgives the widget its native id (:654-684,:1487), and the prepared event is taken only for the current id (:1199-1203). The app makes the widget with autoplay off and callsbegin_playback()once it exists.The demo's vanished window
During a visible demo of this branch at d953aa3, Quick Deck's window went away and the shell stayed up. This is a close, not a crash.
wm: removing client 1thenwm: module instance card.1 torn down; isolate SplashVmId(1) freed, and nothing else.close_client→remove_client→module_host.rsteardown).wm: contained a module panic…(module_host.rs:114) and leaves a failed tile (:1219);[E] splashlines;Videoerror logsError decoding video….binds.rs:151-152).close_requestedwith Allow), so a close also drops unsaved edits;Tests
--remote, 64 ms budget):missing,fixture,statesandplaybackall passed, light and dark, with no script error or overrun in the final run.card-host/p2-playing-pair.pngandp8-present-pair.png, plus thedvariants.playbackpassed, light then dark..pptxcopy's clip didn't start by itself, and played on a tap (12,814).deck.mediawrote.cache/media/<sha256>.mp4(the clip's own bytes), and closing the deck removed it. The sound was never played.macos/.shellandrestartpassed unchanged.playbackpassed, light then dark:deck.mediawrote the.mp4and.wav, and closing the deck removed them.qd_toolbarrename.cargo test --locked -p octosense-shell --lib quickdeck_modelpassed 35 on Linux, and 36 with--features app-hub.hub check --system-appon a stamped copy refuses only for the missinglisting.json, as for every system app.ci-localwas not run.Not verified
deck.slidelists top-level shapes only.🤖 Generated with Claude Code