Skip to content

feat(droid-control): render showcases with fframes instead of Remotion - #55

Open
factory-droid-dev[bot] wants to merge 4 commits into
masterfrom
ain3sh/droid-control-ffframes
Open

factory-droid-dev[bot] wants to merge 4 commits into
masterfrom
ain3sh/droid-control-ffframes

Conversation

@factory-droid-dev

@factory-droid-dev factory-droid-dev Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Description

What

This PR moves the droid-control showcase renderer from Remotion (React and headless Chrome) to fframes (Rust and SVG), with the same composition, props contract and render-showcase.sh flags. Existing props JSON and render commands work unchanged. On the 761-frame 1080p demo the render takes 15.6s on 4 threads, and its output matches stills compared frame by frame against Remotion renders of the same props.

Area Before After
Engine remotion/ (React, Node, npm deps, Chrome) fframes/ crate, droid-showcase binary
Pipeline owner 273 lines of bash and inline Python in render-showcase.sh The binary: props validation and defaults, clip staging, .cast via agg, ffprobe, parallel render, libx264 encode, cleanup
render-showcase.sh Whole pipeline 32 lines: cargo build (a no-op once built) and exec, same flags
Fonts Never loaded, so output used whatever system sans-serif existed Geist and Geist Mono embedded; system fonts only back missing glyphs (CJK etc.)
Cancellation A signal to the script waited until npx exited SIGINT/SIGTERM to the PID or group removes the work dir and exits 130/143
spotlight effect Its CSS mask failed in headless Chrome, so renders never showed the dim Dims the frame outside the spotlit region
Outro wordmark ASCII D drawn as a square box (reads as O); R had a one-character leg D and R redrawn with distinct shapes; O stays square
Prereqs Node >= 18, npm install, Chrome Rust stable, clang/libclang, libx264 (first build only), ffmpeg
License Remotion company license for teams over 3 MIT (fframes), MPL-2.0 (svgr), OFL (Geist)

Where the 17k lines are. Most of the diff is the deleted Remotion app and two lockfiles. 23 files carry behavior.

Share Lines Files What it is What to do with it
32.2% 5,526 1 Deleted remotion/package-lock.json Skip.
19.4% 3,326 34 Deleted Remotion source, config and its one test Skip, or open a component next to its Rust port.
16.2% 2,781 1 New fframes/Cargo.lock Skip.
28.0% 4,795 23 Core logic. The fframes crate and the wrapper script This is the review. Breakdown below.
2.2% 373 1 tests/test_render_showcase.py, rewritten against the real binary Read for the invariants listed under Verification.
2.0% 345 15 CI workflow, docs and skill atoms, diagrams, Geist license, config Skim.

Also binary: seven Geist font files added and the background image moved into fframes/media/.

The 28% that carries behavior, grouped by the change it makes:

  1. Render pipeline (1,365 lines): main.rs, props.rs, clips.rs, stage.rs, timing.rs, render.rs, Cargo.toml, lib.rs, and render-showcase.sh (273 to 32 lines). The binary validates props, stages clips, probes durations, renders frames on every core and encodes them.
  2. Composition layers (1,889 lines): showcase.rs, scenery.rs, title.rs, content.rs, overlays.rs, transition.rs, outro.rs, code.rs. Each is the port of one or more Remotion components.
  3. Rendering primitives (1,541 lines): svg.rs, composite.rs, raster.rs, motion.rs, theme.rs, fonts.rs. These hold SVG helpers, the Rust compositor, easing, palettes and presets, and font loading.

Why

The migration off Remotion was requested with no loss of functionality or tooling. Rendering no longer needs a browser or a Node toolchain, the output looks the same on every machine because the fonts ship with it, and the Remotion team license no longer applies.

How

showcase.rs is the single fframes Video. It composes the layer modules from typed props (props.rs: defaults and validation with field-named errors). Effects with no SVG equivalent are composited in Rust instead of rasterized as SVG filters: the colour grade, layer blending, the frosted backdrop-filter card, and shadows baked once with a box blur. render.rs renders blocks of frames on every core, converts each frame to yuv420p in its worker, and pipes frames in order to ffmpeg.

Related Issue

None. Requested in Slack by @factory-ain3sh.

Reviewer Guide

Diff shape: see the composition table above. 4,795 lines in 23 files are the review; the rest is deletions, lockfiles and docs.
Three passes:

  1. Commit 1 (74280b5, the port): props.rs against the deleted remotion/src/lib/schema.ts, then main.rs, stage.rs and render.rs for the pipeline. Check that every props field keeps its default and validation.
  2. Commits 1 and 2 (b42fb6d, perf): composite.rs and raster.rs, then how content.rs and svg.rs call them. Check that cached chrome and clip frames are keyed on everything that changes them.
  3. The remaining layer modules: read content.rs and transition.rs in full, spot-check the rest against their Remotion components.

Review depth: Deep. This is a full renderer rewrite.
Read order:

  1. fframes/src/showcase.rs: how the layers compose per frame.
  2. fframes/src/props.rs: the props contract.
  3. fframes/src/main.rs, stage.rs, render.rs: the pipeline from CLI to encoded mp4.
  4. fframes/src/composite.rs, raster.rs: the Rust compositor and blur.
  5. Tests: tests/test_render_showcase.py and the unit tests in each module.

Deliberate behavior changes:

  • spotlight now dims the frame (see the table above).
  • Props clips and clipDuration are ignored if present. Clips come only from the positional arguments and the binary probes their durations, as before.
  • Remotion components the composition never rendered (GlitchTitle, ActiveInputPulse, DynamicShadowParallax, GlowLines, WindowReflection) are not ported. GlitchTitle was documented as needing a custom composition.
  • The request said "ffframes"; no project by that name exists, so this uses fframes.

Risk & Impact

  • New first-build prerequisites. Machines that render showcases now need Rust, clang/libclang and libx264 for the first cargo build. Without them render-showcase.sh fails at the build step before staging anything. README, the showcase atom, and CI list the new prereqs.
  • Visual drift from Remotion. Text now uses embedded Geist everywhere, and CSS effects are emulated, so frames are close to but not pixel-identical with Remotion's. Large differences would show up in the frame-by-frame stills, which covered every transition, both layouts, overlays and the outro.
  • Stale cache in the compositor. Window chrome and clip frames are reused while the window's screen transform is unchanged. If a future layer changes chrome without changing that transform, it would render a stale frame. The PSNR comparison against the uncached render (below) shows no such case today.
  • Reverting this PR restores Remotion in full; nothing persistent changes.

Performance Evidence

Workload: the 761-frame 1920x1080 demo: one 17.85s .cast clip, factory preset, compact fidelity, with keystrokes, sections, a callout, zoom, spotlight, a code annotation and the light-leak transition.
Conditions: 2 cores / 4 threads (Xeon), wall time for the whole render-showcase.sh run including staging and encode.

Step Wall time
Port with every layer rasterized as SVG 3m17s
Grade and blending composited in Rust with runtime SIMD dispatch 20.1s
yuv420p conversion in render workers 17.6s
Shadows baked once with a Rust box blur, per-window chrome and clip caching 17.1s
Clips staged as lossless ultrafast x264 15.6s (15.65s re-measured @ 4bb3857)

Per-frame render time on one thread fell from about 872ms to about 36ms (about 19ms on frames where only the clip changes). Most of the remaining time is the final libx264 encode (about 19s of CPU spread across cores).

Quality: PSNR against output from before the perf commit is 49 to 56 dB on stills and 53 to 59 dB per plane on the full video, the level of encoder noise.

Rerun: scripts/render-showcase.sh --props demo.json --output /tmp/demo.mp4 demo.cast.

Verification

Behavior verified @ 4bb3857: the demo renders 761 yuv420p frames at 1920x1080 in 15.65s.
Behavior verified @ 0344b9a: the outro still shows the redrawn D and R next to a square O.
Behavior verified @ 74280b5: stills compared frame by frame with Remotion renders of the same props: title, content, overlays, every transitionStyle at its midpoint, outro, and the side-by-side macOS layout.
Regression coverage:

  • tests/test_render_showcase.py (8 tests, real binary): same-basename clip staging and work-dir cleanup; fidelity mapping to size, CRF and preset, including the --fidelity override; refused inputs fail before staging; concurrent renders keep separate work dirs; SIGTERM cleanup; .cast timeline preserved through agg; and real encodes (pixel format, BT.709 tags, exact frame counts, first and last clip-frame colours, content shorter than a transition).
  • Rust unit tests (38): timing, easing, props defaults and errors, clip probing, fonts, code highlighting, staging, compositing (composite::tests::blur_matches_svgr_gaussian_blur pins the Rust blur to svgr's), and yuv conversion.

Not tested: the five unported components (never rendered by the composition).
Standard validators: cargo fmt --check, cargo clippy --all-targets --release -- -D warnings, cargo test --release and the Python suite pass locally and in CI.

…emotion

The showcase composition now renders through fframes, a Rust SVG video
framework, instead of Remotion + headless Chrome. Every layer of the
Showcase composition is ported 1:1 (presets, palettes, title card, window
chrome and layouts, zoom/spotlight/callout effects, keystrokes, sections,
code annotations, all five transition styles, noise/grade/watermark, Droid
outro), checked frame by frame against Remotion stills.

The droid-showcase binary owns what render-showcase.sh used to do in bash
and Python: props validation and defaults, clip staging, .cast conversion
via agg, probing, rendering on every core, the H.264 encode, and work-dir
cleanup on exit or SIGINT/SIGTERM. render-showcase.sh keeps its CLI and
becomes a thin build-and-exec wrapper. The resolved render plan is printed
as one `showcase plan: {...}` stderr line, which the helper tests assert on
against the real binary instead of an npx shim.

Rendering no longer needs Node, npm deps or Chrome, embeds Geist so output
is identical across machines, and drops the Remotion company license
requirement.

Co-authored-by: factory-ain3sh <ainesh@factory.ai>
factory-droid Bot and others added 2 commits October 10, 2026 15:24
…parallel

Rendering the 25 s demo on a 2-core (4-thread) box goes from 3m17s to
15.6s wall, with the same pixels within 1-2 levels (PSNR vs the previous
stills 49-56 dB, vs the previous video 53-59 dB per plane).

- Frames are composited in Rust on 8-bit premultiplied pixels: the
  background, grade (grain, tint, vignette), windows, shadows and clip
  are blended in fixed point, compiled for every SIMD level with
  runtime dispatch (multiversion). Only vector chrome and overlays go
  through svgr.
- Static layers are baked once (shadows, blurred with the Rust box blur
  that matches svgr's) and resampled once to the pixels they cover.
  Window chrome and clip frames are reused while the window holds still.
- Frosted backdrops downscale and blur just the region a panel shows.
- Render workers convert frames to yuv420p themselves, so ffmpeg only
  runs x264 and the pipe carries 3 MB per frame instead of 8 MB.
- Casts are staged losslessly with the ultrafast preset: no second lossy
  encode, and staging drops from ~8 s to ~1.5 s of CPU.

Co-authored-by: factory-ain3sh <ainesh@factory.ai>
The ASCII art inherited from Remotion drew D as a square box (reading as O)
and gave R a one-character-wide leg. D now has a flat stem and a stepped
curve, O has rounded corners on both sides, and R has a closed bowl with a
full-weight diagonal leg. Letter gaps are uniform and the block no longer
carries leading spaces that pushed it off-centre.

Co-authored-by: factory-ain3sh <ainesh@factory.ai>
@factory-droid-dev factory-droid-dev Bot changed the title droid-control: render showcase videos with fframes instead of Remotion feat(droid-control): render showcases with fframes instead of Remotion Oct 10, 2026
The square O reads heavier next to the redrawn D and R, which still keep
their distinct shapes.

Co-authored-by: factory-ain3sh <ainesh@factory.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant