Skip to content

build: optimize ffmpeg into a minimal per-platform decoder build - #23158

Merged
bolichen97 merged 1 commit into
mainfrom
build/ffmpeg-decoder
Oct 23, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
build/ffmpeg-decoder

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Ship a small, audio-only ffmpeg on every desktop platform, sized to what the app actually uses.

The desktop app bundled the full imageio-ffmpeg executable (50–90 MB per platform) only to decode voice input, and the source-install decoder store fetched the same wheel. The speech path needs a handful of audio formats.

Why it matters

Each desktop bundle carried tens of MB of ffmpeg it never runs, and the source-install store downloaded a ~30 MB wheel to extract one file.

Not a goal

  • Changing runtime ffmpeg resolution order or the digest/signature trust model.
  • Shipping a decoder for the macOS Intel backend (it stays decoderless and uses a system FFmpeg).
  • (Scope note: switching the desktop bundle and store was folded into this PR at the maintainer's request.)

What changed (motivation → approach → change)

  • Build: scripts/build-ffmpeg.sh --target builds a ~2–4 MB audio-only ffmpeg for Linux x64/arm64 (static), macOS arm64/x64 and Windows x64 (mingw), and checks its configuration and core decoders. .github/workflows/build-ffmpeg-decoder.yml builds all five, runs the Windows one, and can publish them with the source tarball and SHA256SUMS. It also runs on PRs that touch it.
  • Release: the five binaries are published as ffmpeg-decoder-7.1.1-r1, built by that workflow.
  • Pins: stt.decoder.ARTIFACTS pins each release asset by size and SHA-256, downloaded as itself (no wheel, no zip extraction). New python -m kiro_crew.stt.decoder --install-into DIR.
  • Desktop: build-desktop.sh installs the decoder through that CLI into site-packages/kirocrew_ffmpeg/bin instead of pip install imageio-ffmpeg; the resolver looks there. Mac signer set and the Windows signing exemption use the new filenames.
  • src/kiro_crew/transcribe_aws.py: the webm→ogg remux now passes -vn, so a Slack video/webm clip with a video track remuxes with the audio-only build (the old full build silently re-encoded video to theora).
  • Docs that named the old decoder now describe the new one: docs/system-specs/modules/stt-streaming.md, docs/build/desktop-app.md, docs/system-specs/modules/cli.md, docs/system-specs/modules/slack-gateway.md, docs/system-specs/modules/learn-cron-dashboard.md, docs/reference/kiro-cli/chat/voice.md, src/kiro_crew/docs/configuration.md, and the src/kiro_crew/transcribe.py module docstring. NOTICE and THIRD-PARTY-NOTICES list the bundled decoder.

Backwards compatibility

Compatible: the runtime resolves and authenticates the decoder exactly as before; only the pinned bytes and bundle path change. A store decoder fetched under an older pin is no longer resolvable by name, so when nothing else resolves, the transcode path fetches the current pin once per process (stt.decoder.migrate_retired_decoder) and removes the retired file.

Reader: src/kiro_crew/transcribe_ffmpeg.py:_open_packaged_ffmpeg_resource -- release -- now reads kirocrew_ffmpeg/bin; test_transcribe_ffmpeg.py
Reader: src/kiro_crew/stt/decoder.py:DecoderStore.ensure -- app -- fetches the release asset; test_stt_decoder.py
Reader: src/kiro_crew/stt/decoder.py:RETIRED_FILENAMES -- chat/app -- a store with an old-pin file migrates on the first transcode; TestRetiredDecoderMigration
Reader: packaging/build-desktop.sh:install_desktop_decoder -- release -- test_build_desktop_decoder_requirement.py, packaging.test.js

Tests

  • test_stt_decoder.py: direct pinned download (install, progress, no .part left, existing target kept on failure), store flows, main() exit codes.
  • test_transcribe_ffmpeg.py: pins cover exactly the shipped platforms and match the release's SHA256SUMS; resolver fixtures use the new bundle path.
  • test_build_desktop_decoder_requirement.py: every backend but macOS Intel installs via the pin table into the resolved directory; no imageio-ffmpeg left in the build.
  • test_ffmpeg_decoder_only_publish_can_write: workflow permissions.

Manual verification

  • CI built all five; verify-windows ran the .exe on Windows.
  • Linux static binary runs on CentOS 7, Debian 10, Amazon Linux 2 and AL2023.
  • macOS arm64 and x86_64 (Rosetta) and Windows x64 checked on real machines: 11 audio formats decode, remux, silencedetect, seek, signing, CJK/space paths.
  • python -m kiro_crew.stt.decoder --install-into fetched and verified the Linux asset from the release.

Related Issues

no linked issue: build-tooling change with no tracked issue

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@bolichen97

Copy link
Copy Markdown
Contributor Author

Intent: Build a small audio-only ffmpeg for every desktop platform, with a manual workflow that can publish the binaries for the desktop bundle to pin.
Not a goal: Switching the desktop bundle or the source-install store to these binaries; changing runtime ffmpeg resolution or arguments.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 22, 2026
@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 3 times, most recently from f35f3e1 to 1e6f6f7 Compare October 22, 2026 18:32
@bolichen97
bolichen97 enabled auto-merge (squash) October 22, 2026 18:35
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 4495baad3b6904382d74790bcecc8fdb49f95ecb — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. The diff changes build, packaging and decoder fetching only. It adds or changes no dashboard control, label or state, so having no screenshots leaves no evidence gap.

[UX-REVIEWED] 4495baa

Description read

[DESCRIPTION-READ] 75344baa994ac0983873e27602a9a310ccfcb406dc8fe652423ccac062fc45d3

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 4495baad3b6904382d74790bcecc8fdb49f95ecb — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. One flag on the new CLI has no caller outside tests.

Notes

  • Item 4 — one consumer, generalized: the only real caller, packaging/build-desktop.sh:204-205, runs --install-into without --platform. Only test/test_stt_decoder.py passes --platform. Subtraction: delete the --platform argument from decoder.main and always use artifact_for().

What this change ships

Inventory (8 items) — 7 justified

Intent: ship a much smaller, audio-only ffmpeg in desktop bundles and in the source-install store. This is an ADDITION; the bundle and store switch was folded in at the maintainer's request.

  1. Desktop bundles carry a ~2–4 MB audio-only decoder instead of the 50–90 MB imageio-ffmpeg — justified
  2. Source installs download the bare executable from a GitHub release, with no PyPI wheel and no zip extraction — justified
  3. A build script and workflow build the five binaries, run the Windows one, and publish the release — justified
  4. New python -m kiro_crew.stt.decoder --install-into DIR [--platform KEY] — one consumer, generalized: --platform is used only by tests
  5. A store holding an old-pin decoder fetches the current pin once per process when a transcode finds nothing, then deletes the old file — justified
  6. The webm→ogg remux drops the video track (-vn), so Slack video clips remux with the audio-only build — justified
  7. The bundled decoder moves to site-packages/kirocrew_ffmpeg/bin, and the macOS and Windows signing names follow — justified
  8. Two store error codes (decoder_wheel_unverified, decoder_member_missing) are removed — justified

[FIRST-PRINCIPLES-REVIEWED] 4495baa

Description read

[DESCRIPTION-READ] 6751644be55cab1ca589d6b8d4c89e349fe124eadb7743c42e65be786a97b7ce

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 4495baad3b6904382d74790bcecc8fdb49f95ecb — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. The audio-only build covers every demuxer, decoder, muxer and filter the runtime decode paths call, and the old store pins have a migration.

Notes

  • The fetch source moves from PyPI to GitHub release assets (_RELEASE_BASE_URL). A host that can reach PyPI but not objects.githubusercontent.com loses the store download. The error it shows should name the host it could not reach.
  • The bundled interpreter no longer has imageio_ffmpeg. dev_fleet/.../feature-demo-recording/references/deps.py:_imageio_ffmpeg falls back to telling the user to pip install imageio-ffmpeg, so nothing breaks, but the probe no longer finds it in the bundle.

[DESIGN-REVIEWED] 4495baa

Description read

[DESCRIPTION-READ] f2628ecbe35d671442c03d51a37e7e13d4e9bcebf5ee13e4c2120e905dc7cc5c

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 4495baad3b6904382d74790bcecc8fdb49f95ecb — this comment is updated in place on each push.

Review details

No findings.

Notes:

  • scripts/build-ffmpeg.sh:174: the decoder list leaves out WMA Pro/Lossless/Voice, ADPCM/GSM WAV, Speex and AC-3. The import gate still accepts files with those suffixes, so such files now fail at decode time.
  • src/kiro_crew/stt/decoder.py:121: the comment says the release carries a macos-x86_64 asset, but the workflow matrix builds none.
  • src/kiro_crew/stt/decoder.py:444: _remove_retired runs only after a fresh fetch. If a system FFmpeg resolves, or the current pin is already accepted, the retired executable stays in the store.

[OPUS-REVIEWED] 4495baa

Verdict parsed from the review's SHA-scoped output markers for commit 4495baad3b6904382d74790bcecc8fdb49f95ecb.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4495baad3b6904382d74790bcecc8fdb49f95ecb: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 22, 2026
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 4495baad3b6904382d74790bcecc8fdb49f95ecb and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4495baa

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4495baad3b6904382d74790bcecc8fdb49f95ecb: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from 1e6f6f7 to 0042cda Compare October 22, 2026 18:49
@bolichen97

Copy link
Copy Markdown
Contributor Author

span=6b1830b79398 disposition: fixed in 0042cda

Relative --out/--source-out are now made absolute against the caller cwd before the script cd s into the temp build dir, so outputs survive cleanup.
The empty TOOLCHAIN array now expands as ${TOOLCHAIN[@]+"${TOOLCHAIN[@]}"}, legal under set -u on macOS bash 3.2.
Verified: a relative-path --target linux-x86_64 build from another cwd wrote both files into ./dist.
self-added: yes
mechanism: absolute output paths; set -u safe empty-array expansion

@bolichen97

Copy link
Copy Markdown
Contributor Author

span=1207bfa4adae disposition: fixed in 0042cda

abs_path anchors relative --out and --source-out to the caller cwd before the temp-dir cd, so the workflow dist/... paths land in the checkout, not the deleted temp dir.
Verified with a relative-path --target linux-x86_64 build: binary and source tarball both present under ./dist.
self-added: yes
mechanism: absolute output paths

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 22, 2026
@bolichen97

bolichen97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

span=6b1830b79398 disposition: fixed in 09aba47

make now builds "$EXE_NAME" (ffmpeg.exe on mingw), the only program rule there.
A full --target windows-x86_64 cross-build in ubuntu:22.04 then hit two more link-time gaps, also fixed: mingw needs -fstack-protector-strong at link time for libssp, and the workflow now installs the host gcc configure requires.
Verified: the container build finishes; the .exe imports only ADVAPI32, bcrypt, KERNEL32, msvcrt, PSAPI and SHELL32. The Linux build is unchanged.
self-added: yes
mechanism: target-specific make goal; libssp link for mingw

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 2 times, most recently from 09aba47 to 1721264 Compare October 22, 2026 19:23
@bolichen97

bolichen97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Retrospective (round 2, head 09aba47 -> 5411204), measured against the frozen intent.

  • Kept abs_path for --out/--source-out: the build cds into a temp dir deleted on exit, so a relative path resolved after that cannot reach the caller.
  • Kept ${TOOLCHAIN[@]+...}: the macOS legs run /bin/bash 3.2, and TOOLCHAIN is empty on every native leg.
  • Kept make "$EXE_NAME": on mingw, ffmpeg.exe is the only program rule.
  • Kept mingw -fstack-protector-strong at link time: CFLAGS always carry it, and mingw links no libssp by default.
  • Smaller: the workflow installs host gcc only. libc6-dev is already a gcc dependency.
  • New fix: every ffmpeg ... | grep -q / | head check now captures output first, so an early-exiting reader cannot SIGPIPE the writer and fail under pipefail. Applied in build-ffmpeg.sh and verify-windows.
  • Kept the macOS -arch cross branch: CI uses native runners, but it is the path for building the Intel slice on an Apple-silicon Mac, and the manual macOS verification builds both slices on one host.

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 2 times, most recently from 5411204 to c94b5e5 Compare October 22, 2026 19:54
@bolichen97

bolichen97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

span=b79a0ae5b2e8 disposition: fixed in c94b5e5

The build matrix now sets max-parallel: 4, so test_workflow_matrix_max_parallel passes and a dispatch never starts all five legs at once.
Verified: ran all 104 test files that read .github/workflows (7199 tests); this was the only failure, and it now passes.
self-added: no
mechanism: matrix max-parallel cap

@bolichen97

bolichen97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

span=6b1830b79398 disposition: fixed in 0f655d4

The header and the final message no longer claim the gateway auto-detects ~/ffmpeg/ffmpeg, and the "Add to PATH" hint is gone.
A host build now prints sudo install -m 0755 <out> /usr/local/bin/ffmpeg. /usr/local/bin is in transcribe_ffmpeg._FFMPEG_CANDIDATE_DIRS, so that is the path the resolver actually trusts.
The default output location is unchanged, so the no-sudo bash build-ffmpeg.sh that Settings suggests still works.
Verified: ran the no-argument build with HOME pointed at a temp dir; it printed the install line for that path.
self-added: no
mechanism: accurate post-build install instruction

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from c94b5e5 to 0f655d4 Compare October 22, 2026 20:14
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 23, 2026
@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 2 times, most recently from d8af159 to 6964f64 Compare October 23, 2026 01:36
@bolichen97

Copy link
Copy Markdown
Contributor Author

Scope note (maintainer request): the desktop and source-install switch to the new decoder is now in this PR instead of a follow-up. The binaries are published as ffmpeg-decoder-7.1.1-r1 from workflow run 39881015516 and pinned in stt.decoder.ARTIFACTS. The Linux builds are now static; the first dynamic build needed glibc 2.35.

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from 6964f64 to 525fe14 Compare October 23, 2026 01:50
@bolichen97

Copy link
Copy Markdown
Contributor Author

span=6b1830b79398 disposition: fixed in 525fe14

transcribe_aws.py's webm→ogg remux now passes -vn, so only the audio stream is mapped. Fixed at the caller, as suggested; the build keeps its audio-only feature set.
Reproduced with the released linux-x86_64 decoder on a VP8+Opus webm: without -vn it exits 8 (theora encoder missing); with -vn it exits 0. The other decode paths write WAV (no default video codec) or already pass -vn; a cover-art m4a to WAV also exits 0.
Regression test test_remux_maps_audio_only fails without the flag and passes with it.
self-added: no
mechanism: audio-only stream mapping on the ogg remux

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from 525fe14 to d126637 Compare October 23, 2026 02:04
@bolichen97

bolichen97 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor Author

span=2c3cb4397785 disposition: fixed in d126637

New stt.decoder.RETIRED_FILENAMES lists the four old pinned names. When nothing resolves, _resolve_ffmpeg_for_execution calls migrate_retired_decoder(). On a store-using install (no bundled decoder, a pinned platform) that still holds a retired file, it runs store().ensure() once per process and then retries resolution once.
DecoderStore.ensure removes retired files after a successful fetch, so the old 50-90 MB executable does not stay behind. Once per process, so a host that cannot reach the release does not wait out a fetch on every memo; the Download button still retries.
The stale desktop_decoder_requirement comment now names install_desktop_decoder.
Tests: TestRetiredDecoderMigration (fetch then remove, once per process, no network without a retired file, bundled/unpinned skip, the resolver retries only after a migration).
self-added: yes
mechanism: one-shot retired-pin migration on resolve

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 2 times, most recently from c4f8db5 to cd0bbe1 Compare October 23, 2026 02:25
@bolichen97

Copy link
Copy Markdown
Contributor Author

span=374f067f6e45 disposition: fixed in cd0bbe1

migrate_retired_decoder now keeps the migration as one process-wide task. A concurrent transcode awaits that same task (shielded, so one caller cancelling does not stop the transfer) instead of returning None during the fetch. After the task finishes it never starts again, and later callers get its result.
A task bound to another event loop is treated as done, so a test or second loop never awaits it across loops.
Test test_concurrent_callers_share_one_transfer: two concurrent calls make one fetch, and both get the installed path.
self-added: yes
mechanism: shared in-flight migration task

@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch 2 times, most recently from 2c56819 to cb7db03 Compare October 23, 2026 02:59
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 23, 2026
@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from cb7db03 to 1065f67 Compare October 23, 2026 04:03
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 23, 2026
Extend build-ffmpeg.sh to build the small audio-only ffmpeg for every
desktop platform (Linux x64/arm64, macOS arm64/x64, Windows x64) and
cover every input the speech path decodes. Add a manual workflow that
builds and verifies all five and can publish them as one release.
@bolichen97
bolichen97 force-pushed the build/ffmpeg-decoder branch from 1065f67 to 4495baa Compare October 23, 2026 04:14
@bolichen97

Copy link
Copy Markdown
Contributor Author

span=0022253b0952 disposition: fixed in 4495baa

The macos-x86_64 leg is gone from the workflow matrix, and --target macos-x86_64 from build-ffmpeg.sh. Only the four pinned platforms are built now; the macOS Intel backend ships no decoder.
macos-aarch64 builds on an Apple-silicon host only, because configure and the verify step run the binary.
The ffmpeg-decoder-7.1.1-r1 release still has the asset from its first run. Nothing pins or fetches it, and a later -rN release will not include it.
self-added: yes
mechanism: removed unconsumed macOS Intel build leg

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by Crew Captain on head 4495baa: Design, First Principles and UX PASS on this head, Opus and GPT no blocking findings, no merge conflict. Auto-merge waits for Coverage Combine and PR Readiness.

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 23, 2026 05:08
@bolichen97
bolichen97 disabled auto-merge October 23, 2026 05:08
@bolichen97
bolichen97 merged commit 4d036ec into main Oct 23, 2026
113 of 114 checks passed
@bolichen97
bolichen97 deleted the build/ffmpeg-decoder branch October 23, 2026 05:08
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants