Skip to content

Build a checkout run's catalog apps in the background - #549

Merged
ymote merged 2 commits into
mainfrom
feat/prebuild-catalog-apps
Oct 20, 2026
Merged

ymote merged 2 commits into
mainfrom
feat/prebuild-catalog-apps

Conversation

@ymote

@ymote ymote commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Why

Run from a checkout, opening a catalog process app the first time waits for its release build (clients::launch_plan: cargo build --release) before the app starts. Measured on an M-series Mac:

App First open, cold build
pdf 1 min 38 s on a quiet machine (120 crates); 3 min 14 s under load
route 110 s
files 38 s

An installed shell compiles nothing (its apps are linked modules, or binaries beside it), so this only affects development and demo runs. It is still every run of OctoSense from a source tree.

What

crates/shell/src/prebuild.rs runs those same builds once, in the background, 20 s after the shell starts. A first open then finds its binary built.

  • Exactly the launch's build. For each listed catalog app it runs clients::launch_plan's build step through clients::build_command (now pub(crate)): same args, environment allow-list and PATH. Each identical build runs once, one at a time, in catalog order.
  • Background priority: darwinbg on macOS (efficiency cores, throttled disk), nice 19 on other Unix, below normal on Windows.
  • Steps aside for the person:
    • The WM tick publishes how many launches are building (ClientSlot::building).
    • While any is, the prebuild stops its own cargo (the build has its own process group, so rustc goes too), waits, then starts that build again. Cargo keeps what it finished.
    • It logs prebuild: waiting while a launch builds and … stepped aside for a launch's build.
  • Quitting stops it: Event::Shutdown calls prebuild::shutdown, which kills the running build's group.
  • Off switch: MAKEPAD_WM_NO_PREBUILD=1, documented in the desktop README in both languages. An installed shell has no build step, so it never starts.

Verified (macOS 26, Apple silicon)

  • cargo test --locked --features mobile-apps -p octosense-shell --lib prebuild (from phone/): 4/4.
  • End to end: a hidden shell from this checkout, MAKEPAD_WM_ROOT set to it, a fresh test profile, catalog apps building into an empty target dir.
    • Cold: prebuild: 15 catalog apps build in the background, then 15 of 15 built in 1486 s at background priority. Per app: reference 345 s (the shared crates), browser 402, route 289, files 142, pdf 100, the rest 11–34 s.
    • Payoff: restarted with MAKEPAD_WM_TEST_APP=pdf, the log reads pdf client 1 first frame in 3290 ms.
    • Warm restart: 15 of 15 built in 74 s (up-to-date checks).
    • Launch first: with an empty target and MAKEPAD_WM_TEST_APP=calculator, the prebuild logged waiting while a launch builds until Calculator started (first frame 53 s, its cold build), then went on.
    • Quit mid-build: with two prebuild cargo processes running, /quit left no cargo and no rustc behind.
    • Not exercised end to end: the stop-mid-build path (a launch starting while a prebuild build runs). It shares wait_or_step_aside with the quit path, which was.
  • Windows: unverified. There, prebuild::shutdown only marks the worker stopped, and its child ends when the worker sees it.

🤖 Generated with Claude Code

ymote and others added 2 commits October 20, 2026 03:33
Run from a checkout, a process app's first open waits for its release build:
Makepad's pdf app took 1 min 38 s cold on a quiet M-series Mac (120 crates),
route 110 s more, and minutes under load. The shell now runs those same
builds once in the background, 20 s after it starts (prebuild.rs), so the
first open finds the binary built: pdf's first frame came 3.3 s after the
launch instead.

- Each listed catalog app's own launch build (clients::launch_plan and
  build_command, now pub(crate)), each identical build once, in catalog
  order, one at a time.
- Background priority: darwinbg on macOS (efficiency cores, throttled disk),
  nice 19 elsewhere, below normal on Windows; its own process group.
- It steps aside for a launch: the WM counts building launches every tick
  (ClientSlot::building); the prebuild then stops its cargo group, waits, and
  starts that build again, cargo keeping what it finished.
- Event::Shutdown stops it (prebuild::shutdown), group and all.
- An installed shell has no build and never starts it;
  MAKEPAD_WM_NO_PREBUILD=1 turns it off (desktop README, both languages).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ymote
ymote marked this pull request as ready for review October 20, 2026 01:19
@ymote

ymote commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Local CI passed on 86e164c (tools/ci-local.sh --only all --linux-host --offload, 14m48s, Darwin arm64; the ubuntu jobs and the macOS jobs' portable steps (their macOS steps ran here) on the Linux build host, Linux x86_64).

GitHub would run for this PR: desktop.yml, phone.yml, apps.yml. Merged on this local pass (macOS runner queue); GitHub CI runs on the merge commit on main.

Job Step Result Time
ci-local / drift The local mapping fits the workflows PASS -
desktop.yml / desktop Prepare the pinned framework sources PASS 1s
desktop.yml / desktop Compile the desktop with the shared runtime PASS 3m53s
desktop.yml / desktop Desktop tools and scripts PASS 20s
desktop.yml / desktop (linux) Prepare the pinned framework sources PASS 1s
desktop.yml / desktop (linux) One copy of every shell source PASS -
desktop.yml / desktop (linux) In-app updater and installer handoff PASS 1m40s
desktop.yml / desktop (linux) One Makepad, App Hub, octos and Rinx in the locked graph PASS 2s
desktop.yml / desktop (linux) The native apps' generated places match native-apps.json PASS 0s
desktop.yml / native-host-api Prepare the pinned framework sources PASS 1s
desktop.yml / native-host-api Build the native acceptance host and pinned Hub CLI PASS 47s
desktop.yml / native-host-api Signed app tool calls a real OS permission API PASS 3s
phone.yml / home Prepare pinned sources and the reviewed runtime patch PASS 1s
phone.yml / home Validate launcher icons and Apple asset catalogs PASS 4s
phone.yml / home Compile Home and its bundled modules PASS 3m30s
phone.yml / home (linux) Prepare pinned sources and the reviewed runtime patch PASS 0s
phone.yml / home (linux) Verify the single runtime graph PASS 2s
phone.yml / home (linux) The native apps' generated places match native-apps.json PASS 0s
phone.yml / home (linux) The pinned octos revision PASS 0s
phone.yml / home (linux) Build octos (unless cached) PASS -
phone.yml / home (linux) Build Home, App Hub admission and runtime policy tests PASS 3m23s
phone.yml / home (linux) Test Home, App Hub admission and runtime policy PASS 42s
phone.yml / home (linux) Two-lane scenario (real kernel, scripted model) PASS 28s
phone.yml / home (linux) Build opt-in AppCard tests PASS 2m48s
phone.yml / home (linux) Test opt-in AppCard PASS 1s
phone.yml / home (linux) Build craft-engine relay and system-chat tests PASS 2m57s
phone.yml / home (linux) Test craft-engine relay and system chat PASS 13s
phone.yml / home (linux) Build linked Terminal tests PASS 2m30s
phone.yml / home (linux) Test linked Terminal PASS 1s
phone.yml / home (linux) Build Wasm and runtime policy tests PASS 1m15s
phone.yml / home (linux) Test Wasm and runtime policy PASS 11s
apps.yml / services (linux) python3 tools/setup.py PASS 0s
apps.yml / services (linux) Shared system app interface PASS 0s
apps.yml / services (linux) Kernel service PASS 19s
apps.yml / services (linux) OctoSense release updater PASS 6s
apps.yml / services (linux) App peers PASS 29s
apps.yml / services (linux) AI providers config PASS 5s
apps.yml / services (linux) Connected provider protocols and Linux host adapter PASS 47s
apps.yml / services (linux) L0 in-card chat (sys.chat, host side) PASS 9s
apps.yml / services (linux) System toolbox (workflow templates) PASS 1m29s
apps.yml / kernel-security (linux) python3 tools/setup.py PASS 0s
apps.yml / kernel-security (linux) The pinned octos revision PASS 0s
apps.yml / kernel-security (linux) Build octos (unless cached) PASS -
apps.yml / kernel-security (linux) Real-kernel tests PASS 40s
apps.yml / apps (linux) python3 tools/setup.py PASS 0s
apps.yml / apps (linux) Host services (Mail, AI providers, News) PASS 2m04s
apps.yml / apps (linux) Connected accounts and Rinx editor PASS 1m31s
apps.yml / apps (linux) Shell AI services (crates/ai-host) PASS 2m57s
apps.yml / apps (linux) AppCard runtime setup safeguards PASS 0s
apps.yml / apps (linux) AppCard clippy (own crates; shared runtime compiles) PASS 27s
apps.yml / apps (linux) AppCard transport and store PASS 7s
apps.yml / apps (linux) One Makepad, App Hub, octos and Rinx in the locked graph PASS 3s
rom.yml / product (linux) python3 -m unittest discover -s tests -v SKIPPED (!) -
rom.yml / product (linux) Check generated Agent Binder client SKIPPED (!) -
rom.yml / product (linux) bash -n scripts/stage-forks.sh scripts/publish-release.sh scripts/apply-to-tree.sh scripts/build-rom.sh PASS -
rom.yml / web-installer (linux) npm ci --ignore-scripts --no-audit --no-fund SKIPPED (!) -
rom.yml / web-installer (linux) npm test SKIPPED (!) -
rom.yml / web-installer (linux) npx playwright install --with-deps chromium SKIPPED (!) -
rom.yml / web-installer (linux) npm run test:browser SKIPPED (!) -
rom.yml / web-installer (linux) python3 -m unittest discover -s tests -p test_installer_manifest.py -v SKIPPED (!) -
rom.yml / web-installer (linux) bash -n scripts/release.sh SKIPPED (!) -
linux-host / sandbox (linux) python3 tools/setup.py PASS 0s
linux-host / sandbox (linux) Process sandbox on Linux (Landlock, seccomp) PASS 1m48s

Skipped (these did NOT pass; (!) marks a skip that blocks tools/ci-local-merge.sh):

  • rom.yml / product (linux): python3 -m unittest discover -s tests -v: javac -version does not work here: install a JDK (17 or newer) and put its bin/ on PATH or set JAVA_HOME
  • rom.yml / product (linux): Check generated Agent Binder client: needs build-tools/35.0.0/aidl, platforms/android-35/framework.aidl in the Android SDK (no Android SDK found): install the SDK packages 'build-tools;35.0.0' and 'platforms;android-35' (sdkmanager) and set ANDROID_HOME
  • rom.yml / web-installer (linux): npm ci --ignore-scripts --no-audit --no-fund: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
  • rom.yml / web-installer (linux): npm test: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
  • rom.yml / web-installer (linux): npx playwright install --with-deps chromium: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
  • rom.yml / web-installer (linux): npm run test:browser: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
  • rom.yml / web-installer (linux): python3 -m unittest discover -s tests -p test_installer_manifest.py -v: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH
  • rom.yml / web-installer (linux): bash -n scripts/release.sh: no node, npm, npx on PATH (the workflow uses node 24); put node on PATH

Notes:

  • GitHub runs Python 3.12; this run used Python 3.9.6
  • GitHub runs Python 3.12; the Linux host used Python 3.14.4

@ymote
ymote merged commit d26ae16 into main Oct 20, 2026
2 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant