refactor(http): build the web UI with Vite and embed it gzipped in the firmware

.gitignore

@@ -1,7 +1,7 @@
 .DS_Store
 __pycache__/
 data/*.*
-data/ui/
+data/*/
 artifacts/
 
 # Cursor hook event log

AGENTS.md

@@ -11,7 +11,7 @@ Tiny Engineer is an open-source ESP32-C3 Wi-Fi desk robot: 3D-printed mechanics,
 | Path | Role |
 | --- | --- |
 | `src/`, `include/`, `lib/`, `data/` | Firmware (PlatformIO) |
-| `ui/` | Web control panel (Vite + ES modules); `pio run` builds, minifies and gzips it onto LittleFS |
+| `ui/` | Web control panel (Vite + ES modules); `pio run` builds it into one minified page and embeds it gzipped in the firmware |
 | `packages/` | HTTP / hook CLIs (Cursor, Antigravity, Claude Code, …) — scope `integrations` |
 | `skills/` | Agent skills shared by Claude Code, Cursor and Antigravity — scope `integrations` |
 | `3d_models/` | CAD and printables (CERN-OHL-S) |

docs/api.md

@@ -6,7 +6,7 @@ Listens on **port 80** after STA Wi-Fi connects, or during setup AP mode at `htt
 
 Optional auth: when an `access_token` is configured in settings, all JSON API routes require `Authorization: Bearer <token>`. Empty token (default) means no auth. `GET /auth` is always public and reports whether auth is required. Missing/wrong token → **401** `{"ok":false,"error":"unauthorized"}`. HTML panel routes stay public (the UI prompts for the token). `Content-Type: application/json`. CORS: `Access-Control-Allow-Origin: *`, `Access-Control-Allow-Headers: Authorization`.
 
-When WiFi credentials are not saved yet, control APIs (`/anim`, `/test/*`) return **503** `{"ok":false,"error":"wifi not configured"}`. Setup routes (`/`, `/config`, `/ui/*`, `/auth`, `/health`, `/settings`, `/setup/servo`, `/setup/led`, `/setup/audio`, `/setup/oled`) stay available on the setup AP.
+When WiFi credentials are not saved yet, control APIs (`/anim`, `/test/*`) return **503** `{"ok":false,"error":"wifi not configured"}`. Setup routes (`/`, `/config`, `/auth`, `/health`, `/settings`, `/setup/servo`, `/setup/led`, `/setup/audio`, `/setup/oled`) stay available on the setup AP.
 
 If boot WiFi credentials are missing, the device opens setup AP mode (`TinyEngineer-XXXX`) and serves a five-step setup wizard at `/config` (servo calibration, OLED orientation, RGB LED, speaker test, then Wi-Fi). If saved credentials fail, it reopens setup AP mode. Hardware tests: [`docs/hardware/testing.md`](hardware/testing.md). How to wire this API into AI tools: [`integration.md`](integration.md).
 
@@ -24,11 +24,7 @@ Open in a browser:
 curl http://tiny-engineer.local/
 ```
 
-Returns `Content-Type: text/html; charset=utf-8`, gzip-encoded. The page is served from LittleFS (`/ui/index.html.gz`); if the filesystem image is missing, or its `/ui/version.txt` does not match the firmware version, it returns **503** with flashing instructions.
-
-### `GET /ui/*`
-
-Web UI assets from LittleFS, gzip-encoded. Bundled files under `/ui/assets/` have content-hashed names and are sent with `Cache-Control: max-age=31536000, immutable`; anything else under `/ui/` is `no-cache`. Always public. Source lives in [`ui/`](../ui/); `pio run` builds it with Vite and gzips the output into `data/ui/`.
+Returns `Content-Type: text/html; charset=utf-8`, gzip-encoded. The page is built from [`ui/`](../ui/) with Vite and embedded in the firmware, so it always matches the API the firmware serves.
 
 ### `GET /auth`
 

docs/flash.md

@@ -41,7 +41,7 @@ Firmware is Arduino on [PlatformIO](https://platformio.org/) ([pioarduino](https
 ### Install and connect
 
 1. Install [PlatformIO Core](https://docs.platformio.org/en/latest/core/installation.html) (or the PlatformIO IDE extension).
-   Also install [Node.js](https://nodejs.org/) 20.19+ on 20.x, or 22.12+: `pio run` builds the web UI in [`ui/`](../ui/) with npm.
+   Also install [Node.js](https://nodejs.org/) 20.19+ on 20.x, or 22.12+: `pio run` builds the web UI in [`ui/`](../ui/) with npm and embeds it in the firmware.
 2. Use a USB-C **data** cable. Charge-only cables fail upload and serial.
 3. Plug the data cable into the **main control board USB-C** once the ESP32 is seated on its headers. Advanced breakout build: use the Adafruit 5993, or the C3-Zero onboard USB-C before that breakout is wired — [hardware/wiring.md](hardware/wiring.md).
 
@@ -55,7 +55,7 @@ pio run -t upload       # flash firmware + LittleFS
 pio device monitor      # serial (115200)
 ```
 
-After firmware upload, a post-script also uploads **LittleFS** ([`scripts/upload_fs_after_upload.py`](../scripts/upload_fs_after_upload.py)) so the web UI and WAV assets (`welcome`, `bell`, and friends) land on the board. If the web page says the UI is missing, or animations move but stay silent, run `pio run -t uploadfs` once.
+After firmware upload, a post-script also uploads **LittleFS** ([`scripts/upload_fs_after_upload.py`](../scripts/upload_fs_after_upload.py)) so WAV assets (`welcome`, `bell`, and friends) land on the board. If animations move but stay silent, run `pio run -t uploadfs` once.
 
 Several serial ports:
 
@@ -119,16 +119,15 @@ After that, with the robot on your home Wi-Fi:
 ```bash
 export TINY_ENGINEER_URL=http://192.168.x.x   # default: tiny-engineer.local
 export TINY_ENGINEER_TOKEN=...                # only if access_token is set
-pio run -e ota -t ota       # LittleFS (web UI, WAV assets), then firmware
+pio run -e ota -t ota       # firmware
+pio run -e ota -t otafs     # LittleFS (WAV assets)
 ```
 
-Firmware and filesystem are one release: the web UI is stamped with the firmware version it was built with, and the firmware refuses to serve a UI from another build (`/` returns **503** naming both versions). `-t ota` therefore uploads the filesystem first, waits for the robot to reboot, then uploads the firmware. `pio run -e ota -t otafs` uploads the filesystem alone, for recovery.
-
 These are the same variables the agent integrations read ([integration.md](integration.md)). OTA listens on UDP/TCP port 3232 and only runs while connected to home Wi-Fi, not in setup AP mode. When `access_token` is set it is also the OTA password; changing it applies to OTA without a reboot.
 
 An update is written to the inactive slot and booted once; it becomes permanent only after it connects to Wi-Fi and starts its OTA listener. Firmware that never gets that far is rolled back to the previous slot on the next reset or power cycle. A firmware that hangs needs that power cycle to recover.
 
-A filesystem update that fails midway reboots the robot; re-run `pio run -e ota -t ota` or fall back to `pio run -e ota -t upload` over USB.
+A filesystem update that fails midway reboots the robot; re-run `pio run -e ota -t otafs` or fall back to `pio run -e ota -t uploadfs` over USB.
 
 Partition changes, including a switch back to the default build, cannot be applied over the air. Flash those with `pio run -t upload` over USB.
 

docs/hardware/testing.md

@@ -106,7 +106,6 @@ curl -X POST "http://tiny-engineer.local/test/servo?index=0&angle=90"
 | Method | Path | Body |
 | --- | --- | --- |
 | `GET` | `/` | HTML endpoint index |
-| `GET` | `/ui/*` | Web UI assets from LittleFS (gzip) |
 | `GET` | `/auth` | Auth status (`ok`, `required`, `wifi_configured`, `provisioning`) — always public |
 | `GET` | `/health` | Health JSON — full field list in [api.md](../api.md#get-health) |
 | `GET` / `POST` | `/anim` | Current animation / start one (`name`, optional `interrupt`) — full params in [api.md](../api.md#post-anim) |

docs/settings.md

@@ -91,7 +91,7 @@ Flash only when you want to try it on hardware (`pio run -t upload`).
 - **JSON buffer:** `sendSettingsJson` uses a fixed `char` buffer — bump size when adding fields.
 - **Factory reset:** `factoryResetSettings()` in [`reset.cpp`](../src/settings/reset.cpp) clears NVS namespace `te` and writes defaults, including WiFi credentials. Servo min/max (`sranges`), RGB LED mapping (`rgb_ord`), and OLED rotation (`oled_rot`) are written back unchanged. Exposed as `POST /settings/reset`. After reset, power-cycle into setup AP mode to configure WiFi again (WiFi is not editable on the normal Config page). Servo ranges, LED mapping, and screen rotation can be retuned in that wizard.
 - **Hostname-style settings:** freeze the boot value separately if live change cannot apply (see `settingsBootHostname()` / `reboot_required`).
-- **UI size:** the panel ships gzipped on LittleFS next to the WAVs; keep controls compact.
+- **UI size:** the panel is embedded gzipped in the firmware; keep controls compact.
 - **Doc drift:** HTML param tables must match `api.md` exactly.
 
 ## Related

partitions.csv

@@ -1,6 +1,6 @@
 # Name,   Type, SubType, Offset,  Size, Flags
 # 4MB: one app slot (USB flash, no OTA). Dual 0x150000 OTA apps overflowed
-# the firmware (~1.3MB). LittleFS is ~1.81MB so a mod can replace
+# the firmware (~1.38MB HTML UI). LittleFS is ~1.81MB so a mod can replace
 # some WAVs with longer clips and keep the stock files it does not ship.
 # NVS stays at 0x9000. spiffs moved; reflash app + filesystem together.
 nvs,      data, nvs,     0x9000,  0x5000,

platformio.ini

@@ -39,6 +39,7 @@ custom_audio_mod =
 extra_scripts =
     pre:scripts/firmware_version.py
     pre:scripts/copy_assets.py
+    pre:scripts/embed_ui.py
     post:scripts/upload_fs_after_upload.py
 
 ; Host Unity tests — never flash. Run: pio test -e native

scripts/audio_pack.py

@@ -258,6 +258,11 @@ def pack_audio(assets_dir, dest_dir, partition_bytes, mod_dir=None):
         if leftover.is_file():
             leftover.unlink()
 
+    # The firmware reads only top-level files, so any directory here is stale build output.
+    for leftover in dest_dir.iterdir():
+        if leftover.is_dir():
+            shutil.rmtree(leftover)
+
     for name in STOCK_WAVS:
         source = assets_dir / name
         if not source.is_file():
@@ -275,16 +280,16 @@ def pack_audio(assets_dir, dest_dir, partition_bytes, mod_dir=None):
 
     total = sum(
         path.stat().st_size
-        for path in dest_dir.rglob("*")
+        for path in dest_dir.iterdir()
         if path.is_file()
     )
     limit = partition_bytes - HEADROOM_BYTES
     if total > limit:
         raise AudioPackError(
-            f"Filesystem image is {total} bytes; "
+            f"Audio image is {total} bytes; "
             f"spiffs allows {limit} after {HEADROOM_BYTES} bytes headroom"
         )
-    print(f"Filesystem image {total} bytes (limit {limit})")
+    print(f"Audio image {total} bytes (limit {limit})")
 
 
 def _overlay_mod(mod_dir, dest_dir):

scripts/copy_assets.py

@@ -9,7 +9,6 @@
     sys.path.insert(0, str(scripts_dir))
 
 from audio_pack import pack_audio, spiffs_size_bytes, validate_mod_name
-from ui_pack import build_ui, pack_ui
 
 mod_name = env.GetProjectOption("custom_audio_mod", "")
 mod_name = "" if mod_name is None else str(mod_name).strip()
@@ -19,9 +18,6 @@
     mod_dir = project_dir / "mods" / mod_name / "assets"
     print(f"Audio mod overlay: {mod_name}")
 
-# Before pack_audio so its size check covers the UI files too.
-pack_ui(build_ui(project_dir / "ui"), project_dir / "data" / "ui", env["TE_FW_VERSION"])
-
 pack_audio(
     project_dir / "assets",
     project_dir / "data",

scripts/embed_ui.py

@@ -0,0 +1,58 @@
+Import("env")
+
+import gzip
+import shutil
+import subprocess
+from pathlib import Path
+
+project_dir = Path(env.subst("$PROJECT_DIR"))
+ui_dir = project_dir / "ui"
+generated_dir = Path(env.subst("$BUILD_DIR")) / "generated"
+header = generated_dir / "web_ui.h"
+
+
+def npm(*args):
+    executable = shutil.which("npm")
+    if executable is None:
+        raise RuntimeError("npm not found; building the web UI needs Node 20.19+ on 20.x, or 22.12+")
+    subprocess.run([executable, *args], cwd=ui_dir, check=True)
+
+
+def newest_source_mtime():
+    sources = [ui_dir / "index.html", ui_dir / "vite.config.js", ui_dir / "package.json", ui_dir / "package-lock.json"]
+    sources += [path for path in (ui_dir / "src").rglob("*") if path.is_file()]
+    return max(path.stat().st_mtime for path in sources)
+
+
+env.Append(CPPPATH=[str(generated_dir)])
+
+# pio runs pre-scripts for every target (uploadfs, otafs, ...), so skip Node when ui/ is unchanged.
+if header.is_file() and header.stat().st_mtime >= newest_source_mtime():
+    Return()
+
+# npm writes this marker on install; an older one means the lockfile changed since.
+installed = ui_dir / "node_modules" / ".package-lock.json"
+if not installed.is_file() or installed.stat().st_mtime < (ui_dir / "package-lock.json").stat().st_mtime:
+    npm("ci")
+npm("run", "build")
+
+page = (ui_dir / "dist" / "index.html").read_bytes()
+# mtime=0 keeps the bytes identical for identical sources, so an unchanged UI does not rebuild index_page.cpp.
+packed = gzip.compress(page, compresslevel=9, mtime=0)
+rows = ",\n".join(
+    "  " + ", ".join(f"0x{byte:02x}" for byte in packed[offset:offset + 16])
+    for offset in range(0, len(packed), 16)
+)
+source = (
+    "// Generated by scripts/embed_ui.py from ui/. Do not edit.\n"
+    "#pragma once\n\n"
+    "#include <Arduino.h>\n\n"
+    f"const uint8_t kWebUiGzip[] PROGMEM = {{\n{rows}\n}};\n"
+)
+
+generated_dir.mkdir(parents=True, exist_ok=True)
+if header.is_file() and header.read_text() == source:
+    header.touch()
+else:
+    header.write_text(source)
+print(f"Embedded web UI: {len(page)} -> {len(packed)} bytes gzipped")

scripts/firmware_version.py

@@ -35,5 +35,3 @@ def _sanitize(raw: str) -> str:
 version = _sanitize(_git_describe(project_dir))
 print(f"Firmware version: {version}")
 env.Append(CPPDEFINES=[("FW_VERSION", '\\"%s\\"' % version)])
-# copy_assets.py stamps the web UI with the same string so the firmware can detect a mismatched filesystem.
-env.Replace(TE_FW_VERSION=version)

scripts/ota.py

@@ -2,15 +2,10 @@
 
 import os
 import subprocess
-import time
-import urllib.error
-import urllib.request
 from pathlib import Path
 from urllib.parse import urlparse
 
 DEFAULT_HOST = "tiny-engineer.local"
-REBOOT_TIMEOUT_S = 90
-FIRMWARE_ATTEMPTS = 3
 
 
 def ota_host():
@@ -20,7 +15,7 @@ def ota_host():
     return host or DEFAULT_HOST
 
 
-def espota(image, filesystem, check=True):
+def espota(image, filesystem):
     command = [
         env.subst("$PYTHONEXE"),
         str(Path(env.PioPlatform().get_package_dir("framework-arduinoespressif32")) / "tools" / "espota.py"),
@@ -38,59 +33,29 @@ def espota(image, filesystem, check=True):
         command.append("--spiffs")
 
     print(f"OTA {'filesystem' if filesystem else 'firmware'} -> {ota_host()}")
-    return subprocess.run(command, check=check).returncode == 0
+    subprocess.run(command, check=True)
 
 
-def wait_for_reboot():
-    # The robot restarts after an OTA write; any HTTP answer (even 401) means it is back.
-    time.sleep(5)
-    deadline = time.monotonic() + REBOOT_TIMEOUT_S
+def upload_firmware(source, target, env):
+    espota(env.subst("$BUILD_DIR/${PROGNAME}.bin"), filesystem=False)
 
-    while time.monotonic() < deadline:
-        try:
-            urllib.request.urlopen(f"http://{ota_host()}/health", timeout=3)
-            return
-        except urllib.error.HTTPError:
-            return
-        except OSError:
-            time.sleep(2)
 
-    raise RuntimeError(f"{ota_host()} did not come back within {REBOOT_TIMEOUT_S} s")
-
-
-def build_filesystem():
+def upload_filesystem(source, target, env):
     project_dir = env.subst("$PROJECT_DIR")
     # The platform only wires up the LittleFS image builder when buildfs is a command-line target.
     subprocess.run(
         [env.subst("$PYTHONEXE"), "-m", "platformio", "run", "-t", "buildfs", "-e", env.subst("$PIOENV"), "-d", project_dir],
         check=True,
     )
-    return env.subst("$BUILD_DIR/${ESP32_FS_IMAGE_NAME}.bin")
-
-
-def upload_release(source, target, env):
-    # Filesystem first: until the firmware follows, the robot keeps its old firmware and the
-    # web UI refuses to load on the version mismatch instead of calling an older API.
-    espota(build_filesystem(), filesystem=True)
-    wait_for_reboot()
-
-    # The OTA listener starts just after the HTTP server, so the first attempt can be early.
-    for attempt in range(1, FIRMWARE_ATTEMPTS + 1):
-        if espota(env.subst("$BUILD_DIR/${PROGNAME}.bin"), filesystem=False, check=attempt == FIRMWARE_ATTEMPTS):
-            return
-        time.sleep(3)
-
-
-def upload_filesystem(source, target, env):
-    espota(build_filesystem(), filesystem=True)
+    espota(env.subst("$BUILD_DIR/${ESP32_FS_IMAGE_NAME}.bin"), filesystem=True)
 
 
 env.AddCustomTarget(
     name="ota",
     dependencies="$BUILD_DIR/${PROGNAME}.bin",
-    actions=upload_release,
+    actions=upload_firmware,
     title="Upload OTA",
-    description="Upload filesystem, then firmware, over Wi-Fi",
+    description="Upload firmware over Wi-Fi",
 )
 
 env.AddCustomTarget(

scripts/test_audio_pack.py

@@ -70,6 +70,12 @@ def test_stock_rebuild_drops_old_cue(self):
         pack_audio(self.assets, self.dest, 2_000_000)
         self.assertFalse((self.dest / "welcome.cue").exists())
 
+    def test_rebuild_drops_stale_directories(self):
+        (self.dest / "ui" / "assets").mkdir(parents=True)
+        (self.dest / "ui" / "assets" / "index.js").write_text("stale")
+        pack_audio(self.assets, self.dest, 2_000_000)
+        self.assertFalse((self.dest / "ui").exists())
+
     def test_stock_copy_has_no_cues(self):
         pack_audio(self.assets, self.dest, 2_000_000)
         for name in STOCK_WAVS: