Skip to content

refactor(http): build the web UI with Vite and embed it gzipped in the firmware - #68

Merged
jamro merged 4 commits into
jamro:mainfrom
Mechazawa:task/embed-ui-in-firmware
Oct 11, 2026
Merged

jamro merged 4 commits into
jamro:mainfrom
Mechazawa:task/embed-ui-in-firmware

Conversation

@Mechazawa

@Mechazawa Mechazawa commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

What

Alternative to #60 with the same frontend work, but the page ships inside the firmware instead of on LittleFS. Pick one of the two.

  • The control panel moves out of the raw string in src/http/index_page.cpp into ui/ (ES modules, Vite, Vitest), exactly as in refactor(http): serve web UI from LittleFS, bundled with Vite #60.
  • Vite with vite-plugin-singlefile builds one self-contained, minified index.html. scripts/embed_ui.py (a PlatformIO pre-script) gzips it into a generated header, $BUILD_DIR/generated/web_ui.h, and the firmware serves those bytes with Content-Encoding: gzip straight from flash.
  • Because the UI is part of the firmware image, it can never be out of step with the API: no /ui/* routes, no version stamp or check, and -t ota / -t otafs / LittleFS work exactly as on main (audio only).
  • The page is 58 KB minified, 15.8 KB gzipped in the firmware, versus 77.7 KB uncompressed on main.
  • embed_ui.py skips Node when nothing under ui/ changed, so uploadfs, otafs and no-op builds do not rebuild the UI; an identical result does not recompile index_page.cpp.
  • Busy state now re-enables only the controls it disabled, and servo range changes notify the servo page and the calibration step, so callers no longer refresh them by hand.
  • Building the firmware needs Node 20.19+ on 20.x, or 22.12+, alongside PlatformIO. The firmware CI job and the release workflow install Node 22.

Checks

  • Title is type(scope): summary
  • Breaking HTTP / pins / NVS / servo defaults / hook CLI called out as type!: plus a BREAKING CHANGE: footer — none; routes and responses are unchanged
  • Firmware: pio run and/or pio test -e native when that code changed — pio run (default, ota, expression-demo), pio test -e native, python3 scripts/test_audio_pack.py, npm run lint --prefix ui, npm test --prefix ui
  • Packages: npm test --prefix packages/tiny-engineer-cursor / …/tiny-engineer-antigravity / …/tiny-engineer-claude-code when that package changed — N/A, no package changes
  • HTTP: HTML index (ui/index.html) + docs/api.md (and README / docs/hardware/testing.md if the route list changed) — route list unchanged; the HTML index now lives in ui/index.html
  • Settings: layers in docs/settings.md; no raw access_token in logs — no new settings; docs/settings.md web UI section points at ui/
  • CAD: .f3d + exported .3mf; CERN-OHL-S; AiEmblem.3mf not used as a branding swap — N/A
  • Pins: include/pins.h + docs/hardware/ together — N/A
  • Servo ranges not widened without a real robot — the setup wizard bar stays display-only; calibration moves only through the step buttons
  • Hardware tested: robot / bench / N/A — robot: firmware flashed over OTA, every page loaded in a headless browser with no JS errors, the served page decompresses byte-identical to the build. The setup wizard (setup AP mode) was not exercised on hardware.
  • No .env, tokens, or Wi-Fi passwords in logs or screenshots

PCB (if this PR changes a board)

N/A — no board changes.

Summary by CodeRabbit

  • New Features
    • The web panel is embedded in the firmware and served from the device’s main page, rather than loaded from the filesystem.
    • Servo controls and calibration update when servo ranges change.
  • Bug Fixes
    • Busy indicators preserve controls that were already disabled.
    • Improved handling of servo ranges and OLED rotation settings during setup.
  • Documentation
    • Updated build, upload, and API guidance to reflect the embedded web panel and separate firmware and filesystem uploads.

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b972be4f-78d7-43d3-b792-ec101290d242

📥 Commits

Reviewing files that changed from the base of the PR and between 87e9754 and 75f1163.


📒 Files selected for processing (3)
  • .gitignore
  • scripts/audio_pack.py
  • scripts/test_audio_pack.py

📝 Walkthrough
📝 Walkthrough

Walkthrough

The UI build now embeds a gzip-compressed single page in firmware, served from GET / instead of LittleFS UI assets. OTA firmware and LittleFS uploads are separate. The frontend also changes servo-range synchronization, wizard settings, authentication loading, and busy-control handling.

Changes

Embedded UI and firmware delivery

Layer / File(s) Summary
Build and embed the UI
.gitignore, platformio.ini, scripts/embed_ui.py, ui/package.json, ui/vite.config.js, scripts/copy_assets.py, scripts/ui_pack.py, scripts/audio_pack.py, AGENTS.md, docs/flash.md, docs/settings.md
Vite builds a single-page UI, and a PlatformIO pre-build script compresses it into a generated C++ header. UI packing into LittleFS is removed. Audio image size counting now includes only files directly in the destination directory.
Serve the embedded page
src/http/index_page.cpp, src/http/index_page.h, src/http/routes.cpp, scripts/firmware_version.py, docs/api.md, docs/hardware/testing.md, ui/index.html, src/http/json.cpp
GET / sends the embedded gzip data with no-cache. LittleFS UI checks, version-mismatch responses, and /ui/* routes are removed. The API documentation no longer lists those routes.
Separate firmware and filesystem uploads
scripts/ota.py, docs/flash.md
The ota target uploads firmware only. The otafs target builds and uploads LittleFS. Documentation describes the two upload commands separately.

Frontend state handling

Layer / File(s) Summary
Synchronize servo and wizard ranges
ui/src/servo-ranges.js, ui/src/servo.js, ui/src/settings.js, ui/src/setup/calibration.js, ui/src/setup/wizard.js
Servo-range updates notify listeners. The servo page and calibration state respond to changes. The wizard uses shared range serialization and an OLED rotation helper.
Update frontend request and busy state
ui/src/shell.js, ui/src/status.js
Authentication loading uses apiGetJson. Busy mode toggles only controls that were enabled when busy mode began.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Refactor

Suggested reviewers: jamro



Merge Risk: ⚪ Minimal · up to 87e97

No confirmed issue from this change currently prevents merging after normal checks.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 4.17% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 12 files. (6 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the primary change: building the web UI with Vite and embedding its gzipped output in firmware.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.


Full details: Docstring Coverage

Explanation

Docstring coverage is 4.17% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 12 files. (6 skipped: 6 unsupported.)




  • Fix all pre-merge checks with AI
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jamro

jamro commented Oct 8, 2026

Copy link
Copy Markdown
Owner

I like this direction... embedding the gzipped UI in the firmware feels like a cleaner fit for Tiny Engineer and removes the firmware/LittleFS version-coupling problem entirely.

Since #60 has now been squash-merged, could you rebase this branch onto the new main so the PR only shows the embedded-UI delta?

One small thing I noticed while reviewing the build script: embed_ui.py uses source mtimes to decide whether the generated header is still valid, but ui/package.json is not included in that list. It would be good to include it as an input as well so changes to the build configuration can’t accidentally reuse a stale web_ui.h.

Other than that, the approach looks solid to me.

Mechazawa and others added 3 commits October 11, 2026 11:26
Vite builds ui/ into one self-contained page (vite-plugin-singlefile),
and scripts/embed_ui.py gzips it into a generated header that the
firmware serves with Content-Encoding: gzip. The UI always matches the
firmware it ships in, so LittleFS only holds audio again: the /ui/*
routes, the version stamp and check, and the filesystem-first OTA order
are gone. The pre-script skips Node when ui/ is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUaQRAXtYTdrXQ4C4us2V5
Busy state re-enables only the controls it disabled, so the wizard no
longer re-renders to undo it. Servo range changes notify the servo page
and the calibration step instead of every caller refreshing both, and
the servo_mins/servo_maxs format lives beside the ranges.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VUaQRAXtYTdrXQ4C4us2V5
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Mechazawa
Mechazawa force-pushed the task/embed-ui-in-firmware branch from cb53e8e to 87e9754 Compare October 11, 2026 09:28
@Mechazawa

Copy link
Copy Markdown
Contributor Author

Rebased onto main, so the PR only shows the embedded-UI delta now, and ui/package.json is an input in embed_ui.py. Thanks for the review!

@Mechazawa
Mechazawa marked this pull request as ready for review October 11, 2026 09:33

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔇 Additional comments (20)
ui/src/servo-ranges.js (1)

13-22: LGTM!

Also applies to: 34-37

ui/src/servo.js (1)

2-2: LGTM!

Also applies to: 12-16, 26-31, 40-40, 50-50, 88-90

ui/src/setup/calibration.js (1)

2-7: LGTM!

Also applies to: 128-130

ui/src/setup/wizard.js (1)

4-4: LGTM!

Also applies to: 32-36, 95-95, 108-108, 130-130

ui/src/shell.js (1)

158-158: LGTM!

ui/src/status.js (1)

6-6: LGTM!

Also applies to: 24-26, 28-28

scripts/embed_ui.py (2)

35-36: npm ci decision can miss a changed package.json.

The marker check compares only package-lock.json against node_modules/.package-lock.json. If a developer edits package.json and the lockfile together, the lockfile mtime rises and the check works. If the lockfile is unchanged but node_modules is missing a dependency, the build fails at npm run build. The failure is clear and recoverable, so no change is required. Keep this in mind if CI caches node_modules.


30-31: 🎯 Functional Correctness

Stale-header check does not cover all possible Vite inputs.

The checked-in UI currently contains no ui/public/ directory or extra top-level build inputs. The listed source paths cover the files present in this change, so the comment’s examples do not establish a current defect. The concern is conditional on adding additional files that Vite reads in the future.

ui/package.json (1)

27-27: LGTM!

ui/vite.config.js (1)

3-3: LGTM!

Also applies to: 25-27, 35-35

platformio.ini (1)

42-42: LGTM!

scripts/audio_pack.py (1)

278-278: LGTM!

Also applies to: 284-284, 287-287

AGENTS.md (2)

14-14: LGTM!


58-58: 📐 Maintainability & Code Quality

AGENTS.md:58 already identifies ui/index.html as the HTML index and lists the endpoint documentation there. The stale-path concern is contradicted, and no wording change is needed for this file.

docs/flash.md (1)

44-44: LGTM!

Also applies to: 58-58, 122-123, 130-130

docs/settings.md (1)

94-94: LGTM!

src/http/index_page.cpp (1)

3-8: LGTM!

docs/api.md (1)

9-9: LGTM!

Also applies to: 27-27

src/http/json.cpp (1)

49-49: LGTM!

scripts/ota.py-36-36 (1)

36-36: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Add a timeout to the subprocess.run calls.

Both calls run external tools with no timeout. If the OTA transfer or the buildfs run hangs, the PlatformIO target blocks forever. The old code had a bounded reboot wait. The new code has no bound.

Add a generous timeout, such as 300 seconds for the OTA upload. Use a longer timeout for the buildfs run. Catch subprocess.TimeoutExpired and exit with a clear message.

The ast-grep and Ruff command-injection hints are false positives here. The command list holds no shell string. The TINY_ENGINEER_URL host comes from the local developer environment and is passed as a single argument.

Proposed fix for the OTA upload call
-    subprocess.run(command, check=True)
+    try:
+        subprocess.run(command, check=True, timeout=300)
+    except subprocess.TimeoutExpired:
+        raise SystemExit("OTA upload timed out after 300 s")

Based on learnings: flag missing timeout arguments on subprocess.run() calls, recommend a sensible default such as 300 s, and handle subprocess.TimeoutExpired.

Also applies to: 46-49

Sources: Learnings, Linters/SAST tools


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0f4a43ab-a01a-49e0-8b92-e303c368d4d3
📥 Commits

Reviewing files that changed from the base of the PR and between 611b554 and 87e9754.

⛔ Files ignored due to path filters (2)
  • partitions.csv is excluded by !**/*.csv
  • ui/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (27)
  • .gitignore
  • AGENTS.md
  • docs/api.md
  • docs/flash.md
  • docs/hardware/testing.md
  • docs/settings.md
  • platformio.ini
  • scripts/audio_pack.py
  • scripts/copy_assets.py
  • scripts/embed_ui.py
  • scripts/firmware_version.py
  • scripts/ota.py
  • scripts/ui_pack.py
  • src/http/index_page.cpp
  • src/http/index_page.h
  • src/http/json.cpp
  • src/http/routes.cpp
  • ui/index.html
  • ui/package.json
  • ui/src/servo-ranges.js
  • ui/src/servo.js
  • ui/src/settings.js
  • ui/src/setup/calibration.js
  • ui/src/setup/wizard.js
  • ui/src/shell.js
  • ui/src/status.js
  • ui/vite.config.js
💤 Files with no reviewable changes (9)
  • ui/index.html
  • src/http/index_page.h
  • docs/hardware/testing.md
  • ui/src/settings.js
  • scripts/firmware_version.py
  • scripts/ui_pack.py
  • scripts/copy_assets.py
  • src/http/routes.cpp
  • .gitignore

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@Mechazawa

Copy link
Copy Markdown
Contributor Author

@coderabbitai re scripts/ota.py:36 (subprocess timeouts): not adding these. espota.py already has its own socket timeouts (10 s for the invitation and connection, 30 s per transfer chunk), so an upload fails rather than hangs. The 90 s bound it replaced was a wait for the robot to reboot between the filesystem and firmware uploads in the combined release. With the UI embedded in the firmware that two-step flow is gone, so there is no reboot to wait on.

@coderabbitai

coderabbitai Bot commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
fd -t f '^ota\.py$|^espota\.py$|^platformio\.ini$'
ast-grep outline scripts/ota.py --items all
nl -ba scripts/ota.py | sed -n '1,180p'
rg -n -C3 'platform|espota|ota|extra_scripts' platformio.ini

Length of output: 4612


@Mechazawa, thanks for clarifying. scripts/ota.py confirms that ota and otafs are separate upload targets. The former 90-second reboot wait does not apply to this flow.

The socket timeouts you described address stalled OTA communication. They are not a total upload deadline, but a separate subprocess deadline is not required for the concern I raised. I withdraw the timeout recommendation.


✏️ Learnings added
Learnt from: Mechazawa
URL: https://github.com/jamro/tiny-engineer/pull/68

Timestamp: 2026-10-11T11:23:43.742Z
Learning: In Tiny Engineer, scripts/ota.py uses the Arduino-ESP32 espota.py tool, which has socket timeouts of 10 seconds for invitation and connection and 30 seconds per transfer chunk, as clarified by the author. These are communication timeouts, not a total upload deadline. The former 90-second bound covered waiting for a reboot between filesystem and firmware uploads in the combined release flow. With the UI embedded in firmware, ota uploads firmware only and otafs uploads LittleFS separately, so that inter-upload reboot wait no longer applies. Do not recommend a subprocess deadline merely to restore the removed reboot-wait bound.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

Comment thread .gitignore Resolved
Comment thread scripts/copy_assets.py Resolved
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Mechazawa
Mechazawa requested a review from jamro October 11, 2026 12:15
@jamro

jamro commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks for the follow-up! the LittleFS cleanup (rmtree on stale dirs under data/) plus data/*/ in .gitignore address the shipping concerns from review.

CI looks good on my side. Merging...

@jamro
jamro merged commit c7afe96 into jamro:main Oct 11, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants