Repository navigation
Conversation
The aarch64 Docker images were built and tested under QEMU-user emulation, where the ASan-instrumented binary aborts at startup (CHECK failed: sanitizer_allocator_primary64.h) because QEMU's emulated address space cannot host libasan's fixed 64-bit allocator region. ASan was therefore disabled on arm64 via NO_SANITIZERS=2. Run the arm64 build/test on native ubuntu-24.04-arm runners instead of under QEMU, where ASan initialises normally, and drop arm64 from noasan_fallback_platforms so it builds with ASan again (ubuntu+debian). - publish_docker_matrix_base.yml: route linux/arm64/v8 to a native ubuntu-24.04-arm runner; skip the multiarch/qemu-user-static binfmt registration on that native runner (it is amd64-only and fails with "Exec format error" there, and is unneeded since arm64 containers run natively); lower vm.mmap_rnd_bits to 28 before the arm64 test as cheap CI-only insurance for older base-image libasan. - publish_docker_matrix_all.yml: remove linux/arm64/v8 from noasan_fallback_platforms (ubuntu, debian); narrow the AsanTest canaries to the platforms still under QEMU (armv7, ppc64le). - Makefile: drop the dead -DSANITIZER_CAN_USE_ALLOCATOR64=0 blocks (the define only affects libasan's own build, not the application) and document the real cause and fix.
Adds a minimal, standalone "listened" flag for library files, kept deliberately separate from contentJson_t (which describes a Tonie tag's content assignment - source, live, cache, cloud auth, ... - none of which applies to a plain library audio file): - New GET field "listened" on /api/fileIndexV2 for library entries. - New POST /api/fileSetListened to toggle it for an arbitrary file. - Sidecar (<file>.json) reads/writes preserve any existing foreign keys already in the file (e.g. from the "migrate to library" feature) instead of overwriting them. - New "cloud.autoMarkListenedOnSync" setting (on by default): marks a library-sourced tag's file as listened once a Toniebox actually downloads/plays it locally. - Fixes a pre-existing duplicate-key bug in fileIndexV2's directory listing branch (the "hide" field was being added once per scanned subentry instead of once per directory). - Orphaned sidecars (whose original file was deleted, e.g. by cache eviction) are intentionally left visible in listings rather than hidden, so they can be spotted and cleaned up manually. Pairs with the corresponding frontend changes in teddycloud_web (feature/listened branch).
…ative-runner Re-enable AddressSanitizer on aarch64 via native arm64 runners
Protects only the web interface (off by default), stores salted password hashes, and leaves Toniebox APIs unchanged.
httpSendResponseStreamUnsafe() serves a stream file while the encoder task is still appending to it. Once the reader catches up, fread() sets the stdio EOF indicator, and that indicator is sticky: every later fread() on the same FILE returns 0 without going to the fd again. The retry path in the send loop therefore spins in osDelayTask() forever and never sends another byte, although the file keeps growing. Seek to the current position before retrying, which clears the indicator. The taf_chapter_split path was unaffected because it already seeks on every iteration.
With encode.ffmpeg_stream_restart enabled, a range request on a stream is answered with the real file size instead of stream_max_size, so the box discards its cached prefix and requests the stream again from offset 0. teddyCloud started the encoder for that request anyway and waited for the sweep and prebuffer delays before answering, so every restart ran the full startup sequence twice and threw the first run away. Skip the encoder for that request and answer immediately. Fixes #407
The content handler ignored the tracks query, so a multi-track download could not return the chosen chapters.
- rename auth user routes to users/get, users/create, users/delete, users/updatePassword - drop the route ordering comment - README: recovery hint and warning against public exposure Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The multi-track export shells out to zip, which was only available in the build stage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mkdtemp() is POSIX-only and doesn't exist on MSVC, which broke the Windows build with an unresolved external symbol. Create the temp directory under settings.internal.cachedirfull using the existing cross-platform fs_ext helpers instead, and size the related path buffers to fit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The streaming pipeline was only documented by the one-line setting descriptions the web UI shows. Nothing explained how they interact, why playback is not live, or why a returning tag replays cached audio. Add docs/webradio-streaming.md covering the pipeline, the constraints the box imposes on it, the encode.* settings and where startup latency comes from.
fix: keep serving stream files past the first EOF
…coder-on-restart perf: skip encoding for range requests that force a box restart
Runs a program with an argv array (no shell), so callers don't need to build and escape a command-line string. Mirrors the existing osPopen per-platform split: fork/execvp/waitpid on POSIX, _spawnvp on Windows. Intended for #484 (export TAF tracks), which currently builds a `zip` command as a hand-escaped shell string. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…arness Moves the existing Python integration tests into tests/py/ and extracts the start-server/wait-ready/stop-server logic (duplicated once already between test_api_custom_json_with_server and #477's auth test) into tests/py/with_server.sh, shared by any test needing a running server. Adds tests/c/ for pure C unit tests that don't need a server: a small assert()-based runner plus a first real test for osSpawnvp. `make test` runs both suites; `make test_c` runs just the C side. Documents the layout in docs/architecture.md and CONTRIBUTING.md.
feat: track listened status for library files
CC3235 uses the Boxine CA and v1 API like CC3200/ESP32, so it must be GENERATION_TB1, not TB2. Fixes #483
… correct PR count for teddycloud_web
feat: add optional web UI login with cookie sessions
cert_load_ca allocated the CA certificate DER and never freed it, so every RSA certificate signed against the CA leaked it. It also leaked on each error path. Sanitizer builds report this and exit non-zero after --generate-client-cert, even though the certificate is written.
connection->private.authenticated is set true when a client certificate authenticates a request, but httpServerRequestCallback never resets it to false. Connection objects come from a static 32-slot pool (HTTP_SERVER_MAX_CONNECTIONS) handed out from index 0, so a slot that last served a certificate-authenticated request stays authenticated=true for whichever unrelated client's connection gets that slot next - skipping client-certificate verification entirely at the api_access_only + boxCertAuth gate. Reset it alongside api_access_only, the other per-request field this callback already refreshes, so every request starts from a clean slate before the certificate block re-establishes it if warranted.
fix(server): clear authenticated on every pooled connection reuse
with_server.sh now starts the server with --base_path in a copy of a template directory, so test runs no longer touch the dev config, data or certs. The sandbox is exported as TC_SANDBOX and removed afterwards; the config backup/restore is gone. - The template holds the generated server certs (minutes of RSA keygen). It is created once in bin/test-template, stamped with a hash of cert.c/cert.h, and regenerated only when those change. CI caches it with the same key. - Ports of 0 are picked free, so tests no longer collide with other services; TEDDYCLOUD_BASE_URL and the ports are exported. - test_hardening.py locates firmware/data through TC_SANDBOX instead of the repo tree, and the settings count check no longer assumes a populated dev config. - Ignore tmp/ for scratch notes.
make test_py starts a fresh sandboxed server per entry (each test file and each hardening class) and runs pytest against it. TESTS=<substr> selects entries, PYTEST_ARGS passes extra arguments. The per-file Makefile targets are removed; docs, docstrings and the CI install (python3-pytest) are updated. run_auth_pool_reuse.sh now generates the client cert with the sandbox base path instead of the dev one, so it no longer depends on the dev certificates.
tests/sim simulates a box: a client certificate signed by the sandbox CA (cached in the template, RSA keygen is slow), TLS without SNI, HTTPS API requests and raw RTNL frames (hand-rolled protobuf encoder, no extra dependency), plus an SSE listener. tests/py/test_box_sim.py checks box authentication, listing of new boxes, allowNewBox, delivery of RTNL log2/log3 events and that malformed RTNL input (zero/oversized length, garbage, huge/short field6) does not kill the server. with_server.sh: export TC_TEMPLATE, create the TB2 cert directories, make the server log line-buffered and print it when the test command fails.
tests/sim/profiles.py describes CC3200, CC3235, ESP32 (old/new user agent) and TB2: user agent, expected hardware detection, RTNL function codes and TLS parameters (version range, ciphers, SNI). The CC3200 connects with legacy TLS (assumed values until captured). Box applies the profile when connecting. test_box_profiles.py checks hardware detection from the user agent, the TLS handshake and each legacy cipher suite, and RTNL tag events with the hardware's function codes. Behaviour that needs a real capture (OTA, content, freshness, RTNL stream, TLS stack, original client certificate, TB2) is a documented skipped stub per profile.
tests/sim/upstream.py is a local HTTPS "Boxine cloud" (certificate signed by the sandbox CA, records requests, scriptable per test). test_cloud_proxy.py checks forwarding of /v1/time, the cloud-disabled and unreachable-cloud fallbacks, and that hostile redirect Location headers (very long, relative, empty, no URL) and redirect loops do not crash the server. Sandbox: start without the tonies.json auto-update (no network in tests), provide a client certificate and CA for the cloud client in the template, and print ASan reports / save the full server log when a test fails.
Request threads read settings strings without a lock while the main loop reloads the config (every config change, including a new box saving its overlay) and freed them: use-after-free, e.g. in web_request reading cloud.remote_hostname. Replaced strings, arrays and config paths are now parked and released after a grace period (settings_loop / shutdown). A reload keeps the old pointers and the option map valid until the new values are in, and the internal.*dirfull paths are built aside and swapped in. test_settings_reload_while_cloud_requests_run forces reloads during cloud requests and fails on the old code with the ASan report.
A user supplied URL (pic of a custom tonie, img_src of a box) was downloaded into the image cache and served back from /cache/, so any host reachable by the server (loopback, LAN, cloud metadata) could be read. Redirects of cloud requests were followed to any host as well. web_request now refuses non-cloud requests whose resolved address is not public (ipv4_is_public), checked on the address that is connected to. core.allowPrivateFetch (default off) allows private targets, e.g. for images hosted on a NAS. Tests: ipv4_is_public range edges; PrivateFetch checks that a local service is not reached by default and is when allowed.
A resumed TLS session carries no client certificate, so the box port and the MQTT server could not identify a box on it and answered 401. Both no longer use the session cache; the web port keeps it.
A box overlay created on first contact got its own core.certdir, but internal.certdirfull still pointed to the global certs/client until the next settings reload, so certificates uploaded for the box overwrote the global client.der. Uploading into a not yet existing directory created it but answered 500 without saving the file. Uploaded certificates now also take effect right away (only the client certificates of the box are reloaded).
The boxes send no SNI, so a proxy in front of several TeddyCloud instances can only route by IP or port. This patches the port in the ESP32 image (v5.231.0 to v5.237.0): the connect function hands host and port to esp_tls_conn_new_sync() and enables TLS only for 443, so the patch sets the port and keeps TLS on. Ports 1..32767; 443 restores the original code; patched images can be patched again. /api/esp32/patchFirmware?port=, --esp32-portpatch <FILE> --port <PORT>.
Adds tafHeader.lengthSeconds (fileIndexV2) and lengthSeconds (getTagIndex).
--generate-client-cert signs with the CA in certs/server below the working directory. On a fresh checkout (CI) that does not exist.
Encodes two white-noise tracks and checks tafHeader.lengthSeconds from fileIndexV2 (#537).
Bumps [actions/cache](https://github.com/actions/cache) from 4 to 6. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4...v6) --- updated-dependencies: - dependency-name: actions/cache dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
KeyPermissions512 ran `--generate-client-cert` without --base_path, so it only passed where the repo root happened to contain a CA. In CI the CA is only in bin/test-template, cert_load_ca failed and the command exited 1. Pass --base_path BASE_DIR like the other tests do. Also free the two path strings in cert_generate_mac when cert_generate_signed fails; AddressSanitizer reported them as leaked. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…break startup ubuntu:latest ships a default `ubuntu` account at uid/gid 1000, so `useradd -u 1000 teddy` fails on the duplicate uid and the `usermod` fallback can't fire because `teddy` was never created. Both errors were swallowed, leaving `gosu teddy` with nothing to switch to. Reuse whatever account already owns the requested ids and hand gosu/su-exec the numeric uid:gid directly -- neither needs a passwd entry to drop privileges to a specific id. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The CC3200 and ESP32 profiles were stubs or assumptions; they are now filled in from decrypted captures of real boxes on their original firmware (CC3200 3.1.0 BF2, 3.3.0, 3.4.0; ESP32 v5.233.0) and from giving a CC3200 (3.4.0) and an ESP32 (v5.233.0) wrong answers on purpose. Payloads that carry MACs, tonie UIDs or the WLAN name are not kept. New tests: - the captured ClientHello sent as it is: suite, extensions, ServerKeyExchange curve and signature, CertificateRequest - a full handshake with the box's signature algorithms (CC3200: CertificateVerify RSA/SHA-1) - the boot requests on one keep-alive connection, freshness checks in the boxes' sizes - every RTNL frame shape of the captures, cut into TLS records the way each box does it - content download, resume with Range and If-Range, connection end (close_notify or FIN) - what the boxes were seen to accept: claim status, header and freshness answer sizes, 404 for missing content - OTA passthrough with Boxine's answer format; the default cacheOta on, localOta off Documented xfails: - claim: the boxes close the connection after a 200 and keep it after a 204; the server sends 200 - POST /v1/log gets no answer; the box closes after a few seconds - a cloud /v1/time that is no number is passed on; a CC3200 that got one blinked red and switched off - a first RTNL record without a line feed waits for more data; the first record of a CC3200 on 3.1.0 BF2 has none - DHE-RSA is not offered (no DH parameters); the CC3200 offers enough other suites
…ime checked Seen on real boxes (CC3200 3.4.0, ESP32 v5.233.0) by giving them wrong answers on purpose: - after a claim answered with 200 both close the connection, after a 204 they keep it and use it for the next request. TeddyCloud's own claim answer is 204 now. - with RTNL refused they post /v1/log and close after a few seconds without an answer (CC3200: 2.9 s and 7.7 s). TeddyCloud answers 200 when it does not forward the log. - a CC3200 that got "teddycloud" as /v1/time blinked red and switched off. A cloud answer that is not only digits or earlier than 2020 (an error page, say) is replaced with the local time. The status line of the cloud answer is still passed on. Not seen on a CC3235 or a Toniebox 2.
Until now any client certificate whose common name looked like a box was accepted. Now: - Certificates issued by a TeddyCloud CA (TB1, TB2) are trusted. - Original Boxine certificates cannot be checked against a CA (the box only sends its own certificate, issued by "Boxine Factory SubCA nn", which is not available). If the original client.der of the box is uploaded, the box has to present exactly that certificate; without it, the first certificate a known box presents is pinned in toniebox.certPin (also with core.allowNewBox off, so nobody is locked out by the update). A box with a trusted certificate is pinned too, so no self-made certificate takes its place later. - core.boxCertAuth (default on) refuses boxes whose certificate does not match; off, they are only logged. - Result per box in internal.boxCertStatus and internal.boxCertSha256. - Same check for the MQTT server; a refused box gets CONNACK "not authorized". An unknown box id only counts as a box there while core.allowNewBox is on. The issuer check and the box id from the certificate subject are in box_cert.c, the check in box_cert_check.c. The TLS context keeps the SHA-256 fingerprint of the box certificate. Refs #85
The global setting raced with the box overlay (401 instead of 200 in about 1 of 8 runs). Setting it on the overlay of the box is reliable (0 of 26 runs failed).
Pick up tests/py/test_*.py and the test_hardening.py classes automatically, so adding a test no longer needs a Makefile edit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.