ci: notice when the container base falls behind Debian stable - #733
Draft
HuggeK wants to merge 9 commits into
Draft
ci: notice when the container base falls behind Debian stable#733HuggeK wants to merge 9 commits into
HuggeK wants to merge 9 commits into
Conversation
Go never resolves ".local" itself. net/conf.go routes those names to libc only when cgo is available, and every FTW build sets CGO_ENABLED=0, so the pure Go resolver is always selected: a configured "zap.local" became a unicast DNS query to the site router and failed. That holds on every base image and every libc, musl and glibc alike, so the container's distro was never the variable here. Add internal/mdnsresolve, which answers those names over multicast DNS, and route every driver transport through it: Modbus TCP, MQTT (driver and Home Assistant bridge), HTTP including the TLS-pinned client, WebSocket and raw TCP. Only ".local" names take the new path; literal IPs and ordinary DNS names dial exactly as before. Resolution runs per dial rather than once at startup, so a device that moves to a new DHCP lease is found again on the next reconnect with no config edit. Answers are cached for the record TTL clamped to 30-120s so reconnect loops cannot flood the LAN, and failures are cached for 5s so a still-booting device is retried soon. Failures log "mDNS resolution failed" and name the mechanism rather than surfacing as a generic dial error. The scanner's reverse PTR lookup moves into the same package so there is one mDNS implementation instead of two. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Core was alpine:3.22 and the updater sidecar was docker:27-cli (also alpine), while the optimizer was already python:3.12-slim-bookworm. That is two libcs and two security streams in one deployment, and two base rootfs blobs pulled per host. Put all three on debian:bookworm-slim so the layer is pulled once and tracked once. glibc additionally lets the image run ordinary prebuilt vendor binaries, which musl cannot, and a full userland makes on-site docker exec debugging practical. libnss-mdns is installed and wired into nsswitch.conf so .local resolves for glibc tools inside the container when an avahi socket is mounted; the FTW process does not depend on that, because a CGO_ENABLED=0 binary bypasses NSS entirely and resolves .local in Go instead. The binary stays fully static and still cross-compiles on the build platform, so only the small runtime layer is emulated for arm64. Measured: 79s for a full arm64 build, image 148MB (amd64 128MB, up from 53MB). wget is now installed explicitly and asserted by the boundary test. It is contractual rather than incidental: ftw-updater docker-execs it inside the core image to decide whether an update commits, and updaters already in the field will keep doing so, so the core image cannot stop shipping it. The boundary test's `^FROM alpine:` line is replaced by the invariant it was actually standing in for -- core must not build on a Python base -- and given a real error message. Also embed tzdata in the binary as a fallback. Production code reads time.Local, which silently degrades to UTC when zoneinfo is missing, and the tzdata tests skip rather than fail, so that regression would ship green. Verified uid 100 / gid 101 have no passwd entry on this base, and corrected the two comments that attributed them to an alpine adduser. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
bookworm is oldstable. Move all three images to trixie so the deployment tracks the suite that is actually receiving full security support, and so the base matches the Raspberry Pi OS release the SD image is built from (deploy/pi-gen/config: RELEASE=trixie). This also brings the optimizer along. Leaving it on a bookworm-derived python image would have split the shared base layer, which is the whole reason the images were aligned -- verified after the move that core, updater and optimizer resolve to one identical base layer digest. Verified on trixie: CVXPY 1.9.2 solves and highspy imports under Python 3.12.13; the updater's copied docker CLI 27.5.1 and compose plugin v2.33.0 both run; core has wget, the CA bundle, zoneinfo and an nsswitch.conf wired for mdns, and runs as uid 100 / gid 101. Pinned to the codename rather than a stable alias, so a major-version jump can never arrive silently on a rebuild. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
The images pin a Debian codename rather than a suite alias, so a major version jump can never arrive silently on a rebuild. The cost of pinning is that nothing notices when a new stable ships, and a suite quietly stops receiving full security support once it becomes oldstable -- which is exactly the state bookworm was already in. Truth comes from Debian's own Release file for the stable suite, not from registry tag listings: tags are noisy, rate-limited, and say nothing about which suite Debian considers stable. The check reads the pin out of the three Dockerfiles rather than hard- coding it, so it cannot drift from what actually ships, and it fails the run rather than reporting "current" if it could not reach Debian. It also flags the images having drifted apart, since a split pin silently costs the shared base layer they were aligned for. When a new stable is found it reports whether debian:<suite>-slim and the matching python:<ver>-slim-<suite> are actually published yet, because the python image can lag by days and moving core alone would split the pin. Follows the bundled-driver-pin workflow: one issue, rewritten each run and closed automatically once the pin is current, so a suite left alone for a year costs one thread rather than fifty-two. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
This was referenced Jul 31, 2026
The optimizer releases independently of core, and optimizer-release.yml skips the build entirely when an image already exists for the requested version, then asserts that the published image's org.opencontainers.image.revision label matches the commit being released. Changing Dockerfile.optimizer without moving the version therefore does not merely produce two different artifacts under one number -- it cannot ship at all. Releasing 1.3.2 from this commit would either reuse the bookworm image or fail metadata verification. Minor rather than patch: the Python surface is untouched, but the runtime underneath it moves a Debian major version, so consumers are getting a materially different artifact. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
@ The base move and the resolver were stacked in one branch, which made the resolver look like a consequence of choosing Debian. It is not: the two are independent, and the resolver is the one with the device-facing risk. Take the resolver back out so this branch is only the base move, and restate what libnss-mdns actually buys here — tools inside the image, not the FTW process, which is CGO_ENABLED=0 and never consults NSS. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com> @
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
This PR assumes the #731 route. #746 is an alternative to #731 + #728 —
one
alpine:3.22base for core and the updater, with.localresolvednatively in Go. If #746 is chosen instead, this check does not just become
redundant, it breaks.
scripts/check-debian-base.shreads a Debian pin out of all three Dockerfilesand treats disagreement between them as a failure (exit 1), because a split
pin is what silently un-shares the base layer. Under #746
DockerfileandDockerfile.updaterare Alpine, so two of those three reads come back emptyand the check fails on every run.
Reworking it for that route means scoping the Debian check to
Dockerfile.optimizer— the only image that cannot leave Debian, since CVXPYpublishes no musllinux wheels — and adding a separate Alpine currency check
for the other two. Worth doing either way, but it is a different script.
Safeguard requested alongside #731: something that notices when a newer Debian
stable ships and asks for the pin to move.
Blocked by #731 — merge that first. (#728 is now a sibling of this PR, not a dependency: the stack was reordered so the base move goes in ahead of the resolver.) The check reads the pin
out of the Dockerfiles, so it needs those to be on Debian first.
Why this is needed
#731 pins a codename (
debian:trixie-slim) rather than a suite alias likedebian:stable-slim, so a major-version jump can never arrive silently on somefuture rebuild. The cost of pinning is that nothing notices when a new stable
ships — and a suite quietly stops receiving full security support once it
becomes oldstable. That is not hypothetical: it is precisely the state
bookwormwas already in when this work started.How it decides
Truth comes from Debian's own
Releasefile for thestablesuite, not fromregistry tag listings — tags are noisy, rate-limited, and say nothing about which
suite Debian actually considers stable.
scripts/check-debian-base.shreads the pin out ofDockerfile,Dockerfile.updaterandDockerfile.optimizerrather than hard-coding it, sothe check cannot drift from what ships. Exit codes match the existing
sync-bundled-drivers.sh --behindconvention:Two deliberate behaviours:
optimizer share one base layer; if they drift apart that benefit is silently
lost, so the check reports it.
python:<ver>-slim-<suite>can lag
debian:<suite>-slimby days, and moving core alone would split thepin — so the report tells you to wait rather than half-move.
Verified by actually running it
All three paths, against live Debian infrastructure:
Simulating a future stable by pointing the check at the
testingsuite — noteit correctly caught that the python image is not published yet:
And an unreachable source correctly exits 2 rather than reporting "current".
The workflow
.github/workflows/debian-base-currency.yml— weekly, Mondays 07:00 UTC. Debianstable ships roughly every two years, so a daily check would be pure noise;
weekly still surfaces a release within days of it landing.
It follows
bundled-driver-pin.ymlexactly: one issue, rewritten on eachrun and closed automatically once the pin is current, so a suite left alone for
a year costs one thread rather than fifty-two. A network failure fails the run
loudly instead of quietly passing.
Also adds the script to the
bash -nsyntax gate incompose-migration-test.On the changeset gate
scripts/andMakefileare outside thechangeset-check.ymlallowlist, sothis would normally need its own entry. It does not get a redundant one: the
changeset in #731 already documents this workflow as part of the pinning
decision, and a CI watcher in the user-facing changelog would be noise. Because
this branch is stacked, that changeset is present in the diff and the gate is
satisfied.
🤖 Generated with Claude Code