diff --git a/.changeset/one-base-across-the-stack.md b/.changeset/one-base-across-the-stack.md new file mode 100644 index 00000000..dae4bc95 --- /dev/null +++ b/.changeset/one-base-across-the-stack.md @@ -0,0 +1,49 @@ +--- +"ftw": minor +--- + +Move the whole container stack to one base: Debian 13 "trixie" slim. + +Core was `alpine:3.22`, the updater sidecar was `docker:27-cli` (also alpine), +and the optimizer was `python:3.12-slim-bookworm` — two libcs, three unrelated +base images and three security streams in one deployment. Core, updater and +optimizer now all sit on `debian:trixie-slim`, verified to share a single base +layer, so a host pulls that rootfs once and there is one suite to track. It also +matches the Raspberry Pi OS release the SD image is built from. + +What the move buys: glibc, so the image can run ordinary prebuilt vendor +binaries, which musl cannot; a full userland, which makes `docker exec` +debugging on a live site practical; and `libnss-mdns` wired into +`/etc/nsswitch.conf`, so `.local` names resolve for glibc tools inside the +container — `getent hosts zap.local`, `curl`, `wget` — once an avahi socket is +mounted. Alpine has no NSS plugin mechanism at all, so none of that was +available before. + +That covers tools in the image, not the FTW process itself: the binary is built +`CGO_ENABLED=0` and so never consults NSS. It stays fully static and still +cross-compiles on the build platform. + +`wget` is now installed explicitly and asserted by the container boundary test. +It is contractual rather than incidental: `ftw-updater` `docker exec`s it inside +the core image to decide whether an update commits, and updaters already +deployed in the field will keep doing so. The zoneinfo database is also embedded +in the binary as a fallback, so a base image without `tzdata` can never silently +push `time.Local` to UTC and mis-time price and plan windows. + +Size: the core image grows from about 53 MB to about 133 MB uncompressed. The +updater is unchanged at about 203 MB — `docker:27-cli` was never a small image — +and now shares its base with the other two, so the per-host download is well +below the nominal sum. + +Data ownership is unchanged: the process still runs as the bare numeric uid 100 +/ gid 101, and gid 101 is still what grants access to the optimizer's socket. + +The optimizer is versioned independently and moves to 1.4.0, because its image +is a materially different artifact once the base changes. Its release workflow +verifies that a published image's revision label matches the commit it claims, +so the new base could not have shipped under the old version number at all. + +The base is pinned to the `trixie` codename rather than a `stable` alias so a +major-version jump can never arrive silently on a rebuild. A new scheduled +`debian base currency` workflow watches for a newer Debian stable and opens a +tracking issue when one appears. diff --git a/.changeset/resolve-local-device-names.md b/.changeset/resolve-local-device-names.md new file mode 100644 index 00000000..3f496148 --- /dev/null +++ b/.changeset/resolve-local-device-names.md @@ -0,0 +1,37 @@ +--- +"ftw": minor +--- + +Resolve device `.local` names, so devices can be configured by name instead of a DHCP-assigned IP. + +Go never resolves `.local` itself: it hands those names to libc only when cgo is +available, and FTW builds with `CGO_ENABLED=0`, so a configured `zap.local` +became a unicast DNS query to the site router and failed. That is true on every +base image and every libc — shipping `libnss-mdns` changes what `getent` and +`curl` resolve inside the container, not what this process resolves. + +FTW now asks the host's own mDNS responder, `avahi-daemon`, over its +simple-protocol socket — the same daemon and the same socket +`libnss_mdns4_minimal.so.2` uses, so a name resolves identically whether FTW +dials it or an operator checks it from a shell in the container. Where that +socket cannot be reached, FTW queries the LAN directly instead. The socket has +to be bind-mounted, and under the Home Assistant Supervisor an add-on cannot +mount arbitrary host paths at all, so the direct path is what makes the feature +work there; it is also the default in Compose, where mounting a host runtime +directory is left to the operator. A successful lookup logs which one answered. + +Every driver transport uses it — Modbus TCP, MQTT (driver and Home Assistant +bridge), HTTP including TLS-pinned clients, WebSocket and raw TCP. + +Resolution happens per dial rather than once at startup, so a device that moves +to a new DHCP lease is found again on the next reconnect without a config edit. +Answers are cached (30–120 s, following the record TTL where there is one) so +reconnect loops do not flood the LAN, and failures are cached briefly so a +device that is still booting is retried soon. A failed resolution logs +`mDNS resolution failed` and names the mechanism, instead of surfacing as a +generic dial error. + +Only `.local` names take this path; literal IPs and ordinary DNS names dial +exactly as before. Multicast still has to reach the LAN, which the Linux +Compose topology has via `network_mode: host`. Under `docker-compose.macos.yml` +the container is bridged, so configure devices by IP there. diff --git a/Dockerfile b/Dockerfile index 6526349c..7cc31194 100644 --- a/Dockerfile +++ b/Dockerfile @@ -22,8 +22,10 @@ RUN cd go && go mod download COPY go/ ./go/ -# Cross-compile by mapping TARGETARCH → GOARCH. CGO is off so the -# binary is fully static and runs on alpine without glibc. +# Cross-compile by mapping TARGETARCH → GOARCH. CGO stays off: the binary is +# fully static, so it is the runtime's *userland* we are choosing below, not a +# libc the binary depends on. Keeping CGO off is what lets the toolchain run +# natively on the build platform instead of under emulation. ARG TARGETOS=linux ARG TARGETARCH ARG VERSION=dev @@ -36,11 +38,42 @@ RUN cd go && \ go build -trimpath -ldflags="-s -w -X main.Version=${VERSION}" \ -o /out/ftw-backup ./cmd/ftw-backup # --- Runtime --------------------------------------------------------------- -FROM alpine:3.22 +# Debian trixie-slim — current Debian stable (13), and the same suite as +# Dockerfile.updater and Dockerfile.optimizer's python:3.12-slim-trixie. One +# rootfs blob is pulled once and shared by all three images, so the extra bytes +# over alpine are paid a single time per host rather than per image, and there +# is one libc and one security stream to track. It also matches the Raspberry Pi +# OS release the SD image is built from (deploy/pi-gen/config: RELEASE=trixie). +# +# Pinned to the codename, not `stable-slim`: a suite alias would silently jump +# major versions on some future rebuild. The `debian base currency` workflow +# watches for a new stable and files an issue, so the bump stays deliberate. +# +# glibc also means the image can run ordinary prebuilt vendor binaries, which +# musl cannot, and ships a full userland for on-site debugging. +FROM debian:trixie-slim -# HTTPS integrations and timezone-aware price/plan windows need these at -# runtime. BusyBox wget provides the health check without adding Python/curl. -RUN apk add --no-cache ca-certificates tzdata +# ca-certificates — HTTPS integrations. +# tzdata — timezone-aware price/plan windows. Without a zoneinfo tree +# time.Local silently degrades to UTC and mis-times plan +# boundaries with no error, so this is load-bearing. +# wget — the HEALTHCHECK below AND ftw-updater's readiness probe, +# which `docker exec`s wget in THIS image to decide whether +# an update commits. Debian slim ships neither wget nor curl, +# so it must be installed explicitly; dropping it would make +# every self-update fail its health gate and roll back. +# libnss-mdns — resolves ".local" for glibc programs in the image (getent, +# curl, wget), so in-container debugging agrees with the +# host. apt wires mdns4_minimal into /etc/nsswitch.conf on +# install. At run time it forwards to avahi-daemon over +# /run/avahi-daemon/socket, which must be bind-mounted; see +# docs/operations.md. It does nothing for the FTW binary +# itself, which is CGO_ENABLED=0 and therefore never consults +# NSS — see the note on the builder stage above. +RUN apt-get update && \ + apt-get install -y --no-install-recommends \ + ca-certificates tzdata wget libnss-mdns && \ + rm -rf /var/lib/apt/lists/* # Image layout: # /app/ftw binary (immutable, replaced on upgrade) @@ -79,7 +112,13 @@ EXPOSE 8080 # with each release. # # UID note: the process runs as uid 100 / gid 101 for compatibility with -# existing bind mounts. Named docker volumes inherit ownership from the image +# existing bind mounts. These are deliberately NUMERIC — no account is created +# and none is needed, which is why ENV HOME above is load-bearing. Verified on +# this base: uid 100 and gid 101 have no passwd/group entry, so ownership simply +# renders numerically. Do not renumber: gid 101 is what grants access to the +# optimizer's 0660 socket, and existing installs (and every flashed SD card) +# already own their data dir as 100:101. +# Named docker volumes inherit ownership from the image # automatically and just work. For HOST BIND MOUNTS, the host # directory must be owned by uid 100 (or world-writable) before the # container starts: diff --git a/Dockerfile.optimizer b/Dockerfile.optimizer index f8847321..1afb7f3c 100644 --- a/Dockerfile.optimizer +++ b/Dockerfile.optimizer @@ -1,11 +1,11 @@ # Independently releasable FTW mathematical optimizer. -FROM python:3.12-slim-bookworm AS build +FROM python:3.12-slim-trixie AS build COPY optimizer/ /src/optimizer/ RUN python -m venv /opt/venv && \ /opt/venv/bin/pip install --no-cache-dir /src/optimizer -FROM python:3.12-slim-bookworm +FROM python:3.12-slim-trixie ARG VERSION=dev ARG BUILD_SHA="" diff --git a/Dockerfile.updater b/Dockerfile.updater index e00b5d5c..b002dbbe 100644 --- a/Dockerfile.updater +++ b/Dockerfile.updater @@ -28,12 +28,24 @@ RUN cd go && \ -o /out/ftw-updater ./cmd/ftw-updater # --- Runtime --------------------------------------------------------------- -# docker:27-cli bundles the `docker` binary + `docker compose` plugin + -# ca-certificates + tzdata, which is exactly what we need to execute -# compose pulls against the host daemon. -FROM docker:27-cli +# Same debian:trixie-slim rootfs as Dockerfile and Dockerfile.optimizer, so +# the whole stack pulls one base layer instead of three. docker:27-cli is +# alpine-based and was the last thing keeping a second libc in the deployment. +# +# The docker CLI and the compose plugin are copied straight out of the official +# CLI image rather than installed from Docker's apt repository: it is the same +# upstream artifact, it pins the version explicitly, and it keeps apt out of the +# emulated arm64 layer. +FROM debian:trixie-slim + +RUN apt-get update && \ + apt-get install -y --no-install-recommends ca-certificates tzdata && \ + rm -rf /var/lib/apt/lists/* + +COPY --from=docker:27-cli /usr/local/bin/docker /usr/local/bin/docker +COPY --from=docker:27-cli /usr/local/libexec/docker/cli-plugins/docker-compose \ + /usr/local/libexec/docker/cli-plugins/docker-compose -# Compose pull hits GHCR over TLS; docker:cli already bundles ca-certs. COPY --from=builder /out/ftw-updater /usr/local/bin/ftw-updater COPY LICENSE NOTICE /usr/share/doc/ftw/ diff --git a/config.example.yaml b/config.example.yaml index 0999020f..ff047809 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -91,7 +91,7 @@ drivers: # battery_telemetry_only: true # capabilities: # http: - # allowed_hosts: ["zap.local"] # use the LAN IP if mDNS is unavailable + # allowed_hosts: ["zap.local"] # .local is resolved by FTW over mDNS # config: # host: zap.local # # meter_serial: p1m-... # optional; P1 is auto-selected diff --git a/deploy/pi-gen/stage-ftw/01-ftw-setup/00-run.sh b/deploy/pi-gen/stage-ftw/01-ftw-setup/00-run.sh index 9e8aaa23..581a86d8 100755 --- a/deploy/pi-gen/stage-ftw/01-ftw-setup/00-run.sh +++ b/deploy/pi-gen/stage-ftw/01-ftw-setup/00-run.sh @@ -69,8 +69,9 @@ EOF # records /opt/ftw automatically. COMPOSE_PROJECT_NAME # stays the literal `ftw`. # -# data/ is owned 100:101 because the in-container ftw user (alpine -# `adduser -S`) needs to own it before SQLite can create state.db. +# data/ is owned 100:101 because the container process runs as that bare +# numeric uid/gid (no account exists in the image) and needs to own the +# directory before SQLite can create state.db. # Same UID/GID mapping as scripts/install.sh. install -d -m 0755 "${ROOTFS_DIR}/opt/ftw" install -d -m 0755 -o 100 -g 101 "${ROOTFS_DIR}/opt/ftw/data" diff --git a/docker-compose.yml b/docker-compose.yml index 414b4fab..a4c75a2e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -73,6 +73,21 @@ services: # here to render update progress in the UI; it never writes to it. - update-ipc:/run/ftw-update - optimizer-ipc:/run/ftw-optimizer + # OPTIONAL — avahi-daemon's runtime directory. Host networking shares + # ports, not Unix sockets, so this is the only way in. + # + # With it, FTW asks the host's mDNS responder to resolve `.local` device + # names (and `getent hosts zap.local` works inside the container, via + # libnss-mdns). Without it FTW queries the LAN itself, which needs no + # host software and is why this stays commented out by default. + # + # Uncomment only if the host runs avahi-daemon — the Raspberry Pi image + # does. Mount the DIRECTORY, not the socket inside it: if the path is + # missing Docker creates it, and an empty directory there is harmless, + # whereas a directory created where the *socket* belongs stops + # avahi-daemon from ever starting. Restarting avahi after the container + # detaches the mount, so restart FTW too if you do. + # - /run/avahi-daemon:/run/avahi-daemon:ro ftw-optimizer: image: ghcr.io/srcfl/ftw-optimizer:${FTW_OPTIMIZER_IMAGE_TAG:-latest} diff --git a/docs/operations.md b/docs/operations.md index 8e95e98a..e6963223 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -180,6 +180,58 @@ Verify the broker address from the same network namespace as core, then inspect broker and driver logs. Device credentials and topic mappings belong to the driver configuration. +### A device `.local` name does not resolve + +FTW resolves `.local` device names itself; the OS resolver is not involved, +because a `CGO_ENABLED=0` Go binary never consults NSS. There are two places an +answer can come from, and the log line for a successful lookup says which: + +``` +resolved host over mDNS host=zap.local addr=192.168.1.42 via=avahi +``` + +**`via=avahi`** — the host's `avahi-daemon` answered over its socket. This is +preferred when available: avahi already holds a record cache, and it is the +same daemon that `getent hosts zap.local` inside the container goes through, so +FTW and your shell cannot disagree about an address. + +**`via=multicast`** — FTW queried the LAN directly. This is what happens when +the avahi socket is not mounted, which is the default, and it needs no host +software at all. + +Failures log `mDNS resolution failed`, and the message names both backends when +both were tried. + +Either way multicast has to reach the LAN, which the Linux Compose topology +provides through `network_mode: host`. Under `docker-compose.macos.yml` the +container is bridged and multicast does not reach the LAN, so configure devices +by IP there. + +#### Letting FTW use avahi + +Host networking shares ports, not Unix sockets, so avahi has to be bind-mounted +in. `docker-compose.yml` carries the line commented out: + +```yaml + volumes: + - /run/avahi-daemon:/run/avahi-daemon:ro +``` + +Mount the *directory*, not the socket file inside it. If the host path is +missing Docker creates it, and an empty directory is harmless — whereas a +directory created where the socket belongs stops `avahi-daemon` from ever +starting. Restarting avahi detaches the mount, so restart FTW after you do. + +This is an optimisation, not a requirement: device connectivity is unchanged +without it. What it does add is `libnss-mdns` working for ordinary tools in the +image — `getent hosts zap.local`, `curl`, `wget` — which is the quickest way to +check a name from inside the container. + +Under the Home Assistant add-on none of this applies: Supervisor mounts only a +fixed set of named paths, so the socket cannot be provided and FTW always +queries the LAN directly. The add-on runs with `host_network: true`, which is +what makes that work. + ### Configuration rejected Read the validation error, compare with diff --git a/docs/sourceful-zap.md b/docs/sourceful-zap.md index 75f98d60..30121f2c 100644 --- a/docs/sourceful-zap.md +++ b/docs/sourceful-zap.md @@ -76,6 +76,15 @@ go test ./internal/drivers -run 'Zap|zap' - not found: confirm Zap is on Wi-Fi and reachable at `http://zap.local/api/system` from the FTW host; +- `.local` name does not resolve: FTW resolves `.local` itself — via the host's + `avahi-daemon` where its socket is mounted, otherwise by querying the LAN — + rather than through the OS resolver, so it needs to be on the same L2 segment + as the device. That is the case with the Linux Compose topology + (`network_mode: host`); under `docker-compose.macos.yml` the container is + bridged and multicast does not reach the LAN, so configure the device by IP + there. The log line naming the failure is `mDNS resolution failed`, and a + successful lookup logs `via=avahi` or `via=multicast`; see + [docs/operations.md](operations.md); - no meter: inspect Zap's `/api/devices` and pin `meter_serial` when needed; - duplicate PV/battery: disable the overlapping Zap DER; - visible battery is not controlled: expected for the telemetry-only driver. diff --git a/go/cmd/ftw/main.go b/go/cmd/ftw/main.go index 509f7946..de9f05ca 100644 --- a/go/cmd/ftw/main.go +++ b/go/cmd/ftw/main.go @@ -25,6 +25,14 @@ import ( "syscall" "time" + // Embed the zoneinfo database as a fallback. The image's system tree still + // wins whenever it is present; this exists so time.Local can never silently + // degrade to UTC and mis-time price and plan windows if a base image ships + // without tzdata. The failure it guards against is invisible — production + // code reads time.Local, and the tzdata tests skip rather than fail — so the + // ~450 KB is worth it. + _ "time/tzdata" + "github.com/srcfl/ftw/go/internal/api" "github.com/srcfl/ftw/go/internal/arp" "github.com/srcfl/ftw/go/internal/battery" diff --git a/go/internal/drivers/lua.go b/go/internal/drivers/lua.go index bb62185c..f9cdc9d5 100644 --- a/go/internal/drivers/lua.go +++ b/go/internal/drivers/lua.go @@ -69,6 +69,8 @@ import ( "time" lua "github.com/yuin/gopher-lua" + + "github.com/srcfl/ftw/go/internal/mdnsresolve" ) // LuaDriver wraps a running Lua VM bound to a HostEnv. @@ -1150,8 +1152,16 @@ func registerHost(L *lua.LState, env *HostEnv) { return false, fmt.Sprintf("host %q (port %s) not in allowed_hosts", host, port) } + // Drivers routinely address a device by its ".local" name, which the + // stdlib resolver cannot answer. Clone the default transport so proxying, + // HTTP/2 and connection pooling are all unchanged — only the dial step + // differs, and only for ".local" hosts. + transport := net_http.DefaultTransport.(*net_http.Transport).Clone() + transport.DialContext = mdnsresolve.DialContext + httpClient := &net_http.Client{ - Timeout: 15 * time.Second, + Timeout: 15 * time.Second, + Transport: transport, CheckRedirect: func(req *net_http.Request, via []*net_http.Request) error { if len(via) >= 10 { return fmt.Errorf("stopped after 10 redirects") @@ -1180,7 +1190,10 @@ func registerHost(L *lua.LState, env *HostEnv) { // CA. Drivers WITHOUT a pin keep Go's default transport untouched, so // nothing about existing HTTP drivers changes. if pin := tlsPin; pin != "" { - tr := net_http.DefaultTransport.(*net_http.Transport).Clone() + // Clone the transport built above so the pinned client keeps the same + // mDNS-aware dialer — a pinned device is usually a local appliance + // addressed by its ".local" name, which is exactly the case that needs it. + tr := transport.Clone() tr.TLSClientConfig = &tls.Config{ // We replace chain/hostname verification with our own exact // fingerprint check below, so the stdlib check must be off. diff --git a/go/internal/drivers/tcp_cap.go b/go/internal/drivers/tcp_cap.go index 4153f3cc..cba42622 100644 --- a/go/internal/drivers/tcp_cap.go +++ b/go/internal/drivers/tcp_cap.go @@ -6,6 +6,8 @@ import ( "strings" "sync" "time" + + "github.com/srcfl/ftw/go/internal/mdnsresolve" ) // TCPCap is the host's raw TCP socket capability. One driver = one upstream @@ -93,7 +95,7 @@ func (n *netTCP) Open(addr string) error { return fmt.Errorf("tcp: %s", reason) } - conn, err := net.DialTimeout("tcp", addr, 10*time.Second) + conn, err := mdnsresolve.DialTimeout("tcp", addr, 10*time.Second) if err != nil { return fmt.Errorf("tcp dial: %w", err) } diff --git a/go/internal/drivers/ws_cap.go b/go/internal/drivers/ws_cap.go index 979d5685..bd23736d 100644 --- a/go/internal/drivers/ws_cap.go +++ b/go/internal/drivers/ws_cap.go @@ -9,6 +9,8 @@ import ( "time" "github.com/gorilla/websocket" + + "github.com/srcfl/ftw/go/internal/mdnsresolve" ) // gorillaWS is the production WSCap implementation. One per driver. @@ -73,6 +75,8 @@ func (g *gorillaWS) Open(url string, headers map[string]string) error { } dialer := *websocket.DefaultDialer dialer.HandshakeTimeout = 15 * time.Second + // ".local" hosts need mDNS; everything else falls through to a plain dial. + dialer.NetDialContext = mdnsresolve.DialContext if len(subprotocols) > 0 { dialer.Subprotocols = subprotocols } diff --git a/go/internal/ha/bridge.go b/go/internal/ha/bridge.go index c39e6f57..dd05fd77 100644 --- a/go/internal/ha/bridge.go +++ b/go/internal/ha/bridge.go @@ -12,6 +12,8 @@ import ( "encoding/json" "fmt" "log/slog" + "net" + "net/url" "sort" "strconv" "strings" @@ -22,6 +24,7 @@ import ( "github.com/srcfl/ftw/go/internal/config" "github.com/srcfl/ftw/go/internal/control" + "github.com/srcfl/ftw/go/internal/mdnsresolve" "github.com/srcfl/ftw/go/internal/telemetry" ) @@ -249,6 +252,13 @@ func (b *Bridge) connectAndStart(cfg *config.HomeAssistant, driverNames []string opts := paho.NewClientOptions(). AddBroker(fmt.Sprintf("tcp://%s:%d", cfg.Broker, cfg.Port)). + // A Home Assistant broker is very often reached as homeassistant.local, + // which the stdlib resolver cannot answer. See internal/mqtt for why a + // TCP-only replacement is complete here. + SetCustomOpenConnectionFn(func(uri *url.URL, o paho.ClientOptions) (net.Conn, error) { + d := mdnsresolve.Dialer{Dialer: net.Dialer{Timeout: o.ConnectTimeout}} + return d.Dial("tcp", uri.Host) + }). SetClientID("forty-two-watts-ha"). SetAutoReconnect(true). SetConnectRetry(true). diff --git a/go/internal/mdnsresolve/avahi.go b/go/internal/mdnsresolve/avahi.go new file mode 100644 index 00000000..01b281d6 --- /dev/null +++ b/go/internal/mdnsresolve/avahi.go @@ -0,0 +1,127 @@ +package mdnsresolve + +import ( + "bufio" + "context" + "fmt" + "net" + "net/netip" + "os" + "strings" +) + +// avahiSocket is avahi-daemon's simple-protocol socket. This is not a path we +// invented: libnss_mdns4_minimal.so.2 has the same string compiled into it, so +// asking here is exactly what `getent hosts foo.local` does one layer down. +// +// A var so tests can point at a socket they control. +var avahiSocket = "/run/avahi-daemon/socket" + +// avahiDial is swappable in tests. +var avahiDial = func(ctx context.Context, path string) (net.Conn, error) { + var d net.Dialer + return d.DialContext(ctx, "unix", path) +} + +// avahiTTL is how long an avahi answer is cached here. The protocol carries no +// TTL — avahi keeps its own record cache and re-queries the LAN when its copy +// ages out, so this number only decides how often we ask it, not how fresh the +// answer is. It matches the floor used for a raw multicast answer. +const avahiTTL = minTTL + +// avahiAvailable reports whether the socket is present and is really a socket. +// +// The type check earns its place: the socket has to be bind-mounted into the +// container, and when the host path does not exist Docker helpfully creates a +// *directory* at the mount point instead. That looks present to a bare +// os.Stat, connects to nothing, and is the single most likely misconfiguration +// on this path. +// A var so tests can assert both branches without needing a real socket on +// whatever platform `go test` is running on. +var avahiAvailable = func() bool { + fi, err := os.Stat(avahiSocket) + return err == nil && fi.Mode()&os.ModeSocket != 0 +} + +type avahiResult struct { + addr netip.Addr + err error +} + +// avahiLookup asks avahi-daemon to resolve name, preferring IPv4. +// +// The two address families are asked concurrently on separate connections +// because a name that exists in one family and not the other costs a full +// avahi resolve timeout to answer negatively — serialising them would put that +// wait in front of every dial to a v4-only device. IPv4 is still the answer +// preferred when both arrive. +func avahiLookup(ctx context.Context, name string) ([]netip.Addr, error) { + v4 := make(chan avahiResult, 1) + v6 := make(chan avahiResult, 1) + go func() { v6 <- avahiResolve(ctx, "RESOLVE-HOSTNAME-IPV6", name) }() + go func() { v4 <- avahiResolve(ctx, "RESOLVE-HOSTNAME-IPV4", name) }() + + r4 := <-v4 + if r4.err == nil { + return []netip.Addr{r4.addr}, nil + } + if r6 := <-v6; r6.err == nil { + return []netip.Addr{r6.addr}, nil + } + return nil, fmt.Errorf("avahi: %s: %w", name, r4.err) +} + +// avahiResolve runs one request/response exchange over the socket. +// +// The wire format is a single line each way. A success is +// +// +
+// +// and a failure is "- ". There is no framing, no length +// prefix and nothing to decode, which is the point of using it: no DNS wire +// format is parsed anywhere on this path. +func avahiResolve(ctx context.Context, command, name string) avahiResult { + conn, err := avahiDial(ctx, avahiSocket) + if err != nil { + return avahiResult{err: fmt.Errorf("connect %s: %w", avahiSocket, err)} + } + defer conn.Close() + + if deadline, ok := ctx.Deadline(); ok { + _ = conn.SetDeadline(deadline) + } + + if _, err := fmt.Fprintf(conn, "%s %s\n", command, name); err != nil { + return avahiResult{err: fmt.Errorf("write request: %w", err)} + } + + scanner := bufio.NewScanner(conn) + if !scanner.Scan() { + if err := scanner.Err(); err != nil { + return avahiResult{err: fmt.Errorf("read reply: %w", err)} + } + return avahiResult{err: fmt.Errorf("no reply")} + } + return parseAvahiReply(scanner.Text()) +} + +func parseAvahiReply(line string) avahiResult { + fields := strings.Fields(line) + if len(fields) == 0 { + return avahiResult{err: fmt.Errorf("empty reply")} + } + if fields[0] != "+" { + // "- 15 Timeout reached" and friends. Hand the daemon's own wording + // back rather than inventing one, so the log says what avahi said. + return avahiResult{err: fmt.Errorf("%s", strings.TrimSpace(strings.TrimPrefix(line, "-")))} + } + // +
+ if len(fields) < 5 { + return avahiResult{err: fmt.Errorf("short reply %q", line)} + } + addr, err := netip.ParseAddr(fields[4]) + if err != nil { + return avahiResult{err: fmt.Errorf("unparsable address %q", fields[4])} + } + return avahiResult{addr: addr.Unmap()} +} diff --git a/go/internal/mdnsresolve/avahi_test.go b/go/internal/mdnsresolve/avahi_test.go new file mode 100644 index 00000000..236b95c7 --- /dev/null +++ b/go/internal/mdnsresolve/avahi_test.go @@ -0,0 +1,214 @@ +package mdnsresolve + +import ( + "bufio" + "context" + "errors" + "io" + "net" + "net/netip" + "strings" + "testing" + "time" + + "golang.org/x/net/dns/dnsmessage" +) + +// fakeAvahi stands in for avahi-daemon. reply is handed the command and the +// name and returns the line the daemon would write back; returning "" makes it +// hang up without answering. +// +// net.Pipe rather than a real unix socket: the protocol is what is under test, +// and the test then runs the same way on every platform the suite runs on. +func fakeAvahi(t *testing.T, reply func(command, name string) string) { + t.Helper() + origAvail, origDial := avahiAvailable, avahiDial + avahiAvailable = func() bool { return true } + avahiDial = func(ctx context.Context, path string) (net.Conn, error) { + client, server := net.Pipe() + go func() { + defer server.Close() + sc := bufio.NewScanner(server) + if !sc.Scan() { + return + } + fields := strings.Fields(sc.Text()) + if len(fields) < 2 { + return + } + if out := reply(fields[0], fields[1]); out != "" { + _, _ = io.WriteString(server, out+"\n") + } + }() + return client, nil + } + t.Cleanup(func() { + avahiAvailable, avahiDial = origAvail, origDial + Flush() + }) +} + +func TestParseAvahiReply(t *testing.T) { + cases := []struct { + name string + line string + want string // empty means the reply must be rejected + }{ + // The exact shape avahi-daemon returns, confirmed against the daemon. + {"ipv4", "+ 2 0 zap.local 192.168.1.42", "192.168.1.42"}, + {"ipv6", "+ 2 1 zap.local fe80::1", "fe80::1"}, + // A v4-mapped answer must dial as plain IPv4. + {"v4 mapped", "+ 2 1 zap.local ::ffff:192.168.1.42", "192.168.1.42"}, + {"failure", "- 15 Timeout reached", ""}, + {"empty", "", ""}, + {"truncated", "+ 2 0 zap.local", ""}, + {"unparsable address", "+ 2 0 zap.local not-an-address", ""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + got := parseAvahiReply(c.line) + if c.want == "" { + if got.err == nil { + t.Fatalf("accepted %q, got %v", c.line, got.addr) + } + return + } + if got.err != nil { + t.Fatalf("rejected %q: %v", c.line, got.err) + } + if got.addr.String() != c.want { + t.Fatalf("addr = %v, want %v", got.addr, c.want) + } + }) + } +} + +// A failure line must carry avahi's own wording, so the log says what the +// daemon said rather than something this package made up. +func TestParseAvahiReplyKeepsDaemonWording(t *testing.T) { + got := parseAvahiReply("- 15 Timeout reached") + if got.err == nil { + t.Fatal("expected an error") + } + if !strings.Contains(got.err.Error(), "Timeout reached") { + t.Fatalf("error %q dropped the daemon's message", got.err) + } +} + +func TestLookupPrefersAvahi(t *testing.T) { + Flush() + fakeAvahi(t, func(command, name string) string { + if command == "RESOLVE-HOSTNAME-IPV4" && name == "inverter.local" { + return "+ 2 0 inverter.local 192.168.1.42" + } + return "- 15 Timeout reached" + }) + + orig := listenPacket + listenPacket = func() (*net.UDPConn, error) { + t.Error("queried the LAN directly even though avahi answered") + return nil, errors.New("should not be called") + } + t.Cleanup(func() { listenPacket = orig }) + + addrs, err := Lookup(context.Background(), "inverter.local") + if err != nil { + t.Fatalf("Lookup: %v", err) + } + if len(addrs) != 1 || addrs[0] != netip.MustParseAddr("192.168.1.42") { + t.Fatalf("addrs = %v, want [192.168.1.42]", addrs) + } +} + +// IPv4 wins when both families answer, so the address a device is dialled on +// does not depend on which goroutine happened to finish first. +func TestAvahiLookupPrefersIPv4(t *testing.T) { + fakeAvahi(t, func(command, name string) string { + if command == "RESOLVE-HOSTNAME-IPV6" { + return "+ 2 1 dual.local fe80::1" + } + return "+ 2 0 dual.local 192.168.1.7" + }) + + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + addrs, err := avahiLookup(ctx, "dual.local") + if err != nil { + t.Fatalf("avahiLookup: %v", err) + } + if len(addrs) != 1 || addrs[0] != netip.MustParseAddr("192.168.1.7") { + t.Fatalf("addrs = %v, want [192.168.1.7]", addrs) + } +} + +// A device that only advertises IPv6 still resolves. +func TestAvahiLookupFallsBackToIPv6(t *testing.T) { + fakeAvahi(t, func(command, name string) string { + if command == "RESOLVE-HOSTNAME-IPV6" { + return "+ 2 1 v6only.local fe80::2" + } + return "- 15 Timeout reached" + }) + + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + addrs, err := avahiLookup(ctx, "v6only.local") + if err != nil { + t.Fatalf("avahiLookup: %v", err) + } + if len(addrs) != 1 || addrs[0] != netip.MustParseAddr("fe80::2") { + t.Fatalf("addrs = %v, want [fe80::2]", addrs) + } +} + +// A socket that is present but unhelpful — daemon starting, wedged, or simply +// without the record — must not be a dead end. +func TestLookupFallsBackToMulticastWhenAvahiFails(t *testing.T) { + Flush() + fakeAvahi(t, func(command, name string) string { return "- 15 Timeout reached" }) + startResponder(t, []dnsmessage.Resource{aResource(t, "inverter.local.", [4]byte{10, 0, 0, 5}, 60)}) + + addrs, err := Lookup(context.Background(), "inverter.local") + if err != nil { + t.Fatalf("Lookup: %v", err) + } + if len(addrs) != 1 || addrs[0] != netip.MustParseAddr("10.0.0.5") { + t.Fatalf("addrs = %v, want [10.0.0.5]", addrs) + } +} + +// When neither backend answers, the error has to say that avahi was asked too +// — "there is no avahi" and "avahi said no" call for different fixes. +func TestLookupErrorNamesBothBackends(t *testing.T) { + Flush() + fakeAvahi(t, func(command, name string) string { return "- 15 Timeout reached" }) + startResponder(t, nil) // never answers + + ctx, cancel := context.WithTimeout(context.Background(), 150*time.Millisecond) + defer cancel() + _, err := Lookup(ctx, "missing.local") + if err == nil { + t.Fatal("expected a failure") + } + if !strings.Contains(err.Error(), "avahi") { + t.Fatalf("error %q does not mention avahi", err) + } +} + +// A socket that does not exist must not be dialled at all: the probe is what +// keeps a stray connect attempt out of every lookup on a host without avahi. +func TestAvahiSkippedWhenSocketAbsent(t *testing.T) { + Flush() + origAvail, origDial := avahiAvailable, avahiDial + avahiAvailable = func() bool { return false } + avahiDial = func(ctx context.Context, path string) (net.Conn, error) { + t.Error("dialled the avahi socket although it is not available") + return nil, errors.New("should not be called") + } + t.Cleanup(func() { avahiAvailable, avahiDial = origAvail, origDial }) + + startResponder(t, []dnsmessage.Resource{aResource(t, "inverter.local.", [4]byte{10, 0, 0, 6}, 60)}) + if _, err := Lookup(context.Background(), "inverter.local"); err != nil { + t.Fatalf("Lookup: %v", err) + } +} diff --git a/go/internal/mdnsresolve/mdnsresolve.go b/go/internal/mdnsresolve/mdnsresolve.go new file mode 100644 index 00000000..07d8d907 --- /dev/null +++ b/go/internal/mdnsresolve/mdnsresolve.go @@ -0,0 +1,231 @@ +// Package mdnsresolve resolves RFC 6762 ".local" host names for outbound +// connections, and provides a dialer that uses it. +// +// Go will not do this itself. net/conf.go routes a ".local" lookup to libc +// only when cgo is available, and every FTW build sets CGO_ENABLED=0, so the +// pure Go resolver is always selected: it reads /etc/resolv.conf and sends a +// *unicast* query to the site router, which has no idea what "inverter.local" +// is. That holds on every base image and every libc, glibc included — the +// image shipping libnss-mdns changes what `getent` and `curl` can resolve +// inside the container, not what this process can. +// +// # Where the answer comes from +// +// First choice is to ask the machine's own mDNS responder, avahi-daemon, over +// its simple-protocol socket. One line out, one line back, no DNS wire format +// decoded here — and it is the same daemon over the same socket that +// libnss_mdns4_minimal.so.2 uses, so a name resolves to the same address +// whether FTW dials it or an operator runs `getent hosts zap.local` in the +// container. avahi also holds a record cache, so a repeat lookup usually costs +// no LAN traffic at all. +// +// Second choice, in multicast.go, is to query the LAN directly. It exists +// because the socket cannot always be reached: it has to be bind-mounted, and +// under the Home Assistant Supervisor an add-on cannot mount arbitrary host +// paths at all — only a fixed set of named ones. FTW ships as an add-on, so a +// resolver that required that socket would simply not work there. The fallback +// also covers any Docker host that does not run avahi. +// +// Only ".local" names take either path. Literal IPs and ordinary DNS names are +// handed straight to the standard dialer. +package mdnsresolve + +import ( + "context" + "fmt" + "log/slog" + "net" + "net/netip" + "strings" + "sync" + "time" +) + +// now is swappable so cache-expiry tests do not have to sleep. +var now = time.Now + +const ( + // queryTimeout bounds each backend separately, so falling back costs at + // most two of these rather than an unbounded wait. It is long enough for a + // sleepy device and short enough that a driver dial does not stall a + // control tick. + queryTimeout = 900 * time.Millisecond + + // A responder's TTL is advisory here. The floor stops a device that + // advertises a very short TTL from turning every Modbus reconnect into a + // multicast storm; the ceiling keeps a DHCP move from taking effect + // arbitrarily late, which is the whole point of binding by name. + minTTL = 30 * time.Second + maxTTL = 120 * time.Second + + // negativeTTL is deliberately short: a device that was off when we first + // looked should become reachable soon after it boots. + negativeTTL = 5 * time.Second +) + +type cacheEntry struct { + addrs []netip.Addr // empty means a cached negative answer + expires time.Time +} + +var ( + cacheMu sync.Mutex + cache = map[string]cacheEntry{} +) + +// IsLocal reports whether host is a ".local" name that mDNS should resolve. +// A literal IP is never one, so a configured "192.168.1.5" keeps the plain +// dial path and never touches the network for resolution. +func IsLocal(host string) bool { + if host == "" || net.ParseIP(host) != nil { + return false + } + return strings.HasSuffix(strings.ToLower(strings.TrimSuffix(host, ".")), ".local") +} + +func canonical(name string) string { + return strings.ToLower(strings.TrimSuffix(name, ".")) +} + +func cacheLookup(key string) ([]netip.Addr, bool) { + cacheMu.Lock() + defer cacheMu.Unlock() + entry, ok := cache[key] + if !ok || now().After(entry.expires) { + return nil, false + } + return entry.addrs, true +} + +func cacheStore(key string, addrs []netip.Addr, ttl time.Duration) { + cacheMu.Lock() + defer cacheMu.Unlock() + cache[key] = cacheEntry{addrs: addrs, expires: now().Add(ttl)} +} + +// Flush drops every cached answer. Tests use it; nothing in production does. +func Flush() { + cacheMu.Lock() + defer cacheMu.Unlock() + cache = map[string]cacheEntry{} +} + +// Lookup resolves a ".local" name to its advertised addresses, asking +// avahi-daemon first and the LAN directly if that is not answerable. +func Lookup(ctx context.Context, name string) ([]netip.Addr, error) { + key := canonical(name) + if addrs, ok := cacheLookup(key); ok { + if len(addrs) == 0 { + return nil, fmt.Errorf("no mDNS responder for %s (cached)", name) + } + return addrs, nil + } + + addrs, ttl, via, err := resolve(ctx, key) + if err != nil || len(addrs) == 0 { + cacheStore(key, nil, negativeTTL) + if err == nil { + err = fmt.Errorf("no mDNS responder for %s", name) + } + return nil, err + } + + cacheStore(key, addrs, ttl) + // Logged on a cache miss only, so this is at most one line per TTL per + // device rather than one per reconnect. "via" is here because the two + // backends fail for completely different reasons, and an operator reading + // a support report needs to know which one was in play. + slog.Info("resolved host over mDNS", "host", key, "addr", addrs[0].String(), "ttl", ttl, "via", via) + return addrs, nil +} + +// resolve tries avahi, then a direct query. An avahi socket that is present +// but unhelpful still falls through: a daemon that is starting up, wedged or +// misconfigured should degrade to a slower answer, not to no answer. +func resolve(ctx context.Context, key string) ([]netip.Addr, time.Duration, string, error) { + var avahiErr error + if avahiAvailable() { + actx, cancel := context.WithTimeout(ctx, queryTimeout) + addrs, err := avahiLookup(actx, key) + cancel() + if err == nil && len(addrs) > 0 { + return addrs, avahiTTL, "avahi", nil + } + avahiErr = err + } + + addrs, ttl, err := queryAddrs(ctx, key) + if err != nil && avahiErr != nil { + // Both were tried; report both, because "avahi said no and the LAN + // said nothing" and "there is no avahi and the LAN said nothing" call + // for different fixes. + return nil, 0, "", fmt.Errorf("%w (avahi: %v)", err, avahiErr) + } + if err != nil { + return nil, 0, "", err + } + return addrs, ttl, "multicast", nil +} + +// Dialer dials TCP addresses, resolving ".local" host names over mDNS first. +// The embedded net.Dialer carries timeout and keep-alive; anything that is not +// a ".local" name is handed straight to it. +// +// Resolution happens per dial, not once at startup. That is what makes binding +// a device by name survive a DHCP lease change: callers that rebuild their +// connection from the original address string pick up the new IP on reconnect. +type Dialer struct { + net.Dialer +} + +// DialContext resolves address if it names a ".local" host, then dials it. +func (d *Dialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) { + host, port, err := net.SplitHostPort(address) + if err != nil || !IsLocal(host) { + return d.Dialer.DialContext(ctx, network, address) + } + + addrs, err := Lookup(ctx, host) + if err != nil { + // Name the mechanism. Without this the operator sees a bare dial + // failure and has no way to tell that resolution was the reason. + slog.Warn("mDNS resolution failed; check the device is on this LAN and the container uses host networking", + "host", host, "err", err) + return nil, fmt.Errorf("resolve %s over mDNS: %w", host, err) + } + + var firstErr error + for _, a := range addrs { + conn, err := d.Dialer.DialContext(ctx, network, net.JoinHostPort(a.String(), port)) + if err == nil { + return conn, nil + } + if firstErr == nil { + firstErr = err + } + } + return nil, fmt.Errorf("dial %s over mDNS: %w", host, firstErr) +} + +// Dial is the context-free form, for callers that have no context to pass. +func (d *Dialer) Dial(network, address string) (net.Conn, error) { + ctx := context.Background() + if d.Timeout > 0 { + var cancel context.CancelFunc + ctx, cancel = context.WithTimeout(ctx, d.Timeout) + defer cancel() + } + return d.DialContext(ctx, network, address) +} + +// DialContext dials with default settings. +func DialContext(ctx context.Context, network, address string) (net.Conn, error) { + var d Dialer + return d.DialContext(ctx, network, address) +} + +// DialTimeout mirrors net.DialTimeout with mDNS resolution added. +func DialTimeout(network, address string, timeout time.Duration) (net.Conn, error) { + d := Dialer{Dialer: net.Dialer{Timeout: timeout}} + return d.Dial(network, address) +} diff --git a/go/internal/mdnsresolve/mdnsresolve_test.go b/go/internal/mdnsresolve/mdnsresolve_test.go new file mode 100644 index 00000000..65a6eaf3 --- /dev/null +++ b/go/internal/mdnsresolve/mdnsresolve_test.go @@ -0,0 +1,290 @@ +package mdnsresolve + +import ( + "context" + "errors" + "net" + "net/netip" + "strings" + "testing" + "time" + + "golang.org/x/net/dns/dnsmessage" +) + +func TestIsLocal(t *testing.T) { + cases := []struct { + host string + want bool + }{ + {"inverter.local", true}, + {"INVERTER.LOCAL", true}, + {"inverter.local.", true}, + {"zap.local", true}, + // A literal address must never trigger a lookup. + {"192.168.1.5", false}, + {"::1", false}, + {"example.com", false}, + {"localhost", false}, + {"local", false}, + {"notlocal", false}, + {"", false}, + } + for _, c := range cases { + if got := IsLocal(c.host); got != c.want { + t.Errorf("IsLocal(%q) = %v, want %v", c.host, got, c.want) + } + } +} + +func mustDNSName(t *testing.T, s string) dnsmessage.Name { + t.Helper() + n, err := dnsmessage.NewName(s) + if err != nil { + t.Fatalf("NewName(%q): %v", s, err) + } + return n +} + +func aResource(t *testing.T, name string, ip [4]byte, ttl uint32) dnsmessage.Resource { + t.Helper() + return dnsmessage.Resource{ + Header: dnsmessage.ResourceHeader{ + Name: mustDNSName(t, name), Type: dnsmessage.TypeA, + Class: dnsmessage.ClassINET, TTL: ttl, + }, + Body: &dnsmessage.AResource{A: ip}, + } +} + +func packAnswer(t *testing.T, qname string, answers []dnsmessage.Resource) []byte { + t.Helper() + msg := dnsmessage.Message{ + Header: dnsmessage.Header{Response: true, Authoritative: true}, + Questions: []dnsmessage.Question{{Name: mustDNSName(t, qname), Type: dnsmessage.TypeA, Class: dnsmessage.ClassINET}}, + Answers: answers, + } + packet, err := msg.Pack() + if err != nil { + t.Fatalf("pack: %v", err) + } + return packet +} + +func TestParseAddrAnswer(t *testing.T) { + qname := "inverter.local." + packet := packAnswer(t, qname, []dnsmessage.Resource{aResource(t, qname, [4]byte{192, 168, 1, 42}, 60)}) + + addrs, ttl, ok := parseAddrAnswer(packet, qname) + if !ok { + t.Fatal("parseAddrAnswer did not accept a valid answer") + } + if len(addrs) != 1 || addrs[0].String() != "192.168.1.42" { + t.Fatalf("addrs = %v, want [192.168.1.42]", addrs) + } + if ttl != 60*time.Second { + t.Fatalf("ttl = %v, want 60s", ttl) + } + + // An answer for a different name must be ignored. + if _, _, ok := parseAddrAnswer(packet, "other.local."); ok { + t.Fatal("accepted an answer for a different name") + } + // Garbage must not panic or resolve. + if _, _, ok := parseAddrAnswer([]byte{1, 2, 3}, qname); ok { + t.Fatal("accepted a malformed packet") + } +} + +func TestParseAddrAnswerClampsTTL(t *testing.T) { + qname := "inverter.local." + for _, c := range []struct { + name string + ttl uint32 + want time.Duration + }{ + // A device advertising a 1 s TTL must not make every Modbus reconnect + // re-query the LAN. + {"below floor", 1, minTTL}, + // A very long TTL must not outlive a DHCP move. + {"above ceiling", 86400, maxTTL}, + {"inside range", 90, 90 * time.Second}, + } { + t.Run(c.name, func(t *testing.T) { + packet := packAnswer(t, qname, []dnsmessage.Resource{aResource(t, qname, [4]byte{10, 0, 0, 1}, c.ttl)}) + _, ttl, ok := parseAddrAnswer(packet, qname) + if !ok { + t.Fatal("answer rejected") + } + if ttl != c.want { + t.Fatalf("ttl = %v, want %v", ttl, c.want) + } + }) + } +} + +// disableAvahi forces the fallback path. Without it these tests would behave +// differently on a developer machine that happens to run avahi-daemon. +func disableAvahi(t *testing.T) { + t.Helper() + orig := avahiAvailable + avahiAvailable = func() bool { return false } + t.Cleanup(func() { avahiAvailable = orig }) +} + +// startResponder points the package at a loopback UDP socket that answers one +// query, so the real send/parse path is exercised without touching the LAN. +func startResponder(t *testing.T, answers []dnsmessage.Resource) { + t.Helper() + rc, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0}) + if err != nil { + t.Fatalf("listen responder: %v", err) + } + + origAddr, origListen := mdnsAddr, listenPacket + mdnsAddr = rc.LocalAddr().(*net.UDPAddr) + listenPacket = func() (*net.UDPConn, error) { + return net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1)}) + } + + done := make(chan struct{}) + go func() { + defer close(done) + buf := make([]byte, 1500) + _ = rc.SetReadDeadline(time.Now().Add(2 * time.Second)) + n, from, err := rc.ReadFromUDP(buf) + if err != nil { + return + } + if answers == nil { + return // silent responder: exercises the negative path + } + var p dnsmessage.Parser + hdr, err := p.Start(buf[:n]) + if err != nil { + return + } + q, err := p.Question() + if err != nil { + return + } + resp := dnsmessage.Message{ + Header: dnsmessage.Header{ID: hdr.ID, Response: true, Authoritative: true}, + Questions: []dnsmessage.Question{q}, + Answers: answers, + } + packed, err := resp.Pack() + if err != nil { + return + } + _, _ = rc.WriteToUDP(packed, from) + }() + + t.Cleanup(func() { + _ = rc.Close() + <-done + mdnsAddr, listenPacket = origAddr, origListen + Flush() + }) +} + +func TestLookupResolvesLocalName(t *testing.T) { + Flush() + disableAvahi(t) + startResponder(t, []dnsmessage.Resource{aResource(t, "inverter.local.", [4]byte{192, 168, 1, 42}, 60)}) + + addrs, err := Lookup(context.Background(), "inverter.local") + if err != nil { + t.Fatalf("Lookup: %v", err) + } + if len(addrs) != 1 || addrs[0] != netip.MustParseAddr("192.168.1.42") { + t.Fatalf("addrs = %v, want [192.168.1.42]", addrs) + } + + // The answer must now be cached: a second call cannot need the responder, + // which has already stopped. + again, err := Lookup(context.Background(), "INVERTER.local") + if err != nil { + t.Fatalf("cached Lookup: %v", err) + } + if len(again) != 1 || again[0] != addrs[0] { + t.Fatalf("cached addrs = %v, want %v", again, addrs) + } +} + +func TestLookupCachesNegativeAnswer(t *testing.T) { + Flush() + disableAvahi(t) + startResponder(t, nil) // never answers + + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + if _, err := Lookup(ctx, "missing.local"); err == nil { + t.Fatal("expected a lookup failure when nothing answers") + } + + addrs, ok := cacheLookup("missing.local") + if !ok { + t.Fatal("a failed lookup should be negatively cached") + } + if len(addrs) != 0 { + t.Fatalf("negative cache holds %v, want no addresses", addrs) + } +} + +func TestCacheExpires(t *testing.T) { + Flush() + base := time.Now() + orig := now + now = func() time.Time { return base } + t.Cleanup(func() { now = orig; Flush() }) + + cacheStore("inverter.local", []netip.Addr{netip.MustParseAddr("192.168.1.9")}, 30*time.Second) + if _, ok := cacheLookup("inverter.local"); !ok { + t.Fatal("entry should be live immediately after store") + } + + now = func() time.Time { return base.Add(31 * time.Second) } + if _, ok := cacheLookup("inverter.local"); ok { + t.Fatal("entry should have expired") + } +} + +func TestDialerSkipsResolutionForPlainHosts(t *testing.T) { + origListen, origAvail := listenPacket, avahiAvailable + listenPacket = func() (*net.UDPConn, error) { + t.Error("issued an mDNS query for a host that is not a .local name") + return nil, errors.New("should not be called") + } + avahiAvailable = func() bool { + t.Error("consulted avahi for a host that is not a .local name") + return false + } + t.Cleanup(func() { listenPacket, avahiAvailable = origListen, origAvail }) + + d := Dialer{Dialer: net.Dialer{Timeout: 500 * time.Millisecond}} + // Nothing listens on port 1; the point is that the failure comes from the + // dial, not from resolution. + if _, err := d.Dial("tcp", "127.0.0.1:1"); err == nil { + t.Fatal("expected the dial to fail") + } else if strings.Contains(err.Error(), "mDNS") { + t.Fatalf("plain IP dial went through mDNS: %v", err) + } +} + +func TestDialerReportsResolutionFailure(t *testing.T) { + Flush() + disableAvahi(t) + startResponder(t, nil) // never answers + + d := Dialer{Dialer: net.Dialer{Timeout: 100 * time.Millisecond}} + _, err := d.Dial("tcp", "missing.local:502") + if err == nil { + t.Fatal("expected a failure") + } + // The error must name the mechanism — an operator reading the log has to be + // able to tell resolution apart from an unreachable device. + if !strings.Contains(err.Error(), "mDNS") { + t.Fatalf("error %q does not mention mDNS", err) + } +} diff --git a/go/internal/mdnsresolve/multicast.go b/go/internal/mdnsresolve/multicast.go new file mode 100644 index 00000000..90a9d028 --- /dev/null +++ b/go/internal/mdnsresolve/multicast.go @@ -0,0 +1,161 @@ +package mdnsresolve + +import ( + "context" + "fmt" + "net" + "net/netip" + "strings" + "time" + + "golang.org/x/net/dns/dnsmessage" +) + +// This file is the fallback described in the package comment: a direct RFC +// 6762 query, used only where avahi-daemon's socket cannot be reached. It is +// deliberately the second choice — see avahi.go for the first. + +// mdnsAddr is the RFC 6762 IPv4 multicast group. A var, not a const, so tests +// can aim a query at a loopback responder. +var mdnsAddr = &net.UDPAddr{IP: net.IPv4(224, 0, 0, 251), Port: 5353} + +// listenPacket opens the ephemeral socket a query is sent from. Replaced in +// tests. It deliberately does NOT bind port 5353: where avahi-daemon runs it +// already owns that port, and the QU bit below asks responders to reply +// directly to this socket instead. +var listenPacket = func() (*net.UDPConn, error) { + return net.ListenUDP("udp4", &net.UDPAddr{IP: net.IPv4zero}) +} + +// classQU is IN with the RFC 6762 unicast-response bit set. +const classQU = dnsmessage.Class(0x8001) + +func queryAddrs(ctx context.Context, name string) ([]netip.Addr, time.Duration, error) { + qname, err := dnsmessage.NewName(name + ".") + if err != nil { + return nil, 0, fmt.Errorf("mdns: bad name %q: %w", name, err) + } + // One packet, two questions. RFC 6762 §5.2 allows it and it saves a round + // trip on dual-stack devices. + msg := dnsmessage.Message{Questions: []dnsmessage.Question{ + {Name: qname, Type: dnsmessage.TypeA, Class: classQU}, + {Name: qname, Type: dnsmessage.TypeAAAA, Class: classQU}, + }} + packed, err := msg.Pack() + if err != nil { + return nil, 0, fmt.Errorf("mdns: pack query: %w", err) + } + + var ( + addrs []netip.Addr + ttl time.Duration + ) + err = exchange(ctx, packed, func(packet []byte) bool { + got, gotTTL, ok := parseAddrAnswer(packet, name+".") + if !ok { + return false + } + addrs, ttl = got, gotTTL + return true + }) + if err != nil { + return nil, 0, err + } + return addrs, ttl, nil +} + +// exchange sends one multicast query and feeds every reply to handle until it +// accepts one or the deadline passes. +func exchange(ctx context.Context, packed []byte, handle func([]byte) bool) error { + conn, err := listenPacket() + if err != nil { + return fmt.Errorf("mdns: open socket: %w", err) + } + defer conn.Close() + + deadline := now().Add(queryTimeout) + if d, ok := ctx.Deadline(); ok && d.Before(deadline) { + deadline = d + } + if err := conn.SetDeadline(deadline); err != nil { + return fmt.Errorf("mdns: set deadline: %w", err) + } + if _, err := conn.WriteToUDP(packed, mdnsAddr); err != nil { + return fmt.Errorf("mdns: send query: %w", err) + } + + buf := make([]byte, 1500) + for { + n, _, err := conn.ReadFromUDP(buf) + if err != nil { + return fmt.Errorf("mdns: no usable answer: %w", err) + } + if handle(buf[:n]) { + return nil + } + } +} + +func parseAddrAnswer(packet []byte, qname string) ([]netip.Addr, time.Duration, bool) { + var p dnsmessage.Parser + if _, err := p.Start(packet); err != nil { + return nil, 0, false + } + if err := p.SkipAllQuestions(); err != nil { + return nil, 0, false + } + var addrs []netip.Addr + ttl := maxTTL + // Labelled so a parse error inside the type switch abandons the whole + // packet: once the parser desynchronises, every later record is suspect. +parse: + for { + h, err := p.AnswerHeader() + if err != nil { + break parse + } + if !strings.EqualFold(h.Name.String(), qname) { + if err := p.SkipAnswer(); err != nil { + break parse + } + continue + } + switch h.Type { + case dnsmessage.TypeA: + r, err := p.AResource() + if err != nil { + break parse + } + addrs = append(addrs, netip.AddrFrom4(r.A)) + case dnsmessage.TypeAAAA: + r, err := p.AAAAResource() + if err != nil { + break parse + } + // Unmap so a v4-mapped AAAA dials as plain IPv4. + addrs = append(addrs, netip.AddrFrom16(r.AAAA).Unmap()) + default: + if err := p.SkipAnswer(); err != nil { + break parse + } + continue + } + if d := time.Duration(h.TTL) * time.Second; d < ttl { + ttl = d + } + } + return finishAnswer(addrs, ttl) +} + +func finishAnswer(addrs []netip.Addr, ttl time.Duration) ([]netip.Addr, time.Duration, bool) { + if len(addrs) == 0 { + return nil, 0, false + } + switch { + case ttl < minTTL: + ttl = minTTL + case ttl > maxTTL: + ttl = maxTTL + } + return addrs, ttl, true +} diff --git a/go/internal/modbus/tcp_client.go b/go/internal/modbus/tcp_client.go index 78b7e7ba..b5b790ab 100644 --- a/go/internal/modbus/tcp_client.go +++ b/go/internal/modbus/tcp_client.go @@ -7,6 +7,8 @@ import ( "io" "net" "time" + + "github.com/srcfl/ftw/go/internal/mdnsresolve" ) const ( @@ -39,10 +41,13 @@ func newTCPClient(addr string, timeout, keepAlive time.Duration) *tcpClient { } func (c *tcpClient) Open() error { - dialer := net.Dialer{ + // Resolution happens here, on every Open, so a device configured by its + // ".local" name is found again after a DHCP lease moves it — the reconnect + // path rebuilds the client from c.addr and picks up the new address. + dialer := mdnsresolve.Dialer{Dialer: net.Dialer{ Timeout: modbusDialTimeout, KeepAlive: c.keepAlive, - } + }} conn, err := dialer.Dial("tcp", c.addr) if err != nil { return err diff --git a/go/internal/mqtt/client.go b/go/internal/mqtt/client.go index fb1ec167..a711dd96 100644 --- a/go/internal/mqtt/client.go +++ b/go/internal/mqtt/client.go @@ -4,12 +4,15 @@ package mqtt import ( "fmt" "log/slog" + "net" + "net/url" "sync" "time" paho "github.com/eclipse/paho.mqtt.golang" "github.com/srcfl/ftw/go/internal/drivers" + "github.com/srcfl/ftw/go/internal/mdnsresolve" ) // Capability wraps a paho client to match drivers.MQTTCap. @@ -51,6 +54,14 @@ func Dial(host string, port int, username, password, clientID string) (*Capabili } opts := paho.NewClientOptions(). AddBroker(fmt.Sprintf("tcp://%s:%d", host, port)). + // paho's built-in dialer goes through the stdlib resolver, which never + // answers a ".local" name. Every broker URL built here is tcp://, so a + // TCP-only replacement is complete; non-".local" hosts fall through to + // a plain dial inside mdnsresolve. + SetCustomOpenConnectionFn(func(uri *url.URL, o paho.ClientOptions) (net.Conn, error) { + d := mdnsresolve.Dialer{Dialer: net.Dialer{Timeout: o.ConnectTimeout}} + return d.Dial("tcp", uri.Host) + }). SetClientID(clientID). SetAutoReconnect(true). SetConnectRetry(true). diff --git a/optimizer/pyproject.toml b/optimizer/pyproject.toml index 2611fca7..8ef2a8de 100644 --- a/optimizer/pyproject.toml +++ b/optimizer/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "ftw-optimizer" -version = "1.3.2" +version = "1.4.0" description = "CVXPY planning engine for ftw" requires-python = ">=3.11" dependencies = [ diff --git a/scripts/install.sh b/scripts/install.sh index 33179091..942f28a5 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -149,8 +149,8 @@ fi echo "" echo "==[3/5]== Preparing install directory: $INSTALL_DIR" mkdir -p "$INSTALL_DIR/data" -# The image runs as uid 100 / gid 101 (the `ftw` user created in the -# alpine runtime stage — see Dockerfile). A bind-mounted host dir must +# The image runs as uid 100 / gid 101 — a bare numeric USER, no account is +# created in the image at all (see Dockerfile). A bind-mounted host dir must # match those IDs so SQLite can create state.db inside it. $SUDO chown -R 100:101 "$INSTALL_DIR/data" diff --git a/scripts/test-container-boundaries.sh b/scripts/test-container-boundaries.sh index ea827f6b..1b4c40d5 100755 --- a/scripts/test-container-boundaries.sh +++ b/scripts/test-container-boundaries.sh @@ -9,7 +9,24 @@ if grep -Eq 'COPY optimizer/|--from=optimizer|/opt/venv|FTW_OPTIMIZER_(PYTHON|DI exit 1 fi -grep -q '^FROM alpine:' Dockerfile +# The core runtime must stay a small, Python-free userland. This used to be +# asserted as `^FROM alpine:`, which was a proxy for the real rule and gave no +# diagnostic when it failed — Python originally reached core precisely by way of +# the optimizer's base image, so the check existed to catch a base that drags an +# interpreter in, not to mandate one distro. Assert that directly instead. +if grep -Eq '^FROM .*(python|pypy)' Dockerfile; then + echo "Dockerfile must not build on a Python base image; use Dockerfile.optimizer for Python/CVXPY" >&2 + exit 1 +fi +# wget is contractual, not incidental: the HEALTHCHECK uses it, and +# ftw-updater docker-execs it inside this image to decide whether an update +# commits. An updater already deployed in the field will keep doing so, so the +# core image cannot stop shipping wget without breaking self-update on every +# host running an older sidecar. +if ! grep -Eq 'wget' Dockerfile; then + echo "Dockerfile must provide wget: the HEALTHCHECK and ftw-updater's readiness probe both exec it" >&2 + exit 1 +fi grep -q '^COPY optimizer/' Dockerfile.optimizer grep -q '/out/ftw-backup' Dockerfile grep -q '/app/ftw-backup' Dockerfile