diff --git a/README.md b/README.md index c00b6ed..31c8505 100644 --- a/README.md +++ b/README.md @@ -48,10 +48,15 @@ documents for different readers. ## Highlights - **Network segmented by trust, not by function.** Six VLANs; IoT, media and - guest segments are terminal — egress only, no path to anything else, and each - carries a tripwire that logs anything which gets past that. Default deny holds - everywhere except the trusted workstation segment and the switch LAN, both of - which are listed rather than counted. + guest segments are terminal **outward** — nothing on them initiates anywhere + else, and each carries a tripwire that logs anything which gets past that. + Inbound is a separate question, and since 2026-09-16 CasaBonita is the one + exception: two more-trusted segments reach one host on named ports, so the + televisions can have a media server without the segment ceasing to be terminal + in the direction that matters + ([ADR-0016](docs/adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md)). + Default deny holds everywhere except the trusted workstation segment and the + switch LAN, both of which are listed rather than counted. [Why](docs/adr/0013-segment-access-as-implemented.md) - **Full observability pipeline for a mixed estate.** Grafana Alloy agents push metrics and logs from Linux hosts; `snmp_exporter` polls the four devices that diff --git a/SECURITY.md b/SECURITY.md index a4deacd..299bef2 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -38,6 +38,7 @@ is a very different thing from an overlooked one. Full detail in | What | Status | | --- | --- | +| A used server arrived with Intel AMT enabled on the factory-default credential | Found 2026-09-16 on `smaug`, the ThinkServer TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), during the pre-build firmware pass: the MEBx accepted Intel's default `admin`, so no password had ever been set, and `Manageability Feature Selection` read `Enabled`. The ME network name and domain were blank, which is evidence it had never been provisioned rather than proof. **Closed the same day and before the host carried anything**: a new ME password, `Unconfigure Network Access` → `Full Unprovision` while the feature was still enabled, then the feature disabled, verified by the `` prompt no longer being offered at boot. Disabled rather than hardened, unlike `shiva` under [ADR-0033](docs/adr/0033-keep-the-ilo-on-the-lab-segment.md): that BMC serves a headless hypervisor in a rack, where a remote console earns its keep; this is a tower with a monitor beside it, and an out-of-band plane on the segment with the televisions buys it nothing. The general lesson is the one [`hardware.md`](docs/hardware.md) already records for the CRS326 — a second-hand device arrives carrying whatever its last owner left on it, and the management plane is the part nobody thinks to look at. | | SNMP community committed in plaintext, shared across firewall, switch, UPS and BMC | Removed from `HEAD` and replaced with four distinct per-device SOPS-encrypted values. Rotated on all four devices; each answers to its own new community. The firewall, the UPS and the BMC additionally refuse the old one. **The switch serves GETBULK to any short community without checking, so whether it still holds the previous one is unverified** — see below. The original shared string has been purged from git history, though it must still be treated as public — it was reachable in a public repository and cannot be un-seen. | | Grafana `admin`/`admin` with anonymous Admin access enabled | Fixed — anonymous auth off, password from SOPS | | Passphrase-encrypted TLS private keys under `certificates/` | Removed from `HEAD` and purged from history. A new CA and leaf have been generated with [`scripts/gen-certs.sh`](scripts/gen-certs.sh); the old keys are superseded and should be treated as compromised wherever they were ever trusted. | diff --git a/docs/architecture.md b/docs/architecture.md index cc821d6..c460487 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -190,7 +190,7 @@ hole from the monitoring VLAN into the monitored one. | `odin` (10.0.30.60) | 🟢 30 | [`stacks/soc`](../stacks/soc) | **Not built yet** — Wazuh (indexer, manager, dashboard), Velociraptor and Alloy: the security half of ADR-0007, placed by [ADR-0030](adr/0030-give-the-security-tooling-its-own-guest-and-its-own-stack.md) on a second guest on `Saruman` because `alexander`'s 8 GiB cannot hold both. The stack is authored and CI-validated ahead of the guest, the way `stacks/lab` was ahead of `alexander`; the build is [`build-the-soc-guest.md`](runbooks/build-the-soc-guest.md), behind [#414](https://github.com/Gerrrt/HomeLab/issues/414). The six domain machines report to it as agents; its Alloy pushes to `alexander`, never to VLAN 99, and the indexer's health is the one series that crosses into the lab's Prometheus. Alloy agent (Docker) | | `oracle` (10.0.99.30) | 🔴 99 | *(none — hand-run containers)* | The Lemmiwinks wiki and its Postgres, since 2025-11-12 ([ADR-0011](adr/0011-keep-the-wiki-internal.md)); Alloy agent (Docker, `scripts/deploy-agent.sh`); the off-host copy of the firewall export (`make backup-firewall`). The estate's host for small off-host jobs — [ADR-0015](adr/0015-give-oracle-the-off-host-jobs.md) | | `trinity` (10.0.99.40) | 🔴 99 | [`stacks/sensitive`](../stacks/sensitive) | **Not built yet** — ADR-0008's sensitive tier on the ProDesk 600 G4 of [ADR-0034](adr/0034-run-the-sensitive-tier-on-the-prodesk-and-make-it-the-spare-hardware.md), after the firewall restore is rehearsed on it ([#404](https://github.com/Gerrrt/HomeLab/issues/404)). The foundation is authored: Caddy as the published HTTPS port and step-ca issuing beneath the tier's own root rather than the estate's, which is left untouched ([#129](https://github.com/Gerrrt/HomeLab/issues/129), [#130](https://github.com/Gerrrt/HomeLab/issues/130), [ADR-0037](adr/0037-give-the-sensitive-tier-its-own-root-and-issue-beneath-it-over-acme.md)), with AdGuard Home behind Caddy and publishing 53 to the firewall's forwarder alone ([#135](https://github.com/Gerrrt/HomeLab/issues/135), [ADR-0010](adr/0010-keep-the-resolver-on-the-gateway.md)); Home Assistant ([#134](https://github.com/Gerrrt/HomeLab/issues/134)), Immich — four containers behind Caddy with a memory limit on each ([#132](https://github.com/Gerrrt/HomeLab/issues/132)) — Paperless-ngx with a Postgres and a Valkey of its own ([#133](https://github.com/Gerrrt/HomeLab/issues/133)) and Vaultwarden ([#131](https://github.com/Gerrrt/HomeLab/issues/131)) are authored as well; the rest follow. Alloy agent (Docker, `scripts/deploy-agent.sh`), pushing to `prometheus` like `oracle`'s | -| `smaug` (10.0.40.30) | 🟡 40 | [`stacks/media`](../stacks/media) | **Not built yet** — [ADR-0008](adr/0008-place-services-by-data-trust.md)'s media tier on the ThinkServer TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), placed and addressed by [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) and running TrueNAS rather than Ubuntu Server by [ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md). **The host exists and the pool does not**: TrueNAS 25.10.7 is installed on its boot SSD and it answers on DHCP, but the ZFS mirror waits on two Exos X20 drives, and the static above is not set until [`build-the-nas.md`](runbooks/build-the-nas.md) §0.3. The stack is authored and CI-validated ahead of the storage, the way `stacks/sensitive` was ahead of `trinity`: Jellyfin alone, publishing 8096 to the segment because the televisions reach it natively and no firewall rule is involved at all ([#138](https://github.com/Gerrrt/HomeLab/issues/138)). No Alloy agent — this host is scraped, never pushing, and how is the fork [#256](https://github.com/Gerrrt/HomeLab/issues/256) now has to settle | +| `smaug` (10.0.40.30) | 🟡 40 | [`stacks/media`](../stacks/media) | [ADR-0008](adr/0008-place-services-by-data-trust.md)'s media tier on the ThinkServer TS150 of [#413](https://github.com/Gerrrt/HomeLab/issues/413), placed and addressed by [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) and running TrueNAS rather than Ubuntu Server by [ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md). **The host is built and the pool is not**: TrueNAS is installed on its boot SSD, it holds the static above since 2026-09-16, and the four inbound rules are created and verified in position — but the ZFS mirror waits on two Exos X20 drives and nothing is deployed on it. The stack is authored and CI-validated ahead of the storage, the way `stacks/sensitive` was ahead of `trinity`: Jellyfin alone, publishing 8096 to the segment because the televisions reach it natively and no firewall rule is involved at all ([#138](https://github.com/Gerrrt/HomeLab/issues/138)). Runs no agent of its own: this host is scraped and never pushes, which makes it the estate's first scraped host and is why [#256](https://github.com/Gerrrt/HomeLab/issues/256) exists — that issue now also has a fork in it, because TrueNAS ships its own metrics endpoint where the issue specifies `node_exporter` | | `bahamut` (10.0.30.50) | 🟢 30 | *(none — Windows)* | **Not built yet** — Windows Server 2025 domain controller, PDC emulator and DNS for `ad.matrix.elysium` — Tier 0. Static, because every member finds a DC through DNS and the DCs *are* the DNS. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | | `leviathan` (10.0.30.51) | 🟢 30 | *(none — Windows)* | **Not built yet** — Windows Server 2025 second domain controller and DNS — Tier 0. Static, for the same reason. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | | `titan` (10.0.30.52) | 🟢 30 | *(none — Windows)* | **Not built yet** — Windows Server 2025 file and member server — the shares, and the NTLM relay target that only exists because 2025 requires outbound SMB signing and not inbound — Tier 1. Scraped by `alexander` on `9182`; it pushes nothing, and runs no Alloy ([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)) | diff --git a/docs/hardware.md b/docs/hardware.md index 41bedc3..29b119f 100644 --- a/docs/hardware.md +++ b/docs/hardware.md @@ -40,6 +40,7 @@ quietly swapped. | `Saruman` | HPE ProLiant DL360 Gen9 | 2× Xeon E5-2680 v3 (48 threads) | 128 GB | 2× 1 TB SAS HDD, RAID 1 | Proxmox VE 9 | | `prometheus` | Apple MacBook Pro (2012, Retina 13") | i5/i7 | 8 GB | 256 GB SSD | Ubuntu Server 24.04 LTS | | `oracle` | Dell Inspiron 15-3565 | AMD A6-9200 (2 cores) | 4 GB | 500 GB HDD | Ubuntu Server 24.04 LTS | +| `smaug` | Lenovo ThinkServer TS150 | Xeon E3-1225 v6 (4 cores) | 8 GB ECC | 240 GB SATA SSD (boot) | TrueNAS 25.10 | The observability stack runs on a thirteen-year-old MacBook. It handles four SNMP devices at a 60-second interval, four Alloy agents, and 30 days of metric @@ -157,9 +158,12 @@ revisions of this repository treated `shiva` as the hypervisor itself. 2026-09-09, **in hand since 2026-09-15**. The NAS `smaug` of [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), tracked under [#413](https://github.com/Gerrrt/HomeLab/issues/413). A tower, - not a rack unit, and a 73 W part where the ADRs pictured an N100; it enters - the Compute table when it is racked — or rather placed — addressed and in - `network.md`. The boot disk the TrueNAS install wants + not a rack unit, and a 73 W part where the ADRs pictured an N100. **It entered + the Compute table on 2026-09-16**, which is the trigger this entry set for + itself — placed, addressed at `10.0.40.30`, and in `network.md`. The Storage + column reads the boot disk alone on purpose: the ZFS mirror does not exist + until the two Exos drives land, and a Storage column describing a pool nobody + has created would be the kind of claim this table exists to not make. The boot disk the TrueNAS install wants ([ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md)) and the bracket that carries it in the optical bay are the entries below and landed with it; the two drives for the mirror, bought 2026-09-11, have not. Nothing @@ -177,10 +181,40 @@ revisions of this repository treated `shiva` as the hypervisor itself. in one of four slots. More memory is therefore an add and not a replace, and the part to match is ECC **unbuffered**: a registered DIMM will not run on this board. Onboard NIC `4c:cc:6a:xx:xx:xx`, recorded as an OUI like every - other address here. BIOS `S06KT03R` dated **2017-05-22**, embedded controller - `S06CT01A` — a nine-year-old firmware that predates the Spectre and Meltdown - microcode, worth checking against Lenovo's latest while the box is empty - rather than once it holds the household's library. + other address here. BIOS **`S06KT81L` dated 2024-02-05**, boot block `1.81`, flashed + 2026-09-16 while the box was still empty. It shipped on `S06KT03R` dated + 2017-05-22 with boot block `1.03` — a firmware predating the Spectre and + Meltdown microcode by a year — and the flash was done before the pool existed + precisely so that a reset of `Configure SATA as` or of `CSM` would cost a + re-check rather than an unbootable host with data on it. Both were re-checked + and both survived, as did the machine type-model, the serial, the UUID, the + MAC and the clock. **The embedded controller did not move**: it read + `S06CT01A` before and reads `S06CT01A` after, and whether the package updates + that component at all is unestablished — recorded as an observation rather + than as a failure, because nothing misbehaves and the BIOS half plainly + took. + **Intel AMT was enabled and on its factory-default credential when this box + arrived, and is now off.** Intel ME `v11.6.12.1204`, MEBx `v11.0.0.0012`: the + MEBx accepted `admin` on 2026-09-16, which is Intel's default and means + nobody had ever set one — the same class of thing the CRS326 entry below + warns about, that a used device arrives carrying whatever its last owner left + on it. `Manageability Feature Selection` read `Enabled`, `Password Policy` + the stock `Anytime`, and the ME network name and domain were both **blank**, + which is the evidence it had never been provisioned onto a network rather + than a proof of it. Closed the same day, in the order the firmware requires: + a new ME password (MEBx forces one at first login, and it lives in the + operator's password manager), then `Unconfigure Network Access` → + `Full Unprovision` while the feature was still enabled, then + `Manageability Feature Selection` → `Disabled`. Verified by the `` + prompt no longer being offered at boot. + **Disabled rather than hardened, which is the opposite of what + [ADR-0033](adr/0033-keep-the-ilo-on-the-lab-segment.md) decided for + `shiva`**, and the difference is the host's job rather than a change of + posture: `Saruman` is headless in a rack and a remote console is load-bearing + there, so its iLO was kept and locked down. `smaug` is a tower with a monitor + beside it, on the segment with the televisions and the consoles. Out-of-band + management buys it nothing and would cost a management plane that answers + when the operating system is off. Six SATA ports, all enabled, and two settings that were already right rather than needing changing: `Configure SATA as [AHCI]`, which is the raw-disk access ZFS wants and the thing diff --git a/docs/network.md b/docs/network.md index 128242c..79c0a1d 100644 --- a/docs/network.md +++ b/docs/network.md @@ -274,6 +274,7 @@ Televisions and consoles. Internet only. | morpheus | `10.0.40.1` | `02:26:26:xx:xx:xx` | HP ProDesk 600 G4 Mini | FreeBSD 16.0 | Rack U5 | Firewall | | nibelheim | `10.0.40.10` | `78:c8:81:xx:xx:xx` | Sony PlayStation 5[^PS5] | — | Lower floor | Console | | hyrule | `10.0.40.20` | `00:05:1b:xx:xx:xx` | Nintendo Switch[^Nintendo] | — | Lower floor | Console | +| smaug | `10.0.40.30` | `4c:cc:6a:xx:xx:xx` | Lenovo ThinkServer TS150 | TrueNAS 25.10 | Media room | NAS | | mediatv | `10.0.40.100` | `58:fd:b1:xx:xx:xx` | LG OLED[^OLEDTV] | webOS | Media room | TV | | streambox | `10.0.40.101` | `f0:46:3b:xx:xx:xx` | Xumo Stream Box[^StreamBox] | entOS | Media room | Streaming | @@ -283,13 +284,23 @@ Televisions and consoles. Internet only. - Internet only, no path to any other segment. Smart TVs run unauditable firmware with a permanent internet connection and no patch guarantee, so they get the same trust level as a guest. -- The planned NAS lands here — `smaug` at `10.0.40.30`, decided by - [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md), - bought 2026-09-09 ([#413](https://github.com/Gerrrt/HomeLab/issues/413)) and - not yet built. It does not change the *Reaches* column: nothing on this - segment will initiate anywhere, and the three rules that ADR writes down all - let a more trusted segment reach **in**. That is the direction this row - records, and it is the one that is unchanged. +- **The NAS is here and addressed since 2026-09-16** — `smaug` at + `10.0.40.30`, decided by + [ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) + and running TrueNAS by + [ADR-0040](adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md) + ([#413](https://github.com/Gerrrt/HomeLab/issues/413)). Its ZFS mirror does + not exist yet and neither does `stacks/media`; what exists is a host on its + address. **It does not change the *Reaches* column**, and that is the point + ADR-0016 made in advance: nothing on this segment initiates anywhere, and the + three rules created that day all let a more trusted segment reach **in**. + That is the direction this row records, and it is the one that is unchanged. +- **Inbound is no longer nothing, and that is deliberate.** Since 2026-09-16 + Hicks reaches `10.0.40.30` on `443` and `8096`, and `10.0.99.20` reaches it + on `9100` and `22` — four host-scoped, port-scoped passes above *Block access + to CasaBonita* on their interfaces. Everything else on every other segment is + still refused, and the televisions need no rule at all because they share this + broadcast domain with the server. [^OLEDTV]: [LG OLED TV](https://www.lg.com/us/tvs/oled) [^PS5]: [PlayStation 5](https://www.playstation.com/en-us/ps5/) diff --git a/docs/runbooks/build-the-nas.md b/docs/runbooks/build-the-nas.md index 76ae9a3..6f55019 100644 --- a/docs/runbooks/build-the-nas.md +++ b/docs/runbooks/build-the-nas.md @@ -17,8 +17,19 @@ drives for §1 onward. > DHCP at `10.0.40.100`. Its spec is read off the machine rather than off a > listing and recorded in [`hardware.md`](../hardware.md). > -> Nothing below §0.3 has been done. The pool does not exist, the address is -> still DHCP, and none of the three firewall rules is created. +> **§0 is complete as of 2026-09-16.** BIOS flashed (§0.1), AMT found on its +> factory-default credential and disabled (§0.2), the static set (§0.3), the +> reservation added (§0.4), and the rules created as four host- and port-scoped +> passes rather than three (§0.5) — `443,8096` from Hicks split into two rules, +> `9100` and `22` from `10.0.99.20`. +> +> §0.6 verified from `morpheus` rather than from the UI: every pass sits above +> *Block access to CasaBonita* on its interface (167–168 before 169 on +> `igc0.99`, 194–195 before 196 on `igc0.50`), Winterfell is still correctly +> refused on `443`, and the `igc0.40` tripwire reads **118,621 evaluations and +> zero packets**. +> +> What is left is the drives. The pool does not exist and nothing is deployed. This builds what [ADR-0016](../adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md) placed and [ADR-0040](../adr/0040-run-truenas-on-smaug-and-keep-the-media-stack-in-this-repository.md) @@ -34,6 +45,14 @@ arrival day into §1–§7 rather than an evening. ### §0.1 — Flash the BIOS, or decide not to +> **Done 2026-09-16.** `S06KT03R` (2017-05-22) → **`S06KT81L` (2024-02-05)**, +> boot block `1.03` → `1.81`, by the DOS utility from a FreeDOS stick. `CSM` +> and `AHCI` were both re-checked afterwards and both survived; so did the +> machine type-model, the serial, the MAC and the clock. The embedded +> controller still reads `S06CT01A` — see [`hardware.md`](../hardware.md). The +> section below is kept for the next machine, and for the next time this one +> needs it. + `S06KT03R` dated **2017-05-22**, which predates the Spectre and Meltdown microcode. Lenovo ships DOS, Windows and Linux update utilities for the TS150; there is no network flash on this machine, because it has no BMC. diff --git a/docs/security.md b/docs/security.md index 17587e7..9709cb7 100644 --- a/docs/security.md +++ b/docs/security.md @@ -11,7 +11,7 @@ What this network is actually built to survive: | --- | --- | | A compromised IoT device pivoting to a workstation | VLAN 20 is terminal — no route to any other segment | | A guest on the Wi-Fi enumerating the LAN | VLAN 10 is terminal, client isolation on | -| A smart TV's firmware phoning somewhere unexpected | VLAN 40 is terminal, egress only | +| A smart TV's firmware phoning somewhere unexpected | VLAN 40 is terminal **outward** — egress only, and nothing on it initiates into another segment. Since 2026-09-16 two more-trusted segments reach one host on it on named ports ([ADR-0016](adr/0016-open-casabonita-inward-and-keep-it-terminal-outward.md)), which changes what reaches *in* and not what gets *out*; the `igc0.40` tripwire still reads zero packets | | A corporate laptop carrying something in from outside | Sits on VLAN 50 but has no management access | | A lab VM escaping into the house | VLAN 30 reachable only *from* trusted, never *to* it; the lab tripwire logs anything that gets past the blocks ([#234](https://github.com/Gerrrt/HomeLab/issues/234)) | | An attacker on the lab segment reaching the hypervisor's BMC | **Accepted.** `shiva` stays on VLAN 30 by decision ([ADR-0033](adr/0033-keep-the-ilo-on-the-lab-segment.md)), hardened on 2026-09-09 — IPMI-over-LAN, SSH and Federation off, and its one path out of the segment deleted; a BMC compromise in the lab costs the lab, and the tripwire watches what it initiates | diff --git a/stacks/media/.env.example b/stacks/media/.env.example index 2598d88..e7fad85 100644 --- a/stacks/media/.env.example +++ b/stacks/media/.env.example @@ -29,7 +29,12 @@ JELLYFIN_GID=65534 # # ADR-0040's reopen condition turns on this working, and build-the-nas.md §6 # is where it gets tested rather than assumed. -RENDER_GID= +# +# Read off smaug on 2026-09-16: `107 render`. That the node exists at all is +# worth noting — it means the kernel bound a driver to the P630 and exposed a +# render node, which is one more step of ADR-0040's condition standing up. The +# step it is NOT is a container using it, which is still §6's to prove. +RENDER_GID=107 # The library on erebor, mounted read-only into the container at /media. #