Skip to content

docs(security): smaug's missing logs are a decision, and now they are written down (#255) - #493

Merged
Gerrrt merged 1 commit into
mainfrom
gerrrt/smaug-logging-gap-residual-8fd9b5
Sep 17, 2026
Merged

Gerrrt merged 1 commit into
mainfrom
gerrrt/smaug-logging-gap-residual-8fd9b5

Conversation

@Gerrrt

@Gerrrt Gerrrt commented Sep 17, 2026

Copy link
Copy Markdown
Owner

ADR-0016 reversed the monitoring direction for CasaBonita and said in advance what
it costs: "The NAS gets metrics and no logs ... the one host in the estate whose
compromise is invisible."
The host landed on 2026-09-16, so the trigger on #255
has fired.

Every other accepted residual of this class has a row in SECURITY.md's Known
exposure
table. This one had none — it was written down in ADR-0016, carried
forward by ADR-0040, and mentioned in one clause of a roadmap entry about a
different issue. Closing that asymmetry is the whole change. The gap is not
closed and is not meant to be.

The three boxes

  • A row in SECURITY.md's Known exposure table, in the
    "Accepted residual, not a fix in progress." form of the ingest-port and
    BMC rows. It goes to the bottom of the residual block rather than the top
    of the table — b7f6e62 set that convention when it appended the BMC row,
    and 531b105 set the opposite one for a closed finding.
  • A line in docs/security.md, beside the tripwire paragraph. It is two
    paragraphs rather than a line, because the second one is box 3.
  • The alert decision, below.

TerminalSegmentReachedInternalNetwork is sufficient. No new alert.

security.rules.yaml:314-337 matches {app="filterlog", action="pass"} against
,10\.0\.(10|20|40)\.[0-9]+,(10\.0\.(30|50|99)|10\.7\.7)\.[0-9]+,. One event: a
terminal segment passed toward the lab, trusted, management or switch LAN.

Line that up against what a compromise of smaug looks like — indexer exploited,
segment enumerated, pool read, exfiltration over 443, persistence. Every branch
except the upward pivot is invisible, not merely unalerted, because the host
emits no log lines and Suricata does not watch igc0.40 either (ids.rules.yaml
declares igc0.20 and igc0.10). The rule covers the only branch that leaves a
record. A second rule cannot widen that: the limit is the absence of evidence.

  • An absence rule — rejected, and it cannot be written.
    absent_over_time({host="smaug"}[Nh]) would fire immediately and permanently,
    and be correct to. SuricataLogsStopped became writable only once somebody
    measured what a normal silence looks like — 22 days of stream, a worst gap of
    75 minutes, nine hours chosen as seven times it. The general pattern is
    homelab_suricata_expected_interface: a declared expectation to measure the
    silence against. Here the expectation is zero and always was, so the firing
    condition is the design working.
  • A up == 0 liveness rule on the scrape target — a real gap, deliberately not
    pre-empted.
    It belongs to Add the scrape path for a host that pushes nothing #256, which has not chosen between node_exporter
    on 9100 and TrueNAS's own endpoint; that fork changes the job name, the
    labels and possibly a fifth firewall rule.
  • Documentation only — accepted. The failure being defended against is
    somebody rediscovering this in two years and treating it as an oversight.

The sharpest version, and it is newly true. scripts/check_loki_coverage.py
takes its denominator from the hosts Loki has already heard from — deliberately,
because a hand-maintained table of which rule should see which host drifts. So it
reports full coverage while blind to smaug, and is right to. JournalSourceStopped
and LogEntriesDropped are both keyed on an agent publishing metrics. Nothing
automated in the estate would notice this gap.
The documents are the control.

Two corrections to the issue text

  1. The issue quotes the ingest-port row as ending "and never an untrusted
    segment
    ". That wording was replaced on 2026-09-04 by 6ce11ed; the current
    row says "…and no untrusted segment reaches it at all." The argument is
    unaffected — only the sentence it points at moved.
  2. The SuricataStopped reasoning is no longer at security.rules.yaml:241-250.
    It is at loki/rules/security.rules.yaml:596-617 and
    prometheus/rules/ids.rules.yaml:30-36.

A stale paragraph, found because the new one had to sit beside it

docs/security.md's Segmentation section still said ADR-0016's CasaBonita passes
were uncreated, and that there were three. Four exist and have since
2026-09-16 — c938c37 corrected the threat-model row and left this one behind.
It now splits the two ADRs by stage (CasaBonita's exist, Skids' does not) and says
outright that half of what it claimed stopped being true, which is this file's
habit rather than a quiet overwrite. docs/network.md carried the same error in
one bullet while the next bullet said four.

Deliberately out of scope

Verification

python3 scripts/check_docs.py and ./scripts/lint.sh both pass. These are the
only checks that read SECURITY.md and docs/security.md; check_firewall_posture
is the load-bearing one, because it anchors on the literal "Default deny holds for"
/ "It does not hold for" sentences and their bold runs, which the edits leave
untouched.

./scripts/validate.sh also passes — all checks passed (3 skipped), the three
being the systemd-unit checks that only run in the deployment checkout. It took
about 35 minutes rather than the usual few, because promtool test rules queued
behind a dozen concurrent containers from peer sessions on this shared docker
daemon; that is contention, not a failure.

Closes #255
Refs #483 #256 #413

🤖 Generated with Claude Code

… written down (#255)

ADR-0016 reversed the monitoring direction for CasaBonita so that nothing on 40
initiates upward, and said in advance what that costs: "The NAS gets metrics and
no logs ... the one host in the estate whose compromise is invisible." The host
landed on 2026-09-16. Every other residual of this class has a row in
SECURITY.md's Known exposure table; this one had none, and the only places it was
written down were two ADRs and one clause inside a roadmap entry about a
different issue. That asymmetry is what this commit closes.

The row goes to the bottom of the residual block rather than the top of the
table, which is the convention the BMC row set: a closed finding is news and goes
first, an accepted residual joins the tail. It carries both halves of the issue —
why the obvious fix is worse than the gap, and why there is no alert.

THE ALERT QUESTION, ANSWERED RATHER THAN DEFERRED.
TerminalSegmentReachedInternalNetwork fires on a pass from a terminal VLAN toward
30, 50, 99 or the switch LAN. Line that up against what a compromise of this host
looks like — indexer exploited, segment enumerated, pool read, exfiltration over
443 — and every branch except the upward pivot is invisible, not merely
unalerted. The rule covers the only branch that leaves a record, and no second
rule can widen that, because the limit is the absence of evidence rather than the
absence of a rule.

An absence rule is the obvious candidate and it cannot be written. SuricataLogsStopped
became writable only after somebody measured what a normal silence looks like —
22 days of stream, a worst gap of 75 minutes, nine hours chosen as seven times it
— and the general pattern is homelab_suricata_expected_interface: a declared
expectation to measure the silence against. Here the expectation is zero and
always was, so the firing condition is the design working. A liveness rule on the
scrape target is a real gap and belongs to #256, which has not yet chosen between
node_exporter on 9100 and TrueNAS's own endpoint; written against a guess it is
how the two end up inconsistent.

The sharpest version of the gap is newly true and is in the prose:
check_loki_coverage.py takes its denominator from the hosts Loki has already
heard from, deliberately, because a hand-maintained table of which rule should
see which host drifts. So it reports full coverage while blind to smaug, and is
right to. JournalSourceStopped and LogEntriesDropped are both keyed on an agent
publishing metrics. Nothing automated in the estate would notice this; the
documents are the control, which is the honest way to say it.

Suricata does not watch igc0.40 either — ids.rules.yaml declares igc0.20 and
igc0.10 — so "the tripwire is the only control" is literal rather than
rhetorical. If a third interface is ever declared there, that sentence needs
revisiting.

WRITING IT TURNED UP A STALE PARAGRAPH, and it was the paragraph the new one had
to sit beside. security.md's Segmentation section still said ADR-0016's
CasaBonita passes were uncreated, and that there were three. Four exist and have
since 2026-09-16; c938c37 corrected the threat-model row and left this one
behind. It now splits the two ADRs by stage — CasaBonita's exist, Skids' does not
— and says outright that half of what it used to claim stopped being true, which
is this file's habit rather than a quiet overwrite. network.md carried the same
error in one bullet while the next bullet said four.

The scrape path, the SMART attributes and the patch state are all deliberately
untouched: #256 owns the first, #483 the other two, and this issue is the record
rather than the fix.

Closes #255
Refs #483 #256 #413

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Gerrrt
Gerrrt merged commit 95b7621 into main Sep 17, 2026
3 checks passed
@Gerrrt
Gerrrt deleted the gerrrt/smaug-logging-gap-residual-8fd9b5 branch September 17, 2026 23:33
Gerrrt added a commit that referenced this pull request Sep 17, 2026
Four inbound passes were created for `smaug` on 2026-09-16 — Hicks to `443` and
to `8096`, `10.0.99.20` to `9100` and to `22` — and prose still said three.

`build-the-nas.md` §0.5 was headed "the three rules" over a three-row table
while the status callout at the top of the same file said four were created. The
table now has four rows, one rule per port, and the section says why the ADR's
count was three: that table wrote the Hicks pass as a single row with a port
list, so three counted rows rather than rules.

Updated rather than marked as the as-planned version. §0.1 sets that precedent
for a BIOS procedure kept "for the next machine", but these are four specific
rules for one host and there is no next machine for them — a runbook should
instruct what exists.

`network.md` stated the count in two adjacent bullets, which is the drift vector
rather than an instance of it. #493 corrected the first bullet from three to
four while this branch was open; this goes one step further and stops it
counting them at all. The next bullet enumerates all four and is the authority.

`security.md` needed the same fix and #493 made it better than this branch had:
it separates CasaBonita's four created passes from Skids' uncreated one, records
that position was verified with `pfctl` rather than the web UI, and cites
ADR-0040 for the inert `22`. That version is kept and this branch's edit to the
file is dropped.

ADR-0016 and ADR-0040 are untouched: ADR-0001 makes them immutable, ADR-0040
already carries the ports correction, and ADR-0016's "three" is a true statement
about what it wrote. `roadmap.md` already had it right.

No `check_docs.py` guard. `firewall-claims.yaml` cannot source this count — its
own header says it is "deliberately NOT a list of what each segment can reach",
that host-and-port exceptions live in `network.md` instead, and that rule bodies
are not published there; it is also `--derive`d rather than authored. These four
passes exist in no machine-readable form in the repository, and the checker that
does read rule bodies asks the live firewall, which ADR-0026 puts outside CI by
design. The fix available instead is the one applied here: state the number once
per document, next to the thing it counts.

Refs #413

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gerrrt added a commit that referenced this pull request Sep 18, 2026
Two conflicts, resolved in opposite directions.

docs/security.md — took main's wholesale. It rewrites the paragraph into two
labelled halves, "CasaBonita's exist" and "Skids' does not exist", adds that
the passes were verified in position with pfctl rather than read off the web
UI, notes that the 22 is inert under ADR-0040, points at network.md as the
authority for the list, and cites #134 for the Skids pass. That is strictly
better than this branch's split of the same sentence, so this branch's version
of security.md is dropped entirely.

docs/network.md — kept this branch's. main corrected the first bullet's count
from three to four; this branch removes the count from that bullet altogether
and leaves the next one as the sole authority, because the count being stated
twice is what let the two drift apart in the first place. Correcting both
leaves the drift vector in place. This side also carries the port-15 bullet.

NOTE: check-docs is still red on this branch, entirely from main — the two
ADR-0041 files and README.md's ADR and runbook counts. PR #509 fixes those;
nothing here can.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

smaug has no logs, and that is a decision — record it as a residual

1 participant