docs(runbook): replace the laptop cell in prometheus (#454)

[Gerrrt]

What changed

A runbook for swapping the A1437 cell in prometheus and proving the swap
worked — docs/runbooks/replace-the-laptop-cell.md. Three existing pointers
retargeted at it, and README's runbook count goes 25 to 26 in the same commit
because check_docs.py counts the glob.

Nothing about the running stack changes. This is documentation plus one
Alertmanager silence, described below.

Why

The repository half of #454
landed in #461 on 2026-09-12 and
the cell was bought on 2026-09-13. The procedure that actually settles that
issue — the fit, and the mains-pull test that proves the property the cell was
bought for — existed only as two sentences inside a UPS runbook and one bullet
in hardware.md, for a glued pouch cell in the machine the whole observability
stack runs on.

The issue stays open. It is settled by the fit and the test, not by this page,
so this is Refs and not a closing keyword.

No collector, and the reason is now in two places rather than one. Alloy's
node collector already exports node_power_supply_* from both laptops. The
issue body asked for energy_full / energy_full_design; neither exists on
either machine — both report the charge_* family — so a collector written to
that text would have emitted an empty file.

Two judgement calls worth reviewing:

• The mains pull is deliberately not silenced, and runs twice.
  HostOnBattery is critical on the power route and will really page. That is
  the point: verify-the-alert-path.md proves the transport with Watchdog,
  which is vector(1) and carries no labels, so nothing else exercises this
  rule's for:, its join against node_power_supply_info{type="Mains"}, or its
  routing. Running it first on the old cell — which is also how a cell gets
  discharged before removal — means a failure after the swap is the cell or the
  adapter and nothing else. Both sibling battery runbooks record deleting a
  silence late as their single regret.
• A new section neither sibling needed: what goes blind while the lid is
  off. shiva and mjolnir are not the monitoring host. prometheus is, so
  no alert can fire or be recorded during the window, and the only signal that
  survives it is the off-host healthcheck — which the runbook says to leave
  alone precisely because it will fire.

Blast radius

• No change to network segmentation or firewall rules
• No new port published to a VLAN that could not already reach the service
• No credential added outside secrets/*.sops.yaml

No rules, tests, compose files or scripts are touched, so nothing deploys and
no alert behaviour changes from the diff itself.

One live change, made outside the diff: HostBatteryHealthLow had been
firing unsilenced for oracle since 2026-09-14 at 72 % of design, against a
cell that is unbought and second in line. Silenced
01cb81d7-5e19-4e6d-b386-f5c8c843032b until 2026-10-08, sharing an expiry with
the #351 disk silence on the
same host so one look covers both. Because the rule is < 0.8 and no label
carries the ratio, that silence hides any further decay of the cell as well as
the 72 % it was created for — recorded in the new runbook's status banner and
its open items rather than only in Alertmanager.

Verification

• make validate passes — see the note below
• Deployed to the lab and confirmed working — n/a, documentation
• Docs updated

python3 scripts/check_docs.py prints docs OK — 82 Prometheus + 18 Loki rules, 7 dashboards, 141 panels, 10 assertions and exits 0. Reverting the
README count to 25 makes it print README.md:96 claims 25 runbooks; the repository has 26 and exit 1, so the bump is load-bearing rather than
cosmetic. ./scripts/lint.sh passes all five linters and markdownlint-cli2
reports 0 issues on the new file. Re-run after rebasing onto fc4490e.

The local ./scripts/validate.sh run was abandoned rather than completed.
Thirteen other worktree sessions were running it concurrently against the
shared Docker daemon at load 36, and it had not finished after twenty minutes.
Since this commit touches no rule, test or config, its promtool stages cannot be
affected by the diff, and CI on a clean runner is the better authority — so the
box above is ticked on CI rather than on that run. Flagging it rather than
implying a green local run.

Every metric name in the runbook was read back from the host's own Prometheus
on 2026-09-17 and all twelve return a count. Three baseline figures were
corrected before publishing: charge_ampere, voltage_volt and
temp_celsius are live-varying, and an instantaneous sample was being recorded
as though it were a fixed fact. temp_celsius in particular, which the safety
section leans on, ranges 32.7–39.2 over 24 hours averaging 33.3, so it is now a
band with a note that a single reading near 39 is ordinary. charge_full is
confirmed static at 6.196 across the retained window, so it is a real baseline.

Left for a separate issue

HostBatteryHealthLow's description names
docs/runbooks/fit-the-ups-battery.md, so the alert for a laptop cell points
at the UPS runbook. Correcting it means editing the rule and the
exp_annotations block that asserts the string verbatim — a rules-and-tests
change, not a docs one. Recorded under What is still open.

The dashboards carry no battery panel at all: power_supply appears in no
Grafana JSON, while every other hardware signal has one.

🤖 Generated with Claude Code