docs(runbooks): the Saruman SSD fit becomes a procedure, and the trays gate it

[Gerrrt]

Refs #418, Refs #76, Refs #148 — no close keywords. #418 is a tracker for a
physical fit that has not happened and cannot yet happen; this adds the document
that will make it provable, and records what has actually been decided.

What this does

Adds docs/runbooks/fit-the-saruman-ssds.md, in the shape of
fit-the-ups-battery.md and replace-the-smart-storage-battery.md — the same
machine, the same before/after table, the same silence discipline. Every step is
labelled with the host it runs on, because VLAN 30 is not reachable from the
monitoring host: steps 0, 1, 10 and 11 are Prometheus and Alertmanager, and
everything in between is run by hand from the Mac.

The layout question #418 left to the fit is answered. The SSDs become a
second RAID 1 logical drive on the P440ar, Smart Array managed; the 7.2K mirror
keeps Proxmox, the ISOs and the backups. That is the smaller change, it keeps a
spindle for the things that do not need IOPS, and it leaves
IloDrivePredictiveFailure, IloDriveSmartUnreadable and the
replaceDriveSSDWearOut(4) state reading cpqida.mib exactly as they do today —
so #351's decision to skip smart-state on this host stays right rather than
needing re-checking against an HBA layout.

Three things the live estate said that the documents did not

The runbook's "before" column is read, not assumed — every cpqDa* value came out
of Prometheus on 2026-09-17.

#76 looks worse than one column suggested. cpqDaAccelWriteCachePercent is the
column that issue has been about. The baseline shows cpqDaAccelMemory,
cpqDaAccelReadMemory and both cache percents at 0 against a 2 GiB module, with
cpqDaLogDrvHasAccel{1} reading other rather than enabled. Four independent
columns agreeing that nothing is allocated is much weaker support for "the iLO
simply does not populate the ratio" than one column was. The runbook takes the
ssacli reading that settles it — nobody has ever run that tool on this machine —
and hands the finding to #76 without acting on it: modify cacheratio= is
controller-wide, would land on the array holding every guest, and would move two
variables at once underneath the measurement.

There is exactly one guest — alexander, vmid 140. The migration is a single
online qm move-disk, so no guest stops, HypervisorGuestStopped never fires, and
#418's "guests down or not" question answers itself: no window is needed.

cpqDaPhyDrvType reads 3 (sata) for both existing drives, on a metric whose
enumeration carries a distinct 4: sas, and the model is MM1000GBKAL — while
hardware.md and ADR-0007 both say SAS. Flagged as a check at the rack, not
corrected: ADR-0007 is immutable, and neither document should be rewritten from an
SNMP enum alone.

What is deliberately not in this PR

No ADR changes. ADR-0029's ninety random write IOPS is derived, not measured —
seek plus half a rotation at 7200 rpm — so replacing it with a differently-shaped
measured number would be worse than leaving it alone. The runbook measures both
arrays at the parameters that derivation implies (4 KiB, queue depth 1, one stream)
and three times: HDD loaded, SSD idle, and HDD idle once the guest has moved off —
that last one only obtainable after the move, because until then the array is never
quiet. Until that reading exists, ADR-0029, ADR-0007, ADR-0017 and the roadmap's
#414 paragraph stand as written, and then each gets a dated [!NOTE] rather than a
silent edit. The runbook's "Flipping the documents" section carries the list.

No Compute table change. hardware.md says the Storage column changes when #418
fits them and not before. It has not.

The fit is blocked, and the runbook says so at the top

#498 recorded two SFF trays (651687-001) bought 2026-09-11 and still in transit on
2026-09-17. A Gen9 bay holds a drive only in a carrier, so this is a stop condition
at step 4 rather than something discovered at the rack. Steps 0–3 can all be done
first, and step 2 is the one most worth failing early: getting ssacli onto Debian
13 is something nothing in this repository is evidence of, and it is written as
an unknown with three fallbacks rather than as a command that works.

Fifteen such unknowns are listed at the end. Each is something the fit can settle,
and settling them is most of the value of doing it carefully.

CI

Green. make check-docs, make lint and the rest of make validate pass on this
branch, rebased onto main at 107256f.

This PR originally also carried two README counter fixes, because check-docs was
failing on main itself — stale ADR and runbook counts, plus two files both
claiming ADR-0041 from merges nine minutes apart. #503
has since fixed the duplicate (renumbered the second to ADR-0042) and the ADR
counter on main, so all that remains here is the runbook count going to 28 for the
runbook this adds.

🤖 Generated with Claude Code