Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 36 additions & 31 deletions docs/hardware.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ quietly swapped.
| Host | Hardware | CPU | RAM | Storage | OS |
| --- | --- | --- | --- | --- | --- |
| `morpheus` | HP ProDesk 600 G4 Mini | i5-8500T | 32 GB | 1 TB NVMe SSD | FreeBSD 16.0 (pfSense) |
| `Saruman` | HPE ProLiant DL360 Gen9 | 2× Xeon E5-2680 v3 (48 threads) | 128 GB | 2× 1 TB SAS HDD, RAID 1 | Proxmox VE 9 |
| `Saruman` | HPE ProLiant DL360 Gen9 | 2× Xeon E5-2680 v3 (48 threads) | 128 GB | 2× 1 TB SAS HDD, RAID 1; 2× 960 GB SATA SSD, unassigned | Proxmox VE 9 |
| `prometheus` | Apple MacBook Pro (2012, Retina 13") | i5/i7 | 8 GB | 256 GB SSD | Ubuntu Server 24.04 LTS |
| `oracle` | Dell Inspiron 15-3565 | AMD A6-9200 (2 cores) | 4 GB | 500 GB HDD | Ubuntu Server 24.04 LTS |
| `smaug` | Lenovo ThinkServer TS150 | Xeon E3-1225 v6 (4 cores) | 8 GB ECC | 240 GB SATA SSD (boot) | TrueNAS 25.10 |
Expand Down Expand Up @@ -322,31 +322,43 @@ revisions of this repository treated `shiva` as the hypervisor itself.
it — worth knowing before reading a slow self-test as a sick disk. TrueNAS's
scheduled tests take it from here.
- 2× Samsung SM863a 960 GB (`MZ-7KM960N`), 2.5" SATA 6 Gb/s enterprise
SSDs with power-loss protection[^SM863a] — purchased 2026-09-09, **in hand
since 2026-09-11**, for the ProLiant's SFF bays. Bought against the number
every sizing
decision on `Saruman` starts from: a 7.2K mirror serving about ninety
random write IOPS
SSDs with power-loss protection[^SM863a] — purchased 2026-09-09, delivered
2026-09-11, **fitted 2026-09-18 in bays 3 and 4 of the ProLiant, and not
yet in any logical drive.** Bought against the number every sizing decision
on `Saruman` starts from: a 7.2K mirror serving about ninety random write
IOPS
([ADR-0029](adr/0029-size-the-lab-domain-and-separate-its-namespace-and-clock.md)).
Serials go here when they land — read off the labels before the drives go
into the bays, which is step 4 of
[`fit-the-saruman-ssds.md`](runbooks/fit-the-saruman-ssds.md), because
reading them back afterwards means reading them through the tool the fit is
trying to verify. The Compute table's Storage column changes
when [#418](https://github.com/Gerrrt/HomeLab/issues/418) fits them, and
not before — that issue also names the ADRs whose arithmetic the fit makes
stale. The fit itself is
[`fit-the-saruman-ssds.md`](runbooks/fit-the-saruman-ssds.md), whose step 4
reads both serials off the labels before the drives go into the bays —
reading them back afterwards means reading them through the tool the fit is
trying to verify
- 2× HP 2.5" SFF drive tray, `651687-001`[^Caddy] — bought 2026-09-11, in
transit, quoted to arrive by 2026-09-17 and not here on the morning of it.
The carriers the SM863a pair above needs to sit in `Saruman`'s SFF bays
Serial `S3F3NX0K601487` in Bay 3 and `S3F3NX0K806107` in Bay 4, firmware
`GXM5304Q`, both negotiated at 6 Gb/s — **as the iLO reports them over
SNMP, not as read off the labels.** The fit runbook asked for the labels
first, because reading a serial back through the controller means reading
it through the tool the fit is trying to verify; that was not done, and the
runbook's step 4 says how it still can be while the drives are unassigned.
The iLO gives the model only as `SAMSUNG` — it names a third-party SATA
drive by vendor where it gives the HPE disks their part number — so the part
is proved by the size (`915715` MB), the firmware and the serial prefix
rather than by the model string. Both read solid-state, SMART `ok` and
`notConfigured` from the first scrape they appeared in, at 20:11 UTC, and
nothing alerted. The Compute table's Storage column moved on the day, as
this entry said it would. What has not moved is everything after the bays:
no logical drive, no thin pool, `alexander` still on the HDD mirror, and no
measurement — so
[#418](https://github.com/Gerrrt/HomeLab/issues/418) stays open and the ADRs
whose arithmetic it names stand as written until step 8 of
[`fit-the-saruman-ssds.md`](runbooks/fit-the-saruman-ssds.md) produces a
number. This entry is where the serials live, which is the question
[#148](https://github.com/Gerrrt/HomeLab/issues/148) asked.
- 2× HP 2.5" SFF drive tray, `651687-001`[^Caddy] — bought 2026-09-11,
**arrived and fitted 2026-09-18**, one under each SM863a in bays 3 and 4.
The carriers the pair above needs to sit in `Saruman`'s SFF bays
([#418](https://github.com/Gerrrt/HomeLab/issues/418)). A Gen9 bay holds a
drive only in a tray, so two drives want two trays, and two is what was
bought — worth writing down, because one tray short is one SSD fitted and
one on a shelf.
one on a shelf. The arrival checks this entry asked for are answered by the
bays rather than by the packaging: both trays took a drive, and the iLO
reads the same carrier firmware on all four bays
(`cpqDaPhyDrvSmartCarrierAppFWRev` `11`, bootloader `6`), so they are the
Gen8/Gen9 SmartDrive carrier and not the Gen10 part or the 3.5" LFF one.
**This purchase went unrecorded for six days**, which is the omission this
entry exists to close. It was made in the same sitting as the I226 card, the
Exos pair, and the boot disk's bracket and tape, every one of which has had
Expand All @@ -355,15 +367,8 @@ revisions of this repository treated `shiva` as the hypervisor itself.
rule failing rather than a thing the rule allows. Found on 2026-09-17 while
reading the ProDesk's arrival paperwork for
[#92](https://github.com/Gerrrt/HomeLab/issues/92), which is not a way of
finding purchases that can be relied on.
**Checked on arrival, not assumed:** that they are the Gen8/Gen9 SmartDrive
carrier and not the Gen10 part or the 3.5" LFF one, neither of which fits
this machine; that both arrived; and that each has its screws, since a
carrier without them holds a drive no better than no carrier. `651687-001`
is the listing's part number and not yet the tray's.
**Until they land the SSDs cannot be fitted at all**, which is the sharpest
form of the point #418 is built on: arriving is not fitting, and neither the
Compute table nor ADR-0029's arithmetic moves for a drive on a shelf.
finding purchases that can be relied on. `651687-001` is the listing's part
number, not read off the tray.
- MikroTik CRS326-24G-2S+RM[^CRS326] — 24 × 1 GbE, 2 × SFP+, 1U, dual-boot
RouterOS / SwOS — bought used 2026-09-13; in transit, delivery estimated
2026-09-23, moved out from the 09-16 to 09-21 window quoted at purchase. The
Expand Down
63 changes: 42 additions & 21 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,13 @@ restore onto `trinity` needs
installer stick — between them, the whole of what the rehearsal now waits on;
the two Exos X20 18 TB drives for `smaug`'s mirror
([#413](https://github.com/Gerrrt/HomeLab/issues/413)); the CRS326 that
replaces `neo` ([#444](https://github.com/Gerrrt/HomeLab/issues/444)); and two
HP SFF drive trays for `Saruman`, the carriers its SM863a pair needs to go in
the bays ([#418](https://github.com/Gerrrt/HomeLab/issues/418)). The A1437 cell
for `prometheus` was on this list until 2026-09-18 and is on neither now: it is
fitted ([#454](https://github.com/Gerrrt/HomeLab/issues/454)). These two
paragraphs track money that has not yet become a working part, and that has.
replaces `neo` ([#444](https://github.com/Gerrrt/HomeLab/issues/444)). The
A1437 cell for `prometheus` was on this list until 2026-09-18 and is on neither
now: it is fitted ([#454](https://github.com/Gerrrt/HomeLab/issues/454)). The
two SFF drive trays for `Saruman` left it the same day, and took the SM863a
pair off the paragraph above with them — see the 2026-09-18 note below. These
two paragraphs track money that has not yet become a working part, and that
has.

**2026-09-17: the trays were bought on 2026-09-11 and are only now written
down.** They were paid for in the same sitting as the I226 card, the Exos pair
Expand All @@ -150,6 +151,15 @@ against the order, which was one order for the pair; that both drives were in
it is not separately proved and is
[#418](https://github.com/Gerrrt/HomeLab/issues/418)'s to confirm at the bays.

**2026-09-18: the trays arrived and both SSDs are in the bays.** Confirmed at
the bays, as the paragraph above asked: the iLO reports two new drives at
`Port 1I Box 1 Bay 3` and `Bay 4` from 20:11 UTC, with different serials, so
the pair was a pair. Fitted is not finished — the drives are in no logical
drive, nothing has moved to them, and the measurement the purchase exists for
has not been taken; [#418](https://github.com/Gerrrt/HomeLab/issues/418)
carries that. Neither the SSDs nor the trays appear in either paragraph above
now: they are parts, and [`hardware.md`](hardware.md) has them.

**No row on the table above moves** — arriving is not buying, and this section
tracks money. Nothing was racked, fitted or built by it either. What it changed
is one sentence: the two builds stopped waiting on a van for their machines,
Expand Down Expand Up @@ -883,9 +893,10 @@ what left this one unfireable for months.
DC-plus-workstations domain is the DC itself. The servers run continuously and
the endpoints per session, because a 7.2K mirror serves about ninety random
write IOPS and six idle Windows guests would be most of them — the number
#418's SSDs were bought against, and the one their *fit* re-derives; a drive
on the shelf changes no duty cycle either, and since 2026-09-11 that is
where they are — waiting on the trays that let them into the bays at all.
#418's SSDs were bought against, and the one their *fit* re-derives. The
drives went into bays 3 and 4 on 2026-09-18 and are still unassigned, and an
unassigned drive changes no duty cycle either — the sentence above holds
until the runbook's step 8 measures both arrays.
Three things
that ADR left explicit because they fail quietly: the DC takes its clock from
the gateway, not `time.windows.com` — ADR-0014 named that failure and did not
Expand Down Expand Up @@ -978,7 +989,8 @@ what left this one unfireable for months.
→ [runbook](runbooks/build-the-playground.md)
- **[#418](https://github.com/Gerrrt/HomeLab/issues/418) Fit the two SSDs in
`Saruman`.** Two Samsung SM863a 960 GB SATA enterprise drives, bought
2026-09-09, **delivered 2026-09-11, not fitted and not yet fittable**.
2026-09-09, delivered 2026-09-11, **fitted 2026-09-18 and in no logical
drive yet**.
ADR-0007's constraint — "the
fleet is sized against spindles, not RAM" — became a number in ADR-0029,
about ninety random write IOPS for the whole machine, and that number sized
Expand All @@ -997,14 +1009,23 @@ what left this one unfireable for months.
array holding every guest, so the runbook takes the `ssacli` reading nobody
has ever taken on this machine and #76 owns what to do about it.

**What blocks it is neither of those: the trays.** A Gen9 bay holds a drive
only in a SmartDrive carrier, two were bought `651687-001` on 2026-09-11,
and on the morning of 2026-09-17 they had not arrived. Until they do, the
drives cannot enter the machine, which is why the runbook makes the carriers
a stop condition at step 4 rather than something discovered at the rack. The
same step confirms both drives are physically present — the delivery notice
covers one order, not two units, which the paragraph above leaves to this
issue to prove at the bays.
**The trays blocked it for a day, and the fit stopped at the bays.** A Gen9
bay holds a drive only in a SmartDrive carrier; two `651687-001` were bought
2026-09-11, had not arrived on the morning of 2026-09-17, and were in the
chassis with a drive in each by 20:11 UTC on 2026-09-18 — both drives
present, `Bay 3` and `Bay 4`, different serials, solid-state, SMART `ok`,
`notConfigured`, and nothing alerted. That is the runbook's step 4 and no
step after it: no `ssacli` on the host yet, no logical drive, no thin pool,
`alexander` still on the HDD mirror. Three predictions were wrong on the day
and are corrected in the runbook — the iLO names the drive `SAMSUNG` and not
by part number, the walk did not get slower, and the serials were read back
through the iLO rather than off the labels first, which the runbook now says
can still be put right while the drives are unassigned and not after. One
reading is worse than hoped: the wear and endurance columns are blank on
both SSDs, as they are on the HDDs, and if they stay blank once the drives
are in a logical drive, wear monitoring on the newest drives in the estate
needs `smartctl` through the `hpsa` path, which is an issue the runbook's
closing paragraph already describes.

What is still open after that is the only thing the purchase was for.
**ADR-0029's
Expand All @@ -1013,9 +1034,9 @@ what left this one unfireable for months.
implies, 4 KiB at queue depth 1, and three times: the HDD mirror loaded, the
SSD array idle, and the HDD mirror idle once the guest has moved off it.
Until that reading exists, ADR-0029, ADR-0007, ADR-0017 and the #414
paragraph above all stand as written. The fit makes them stale on the day,
and it gets a dated note on each rather than a silent edit — ADR-0001 makes
them immutable.
paragraph above all stand as written. The measurement makes them stale on
the day it lands, not the day the drives did, and it gets a dated note on
each rather than a silent edit — ADR-0001 makes them immutable.
→ [runbook](runbooks/fit-the-saruman-ssds.md)

## Automation
Expand Down
Loading
Loading