make backup's volume sets stay on the host they protect: ADR-0015's copy to oracle is decided and unbuilt

[Gerrrt]

Decided and unbuilt

docs/roadmap.md's backup paragraph: "The volume sets make backup writes still sit on the host they protect … ADR-0015 sends them to oracle … and leaves only the copying to build." The firewall's configuration already makes that trip (#92's first half, verified 2026-09-03). The volume archives — Prometheus's TSDB, Loki, Grafana, and on trinity one day the tier's — do not.

A backup on the disk it backs up is a copy, not a backup. ADR-0015 chose oracle as the destination and the decision has sat inside #92's roadmap bullet, where it stopped belonging when ADR-0034 re-scoped that issue to the firewall rehearsal.

What to build

• A step in the backup timer, after the archive is written and verified, that copies the set to oracle over the path the firewall copy already uses (same user, atropos; same key; same directory convention).
• Retention on oracle: how many sets, and who prunes them. The firewall copy's answer is probably the right one.
• verify-backups learns to check the remote copy exists and matches, or the BackupCopyMissing-shaped rule that The heartbeat receiver is not a dead man's switch — nothing watches for its absence #359's dead-man pattern suggests.
• restore-the-stack.md gains the sentence "the set is on oracle at …", because a restore runbook that starts from a host that has just died is a runbook that starts nowhere.

Done when

A set written on prometheus is on oracle within the same timer run, a rule fires when one is not, and the restore runbook starts from the remote copy.

Refs #92, ADR-0015.

[Gerrrt]

Built and deployed by #550 (merged 2026-09-19 22:58 UTC).

On oracle now: backups/volumes/observability/ holds the five complete sets (20260829 → 20260906), seeded from the main checkout with make backup ARGS=--copy-only without stopping the stack, and make backup ARGS='--verify-only --all' then passed on both sides — every archive hashes to its MANIFEST entry there, every MANIFEST byte-identical. The daily verify-backups run does the same from tomorrow 05:30.

What closes this: the "same timer run" half of Done when is Sunday 2026-09-20 03:30 — the first homelab-backup-volumes run with the copy step in it. A new stamp under backups/volumes/observability on oracle after it, with homelab_job_last_exit_code{homelab_job="backup-volumes"} == 0, is the proof; a failed copy is ScheduledJobFailed within ten minutes, by design (no new rule — backup.rules.yaml names no job).

Two loose ends, neither blocking:

• The 2026-09-13 run failed on the loki-data sentinel (backup-volumes and verify-backups fail since 2026-09-13: loki-data carries paperless-data's ./index sentinel #468, fixed by fix(backup): a sentinel that is another volume's companion is not, on its own, a crossed mapping (#468) #469 the next day; the live checkout has it). Its incomplete set backups/volumes/20260913T033005Z is still on prometheus for a hand removal, as the script's own warning asks; it was correctly not copied.
• The two unit files changed only in their comments (VOL_OFFHOST named as a host fact; the verify unit now says it reaches oracle). sudo ./scripts/install-timers.sh --install re-copies them when convenient; the running units behave identically without it.

[Gerrrt]

A second directory rides this copy. #554 (for #484) pulls Jellyfin's state off smaug into backups/nas/ on the monitoring host and copies it to oracle under backups/nas — beside backups/volumes/<stack> — by the helpers this issue's PR landed: backup-nas.sh sets VOL_OFFHOST to that directory before it sources backup-volumes.sh, so the value the volume unit puts in /etc/default/homelab-timers is never inherited, and every rule (verify by far-side hash, prune to KEEP, .part-then-mv, no globs over the wire) applies unchanged. make verify-backups walks both directories on both sides. Nothing here changes what this issue still owes: the seed and the first same-run copy on the deployment checkout.

Refs #484, refs #554.