Skip to content

AGENTS.md

Latest commit

 

History

History
148 lines (126 loc) · 8.54 KB

AGENTS.md

File metadata and controls

148 lines (126 loc) · 8.54 KB
Raw

AGENTS.md — Berth

Berth is the VRAM leasing service for every GPU you run. One Go daemon per machine hands a single card to many inference engines in turn: engines lease the card, the holder is drained, Berth waits until the driver reports the memory free, then admits the next engine. Processes holding VRAM without a lease are stowaways, reported, never touched. Linux + NVIDIA; Linux AMD discrete RDNA3/RDNA4 and Phoenix APU (780M) support is experimental. APU leases are exclusive, with separate reserved UMA VRAM/GTT accounting and host-RAM admission checks; APU parking is not supported. Alpha.

Commands

Run from the repo root (a worktree):

# the one command that proves the tree is sound: build, format, vet, all tests
go build ./... && test -z "$(gofmt -l .)" && go vet ./... && go test ./...

# the Windows node must keep cross-building (CI runs this too)
GOOS=windows go build ./... && GOOS=windows go vet ./...

# lint: the exact CI command; only `go` is needed on PATH
go run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.14.0 run ./...

# build the binary (quick start: README.md)
go build -o ~/.local/bin/berth ./cmd/berth

CI (.github/workflows/ci.yml) runs exactly those commands, in that order, on Linux, and a windows job runs build, vet and go test ./... on windows-latest. A test that starts a Unix program or relies on Unix signals carries //go:build unix; everything else must pass on both. Keep the local proof command green before opening a PR; CI must end green at merge.

Packages

  • cmd/berth — CLI entrypoint: main and the release version only.
  • internal/broker — the core: the lease table for one GPU, priorities, the drain ladder, process attribution, stowaway detection, the queue. In-memory only; never kills.
  • internal/cli — the berth command tree: one flag set, one output contract (human, json, yaml) and one exit-code contract (0 done, 1 validation, 2 system) for every command.
  • internal/config — TOML config (~/.config/berth/berth.toml), its defaults and its warnings; berth config check and berth config show.
  • internal/comfy — the ComfyUI HTTP calls Berth makes (queue, free, system stats), once.
  • internal/engines — engine readers (every engine in docs/engines.md's table, custom), port discovery by response fingerprint, and the 24 h request log.
  • internal/export — the store's rows as CSV or JSON for the berth export command.
  • internal/fsx — the one atomic file write (fsync the file and its directory).
  • internal/gpu — the shared Probe contract, NVIDIA nvidia-smi polling, and Linux AMD sysfs/DRM observations (discrete RDNA3/RDNA4 and Phoenix APUs); on Windows, NVIDIA per-process VRAM from the PDH counter Task Manager reads. Guarded AMD reads require fresh driver evidence for admission and release. See contrib/amd/README.md for validation.
  • internal/harbor — the berth harbor web UI server and its collectors; UI assets live in internal/harbor/web/.
  • internal/history — the node's last hour of samples in memory at 5 s resolution.
  • internal/host — host telemetry: CPU, RAM, swap, load, memory pressure (Linux PSI); Windows via GlobalMemoryStatusEx.
  • internal/kvsnap — llama.cpp slot snapshots: the slot client and the budgeted snapshot index.
  • internal/insights — request KPIs, token ledger, leaderboard; one implementation over the node's in-memory log and the harbor's SQLite rows.
  • internal/notify — berth notify: which events become desktop notifications at each level, the burst gate, notify-send on Linux and the Win32 tray on Windows.
  • internal/pagecache — how much of a file the page cache holds (mincore), and prefetch.
  • internal/park — park and resume adapters (ComfyUI, vLLM, SGLang, custom hooks, the Berth park protocol) and the RAM admission and eviction rules. The lease state lives in the broker.
  • internal/procs — process attribution: /proc parent walk and container IDs from cgroups on Linux; Toolhelp snapshots and process times on Windows.
  • internal/proto — NDJSON message types over the Unix socket.
  • internal/runner — the berth run wrapper: own process group, drain ladder, fail-open start.
  • internal/runs — unprotected-run records (daemon down, or gone mid-run).
  • internal/server — the Unix socket server: leases, event log fan-out, stowaway notifications, the loopback /metrics listener.
  • internal/skill — the agent skills berth skill install writes: SKILL.md (berth) and comfyui/SKILL.md. Users' agents act on every command in them, so a change to a flag, a route or a default that a skill names updates the skill in the same PR. Skills are public text: write them like the docs.
  • internal/store — the harbor's SQLite store (modernc.org/sqlite; pure Go, no CGO).
  • internal/storage — model roots: filesystem, free space, the free-space floor.
  • internal/swapcost — measured switch times per engine and tier, and chain plans.
  • test/ui — dev-only Playwright click/hover check; not part of go test ./..., never in CI.
  • contrib/ — systemd user units (and units.go, which embeds them for berth service install), the llama-swap patch, INTEGRATION.md, chaos/ (the failure scenarios), bench/ (the swap benchmark), windows/ (the Windows checklist).

License header

Every .go, .py and .sh file starts with these two lines (# instead of // outside Go), above any //go:build line, followed by a blank line. TestSourceFilesCarryTheLicenseHeader fails without them.

// Copyright 2026 Trevin Tindall (T-Crypt)
// SPDX-License-Identifier: Apache-2.0

What Berth never does (invariants — do not break them)

  • Never kill a process it did not start, or any process by default. Stowaway handling is report-only.
  • Never grant a card because a process exited; grant only after the driver shows the memory free.
  • Never claim memory is reserved when it is only estimated to be free.
  • Never take a card on another user's machine without their consent (on one machine, only the priorities you set decide).
  • No cloud account, no telemetry upload, no other software needed to run.
  • Process identity is PID plus start time; a reused PID never inherits a lease.
  • berth run is fail-open: with the daemon down the engine still starts and is recorded as an unprotected run. Only --fail-closed refuses.
  • The web UI listens on loopback by default; a non-loopback --listen address is the operator's explicit opt-in to expose it (warning at startup, banner on the page, no authentication in this release). --allow-host names a same-machine reverse proxy's hostname for a loopback harbor; the listener stays on loopback. /metrics stays loopback-only.
  • Berth collects every stat itself (driver, /proc, the engines' own APIs). It has no dependency on Speculum or any other tool.

Do not touch

  • No package managers (Makefile, Taskfile, npm) and no JS CI stage; the native Go toolchain is the only harness. test/ui Playwright checks stay dev-only.
  • Do not swap modernc.org/sqlite for a CGO sqlite driver; pure Go is a hard requirement.
  • Do not pin a different Go or linter version than CI does: Go comes from go.mod (1.27.1), the linter from the @v2.14.0 line in AGENTS.md and ci.yml. Bumping either means changing both in the same commit.
  • An engine reader decides up/down only from what the engine serves unconditionally; optional endpoints (a /metrics that is off by default, a list behind a key) add detail and are never a liveness signal.

When you find a Berth bug

If you hit an error while working with Berth (a swap or drain that never completes, a stuck lease, a wrong attribution, a grant onto a full card, a crash, a wrong figure in status, a test that only fails on one OS), do not work around it silently:

  • Prefer a fix: a branch, a test that fails before the fix, the fix, a PR (rules below).
  • If you cannot fix it now, open an issue: gh issue create --repo T-Crypt/berth with what you ran, what happened, what you expected, berth version, and berth events --format json around it. Use the "Drain report" form (.github/ISSUE_TEMPLATE/drain-report.yml) for drain and swap failures. Strip API keys, tokens and private paths first.

Git

  • One change per branch; deliver through a pull request against main. No force-push to shared branches.
  • Commit subjects: <type>(<scope>): <summary> with type in feat, fix, refactor, test, docs, chore, ci, build (e.g. fix(engines): read vLLM's window).
  • Keep the proof command and lint green before opening a pull request.