A small GPU job scheduler I wrote to learn Go and to get a feel for what
GPU cloud control planes actually do. One binary, three roles: gpuq serve
is the control plane (queue + scheduler), gpuq agent runs jobs on a GPU
node via Docker, and the rest of the subcommands are the client.
This is a personal learning project. No auth, state lives in memory, and I only run it on my own two machines. Don't use it for anything real.
go build -o gpuq ./cmd/gpuq
./gpuq serve # terminal 1
./gpuq agent --fake-gpus 2 # terminal 2
./gpuq submit -i ubuntu -- echo hi # terminal 3
./gpuq ps
./gpuq logs j-1scripts/demo.sh local runs that whole scenario by itself.
Install Docker and the NVIDIA Container Toolkit, then run the agent without
--fake-gpus. The deploy/ directory has the systemd units I use on my own
machine. Jobs can carry environment variables (env in the POST /jobs body),
which I ended up needing to work around a CUDA driver/image version mismatch.
A job is a small state machine: pending, scheduled, running, then one of
succeeded/failed/canceled. The scheduler is a single goroutine doing GPU slot
accounting over registered nodes. Agents poll the control plane for work, run
each job with docker run --rm --gpus device=N, and report the exit code and
logs back. The runner is an interface, so the whole system runs and tests on
a laptop with no GPU using a fake runner.
- State is in memory. A control plane restart forgets every job. I learned this the hard way when it got OOM-killed mid-experiment and came back empty.
- Agents short-poll every 500 ms, which puts about a second of dead time into every dispatch.
- Whole-GPU slots only, FIFO only, one scheduler.
- No auth at all. I only reach it over my tailnet.
If I keep working on it: long-poll or a watch stream instead of polling, some minimal persistence, and volume mounts for the runner (weights currently have to come in over HTTP, which is a silly workaround).