Skip to content

[0.3] Add CPU-first host discovery and placement foundation - #16

Merged
pschildgen87-code merged 19 commits into
mainfrom
feat/0.3.0-alpha.3-host-placement
Sep 5, 2026
Merged

pschildgen87-code merged 19 commits into
mainfrom
feat/0.3.0-alpha.3-host-placement

Conversation

@pschildgen87-code

Copy link
Copy Markdown
Contributor

Summary

This is the next Runtime 0.3 slice after the direct llama.cpp path. It establishes the host-resource contracts required before Runtime can make real backend/model activation decisions.

  • adds quantum.runtime/host-profile/v1alpha1 Linux discovery for CPU vendor/model/features, physical/logical CPUs, SMT ratio, NUMA nodes, RAM/huge-page metadata, block/NVMe devices and DRM-visible AMD/NVIDIA/Intel accelerators
  • discovery is fail-soft for optional sysfs data but validates core CPU/memory invariants before exposing the profile
  • adds explicit bounded calibration with a worker-count sweep across physical-core and logical/SMT candidates, recording memory copy/read throughput and the best observed worker count
  • calibration is explicit through POST /v1/host/calibrate; it does not consume resources automatically during daemon startup
  • adds quantum.runtime/placement-plan/v1alpha1 with separate model-weight, MoE-expert, KV-cache, prefix-cache, projector, workspace and cold-tier classes
  • implements the first CPU-first policy: CPU/RAM candidate first, GPU/hybrid only when the hot set does not fit and acceleration is explicitly allowed
  • allows model/MoE weight splitting across RAM/VRAM as a pre-activation candidate while keeping cache/workspace classes whole-tier in this slice
  • adds explicit NVMe placement only for declared cold bytes; active/hot state is never silently spilled to disk and capacity failures return capacity_exceeded
  • adds /v1/host, /v1/host/calibrate and /v1/placement
  • adds resource-control state for the last bounded calibration without mixing hardware discovery, calibration and placement responsibilities
  • documents the current boundaries and Turin Tier-1 relationship in docs/HARDWARE-PLACEMENT.md
  • bumps Runtime to 0.3.0-alpha.3

Validation already performed locally on the new packages

  • gofmt
  • go test ./... for hostprofile, calibration, placement, resourcecontrol
  • tested discovery on Linux with CPU/NUMA/RAM/block-device data
  • placement tests verify CPU-only remains preferred even when a GPU exists, hybrid is only considered after CPU capacity fails, explicit cold NVMe use, and no hot-state disk spilling

The repository PR CI remains the authoritative full-tree validation (gofmt, legal/systemd checks, version consistency, go vet, go test -race ./..., Runtime/installer builds).

Scope boundary

This slice does not yet auto-launch/reconfigure llama-server, persist calibration across restarts, set NUMA/thread affinity, estimate GGUF/KV sizes automatically, measure real-model prefill/decode throughput, dynamically query free VRAM, or claim a hybrid candidate is faster than CPU-only. Those require backend/model-specific conformance and the Turin hardware benchmark track.

Refs #8, #11, #12, #13.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-09-05T21:59:10.277366Z f728c5b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant