Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
06e31e9
Add guest-aware runtime CPU limit discovery
pschildgen87-code Sep 5, 2026
1fe6f36
Test host limit CPU list parsing
pschildgen87-code Sep 5, 2026
e1c1d80
Add deployment profile contract
pschildgen87-code Sep 5, 2026
b503115
Add E.M.B.E.R. production deployment profile
pschildgen87-code Sep 5, 2026
fb8c4da
Test E.M.B.E.R. deployment profile
pschildgen87-code Sep 5, 2026
5e31bd9
Add application profile admission engine
pschildgen87-code Sep 5, 2026
10197e1
Test E.M.B.E.R. admission decisions
pschildgen87-code Sep 5, 2026
4880309
Add real-model CPU benchmark plan contract
pschildgen87-code Sep 5, 2026
112c9ac
Test current 20-core Ember benchmark matrix
pschildgen87-code Sep 5, 2026
f6751a4
Expose deployment admission and benchmark planning APIs
pschildgen87-code Sep 5, 2026
f25b752
Chain deployment and resource routes
pschildgen87-code Sep 5, 2026
1e75443
Expose guest-aware CPU limits with host resources
pschildgen87-code Sep 5, 2026
9771c4f
Test deployment and benchmark APIs
pschildgen87-code Sep 5, 2026
f432dca
Document observed E.M.B.E.R. deployment generations
pschildgen87-code Sep 5, 2026
dfb18a5
Bump Runtime to 0.3.0-alpha.4
pschildgen87-code Sep 5, 2026
b397422
Update build version for alpha.4
pschildgen87-code Sep 5, 2026
36db2df
Document Runtime 0.3.0-alpha.4
pschildgen87-code Sep 5, 2026
795e139
Update README for alpha.4 admission layer
pschildgen87-code Sep 5, 2026
aa6fa54
gofmt benchmark plan
pschildgen87-code Sep 5, 2026
fa4c842
gofmt admission tests
pschildgen87-code Sep 5, 2026
87fea51
gofmt host limit discovery
pschildgen87-code Sep 5, 2026
0284afc
Normalize virtualization evidence before classification
pschildgen87-code Sep 5, 2026
1387441
Document alpha.4 admission and benchmark APIs
pschildgen87-code Sep 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Changelog

## 0.3.0-alpha.4 - 2026-09-06

- Added `quantum.runtime/deployment-profile/v1alpha1` with a builtin `ember-production` application profile kept separate from generic Quantum Runtime minimums.
- Encoded the current supported E.M.B.E.R. production requirements as 64 GiB memory, ECC required, DDR5 preferred, 8 physical cores, GPU optional and MoE architecture mandatory; the historical ~2.6 GHz reference remains advisory only.
- Added `quantum.runtime/admission-result/v1alpha1` with explicit `admitted`, `rejected` and `needs_operator_evidence` decisions and per-requirement evidence instead of guessing missing ECC/provider facts.
- Added guest/process CPU-limit discovery from process affinity, cgroup cpusets and CPU quota plus virtualization evidence so an EPYC host model name cannot imply that all host-physical cores are allocated to a KVM guest.
- Added operator core-budget evidence so application admission can reserve guest CPU capacity for other services without changing global Runtime topology.
- Added `quantum.runtime/model-benchmark-plan/v1alpha1` to generate repeatable CPU worker-count matrices without claiming unmeasured throughput; a 20-core guest with four reserved system cores produces the intended 8/12/16 production candidates plus an optional 20-core full-host comparison.
- Added `/v1/deployment-profiles`, `/v1/admission` and `/v1/benchmark-plan`; `/v1/host`, calibration and placement responses now also expose guest/process CPU limits.
- Added `docs/EMBER-DEPLOYMENT-HISTORY.md` distinguishing historical functional compatibility with small models/RAG-Lite/LTM from the current supported E.M.B.E.R. production/intelligence profile.
- Recorded the RS 4000 G12 (12 dedicated cores, 32 GB DDR5 ECC, 1 TB NVMe, Gemma 3 12B Q4) as a historical practical-use reference and the RS 12000 G12 (20 dedicated guest cores, 96 GB DDR5 ECC, ~3 TB NVMe RAID-10, KVM, Debian 13, 16-core Ember budget) as the current primary reference host.
- Kept exact hardware for the earliest low-cost Netcup root-server generation unknown rather than reconstructing missing values from current product offerings.

## 0.3.0-alpha.3 - 2026-09-05

- Added `quantum.runtime/host-profile/v1alpha1` Linux discovery for CPU vendor/model/features, physical/logical CPUs, SMT ratio, NUMA nodes, RAM/huge-page metadata, block/NVMe devices and visible AMD/NVIDIA/Intel accelerator metadata.
Expand Down Expand Up @@ -140,4 +153,4 @@ Initial Quantum Runtime foundation.
- Quantum native chat API
- OpenAI-compatible API
- GPU scheduling
- Quantum CoreOS TCI runtime integration
- Quantum CoreOS TCI runtime integration
61 changes: 54 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Quantum Runtime is the independent AI runtime and model-service project of Starlight Unit Studios.

Current version: `0.3.0-alpha.3`
Current version: `0.3.0-alpha.4`

## Current alpha scope

Expand All @@ -14,6 +14,7 @@ Quantum Runtime 0.3 adds the first model- and engine-neutral backend capability
2. the versioned model identity and read-only registry contract
3. a non-destructive Linux service package that can be installed beside an existing Ollama service
4. host-resource discovery, bounded calibration and a CPU-first placement contract for RAM/VRAM/NVMe candidates
5. application deployment-profile admission, guest/process CPU-limit discovery and repeatable real-model benchmark planning

Quantum Runtime now has two local execution paths. Ollama remains the default adoption/fallback mode, while a direct llama.cpp path can talk to `llama-server` without an Ollama daemon in the inference request path:

Expand Down Expand Up @@ -43,9 +44,13 @@ The initial llama.cpp adapter deliberately uses the external `llama-server` proc
- Ollama chat/generate/embed compatibility translation to llama.cpp OpenAI-style endpoints
- fail-closed handling for llama.cpp features not yet normalized: vision, tools, reasoning control and structured output
- Linux host discovery for CPU topology/features, NUMA, RAM/huge pages, block/NVMe devices and visible accelerator/VRAM metadata
- guest/process CPU-limit discovery from affinity, cgroup cpusets and CPU quota with virtualization evidence
- explicit bounded memory-bandwidth/worker-count calibration rather than automatic startup benchmarking
- versioned CPU-first placement plans that prefer RAM/CPU when the hot set fits and only consider hybrid acceleration after CPU capacity fails
- explicit NVMe cold-tier placement; hot execution state is never silently spilled to disk
- machine-readable application deployment profiles separate from generic Runtime minimums
- E.M.B.E.R. production admission with explicit `admitted`, `rejected` and `needs_operator_evidence` decisions
- repeatable real-model CPU benchmark-plan generation without claiming unmeasured throughput
- streamed forwarding/translation, cancellation, body limits and backend timeouts
- loopback-only default on `127.0.0.1:11450`
- bearer-token requirement for any non-loopback bind
Expand Down Expand Up @@ -78,6 +83,7 @@ curl -s http://127.0.0.1:11450/v1/backends
curl -s http://127.0.0.1:11450/v1/model-policies
curl -s http://127.0.0.1:11450/v1/upstreams
curl -s http://127.0.0.1:11450/v1/host
curl -s http://127.0.0.1:11450/v1/deployment-profiles
curl -s -X POST http://127.0.0.1:11450/v1/host/calibrate
```

Expand Down Expand Up @@ -116,9 +122,9 @@ The machine-readable schema lives at:
schema/model-manifest-v1alpha1.schema.json
```

Builtin contract/profile examples live under `internal/modelregistry/data/` for a generic model, Ember CoreUI and the future Quantum CoreOS Gemma 4 e4b TCI profile.
Builtin contract/profile examples live under `internal/modelregistry/data/` for a generic model, Ember CoreUI, Gemma 4 26B A4B MoE reference and the future Quantum CoreOS Gemma 4 e4b TCI profile.

The manifest separates canonical model identity from aliases, source revision, backend, artifacts, SHA-256 integrity, capabilities, compatibility, persona package, lifecycle state and provenance. In 0.3, artifacts may additionally declare their own backend/format/role, allowing one canonical identity to acquire multiple backend artifacts without changing the client-facing model ID. Generic architecture metadata now distinguishes dense, MoE and unknown profiles and can carry context-policy and expert-topology data without exposing family-specific fields in the public API. An `unverified` example is a contract/profile reference, not a cryptographic claim about unresolved model artifacts.
The manifest separates canonical model identity from aliases, source revision, backend, artifacts, SHA-256 integrity, capabilities, compatibility, persona package, lifecycle state and provenance. In 0.3, artifacts may additionally declare their own backend/format/role, allowing one canonical identity to acquire multiple backend artifacts without changing the client-facing model ID. Generic architecture metadata distinguishes dense, MoE and unknown profiles and can carry context-policy and expert-topology data without exposing family-specific fields in the public API. An `unverified` example is a contract/profile reference, not a cryptographic claim about unresolved model artifacts.

## Direct llama.cpp backend

Expand All @@ -141,11 +147,51 @@ If the llama.cpp server uses `--api-key`, set `QUANTUM_RUNTIME_LLAMA_CPP_API_KEY

## CPU-first host placement

`0.3.0-alpha.3` adds the first generic hardware/resource contract. `GET /v1/host` reports OS-visible CPU, NUMA, RAM, storage and accelerator metadata. `POST /v1/host/calibrate` runs an explicit bounded synthetic memory/worker sweep. `POST /v1/placement` creates a pre-activation capacity plan for model weights, MoE experts, caches, projectors, workspace and an optional explicit cold NVMe tier.
`0.3.0-alpha.3` added the first generic hardware/resource contract. `GET /v1/host` reports OS-visible CPU, NUMA, RAM, storage and accelerator metadata. `POST /v1/host/calibrate` runs an explicit bounded synthetic memory/worker sweep. `POST /v1/placement` creates a pre-activation capacity plan for model weights, MoE experts, caches, projectors, workspace and an optional explicit cold NVMe tier.

`0.3.0-alpha.4` adds a separate guest/process limit view. CPU affinity and cgroup cpusets define the usable guest/process CPU set; cgroup CPU quota is reported independently. Virtualization evidence is surfaced so a physical host model such as a 96-core EPYC 9645 cannot silently become a 96-core Runtime allocation when a KVM guest was allocated fewer cores.

CPU/RAM remains the baseline. A visible GPU does not automatically win: if the hot working set fits usable RAM, the first planner returns `cpu_only`. Hybrid placement is only considered after CPU-only capacity fails and acceleration was explicitly allowed. NVMe is never used as an implicit escape hatch for hot state.

This is a truthful capacity/host foundation, not yet a claim of measured model throughput. Real backend/model prefill/decode calibration, NUMA affinity, thread pinning, GGUF size estimation and backend activation from placement plans remain later Runtime 0.3 slices. See `docs/HARDWARE-PLACEMENT.md`.
## E.M.B.E.R. production admission

`ember-production` is an **application profile**, not a generic Runtime minimum. The current profile requires 64 GiB memory, ECC, at least eight physical cores and an MoE model. DDR5 is preferred. A GPU is optional. The historical ~2.6 GHz/core figure remains advisory and is never a hard admission cutoff.

Properties that ordinary guest userspace cannot prove reliably remain `unknown`. Operator evidence can be supplied explicitly, for example on the current KVM deployment:

```bash
curl -s -X POST http://127.0.0.1:11450/v1/admission \
-H 'Content-Type: application/json' \
-d '{
"profile":"ember-production",
"model":"gemma4:26b-a4b-reference",
"operator_evidence":{
"ecc_verified":true,
"memory_class":"ddr5",
"dedicated_physical_cores":20,
"core_budget":16
}
}'
```

A dense model fails the MoE production requirement instead of being silently substituted while preserving the E.M.B.E.R. profile identity.

## Real-model CPU benchmark planning

`POST /v1/benchmark-plan` creates the worker-count matrix for a canonical model without running or inventing benchmark results. Example for a 20-core guest where four cores remain reserved for website/game/system workload:

```bash
curl -s -X POST http://127.0.0.1:11450/v1/benchmark-plan \
-H 'Content-Type: application/json' \
-d '{
"model":"gemma4:26b-a4b-reference",
"reserve_system_cores":4,
"minimum_workers":8,
"include_full_host_comparison":true
}'
```

On a 20-core effective guest allocation this yields 8/12/16 production candidates and a 20-core full-host comparison. Prefill and decode measurements still have to be collected separately for the exact backend/model/quantization/context tuple before Runtime can prefer one candidate.

## Ember CoreUI adoption

Expand Down Expand Up @@ -202,7 +248,7 @@ The process rejects a network-wide bind when no bearer token is configured.
```text
Quantum Runtime
model identity, lifecycle, inference APIs, streaming, backend adapters,
host discovery, calibration and placement policy
host discovery, calibration, placement and application-profile admission

Quantum Control
server, hosting, service, database, backup and update management
Expand All @@ -218,6 +264,7 @@ Quantum CoreOS consumes released Quantum Runtime packages rather than maintainin
- `docs/ARCHITECTURE.md`
- `docs/API.md`
- `docs/HARDWARE-PLACEMENT.md`
- `docs/EMBER-DEPLOYMENT-HISTORY.md`
- `docs/INSTALLATION.md`
- `docs/ROADMAP.md`
- `docs/SECURITY.md`
Expand All @@ -230,4 +277,4 @@ Quantum Runtime project-owned code is licensed under the **Starlight Unit Studio

Private/internal use, internal commercial use and Integrated Application Use are permitted under the license conditions. Quantum Runtime itself may not be sold, white-labeled or offered as a paid standalone general-purpose Runtime service. Third-party inference engines, models, model weights, datasets and tools retain their own terms.

The legally controlling German text is `LICENSE.de.md`; `LICENSE.md` is an English convenience translation. This is a custom Source Available license and is not an OSI-approved open-source license.
The legally controlling German text is `LICENSE.de.md`; `LICENSE.md` is an English convenience translation. This is a custom Source Available license and is not an OSI-approved open-source license.
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.3.0-alpha.3
0.3.0-alpha.4
Loading
Loading