Skip to content

[profile] Define E.M.B.E.R. CPU baseline and MoE model requirement #13

Open

Description

@pschildgen87-code

Goal

Define a machine-readable E.M.B.E.R. deployment profile that captures the current minimum supported CPU-only hardware class and requires a Mixture-of-Experts model architecture, without turning those requirements into global Quantum Runtime requirements.

This is an application/profile policy. Quantum Runtime itself remains model-neutral and must continue to support dense models and smaller hardware where a selected model/backend can run correctly.

Current E.M.B.E.R. baseline

Treat the following as the current supported baseline for the E.M.B.E.R. production profile:

  • Memory: 64 GiB DDR5 ECC RAM minimum
  • CPU: 8 physical CPU cores minimum
  • CPU reference clock class: about 2.6 GHz/core or better
  • GPU: optional, never required for the baseline profile
  • Model architecture: MoE required

The 2.6 GHz value is a deployment reference, not a claim that clock frequency alone predicts inference throughput across CPU generations/architectures. Runtime admission should ultimately prefer the bounded host calibration/performance data from #11 when available.

Model-family policy

The profile should express a generic architecture requirement such as:

architecture.required = moe

It must not hardcode the generic Runtime to one model family.

For the current Gemma 4 family, the production E.M.B.E.R. candidate is the 26B A4B MoE variant. The other current Gemma 4 sizes are dense/effective-parameter variants rather than MoE, so they do not satisfy this E.M.B.E.R. profile requirement.

Other compatible MoE families remain valid candidates when their capabilities pass the Runtime conformance/performance ledger. Examples include Qwen MoE variants such as Qwen3-30B-A3B and larger MoE variants.

Runtime representation

Add an application/profile-level requirement contract that can express at least:

profile.id = ember
hardware.memory.minimum_bytes
hardware.memory.ecc_required
hardware.memory.preferred_class = ddr5
hardware.cpu.minimum_physical_cores
hardware.cpu.reference_min_clock_mhz
hardware.accelerator.required = false
model.architecture.required = moe

Where the host cannot reliably prove a property such as ECC from normal userspace discovery, report unknown rather than silently claiming compliance. Operator policy may provide an explicit verified override.

Admission behavior

Before activating the E.M.B.E.R. profile:

  1. Resolve the canonical model and verify that its declared architecture is MoE.
  2. Validate host memory capacity and CPU core count.
  3. Validate ECC/DDR generation where the platform exposes trustworthy data; otherwise surface unknown and apply operator policy.
  4. Prefer the calibrated performance profile from [0.3] Make CPU-first execution and tiered memory placement the Runtime default #11 over raw MHz when available.
  5. Keep CPU-only as a valid first-class placement; an available GPU is only an optional accelerator.
  6. Reject or clearly downgrade from the E.M.B.E.R. production profile when mandatory profile requirements are not met. Do not silently substitute a dense model while preserving the E.M.B.E.R. profile identity.

CoreUI / Game integration

CoreUI and Game PHP backends should consume the Runtime/profile decision instead of accumulating model-specific fixed inference constants. In particular:

  • do not hardcode a global Gemma-only assumption into clients;
  • select/validate the E.M.B.E.R. profile by capability (MoE) rather than by string matching alone;
  • preserve the Gemma 4 + Ollama backend-managed context policy tracked in [0.3] Build universal multi-backend model capability fusion layer #8 instead of blindly injecting num_ctx;
  • allow future compatible MoE families to satisfy the same profile without rewriting client PHP.

Acceptance criteria

  • E.M.B.E.R. requirements are represented separately from generic Runtime minimum requirements
  • profile declares 64 GiB DDR5 ECC, 8 physical cores, ~2.6 GHz reference class, GPU optional
  • MoE is a mandatory model capability for the E.M.B.E.R. production profile
  • Gemma 4 26B A4B is represented as the current Gemma-family MoE candidate, not as a generic Runtime hardcode
  • unrelated compatible MoE families can satisfy the same architecture contract
  • insufficient hardware/model capability produces an explicit admission result
  • unknown ECC/DDR detection is represented honestly rather than guessed
  • CoreUI/Game can consume the profile without maintaining duplicated model-family constants
  • implementation composes with [0.3] Build universal multi-backend model capability fusion layer #8 backend capabilities and [0.3] Make CPU-first execution and tiered memory placement the Runtime default #11 CPU-first host calibration

Related

Activity

  1. pschildgen87-code commented on Sep 4, 2026

    @pschildgen87-code
    ContributorAuthor

    Clarification from deployment history: the ~2.6 GHz/core value is only a historical/reference figure observed across multiple Netcup server upgrade generations. E.M.B.E.R. remained operational across hosts with differing advertised CPU clock figures, so GHz must NOT be treated as a hard admission cutoff.

    Policy consequence:

    • Hard minimums remain the supported memory/core architecture profile and the MoE requirement.
    • Raw clock frequency is advisory metadata only.
    • Admission/performance decisions should prefer measured host calibration, CPU ISA/features, memory bandwidth/topology and backend/model throughput from [0.3] Make CPU-first execution and tiered memory placement the Runtime default #11.
    • A host below 2.6 GHz must not be rejected solely because of MHz/GHz if calibration shows it satisfies the E.M.B.E.R. performance profile.
  2. pschildgen87-code commented on Sep 5, 2026

    @pschildgen87-code
    ContributorAuthor

    Additional deployment-history evidence for the E.M.B.E.R. profile:

    • The E.M.B.E.R. system construct (including RAG-Lite and long-term memory) was already operational on the smallest Netcup root-server tier used by the studio, around EUR 12/month, when paired with sufficiently small models. Exact CPU/RAM values for that historical host are not yet recorded here, so this observation must not be converted into guessed machine-readable hardware limits.
    • The later Netcup RS 4000 G12 is confirmed as a practical historical quality step: AMD EPYC 9645 (Turin) host class, 12 dedicated cores, 32 GB DDR5 ECC, 1 TB NVMe. This is where the construct became practically usable with Gemma 3 12B Q4.
    • Current primary host: Netcup RS 12000 G12, AMD EPYC 9645 (Turin) host class, 20 dedicated guest CPU cores, 96 GB DDR5 ECC, ~3 TB NVMe RAID-10, KVM, Debian 13. 16 of the 20 guest cores are reserved operationally for Ember while the remaining capacity serves the studio site/game/system workload.

    Policy consequence: distinguish functional compatibility from the supported E.M.B.E.R. production profile. RAG/LTM and the application stack can operate on smaller hardware with smaller/dense models, but that does not satisfy the current production MoE/intelligence profile. Runtime must therefore not turn the 64 GiB + MoE production requirement into a generic Runtime minimum, and it must not infer host-physical 96 cores from the EPYC 9645 model name when the KVM guest is allocated only 20 dedicated CPUs. Guest-visible/cgroup/cpuset resources are authoritative for placement.

  3. pschildgen87-code commented on Sep 5, 2026

    @pschildgen87-code
    ContributorAuthor

    Runtime 0.3.0-alpha.4 is now merged/released via PR #17 and implements most of this profile contract:

    • builtin machine-readable ember-production profile remains separate from generic Runtime minimums
    • 64 GiB minimum memory, ECC required, DDR5 preferred, >=8 physical cores, GPU optional
    • historical ~2.6 GHz reference is advisory only and is not a hard admission gate
    • MoE is mandatory through generic architecture capability, not model-name matching
    • Gemma 4 26B A4B remains the current Gemma-family MoE reference candidate; unrelated MoE manifests can satisfy the same architecture contract
    • explicit admitted / rejected / needs_operator_evidence result with per-check evidence
    • missing ECC/provider facts remain unknown until explicit operator evidence is supplied
    • guest/process CPU entitlement and operator core budgets are separated from physical host CPU identity

    I am leaving #13 open for now because the remaining acceptance point is client adoption: CoreUI/Game expose the Runtime endpoint already, but they have not yet been changed to actively consume /v1/admission / ember-production instead of merely being compatible with the Runtime inference path. Real calibrated performance evidence from #11 is also still the next step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions