Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1925,7 +1925,6 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/skills/daslang/
DESTINATION ${DAS_INSTALL_DOCDIR}/skills/daslang
FILES_MATCHING
REGEX "/daslang/README\\.md$" EXCLUDE
REGEX "/LAWS\\.md$" EXCLUDE
PATTERN "*.md"
)
# The review constitution ships with the review skills that route to it, and the
Expand Down Expand Up @@ -2028,7 +2027,6 @@ install(DIRECTORY ${PROJECT_SOURCE_DIR}/skills/daslang/
DESTINATION .claude/skills/daslang
FILES_MATCHING
REGEX "/daslang/README\\.md$" EXCLUDE
REGEX "/LAWS\\.md$" EXCLUDE
PATTERN "*.md"
)
install(FILES ${PROJECT_SOURCE_DIR}/.claude/agents/dragon.md DESTINATION .claude/agents)
Expand Down Expand Up @@ -2380,7 +2378,7 @@ install(CODE [[
get_filename_component(_name "${_f}" NAME)
get_filename_component(_f_abs "${_f}" ABSOLUTE)
if((_name MATCHES "^REVIEW.*\.(md|das)$"
OR _name MATCHES "^(ARCHITECTURE[A-Za-z_]*|LAWS|MASTERPLAN[A-Za-z_]*|PERF_LEDGER|PROFILE|THINKING)\.md$"
OR _name MATCHES "^(ARCHITECTURE[A-Za-z_]*|MASTERPLAN[A-Za-z_]*|PERF_LEDGER|PROFILE|THINKING)\.md$"
OR _name MATCHES "^followup_.*\.md$")
AND NOT _f_abs STREQUAL _root_common
AND NOT _f_abs STREQUAL _root_arch_common)
Expand Down
6 changes: 3 additions & 3 deletions modules/dasLLAMA/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@ re-transcoding `$LCPP/src/unicode-data.cpp`).
- `ARCHITECTURE_GPU_VULKAN.md` - sec.2.2j-2.2q: the Vulkan resident driver - the prefill window
chain, the cm2 decode lanes and tile pick, the class-pipeline build seat, the residency plan,
the GPU-slot marks swap, the Q8 requant byte store, and the MoE expert chain on the cm2 tiles.
- `ARCHITECTURE_GPU_VULKAN_DECODE.md` - sec.2.2r-2.2t: the per-op tier's decode era - the decode
attention block over per-layer K/V mirrors, the streamed expert layer's GPU/CPU split, and the
whole-token decode span.
- `ARCHITECTURE_GPU_VULKAN_DECODE.md` - sec.2.2r-2.2u: the per-op tier's decode era - the decode
attention block over per-layer K/V mirrors, the streamed expert layer's GPU/CPU split, the
whole-token decode span, and the deltanet decode step's per-session resident state.
- `ARCHITECTURE_GPU_MTP.md` - sec.2.28-2.39: the Metal speculative round over the batch driver's
same-slab verify, the box knob that sets the depth a round drafts, and the kernel
argument-alignment contract enforced at every dispatch.
Expand Down
5 changes: 4 additions & 1 deletion modules/dasLLAMA/ARCHITECTURE_ENGINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ file builds an `ArchDesc` (name * `configure` * the `ArchBlocks` fn-ptr quad * `
asr_bench process per cell; macOS only) - the interim footprint instrument until a footprint
leg lands in `gen_bench_records`; its numbers live in `PERF_LEDGER.md`, never the stores.

### 1.9 Serving
### 1.9 Serving {#scheduler-step}

- **`dasllama_scheduler.das`** - the continuous-batching scheduler, the serving layer over the
facade (its one engine require is `dasllama/dasllama`). One synchronous thread: each
Expand All @@ -255,4 +255,7 @@ file builds an `ArchDesc` (name * `configure` * the `ArchBlocks` fn-ptr quad * `
streams' KV pages to the prefix cache. Results flow out as `SchedEvent`s - no HTTP here.
`utils/dasllama-server` owns the writers; `tutorials/dasLLAMA/13_serving.das` is the
teaching consumer; `tests/test_scheduler.das` gates it against `generate()` references.
The step clears its gather arrays (`batch_rows`, `batch_toks`, `batch_idx`) before it reaps
finished streams: `batch_rows` holds borrowed pointers into the sessions the reap deletes,
and a validating heap collect between steps walks every pointer the array still holds.

7 changes: 6 additions & 1 deletion modules/dasLLAMA/ARCHITECTURE_GPU.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,8 @@ that a question answered for one backend has an obvious address in the other. Th
route/mark/want/status state, engine-facing forwarders. Vulkan implements it (per-op offload plus
resident plumbing, and the decode-era seats it alone fills: the cm2 expert chain
`set_moe_gpu_ffn_xf_hooks` / `_async_hooks`, the decode attention block
`set_moe_gpu_attn_dec_hooks`, the decode FFN tail `set_moe_gpu_ffn_tail_hooks`, the
`set_moe_gpu_attn_dec_hooks`, the decode FFN tail `set_moe_gpu_ffn_tail_hooks`, the deltanet
decode step's state seams `set_moe_gpu_dn_state_hooks` (flush, invalidate, release), the
whole-token span `set_moe_gpu_span_dec_hook` - the span rides common's decode override
registry as `vulkan_moe_span`, selected by the MoE placement and declining per token). The
installs are one-way: a test that arms the tier installs the seats and never restores them,
Expand Down Expand Up @@ -136,6 +137,10 @@ Sections 2.28-2.30, the Metal speculative round and the dispatch alignment contr
**The allowed asymmetries between the backends - this list is closed; a new one lands with its
entry here:**

- **The `dasllama_gpu_tier` cooperation SPI is Vulkan-only**: every hook seat the tier
exposes (`install_moe_gpu_tier` and the `set_moe_gpu_*_hooks` setters) is registered by the
Vulkan family alone, and the role row above enumerates the seats; a new seat lands in that
row, not as a new entry here.
- **Metal sits ABOVE `dasllama_common`** (typed `Model`/`Session` access, shapes unconditional);
**Vulkan sits BELOW it** (untyped pointer/array seams - the family never requires common).
Both tiers ENTER from the transformer umbrella (`?das_metal` requires; the single `?vulkan`
Expand Down
10 changes: 9 additions & 1 deletion modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,15 @@ codec, and the device-embed arm). The save moves the installed state out and lea
reading as no-model; the restore is its exact inverse. The whole-model drop clears the same set
and deselects the `"vulkan"` overrides, so a dropped model's prefill and decode take the plain
CPU path and a later re-arm passes `resident_upload`'s no-active-override gate. The three carry
the same set, which is why a model's device state never survives into the next.
the same set, which is why a model's device state never survives into the next. The upload
rail enforces it from its own side: a load that finds marks still installed - a model deleted
without the drop, the shape every test process and single-model tool takes - drops that
model's device state before uploading its own, carrying the load's MoE layer request across
the drop (the one mark the drop's reset would otherwise zero before the rail reads it).
Without that drop the second model's stacks
land beside the first's, and the offset-keyed stack lookup serves whichever model's plane
registered that offset first: the decode attention block asserts on the geometry change, and a
model whose geometry matches decodes the earlier model's weights.

### 2.2p The Q8 requant writers store one quant per byte {#q8-requant-byte-store}

Expand Down
34 changes: 32 additions & 2 deletions modules/dasLLAMA/ARCHITECTURE_GPU_VULKAN_DECODE.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# dasLLAMA Architecture - the Vulkan per-op tier's decode era

Companion to `ARCHITECTURE_GPU_VULKAN.md`; section numbers are `ARCHITECTURE.md`'s. This
document carries sections 2.2r-2.2t: the decode attention block over per-layer K/V mirrors,
the streamed expert layer's GPU/CPU split, and the whole-token decode span. The prefill window
document carries sections 2.2r-2.2u: the decode attention block over per-layer K/V mirrors,
the streamed expert layer's GPU/CPU split, the whole-token decode span, and the deltanet decode
step's per-session resident state. The prefill window
chain, the cm2 tiles and the MoE expert chain these build on are `ARCHITECTURE_GPU_VULKAN.md`
sections 2.2j-2.2q.

Expand Down Expand Up @@ -138,3 +139,32 @@ arch binds a non-standard attention or FFN block declines whole (`span_model_ok`
span's recorder declares those edges by hand (`vhz_dep` on the span's own region bits) after
each requant and after the top-k. The loader carves the router planes with the mirrors
(`set_moe_gpu_dat_need`).

### 2.2u The deltanet decode step's resident state follows its session {#dn-step-owner}

**The deltanet decode step keeps one device copy of a recurrent layer's state and conv ring,
and that copy belongs to one session at a time.** The step (`vk_moe_dn_step`) records, per
layer, the host address of the session state it uploaded (`DnStep.owner_state_addr`, with the
conv history's address beside it): a step from the owner runs on the resident copy; a step from a
different session sends the resident copy home to its owner's buffers when it is dirty, then
cold-uploads its own state and takes the slot. A flush request (`vk_dn_step_flush`) writes only
when the requester is the owner - a foreign session's state is already on its host, so the
request is a no-op there. Two sessions decoding turn about on the tier therefore pay a flush
and an upload per recurrent layer per switch, and each reads its own state.

**A session's identity outlives nothing.** Every Session with deltanet state carries a
`DnOwner` token whose range is that state's host addresses; the token's finalizer - run by the
Session's own finalizer, at every `delete` - releases the device copies the range owns without
writing them (`vk_dn_step_release`), so a later switch never flushes into freed memory and a
new session at the same addresses never inherits a dead one's copy. The token and its
finalizer live beside Session in the engine module: a struct's finalizer is resolved in the
scope of the module that deletes it, and a Session dies in modules that see the engine and
never the tier (the scheduler reaps its streams through the facade). A session's own
position-zero reset releases its own copies the same way (`dn_reset`), and so does its
speculative-round rollback (`mtp_state_restore`): the session's host state is authoritative,
and the other sessions' copies stay theirs. The whole-device invalidate
(`vk_dn_step_invalidate`) remains the seam for a dropped model, where every host copy is
authoritative at once. The engine reaches those seams through the tier's forwarders:
`dn_flush_layer` per layer before any CPU read of `dn_state` or the conv history,
`moe_gpu_dn_release` for one session's copies, and `moe_gpu_dn_invalidate` for every copy at
once.
9 changes: 9 additions & 0 deletions modules/dasLLAMA/PERF_LEDGER.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,15 @@ what it costs today and what the fix would change.
against the 07-25 row; done when both rows sit in `performance/records/zen2.json` and the
site records are regenerated.

- **OWED ROWS - the parity pass's ten 1B pp512 vehicles.** The mirror pass
(`plans/kernel_parity_pass.md`, 2026-09-03) read Q3_K_L at 1.08, IQ3_M at 1.06 and IQ2_XXS
at 0.98 of the reference on the `lcpp_bench --for-debug-purposes --plen 512 --ngen 0 --reps
12` rig bracketed by `llama-bench -p 512 -n 0 -r 6`, stage readings only; the board's rows
for those vehicles are two-rep readings and stand until re-minted. Same refusal as above -
the hardware stamp declines this box while the remote-access daemon runs. Owed: the ten
vehicles' vulkan pp512 rows re-minted by `gen_bench_records`, direction-grade against the
rows they replace; done when they sit in `performance/records/zen2.json`.

- **OPEN (narrowed) - the gemma3v encode residual after the tower flash: ~0.92x vs the
pair.** The slab road closed in three landings: the 96 head pad (guarded AV columns,
668 -> 486 -> 452), then the LIFTED dk72 flash (MetalTowerFlash + the per-head-contiguous
Expand Down
Loading
Loading