Skip to content

proofs/exact-llm: route t3, every product of exact-16 by adds and subtracts; the pinned pair, no multiply instruction in the kernel - #104

Open
lmtlssss wants to merge 12 commits into
mainfrom
t3-rebase-trial
Open

lmtlssss wants to merge 12 commits into
mainfrom
t3-rebase-trial

Conversation

@lmtlssss

Copy link
Copy Markdown
Owner

A fourth route for the exact model, exact16-t3: every product of exact-16's forward pass computed by t3.c in add, subtract, shift by a constant and bitwise logic, from signed-digit weight streams. On the Apple M1 it prints exact-16's pinned pair (b0cc875f…0b4460 / a800ebad…7613cc), MATCH, 0 clamped, also rerun with the kernel and every stream built afresh; t3_nomul.py finds no multiply or divide instruction in the kernel's 511.
The workflow adds three jobs (SmolLM2-135M, --routes t3, then t3_nomul.py) on ubuntu-latest and ubuntu-24.04-arm with gcc and on macos-latest with Apple clang. Divisions, square roots and computed shifts stay NumPy's, as the README states.

… at most 64 threads; a stream only for whole columns

The weight streams were kept by matrix name, first column and width in one folder for every model: the embedding
blocks (model.embed_tokens.weight, columns of 8,192) have the same names in the 135M, 360M and 1.7B, so a second
model's t3 run in the same cache read the first model's logits streams. They are now kept in
T3_DIR/streams-<t3.c's hash>/<model>; the hash also keeps a stream of an older t3.c from being read by a newer one.
register() replaces the matrices registered before and records the model.

t3.c runs at most 64 threads and leaves the rows past the 64th share unset; T3_THREADS is now at most 64.

weight_key() gave a block the whole matrix's stream when the block began in row 0 with the matrix's row stride,
whatever its rows; it now requires all the matrix's rows and columns within it. No caller passed such a block.
…what t3 leaves to NumPy, no M1 seconds

t3_nomul.py matched library multiply and divide calls with \b__mul..., which a Mach-O symbol's extra leading
underscore defeats: a 128-bit divide built on arm64 macOS (a call to ___divti3) passed as NO MULTIPLIER. The
pattern now takes one more underscore and the ARM EABI routines (__aeabi_*mul, *div). Also listed: arm64 fmlal,
fmlal2, fmlsl, fmlsl2, fcmla, fdot, bfmlal; x86-64 pclmul, vpclmul, dpps, dppd, vdpp. The kernel built on the M1
(t3-f6d3700bf905) still holds none: 511 instructions, 496 in the five functions of t3.c and 15 in its call stubs.
Reported: a*b on arm64 (mul), t3.c without its KEEP barriers (mul in run_rows), a*b for rv32i (__mulsi3), the
128-bit divide (___divti3).

README, t3: "every product of exact-16's forward pass"; the division and square-root bracket checks multiply in
NumPy, and the shifts by a computed amount stay NumPy's; the streams' folder; the M1 table without seconds (the M1
was shared during the run; record/t3-m1.txt keeps them); the 511 instructions as functions and stubs, and what
t3_nomul.py reports; t3_rv32.c compiled only, not run.
… and the 14-bit input index

t3_run does not check what pthread_create returns: a thread that fails to start leaves its rows unset, which check
prints as DIFFERENT. A stream word's input index is 14 bits: a product has at most 16,383 inputs, which for the
context products is 16,383 positions.
…-24.04-arm and macos-latest, then t3_nomul.py

Three jobs, one route each in the matrix form of the 1.7B jobs: check --model SmolLM2-135M --routes t3 --no-float
--no-onnx, then t3_nomul.py on the kernel check built. Linux x86-64 and arm64 build t3.c with gcc, macOS arm64 with
Apple clang; the Windows runners have no C compiler. 120 minutes each (the 135M's other jobs keep 90).

The base matrix gets routes: [''] (check's default routes). Without it, an include entry whose model and runner match
a combination of the cross product would be merged into that job (GitHub adds an include's pairs to every combination
whose original values they do not overwrite) and turn the 135M's default job on that runner into a t3 job. With it,
each t3 entry is a job of its own: 27 jobs, the 24 before and these 3.
…t afresh; exact and t3 MATCH, no multiplier

record/t3-m1-rerun.txt (paths under the home folder written as ~): this code (run from 2f5e961, the same files as
this branch's) on the Apple M1 with an empty kernel and stream folder. routes_test.py: 42 cases on four routes, 0
different, every 16-bit value through the stream exact. check --routes t3 --no-float --no-onnx: exact 30ed613b...
MATCH; exact-16 on t3 b0cc875f...0b4460 / a800ebad...7613cc MATCH, 0 clamped; 1,653,064,171,200 weight digit adds
over the 64 prompts. t3_nomul.py on the new kernel: 511 instructions, no multiply or divide. The README names this
record and the first run's (record/t3-m1.txt, record/t3-count.json).
…instead of loading a kernel the runner cannot open
…a barrier, so gcc folds no multiply into it; t3_nomul.py prints each flagged instruction
… built afresh: exact and t3 MATCH, counters as before, 0 multipliers in 517 instructions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant