The same model, the same prompt, the same bits, on every chip.
samebit is an integer-only inference specification with conforming kernels for several backends. A model that follows the spec produces bit-identical intermediate tensors, logits and tokens on every conforming backend, with any batch size, thread count or summation order.
$ python -m samebit
golden e5aa829b49bc2cef tokens [195, 18, 29, 236, 112, 255, 129, 248, 13, 41, 129, 128, 50, 62, 114, 16]
reference e5aa829b49bc2cef PASS (0.29s)
numpy e5aa829b49bc2cef PASS (0.02s)
c e5aa829b49bc2cef PASS (0.08s)
torch e5aa829b49bc2cef PASS (0.06s)
mlx e5aa829b49bc2cef PASS (0.46s)
Each line is a SHA-256 over all 153 intermediate tensors of a 16-token greedy generation. All five implementations are written separately and share no arithmetic code.
Continuous integration reproduces the same digest on x86-64 Linux and on ARM64
Apple-silicon macOS, with Python 3.9 and 3.12: 4 machines × 5 backends, one
digest (run).
On an NVIDIA Tesla T4 (Kaggle, 28 October 2026), the CUDA backend reproduces the same
digest, with its matmul split across thread blocks and combined by atomicAdd, and
all 60 tests pass:
cuda: Tesla T4
cuda e5aa829b49bc2cef PASS (0.27s)
Spec v0.2 (docs/SPEC-v0.2.md) carries a real pretrained model, Qwen2.5-0.5B (494M parameters, 24 layers, grouped-query attention), entirely in integers: int8 weights, int16 activations, an int32 residual stream and int64 accumulators.
Accuracy on WikiText-2 test, against the original float32 model:
| Tokens | Float32 perplexity | samebit integer perplexity | Same top-1 token | |
|---|---|---|---|---|
| Qwen2.5-0.5B | 10,240 (40 × 256) | 22.99 | 23.16 (+0.75%) | 93.25% |
Per-window results: results/qwen2_wikitext2_eval_x86_2026-10-28.txt.
Calibration uses only WikiText-2 train; the test text is never seen during conversion.
Bit-identity across three separately written implementations, checked on all 24 residual
streams and the logits (tools/qwen2/determinism.py):
numpy-blas 35650ae52353be4c (12.1s)
numpy-int64 35650ae52353be4c (111.9s)
torch 35650ae52353be4c (6.5s)
32 tokens, 24 residual streams + logits: IDENTICAL
The NumPy BLAS path does its integer matmuls in float64. That is exact because the spec bounds every partial sum below 2^53. A floating-point unit therefore gives the same bits as an integer unit. Steps to reproduce are in tools/qwen2.
On a Tesla T4 (Kaggle, 28 October 2026), the v0.2 CUDA kernels (src/samebit/v2/ops_cuda.py)
reproduce the x86 CPU's logits exactly, on real WikiText-2 text
(notebook, tools/qwen2/gpu_check.py, full output).
Kaggle reran it independently, and that public run gives the same digests: kaggle.com/code/sankurkundu/….
GPU: Tesla T4
integer model fingerprint e1994bc2e4f9bff7 (same)
window 0: GPU 9709de19df934eb4 CPU 9709de19df934eb4 published x86 CPU 9709de19df934eb4 IDENTICAL
window 1: GPU 204ff68dead1d2ea CPU 204ff68dead1d2ea published x86 CPU 204ff68dead1d2ea IDENTICAL
split-K 1, 4, 16, 64: 9709de19df934eb4 every time
The GPU matmul splits every dot product across up to 64 thread blocks and adds their results
with atomicAdd, so the order of the additions differs on every run. The bits do not change.
Ordinary floating-point inference changes when the input shape changes. Here are the logits of the same first 128 tokens, computed alone and then inside a longer input or a batch:
| Alone vs inside 256 tokens | Alone vs in a batch of 8 | |
|---|---|---|
| samebit integers, T4 | 0 | 0 by construction (each sequence is computed independently) |
| PyTorch float16, T4 | 0.251 | 0.125 |
| PyTorch float32, T4 | 9.5e-5 | 1.3e-4 |
The values are the largest absolute difference in any logit.
Here is the time to process one 256-token prompt, median of 3 runs. The public Kaggle rerun measured 513 tokens/s for samebit and 6,438 for float16:
| Time | Tokens/s | |
|---|---|---|
| PyTorch float16, T4 | 42 ms | 6,033 |
| PyTorch float32, T4 | 80 ms | 3,219 |
| samebit integers, T4 | 482 ms | 531 |
| PyTorch float32, CPU | 1,487 ms | 172 |
| samebit integers, CPU (NumPy) | 7,763 ms | 33 |
The first CUDA kernels are plain: one thread per output column, and no tensor cores. That makes them 11× slower than float16. The T4's int8 tensor cores can compute exact integer dot products, so this gap is the main target of the next stage.
Ask a model the same question twice at temperature 0 and you can get two different answers. Floating-point addition is not associative, so the result of a reduction depends on how a kernel splits it. That split changes with the batch size, the GPU model, the vendor, the kernel library and the version.
examples/why_integers.py sums one 4096-element dot product in five orders:
order float32 result int32 result
forward, blocks of 4096 -31.6259975 (c1fd020b) -316260
forward, blocks of 64 -31.6260014 (c1fd020d) -316260
backward, blocks of 64 -31.6259975 (c1fd020b) -316260
forward, blocks of 7 -31.6260242 (c1fd0219) -316260
backward, blocks of 1 -31.6259823 (c1fd0203) -316260
Four different float32 results, one integer result.
Reproducible inference matters because:
- Reinforcement learning: a sampler and a trainer that compute log-probabilities differently is a known source of instability.
- Audits: evaluation, safety and regulatory work need results anyone can reproduce.
- Verification: if an inference provider's output is fully determined, anyone can check that they ran the model they claim.
- Hardware portability: a model can give the same answer on an NVIDIA GPU, an AMD GPU, an Apple laptop and a phone.
Recent work makes floating-point inference batch-invariant or invariant across tensor-parallel sizes within one software stack. samebit takes a different route: it removes floating point from the runtime, so agreement follows from integer arithmetic and does not depend on any vendor's kernels.
- Integer arithmetic only. int8 weights and activations, exact int32
accumulation and fixed-point requantization (
rshift_round, round half up, defined for negative values too). - Proved bounds.
docs/BOUNDS.mdshows that no intermediate overflows in any summation order. Up to K = 131071, an int8 matmul is exact however it is tiled, threaded or vectorised. - Exact nonlinearities. RMSNorm uses an exact 16-step integer square root.
Softmax, SiLU and rotary embeddings use lookup tables built with 60-digit
decimal arithmetic, so table bytes do not depend on a platform's
libm. - Pinned tie-breaking. argmax returns the lowest tied index on every backend.
The spec is SPEC.md. Its executable form is
src/samebit/reference.py, written in plain Python integers with explicit
bound checks.
| Backend | Where it runs | Status |
|---|---|---|
reference |
Plain Python integers | Conforms; the definition |
numpy |
Any CPU | Conforms |
c |
Portable C99 with optional OpenMP; three accumulation orders | Conforms; all orders agree |
torch |
PyTorch CPU | Conforms |
mlx |
Apple MLX (Metal on Apple silicon, CPU elsewhere) | Conforms on CPU; Metal not yet run |
cuda |
NVIDIA GPUs (CuPy, kernels compiled at run time) | Conforms on a Tesla T4 (Kaggle, 28 Oct 2026), v0.1 and v0.2 (Qwen2.5-0.5B); also checked under CPU emulation on every push |
| ROCm / HIP | AMD GPUs | Planned |
labs/same_bits is an interactive marimo lab, in the
format of the Co-Labs of Harvard's open Machine Learning Systems textbook. It shows why
temperature 0 does not give the same answer twice. Students measure four distinct float32 sums
of the same 4,096 numbers, then 7 of 64 users getting different text only because they were
batched. Then they fix it twice: with batch-invariant kernels, at a speed cost, and with
integer arithmetic, at an accuracy cost. It runs on numpy alone: marimo run labs/same_bits/lab_same_bits.py.
git clone https://github.com/Shakhtar-Sankur/samebit
cd samebit
pip install -e ".[test]" # numpy and pytest
pip install torch mlx # optional extra backends
python -m samebit # conformance check against the golden digest
pytest # op-level and end-to-end tests
python examples/why_integers.py # floats vs integersThe C backend compiles itself on first use with the system C compiler.
Open examples/kaggle_gpu.ipynb in Kaggle, set the accelerator to a
T4 or P100 and run all cells, or locally:
pip install -e ".[test,cuda]"
python -m samebit cudaThe CUDA matmul splits every dot product across SAMEBIT_CUDA_SPLITK thread blocks (default 8)
and combines them with atomicAdd, so the order of the final additions changes on every run. The
digest must not.
Without a GPU, tests/emulated_cupy stands in for CuPy: it compiles the same CUDA source as C++,
runs every block and thread on the CPU, and shuffles the split-K blocks. CI runs the CUDA kernels
this way on every push.
- Op conformance (
tests/test_ops.py): every op on every backend must match the reference exactly. The tests cover the int8 minimum, saturation, rounding of negative halves, all-tied rows, masked softmax and the largest matmul depth allowed by the int32 bound. - End to end (
tests/test_model.py): a 2-layer, Llama-shaped decoder with RMSNorm, rotary attention with a KV cache and SwiGLU must reproduce the golden digest on every backend. Processing the prompt in one batch or one token at a time must give identical logits. - Order independence: the C backend runs ascending, descending and blocked back-to-front accumulation and must agree.
- Stage 1, done: spec v0.1, proved bounds, five conforming backends, and a conformance suite with a golden digest.
- Stage 2: CUDA kernels (done: conform on a Tesla T4) and Metal. Convert a real open model to the spec (done: Qwen2.5-0.5B, spec v0.2, three implementations bit-identical; the CUDA kernels give the same bits on a Tesla T4 at 531 tokens/s). Next: int8 tensor-core kernels to close the gap to float16, and Metal.
- Stage 3: AMD ROCm. Machine-checked proofs that the kernels meet the spec and the bounds. Preprint.
- Stage 4: a public auditor that checks, from outputs alone, whether a hosted model matches its published weights.
Integer-only inference is well established: gemmlowp and TensorFlow Lite fixed-point requantization, and integer-only transformers such as I-BERT (Kim et al., 2021). samebit applies these ideas to a different goal. It turns integer inference into an exact, cross-vendor conformance target with a written spec, proved bounds and a golden digest.
See CITATION.cff.
Apache-2.0. Copyright 2026 Sankur Kundu.