Skip to content

Release opt-in full-model PyTorch wrapper and Metal SiLU fix - #8

Merged
logickeau merged 3 commits into
mainfrom
release/v0.3.3
Nov 13, 2026
Merged

logickeau merged 3 commits into
mainfrom
release/v0.3.3

Conversation

@logickeau

Copy link
Copy Markdown
Contributor

This releases the tested resident full-model PyTorch wrapper and Metal SiLU/SwiGLU correction. The wrapper remains explicitly opt-in: importing vitni_torch does not open a model, replace torch.matmul/normal modules, or change global backend and determinism settings. Callers must construct VitniModel; the default is CPU, and optimized Metal requires backend='metal', optimized=True.

The wrapper retains native model weights/metadata/KV allocations, returns CPU float32 logits and uses the existing generation/receipt driver. Native autorelease scopes release temporary Metal command objects between requests. The SiLU correction preserves the existing CPU rounded formula for finite-input overflow, large-divisor and subnormal cases. CPU numerical rules and published anchors are unchanged.

Validation recorded on the M3 Max research branch:

  • 24.32 million ordinary-model CPU/Metal logit words matched exactly, including maximum contexts and complete standalone receipt comparisons.
  • A diagnostic finite-weight checkpoint exposed 128,000/128,000 wrong but finite baseline logits; the fixed backend matched CPU and the frozen CPU receipt.
  • 57 installed-package tests and 16 native Metal regressions passed. Edge checks compared 687,906 scalar outputs exactly.
  • Nine rotating-order performance pairs showed no material regression: generation medians +0.08% for TinyLlama and +0.10% for Stories versus the frozen optimized engine.

Release preparation adds a fresh-process opt-in check and real TinyLlama full-model CPU coverage to Linux/macOS package CI, with CPU-only mode labeled explicitly. Python remains experimental 0.1.0a2, distributed through GitHub assets/tagged source. The Rust release is v0.3.3. Release notes document eager/unbatched CPU tensor I/O, no CUDA/autograd/arbitrary-model conversion, context/type limits and the tested numerical scope.

The final wheel, rebuilt from the source distribution and installed in isolation, passed 58 CPU/MPS/native Metal tests with strict hardware requirements. Its separate TinyLlama CPU-contract check passed all 11 tests locally. cargo package --allow-dirty --offline verified the Rust package build.

See docs/releases/v0.3.3.md and the preserved research evidence for runtime definitions, raw observations and hashes. No PyTorch maintainer outreach or social posting is part of this release.

@logickeau
logickeau merged commit 854a392 into main Nov 13, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant