Skip to content

Release opt-in Metal optimizations and experimental PyTorch operators - #7

Merged
logickeau merged 3 commits into
mainfrom
release/v0.3.2
Nov 13, 2026
Merged

logickeau merged 3 commits into
mainfrom
release/v0.3.2

Conversation

@logickeau

@logickeau logickeau commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

The tested Metal improvements were available only on the local research branch, and PyTorch users had no explicit way to call Vitni operators. This release exposes the retained Metal preset through a SDK-compatible receipt CLI and adds the experimental vitni-torch 0.1.0a1 package. Default engine behavior and published numerical anchors are unchanged.

The Metal preset combines cooperative Q4/RMS/Q6/attention, metadata caching, contraction control and concurrent dispatch. Rejected experiments remain inactive. The new CLI rejects unsupported projection encodings/dimensions and contexts beyond 512 tokens.

The PyTorch preview registers inference-only CPU/MPS canonical_matmul and raw GGUF q4_k_linear operators, backed by the actual Rust engine and direct MPS shader execution. It supports compilation/export registration without replacing PyTorch's native operations. The tested MPS compiler flushes subnormals; both devices therefore reject inputs outside the documented bounded domain. No arbitrary-model, autograd or CUDA support is claimed.

Validation:

  • Rust core: 153 passed, 27 explicitly ignored model/hardware tests; cargo package verified.
  • Physical M3 Max: nine retained Metal GPU tests passed, including regime-switch replay.
  • Wheel rebuilt from source archive and installed independently: all 40 CPU/MPS tests passed, including independent arithmetic references, opcheck, eager/AOT-eager/Inductor compilation, export/reload and no-CPU-fallback assertions.
  • Five real Q4_K projections: 10,272 output words per device, zero bit mismatches against previously pinned CPU files.
  • Actual Vitnify SDK 0.4.9 bridge: complete CPU/optimized-Metal receipt JSON matched for TinyLlama and Stories; over-context invocation rejected.

Costs and limits are documented in docs/releases/v0.3.2.md with raw evidence. The approximately 2.47x full-model forward ratio is separate from the preview's 4.4–7.2x MPS float32 component ratios. The pre-existing Metal SiLU overflow edge remains unresolved. Physical CUDA, other GPU generations, larger PyTorch models and long contexts were not tested in this release.

Research history remains preserved locally. This release snapshot normalizes machine-specific workspace prefixes in evidence JSON without changing observations or content hashes.

Comment on lines +10 to +42
name: wheel and CPU contract (${{ matrix.os }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-14]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
workspaces: bindings/torch/native
- name: Build tooling
run: python -m pip install build pytest numpy
- name: Install CPU PyTorch on Linux
if: runner.os == 'Linux'
run: python -m pip install torch==2.14.1 --index-url https://download.pytorch.org/whl/cpu
- name: Install PyTorch on macOS
if: runner.os == 'macOS'
run: python -m pip install torch==2.14.1
- name: Build wheel and source distribution
run: python -m build
- name: Install the built wheel
run: python -m pip install --no-deps dist/*.whl
- name: Test installed package
run: python -m pytest -q python/tests
- uses: actions/upload-artifact@v4
with:
name: vitni-torch-${{ matrix.os }}
path: dist/
@logickeau
logickeau merged commit 9cc5ac5 into main Nov 13, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants