Repository navigation
Release opt-in Metal optimizations and experimental PyTorch operators - #7
Merged
Merged
Conversation
Comment on lines
+10
to
+42
| name: wheel and CPU contract (${{ matrix.os }}) | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-14] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.12' | ||
| - uses: dtolnay/rust-toolchain@stable | ||
| - uses: Swatinem/rust-cache@v2 | ||
| with: | ||
| workspaces: bindings/torch/native | ||
| - name: Build tooling | ||
| run: python -m pip install build pytest numpy | ||
| - name: Install CPU PyTorch on Linux | ||
| if: runner.os == 'Linux' | ||
| run: python -m pip install torch==2.14.1 --index-url https://download.pytorch.org/whl/cpu | ||
| - name: Install PyTorch on macOS | ||
| if: runner.os == 'macOS' | ||
| run: python -m pip install torch==2.14.1 | ||
| - name: Build wheel and source distribution | ||
| run: python -m build | ||
| - name: Install the built wheel | ||
| run: python -m pip install --no-deps dist/*.whl | ||
| - name: Test installed package | ||
| run: python -m pytest -q python/tests | ||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: vitni-torch-${{ matrix.os }} | ||
| path: dist/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The tested Metal improvements were available only on the local research branch, and PyTorch users had no explicit way to call Vitni operators. This release exposes the retained Metal preset through a SDK-compatible receipt CLI and adds the experimental
vitni-torch0.1.0a1 package. Default engine behavior and published numerical anchors are unchanged.The Metal preset combines cooperative Q4/RMS/Q6/attention, metadata caching, contraction control and concurrent dispatch. Rejected experiments remain inactive. The new CLI rejects unsupported projection encodings/dimensions and contexts beyond 512 tokens.
The PyTorch preview registers inference-only CPU/MPS
canonical_matmuland raw GGUFq4_k_linearoperators, backed by the actual Rust engine and direct MPS shader execution. It supports compilation/export registration without replacing PyTorch's native operations. The tested MPS compiler flushes subnormals; both devices therefore reject inputs outside the documented bounded domain. No arbitrary-model, autograd or CUDA support is claimed.Validation:
cargo packageverified.Costs and limits are documented in
docs/releases/v0.3.2.mdwith raw evidence. The approximately 2.47x full-model forward ratio is separate from the preview's 4.4–7.2x MPS float32 component ratios. The pre-existing Metal SiLU overflow edge remains unresolved. Physical CUDA, other GPU generations, larger PyTorch models and long contexts were not tested in this release.Research history remains preserved locally. This release snapshot normalizes machine-specific workspace prefixes in evidence JSON without changing observations or content hashes.