Skip to content

Vendor linfa-linalg's LOBPCG and drop the ndarray 0.16 dependency #269

Open

Description

@Trecek

Summary

Copy the LOBPCG code path of linfa-linalg 0.2.1 into a private module of spectral-init, ported to ndarray 0.17, and remove the linfa-linalg and ndarray16 dependencies.

Not a blocker for 0.1.0 or PR #268: the current dependency works, and none of it appears in the public API, so removing it later is not a semver change.

Why

  • linfa-linalg is built on ndarray 0.16, while spectral-init and umap-rs use ndarray 0.17. Every build therefore also compiles ndarray 0.16, rand 0.8 and thiserror 1 (9 crates in the published dependency tree exist only because of it), and src/solvers/nd16_bridge.rs converts arrays between the two ndarray versions on every LOBPCG call.
  • Upstream is not actively maintained: 0.2.1 (2025-05-15) is the last release and last commit, and Dependencies update rust-ml/linfa-linalg#19 (the ndarray 0.17 update) has been open since 2025-11-17 without a review.
  • Owning the code allows later changes to the inner loop (see [Evaluation] Verify linfa-linalg soft-locking behavior #108 on soft locking); today the warm-restart and unconvergence handling has to wrap linfa's loop from outside.

Approach

  • Vendor, do not rewrite. Copy the non-test code the LOBPCG path needs (lobpcg/algorithm.rs plus the Cholesky, symmetric eigendecomposition, triangular-solve, Householder/Givens, norm and indexing helpers, about 1,670 lines) into a private module under src/solvers/, ported to ndarray 0.17. The diff of linfa-linalg#19 shows the library code barely changes for that port.
  • Keep every arithmetic operation in the same order. A new implementation (for example on faer) would change iteration paths and therefore PythonCompat results; that is out of scope here.
  • Lints. Bring the code under this repository's lints (unsafe_op_in_unsafe_fn, clippy::undocumented_unsafe_blocks): the 18 lines with unsafe need SAFETY comments or checked indexing, and the code moves from edition 2018 to 2024.
  • Licence. linfa-linalg is MIT/Apache-2.0, which is compatible, but it ships no LICENSE file. Add an attribution notice naming the source (rust-ml/linfa-linalg 0.2.1; Yuhan Lin and contributors) with the licence text.
  • Clean up. Remove the nd16_bridge.rs conversions, the linfa-linalg@0.2 skip in .config/deny.toml, the ndarray16/linfa rule in .github/scripts/check-purity.sh and the user-guide note about the extra ndarray; update CLAUDE.md, .github/package-allowlist.txt and Cargo.lock.
  • Verify. Compare the Levels 1–3 eigenpairs, old vs new, on every fixture in both ComputeModes; run cargo nextest run --workspace --all-features and clippy with -D warnings; check that cargo tree -p spectral-init -e normal no longer lists ndarray 0.16, linfa-linalg, rand 0.8 or thiserror 1.

Estimated effort: 2–3 days.

Requirements

PythonCompat solver output is identical to the current linfa-linalg path on every reference fixture.
The published spectral-init no longer depends on linfa-linalg or ndarray 0.16.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions