Skip to content

Optimise native CPU FFTs and modal diagnostic scheduling - #16

Merged
jtravs merged 7 commits into
mainfrom
feature/native-cpu-optimisation
Oct 24, 2026
Merged

jtravs merged 7 commits into
mainfrom
feature/native-cpu-optimisation

Conversation

@jtravs

@jtravs jtravs commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Native CPU propagation was spending avoidable time in scalar FFTW codelets, long-row Raman plans and embedded transverse diagnostics at intermediate DOPRI5 stages. This change enables coherent NEON FFTW on Apple silicon, tunes persistent Raman planning/provider selection, and defers coarse modal diagnostics until their consumers need them.

Based on current main at 174be2a, including merged #15. The diff contains only the native optimisation programme.

Retained changes

  • Enable pinned FFTW 3.3.11 NEON codelets in both precisions on macOS arm64, preserving the M1 baseline, deployment target, cycle counter and coherent OpenMP runtime. The existing macOS wheel recipe bundles these libraries; no workflow changes are introduced here.
  • Honour planning effort for long Raman full/tail transforms. With NEON, use FFTW for serial and long rows and retain vDSP for short parallel power-of-two rows where removal would regress performance. Serial MEASURE can increase construction time substantially; SOLPROP_FFTW_PLAN=estimate remains available.
  • Project embedded quadrature only at the final required ordinary DOPRI5 modal stage, then defer its FFT/held-current assembly until accepted scoring. Fine-only projection avoids stacked coarse GEMM work. Private readiness and invalidation preserve rejected-trial, reset, FSAL and peak semantics.
  • Extend numerical/lifetime tests and document the build recipe, measured results and rejected approaches.

Kernel schema 35, frontend signatures, canonical layouts, protected arithmetic and Accelerate BLAS are unchanged by this PR. GPU execution is unchanged. Plasma traversal, Cartesian tile and thread-sizing prototypes were rejected after bounded measurements; none is retained.

Lightweight performance evidence

Frozen baseline 9521a88 versus tested native snapshot b3bdd0f, prescribed ten-step windows, alternating rounds. This is the approved lightweight campaign, not a standard-suite adoption verdict.

Measured set One thread: elapsed-time reduction 18 threads: elapsed-time reduction
Mac M5 Max, retained combination across eight main cases 20.8% 13.5%
Xeon GCC 16.2 / OpenBLAS 0.3.34 / FFTW 3.3.11, scheduling affected set 4.3% 7.7%
Xeon IntelLLVM 2026.1.1 / MKL 2026.1, scheduling affected set 6.2% 7.8%

The production-budget results satisfy the greater-than-5% and conservative-spread criteria. The ledger retains per-case ranges, setup costs, guards, noisy controls and roughly 1% GCC small-window costs. vDSP removal and the regressing Mac plasma traversal were rejected rather than averaged away.

These measurements predate integration with #15 and use their frozen dependency versions. No new timing claim is made for #15's startup or coupling changes. The merge changes no native source relative to the tested snapshot.

Validation

  • Original campaign: Mac focused numerical/lifetime checks, native C++/Python suites on both Xeon bundles, bounded adaptive comparisons against tighter NumPy references, and separate resource/library receipts.
  • GCC native suites: 137 C++ and 378 Python passes; Intel: 133 C++ and 376 Python passes, with provider/platform/device skips recorded.
  • macOS distribution build, delocate repair and clean installation passed: both NEON precisions, one OpenMP runtime, licence files and loader discovery verified.
  • After integrating Shorter start-up: optmat 0.9.1, lazy imports, no numba on the native backend #15 with optmat 0.9.1: 45 native diagnostic/Raman tests, 23 coupling/relaunch tests and 6 startup tests passed against the unchanged validated native binary. Focused Ruff, formatting, native type checking, spelling and diff checks passed.
  • CI on the previous head passed coverage, the native quality gate and native suite, static checks, documentation, both slow lanes, Windows and lowest-dependency tests. The macOS default lane was interrupted by a runner shutdown at 83%, with no reported test failures. Linux and macOS wheel test suites also passed; the Windows wheel suite was still running when the branch was updated.
  • Integrating current main changes only contributor guidance and the locked filelock/virtualenv development dependencies. Lockfile consistency, changed-file spelling, diff checks and native import/contract checks passed. Native sources and CI workflows are unchanged by that merge. Fresh full CI and Wheels runs qualify the updated tree. GPU/device checks were unavailable on the CPU hosts.

Detailed measurements and limitations are in docs/dev/benchmarks.md, with build and policy documentation linked from the changelog. Baseline and rejected-prototype evidence remain preserved separately.

Copilot AI balanced review requested due to automatic review settings October 24, 2026 14:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@jtravs
jtravs changed the base branch from perf/startup-latency to main October 24, 2026 17:46
@jtravs
jtravs merged commit ce4899d into main Oct 24, 2026
24 of 25 checks passed
@jtravs
jtravs deleted the feature/native-cpu-optimisation branch November 6, 2026 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants