Skip to content

test(python): run the test suite in parallel with pytest-xdist - #85

Merged
prrao87 merged 1 commit into
mainfrom
chore/pytest-xdist
Oct 29, 2026
Merged

prrao87 merged 1 commit into
mainfrom
chore/pytest-xdist

Conversation

@prrao87

@prrao87 prrao87 commented Oct 29, 2026

Copy link
Copy Markdown
Member

Why

The Python suite runs one test at a time, so a 4-core CI runner and a multi-core laptop both sit mostly idle during make test. Running the tests across worker processes makes the local loop and the Python Tests CI job faster without changing any test.

What changed

  • pytest-xdist joins the dev dependency group, so make install brings it in.
  • make test and the Python Tests workflow run pytest with -n auto, one worker per core. PYTEST_ARGS="-n 0" still runs the suite serially when debugging a single test, since the later -n wins.
  • The release workflow's wheel tests are unchanged. Each takes about 30 seconds and already runs as a parallel matrix.
  • Nothing changes for users of the package: no API, behaviour or documentation changes.

Validation

Local timings on an Apple Silicon laptop with the debug build, the same 1645 tests passing in each run:

Run Wall time
Serial (before) 25.4s
-n 4 (the CI runner's core count) 10.8s
-n auto 8.1s
  • uv lock --check passes; the lockfile adds only pytest-xdist and execnet.
  • uvx typos@1.26.0 . passes.
  • No benchmark runs, since the change touches only test tooling and no lance-csr code.
  • The timings above come from a local build, not from this branch's own build. The Python Tests run on this PR is the check on this branch; its pytest step took 63s serially on main, and I expect it to drop to about 25s.

🤖 Generated with Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Deploying lance-csr with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3bd182d
Status: ✅  Deploy successful!
Preview URL: https://aa350854.lance-csr.pages.dev
Branch Preview URL: https://chore-pytest-xdist.lance-csr.pages.dev

View logs

@prrao87
prrao87 marked this pull request as ready for review October 29, 2026 17:28
@prrao87
prrao87 merged commit 1dc63aa into main Oct 29, 2026
3 checks passed
@prrao87
prrao87 deleted the chore/pytest-xdist branch October 29, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant