dataframely: Add version 3.0.0 by luhenry · Pull Request #2423 · riseproject-dev/python-wheels

luhenry · 2026-09-28T07:29:41Z

Package: dataframely
Version: 3.0.0
Source: https://github.com/quantco/dataframely
Docs: https://dataframely.readthedocs.io/

Compiles dataframely's own PyO3 extension (dataframely._native), which patches Polars core/arrow onto a pinned pola-rs fork commit and links pyo3-polars. Upstream publishes no riscv64 wheel.

Mirrors upstream's build.yml.

Differs from upstream

Fat LTO/codegen-units=1 dropped, opt-level=1, CARGO_BUILD_JOBS=2 - riscv64 runner cannot afford upstream's release profile on this crate graph
Only linux-riscv64 built - upstream's macOS/Windows/other-Linux legs are out of scope here

Matrix: cp310-abi3 only, single leg - matches upstream's own single-interpreter maturin build (abi3-py310 covers cp310-cp314)

Testing

same as upstream's default (non-optional) unit-test leg, pytest tests -m "not with_optionals"

License: OK

dataframely is not a pure-Python Polars wrapper: it ships its own PyO3 extension (dataframely._native) built with maturin, and its Cargo.toml patches polars/polars-core/polars-arrow/pyo3-polars onto a pinned pola-rs fork commit, so building it compiles a full Polars-scale crate graph from source -- comparable in size to build-polars-runtime.yml's own build. Mirrors upstream's own build.yml (maturin-action, --release -i python3.10, abi3-py310), narrowed to riscv64gc-unknown-linux-gnu on manylinux_2_39, with the same fat-LTO override build-polars-runtime.yml and build-deltalake.yml already carry for a crate graph of this size.

github-actions · 2026-09-28T08:26:01Z

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-02 07:10 UTC

…le tests pypi.riseproject.dev's pyarrow only has riscv64 wheels for cp312+, so uv cannot resolve it for the cp310 test venv. dataframely's own test suite only hard-imports pyarrow in tests/columns/test_arrow_pycapsule.py, so ignore that one module instead of pulling pyarrow in for this abi3 build.

The Test wheel step installs the just-built wheel then runs `python -c 'import dataframely; ...'` and `python -m pytest tests ...` from the checkout root. dataframely's pure-Python wrapper package lives directly at that root, and both invocation forms unconditionally prepend cwd to sys.path[0], so they resolved the checkout's own dataframely/__init__.py (no compiled _native) instead of the wheel's copy in site-packages (gotcha 25/218). Fix: cd /tmp first, and pass the tests dir plus pytest's config file and --ignore path as GITHUB_WORKSPACE-absolute so pytest can still find them. Reproduced and verified the fix locally with a stand-in package tree mimicking the checkout/site-packages split, for both the `python -c` and `python -m pytest` invocations.

tests/benches/conftest.py imports pandas at module load, which pytest collection needs even though the benchmark suite itself is not gated by -m "not with_optionals". pypi.riseproject.dev already publishes a cp310 riscv64 wheel for pandas (up through 2.3.2, before 3.0's requires-python bump to >=3.11), so this resolves to a prebuilt wheel rather than a from-source build.

… no cp310 riscv64 wheel scikit-learn only has cp312+ riscv64 wheels on the registry, so it can't be installed for this cp310-abi3 leg's test dependencies; skip the one directory that needs it, mirroring the existing pyarrow skip.

luhenry · 2026-09-30T14:15:53Z

The Build dataframely 3.0.0 cp310-abi3-manylinux_riscv64 check failed at 14:14 UTC after running ~5.5h, but the job's own step list shows Test wheel still in_progress (no completed_at) and the two steps after it pending when the job was force-marked failure — the runner (rise-riscv-runner-bunz5lxyc) died mid-test, this isn't a real test failure in this PR's code. Build wheel had already completed successfully beforehand. Not this PR's issue; queued a re-run of the failed job via the Actions API.

Generated by Claude Code

luhenry · 2026-09-30T20:09:30Z

Run 36691176068's Test wheel step (this time ran to genuine completion, not the earlier runner death) is a real failure: 9 failed, 1608 passed, 9 skipped, 365 deselected, 5 xfailed.

All 9 failures cluster around the native dataframely._native Rust/polars validation plugin producing a different error-message format than the tests expect, e.g.:

AssertionError: Regex pattern 'examples: \[' does not match 'the plugin failed with message: \n2 rules failed validation for schema ...'

dataframely v3.0.0's own pyproject.toml pins polars>=1.35 with no upper bound, so this environment resolved whatever polars is newest on the registry today — likely newer than what 3.0.0's test suite was actually written against, similar in kind to the ase/pytest version-drift bugs found elsewhere in this campaign (matscipy, kivy). The other failures (test_sample_fuzzy_failure, test_synthesized_categories_name, test_sample_invalid_override_values_raises) look like the same family: assertions tied to specific polars-plugin error text or sampling behavior.

Not fixing this now: pinning a polars version to test the hypothesis costs another ~4-4.5 hour Build wheel + Test wheel cycle on this runner pool (the maturin build alone dominates), and I don't have a way to verify the right polars version without paying that cost per guess. Flagging with this diagnosis rather than guessing blind. Next step for whoever picks this up: identify the polars version dataframely v3.0.0's own CI actually tests against (its pixi lockfile, if it pins one) and cap CIBW/test-install polars to match, the same pattern used for matscipy (ase<3.28) and kivy (pytest<9.1) in this repo.

Generated by Claude Code

…match The test install picked pytest 6.2.5: the registry hosts tomli for cp312+ only, so under uv's default first-index strategy no tomli was available for cp310 and the resolver backtracked to the last pytest that used toml instead. pytest 6's --import-mode=importlib (from upstream's addopts) names test modules by basename, which produced all 9 failures in the last run. The same 9 failures reproduce on x86_64 with upstream's own PyPI wheel under pytest 6.2.5, and the full suite passes under pytest 9.1.1. The bare dist/*.whl install also never consulted the registry, so it built numpy 2.2.6 and polars-runtime-32 1.44.2 from source (~4h); the registry now carries polars-runtime-32 1.44.2. pandas is dropped since only the already-skipped tests/benches import it.

luhenry · 2026-10-02T00:47:31Z

Follow-up on run 36691176068: the 9 failures are not polars drift, and not riscv64-specific. The test venv resolved pytest 6.2.5:

pypi.riseproject.dev hosts tomli 2.4.1 only as cp312+ riscv64 wheels. Under uv's default first-index strategy, the extra index wins for tomli, so there is no tomli at all for cp310 and the resolver backtracks to the last pytest that depended on toml instead (6.2.5).
pytest 6's --import-mode=importlib (from upstream's addopts) names test modules by basename. That explains 'test_categorical:TestSchema' vs 'column_types.test_categorical:TestSchema' directly, and the dy.Config-dependent failures (examples / sampling iterations) only show up in full-suite ordering.
Reproduced on x86_64 with upstream's own PyPI dataframely==3.0.0 wheel + polars==1.44.2 + pytest==6.2.5, using the same command line: the identical 9 failures (9 failed, 1608 passed, 9 skipped, 365 deselected, 5 xfailed). With pytest 9.1.1 (upstream's lockfile has 9.0.3): 1617 passed, 9 skipped, 365 deselected, 5 xfailed.

Pushed 3341c1b: the Test wheel step now installs the wheel and test deps in one uv pip install with UV_EXTRA_INDEX_URL + UV_INDEX_STRATEGY=unsafe-best-match + UV_ONLY_BINARY=numpy. A dry resolve for riscv64/cp310 now gives pytest 9.1.1 + tomli 2.4.1.

This also removes ~4h from the Test step. The old bare uv pip install dist/*.whl never consulted the registry, so it built numpy 2.2.6 and polars-runtime-32 1.44.2 from source (16:04 to 19:58). The registry now has polars_runtime_32-1.44.2-cp310-abi3. pandas is dropped since only the already-ignored tests/benches imports it.

Generated by Claude Code

luhenry added a commit that referenced this pull request Sep 28, 2026

dataframely: queued -> ci-running, PR #2423

811a790

luhenry added 4 commits September 28, 2026 13:44

luhenry force-pushed the main branch 3 times, most recently from 39fb7ba to a75cf68 Compare October 1, 2026 15:22

luhenry marked this pull request as ready for review October 2, 2026 03:01

luhenry merged commit 8554766 into main Oct 2, 2026
9 checks passed

luhenry deleted the dataframely branch October 2, 2026 07:02

Navigation Menu

dataframely: Add version 3.0.0 - #2423

dataframely: Add version 3.0.0#2423
luhenry merged 6 commits into
mainfrom
dataframely

luhenry commented Sep 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 28, 2026 •

edited

Loading

Uh oh!

luhenry commented Sep 30, 2026

Uh oh!

luhenry commented Sep 30, 2026

Uh oh!

luhenry commented Oct 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

luhenry commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

luhenry commented Sep 30, 2026

Uh oh!

luhenry commented Sep 30, 2026

Uh oh!

luhenry commented Oct 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

luhenry commented Sep 28, 2026 •

edited

Loading

github-actions Bot commented Sep 28, 2026 •

edited

Loading