[Nathan-D-R]

Second PR of the final major round. Stacked on #53 (API foundation); the base is that branch and retargets to main when #53 merges.

A new Arrow major no longer forces a new openqvd major, on the Rust side or the Python side.

What changes

Rust

• openqvd has no Arrow dependency and no arrow feature.
• The type rules (DATE/TIMESTAMP/TIME/Int/Float/Utf8) and filters move into the core as Arrow-free API: Qvd::column(i) -> ColumnData (a ColumnType, one converted value per distinct symbol, the symbol index of each row), Qvd::select_rows(&[Filter]), Filter, ColumnFilter.
• New crate openqvd-arrow: features arrow-59 and arrow-60, modules openqvd_arrow::v59 / v60, built from one source file with renamed dependencies. Both can be enabled at once. A new Arrow major is a new feature plus one macro line. Each module has to_record_batch(&Qvd, &BatchOptions), record_batch_to_write_table and record_batches_to_write_table (all batches).

Python

• pyo3-arrow is gone. The extension implements the Arrow PyCapsule interface itself (__arrow_c_stream__, __arrow_c_schema__ on export; __arrow_c_stream__ and __arrow_c_array__ on import). The Arrow dependency is private to the extension and never appears in a signature.
• openqvd.read returns openqvd.Table: num_rows, column_names, warnings, the two capsule methods, and lazy to_pyarrow(), to_polars(), to_pandas(). openqvd.write takes anything that exports Arrow data.
• pyarrow is no longer a dependency (extras: pyarrow, polars, pandas, duckdb). Floors were measured by running the suite against the minimums: pyarrow 15, Polars 1.10, pandas 2.2, DuckDB 1.2.
• abi3 wheels (pyo3 feature abi3-py310); the PyPy classifier is dropped.

Dependabot #38, #39, #40 (arrow-array / arrow-cast / arrow-schema 59 to 60)

Covered: arrow-60 is a feature of openqvd-arrow, and the Python extension builds against Arrow 60 directly (no pyo3-arrow, which is what made those PRs fail). All tests pass with 59 and 60 enabled together. I am closing the three with a pointer to this PR.

API breaks (one commit each: core, arrow crate, Python)

Qvd::to_record_batch* and record_batch_to_write_table at the crate root are removed in favor of openqvd-arrow; Filter/ColumnFilter built with Filter::new; openqvd.read returns Table, not pyarrow.Table; pyarrow no longer installed by default. See CHANGELOG.

Findings fixed here

• S2-4 (Python write kept only the first batch): every batch is converted. Test: test_every_batch_is_written, every_batch_is_converted.
• S2-3, NaN half: filters compare by symbol index, so NaN rows are kept. Test: filters_select_rows_by_symbol_index. The typed-value half (1.0 vs 1) follows in a later PR.
• S2-6 (Arrow/column paths): a column outside ReadOptions::columns is FieldNotDecoded, not an all-NULL column. Test: undecoded_column_is_an_error_not_nulls. Qvd::rows() still yields None for such fields (documented).
• Filtering a table with an all-NULL column failed with an Arrow length error (noted in the audit); fixed and tested.
• Not touched here (the type rules moved unchanged, including the lossy conversions S1-1, S1-4, S1-6 and writer S1-3, S1-5, S1-12): they are the next PRs.

Tests changed

test_core.py reads through .to_pyarrow(); test_duckdb.py uses Table.column_names. New: tests/columns.rs, openqvd-arrow/tests/versions.rs (same suite per Arrow major), test_table.py.

Corpus

Python read of all 1,145 .qvd files in /mnt/nas/data/QVD compared with the 2.0.0 gate run (Oct 20): 0 differences in status, row counts, Arrow types or per-column value hashes. 52 files error in both runs; only the text of the error differs. Rust validate_all: 1,044 ok, 3 known damaged fixtures. Nothing newly errors.

Lenient mode

None added.

Loom

pl.from_arrow(openqvd.read(...)) now returns a Series, not a DataFrame. Loom must switch to table.to_polars() or pl.DataFrame(table); its other uses keep working. Full list in the report.

Docs-site and release-PR edits needed

• docs/: Rust quickstart (openqvd-arrow, ColumnData), Python quickstart (Table, extras, floors), install page (abi3, no PyPy).
• .github/workflows/publish.yml: publish openqvd-arrow after openqvd (cargo publish -p openqvd-arrow); drop --find-interpreter from the wheel build since wheels are abi3.
• RELEASING.md: three crates now share the version; crates/openqvd-arrow dependency line on openqvd.
• ci.yml needs no change (--all-features builds both Arrow majors).

Review fixes

• B1: Qvd::rows() yields Err(FieldNotDecoded) first and ends when any field is undecoded; to_write_table() errors up front (test rows_and_to_write_table_refuse_undecoded_columns). The text above that says rows() still yields None is superseded.
• S1: Dependabot ignores arrow-* majors and groups minors/patches.
• S3: read(columns=[]) returns a zero-column table with the row count (test_empty_column_list_gives_a_table_with_rows_and_no_columns).
• S4: to_relation(path) without con uses DuckDB's default connection; view_name needs con. New CI job python-floors installs pyarrow 15.0.0, polars 1.10.0, pandas 2.2.0, duckdb 1.2.0 on Ubuntu / Python 3.12 and runs pytest; clippy now runs with --all-targets. N9: rust-version is 1.88.