You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cap the bytes of string and binary data blocks; skip chunks a known mask excludes - #106
Two patches for spiceai/spiceai#15323: a point lookup on a wide string column read a whole ~12.5 MiB block to return one row.
Byte-capped data blocks for string and binary columns (WriteStrategyBuilder::with_data_block_max_bytes, default None; SPICE_PATCHES.md row 17).
Chunked reads skip chunks a known mask excludes (MaskFuture::known; row 18).
WHY
effective_block_len sizes data blocks by bytes only for fixed-width types, so a Utf8/Binary column always gets 8,192-row blocks whatever its rows weigh. A block is one segment, and a segment is the smallest unit the reader fetches, so a one-row read of a ~3 KB/row JSON column reads ~12.5 MiB.
Splitting blocks alone is not enough:
Write cost. A block of at most 1,024 rows is its own BtrBlocks sample, so the scheme search trial-compresses every candidate on the whole block. Writes got 3.5x slower.
Point reads. A point read of one row in an 8,192-row range, with the mask already resolved, still fetched every chunk of the range, because FlatReader fetches its segment before it looks at the mask. With split blocks, this made a lookup read more bytes than with no cap.
HOW
RepartitionStrategy::with_block_size_maximum splits a canonical VarBinView chunk whose referenced bytes exceed the maximum. It cuts the chunk into row blocks, then halves any block still over the maximum until each piece fits or holds one row, so a cluster of wide rows can't leave a block oversized.
The split sizes divide the 8,192-row zone, so zone maps and cross-column block boundaries are unchanged.
With no maximum, nothing changes.
CompressionPlan: CascadingCompressor::plan_for chooses the schemes for a whole row block on the same stratified sample compress() would draw from the unsplit block, recording each decision keyed by cascade path; compress_with_plan applies them to every split chunk.
Choosing on the block's first split chunk instead (the first version of this PR) fits the rest of the block to that chunk: with incompressible rows ahead of JSON, the JSON got FSST instead of OnPair, for a 15% larger file and 2.4x the LIKE CPU. compressed::tests::a_group_is_encoded_as_its_unsplit_data_would_be covers this, and fails on the first-chunk version (FSST vs OnPair).
A path the plan never decided falls back to the search.
A replayed scheme is still kept only when it shrinks its input. If it cannot encode the data at all (a sequence replayed on non-sequential integers), the normal search runs instead.
CompressingStrategy::with_group_rows compresses consecutive chunks smaller than N rows as one CPU task. It applies only to Utf8/Binary chunks, and WriteStrategyBuilder enables it only when a maximum is set. Other small chunks, such as a wide fixed-size list's blocks, stay parallel. CompressorPlugin::compress_chunks defaults to one call per chunk and is forwarded explicitly through Arc<dyn CompressorPlugin>.
ChunkedReader skips chunks whose slice of a known mask is all-false, in both projection and filter evaluation.
Evidence
All runs are on Apple M5 Max, macOS, release profile, base 806e48da2.
Write CPU, vortex-btrblocks/benches/small_chunk_compress.rs (98,304 rows of ~3 KB JSON, 1,024-row chunks, thread CPU, median of 5):
Arm
Write CPU
Output
default, 8,192-row blocks
2,804 ms
69.96 MiB
split blocks, full search per block
9,853 ms (3.5x)
74.51 MiB
split blocks, plan recorded on the first block and replayed
2,934 ms (1.05x)
74.51 MiB
split blocks, plan chosen on a 16 × 64-row sample of the whole block
3,138 ms (1.12x)
74.51 MiB
Point reads through the builder (same data, through Spice's bench via a local [patch]):
Cap
Bytes per lookup
Lookup CPU
File size
none
6,136,725
0.267 ms
71.4 MiB
4 MiB
819,273
0.074 ms
76.2 MiB (+7%)
Every row read back byte-identical.
Read path with the skip, narrow columns (vortex-file/benches/scan_mask_paths.rs, in this PR; 4M rows; files byte-identical with and without the skip): full projection, filtered range, exclude-plus-filter and point-read CPU are equal within run-to-run noise.
Tests:vortex-compressor, vortex-btrblocks (golden snapshots included), vortex-layout and vortex-file pass. The new tests:
3 plan tests;
4 split tests;
3 grouping tests, one of which checks that an error mid-group is emitted after the group;
1 file round trip with nulls;
3 chunk-skip cases with a counting segment source.
Negative controls: each of these turned its target tests red, and the sources were then restored:
replay ignoring the plan;
split_rows returning None;
grouping disabled;
the skip removed;
choosing a block's encodings on its first split chunk instead of the whole block's sample (FSST instead of OnPair).
CI at 877e96db6 (re-running at the new head): all Rust lint, docs, MSRV, miri, asan, tsan, musl, Windows, wasm and golden-corpus jobs pass. The base branch's last PR (#102) fails the same other checks, for the following reasons, none of them in this PR's code:
the default and all-features builds time out at 30 minutes;
the coverage job's runner ran out of disk;
the generated-sources check finds vortex-ffi/cinclude/vortex.h out of date, a file this PR does not touch;
the Python, SQL logic test, lint-toml, minimal-dependency and advisory checks fail.
The changelog-label check needs a label.
Review fixes (ed8b9e422..dce56c6ea, one commit per review thread, each with a regression test that failed first):
fall back to the search when a replayed scheme cannot encode the data;
an inspected MaskFuture's slices still run its callback;
group only capped string and binary blocks (an embedding column's write wall time had gone 52 → 187 ms with no maximum set);
halve split blocks until each piece fits;
build the CPU-timing benches only on Unix;
keep coalesced blocks under the maximum (a 0.9 MiB block and a 3.9 MiB one had merged into 5.0 MB under 4 MiB);
compact each split piece, so it stores only its own rows' bytes whatever the compressor (with a pass-through compressor, random binary split in two had written 16.5 MB instead of 8.3 MB).
The measurements below were taken at 877e96db6. For uniform-width strings the split and the encodings are unchanged, and the 3 KB JSON dataset gives the same file size and segment count.
Cases that could be worse
vortex-file/benches/wide_string_cases.rs (in this PR) writes each dataset with and without a 4 MiB maximum in one process. It then reads both arms, interleaved A B B A, through an in-memory reader that coalesces like an object store (1 MiB gap, 16 MiB max) and counts what it is asked for. The values each arm returns are compared and equal.
Commit 877e96db6; Apple M5 Max; load average 4.9–10.6.
Each scan entry is the median CPU of 11 repetitions. Point reads are 10,000 per arm.
The in-memory reader slices instead of copying, so the point-read rows understate the saving on a real pread.
control: dictionary-encoded strings (the maximum does not apply)
0%
+1%
+1% / −1%
101 → 105
+1% / 0%
6.5 → 6.2 ms / −2%
equal
Where it is worse:
Full scans of a wide column: +10–20% CPU.
Range scans projecting it: 1.6–3x CPU, though only a few milliseconds.
Writes: +7–15% CPU.
Object-store requests: up to 7x more per full scan.
Very wide rows:LIKE uses 40% more CPU, spread over more parallel blocks, so its wall time is 5x lower.
Write cost of this commit: choosing on the whole block's sample costs about 7% more write CPU than choosing on the first chunk (uniform 3 KB JSON, 2,062 vs 1,926 ms). That is what removes the regression above: LIKE 75 → 32 ms, file 60.7 → 54.7 MiB.
Spice consumer: spiceai/spiceai#15434, pinned to this branch's head dce56c6ea8b95eebb92633fd65ae6ea271ff1002.
…encodings once per row block
A string or binary column is written in data blocks of `row_block_size` (8,192) rows whatever its
rows weigh: `RepartitionWriterOptions::effective_block_len` shrinks blocks only for types with a
fixed element size. A column of ~3 KB JSON documents gets ~12.5 MiB blocks, and since a reader
never reads less than one block, a one-row lookup reads and copies all of it.
`WriteStrategyBuilder::with_data_block_max_bytes` caps the bytes a block's rows reference. A
block over the cap is split into blocks of a power-of-two number of rows, which divides the row
block size, so split blocks line up with zone statistics and with every other column's blocks.
Blocks are measured by the bytes their views reference, not by their buffers, which slices of a
larger array share. The cap is off by default.
Splitting alone makes writes 3-4x slower on such a column. Choosing an encoding compresses a
sample once per candidate scheme, and a block of at most 1,024 rows is its own sample, so every
small block paid several full compressions. `CascadingCompressor::compress_recording` returns the
schemes a compression chose as a `CompressionPlan`, and `compress_with_plan` replays them without
estimating; a path the plan never decided still searches, and a replayed scheme is still kept only
when it shrinks its input. `CompressingStrategy::with_group_rows` compresses consecutive blocks
smaller than the row count together through the new `CompressorPlugin::compress_chunks`, which
BtrBlocks implements by searching the first block and replaying on the rest, so encodings are
still chosen once per row block. A block of the full row count is always compressed alone, so
columns whose blocks are not split compress exactly as before.
On 98,304 rows of ~3 KB JSON with a 4 MiB cap: 0.82 MB read per single-row lookup instead of
6.1 MB, +7% file size, write CPU within 10% of the default (3.5x without encoding reuse).
…w range
`ChunkedReader` evaluated a filter or projection on every chunk overlapping the row range, and the
flat reader requests its segment before awaiting the mask, so every chunk in range was read even
when the mask selected none of its rows. A point read passes an already-resolved mask over a whole
8,192-row range; once a string column's blocks are split under a byte cap, reading one row of that
range fetched every split block, which made the lookup slower than before the split.
`MaskFuture::known` returns the mask when the future was created from one (`ready`, `new_true`, or
a slice of either). `ChunkedReader` skips a chunk whose known mask slice selects no rows: its
projection contributes nothing and its filter result is all false, so neither its segment nor its
children are requested. A mask that is still pending is evaluated as before, so scans keep issuing
segment requests ahead of the mask.
…block, not its first chunk
Replaying the schemes chosen on a block's first split chunk encoded the rest to suit that
chunk: with incompressible rows ahead of JSON, the JSON got FSST instead of OnPair, a 15%
larger file and 2.4x the LIKE CPU. CascadingCompressor::plan_for chooses on the stratified
sample compress() would draw from the unsplit block, so each split chunk is encoded as the
block would have been.
…he data
A plan replays the schemes chosen on other data, and some schemes accept a dtype yet reject
data they cannot encode: a sequence recorded on 0..n failed the write of random integers with
'cannot sequence encode array'. On that failure the cascade now chooses as if there were no
plan; an error from that choice still fails the write.
inspect kept the known mask, so a slice resolved from it without awaiting the inspected future
and never ran the callback. An inspected future is no longer known.
…plits
Grouping applied to every chunk smaller than a row block, with or without a maximum, so the
blocks of a wide fixed-size list (an embedding column) compressed serially in one task: same
bytes and CPU, 3.5x the write wall time. Only Utf8/Binary chunks group, and only when
with_data_block_max_bytes is set.
The rows per block came from the chunk's average row width, so wide rows clustered at one end
left a block over the maximum (25.7 MB under 4 MiB). Each block over the maximum is now halved
until its pieces fit or hold one row; uniform rows split as before.
The maximum was checked only on incoming chunks, so coalescing could merge blocks that each fit
into one that did not: a 0.9 MiB block and a 3.9 MiB one became a 5.0 MB block under a 4 MiB
maximum. Every block coalescing emits is now split by the same rule.
A split piece was a slice sharing its parent's data buffers, and a buffer is written whole, so
with a compressor that stores chunks as they are each piece carried the whole block: random
binary split in two wrote 16.5 MB instead of 8.3 MB. Each piece is now compacted at threshold
1.0, which slices a contiguously used buffer rather than copying it. The regression covers
BtrBlocks and a pass-through compressor, for a block split as it arrives and one rebuilt by
coalescing.
The P2 arm uses a different sampler from production plan_for: it takes fixed-position runs and repeats rows in short tails, whereas production uses seeded-random runs and keeps blocks of at most 1,024 rows intact. For example, SCC_ROWS=8256 expands the 64-row tail into 544 sampled rows, so the reported CPU and output sizes can reflect different encoding choices and planning work. Call plan_for on the whole block slice instead of duplicating its sampler here.
Addressed the previously missed benchmark issue from Copilot's last review in 51e7a4c: the sampled arm of vortex-btrblocks/benches/small_chunk_compress.rs now calls plan_for on the whole block, so it plans exactly as the writer does. It builds and runs, including SCC_ROWS=8256, and clippy is clean. Benchmark only; no library code changed.
…nt lookup reads one small block (spiceai#15434)
* fix(cayenne): cap the bytes of a string or binary data block so a point lookup reads one small block
A point lookup on a Cayenne table returned one row but read and copied the whole Vortex data block
that held it, for every projected column. Vortex sizes blocks by bytes only for fixed-width types, so
a string or binary column always got 8,192-row blocks: ~12.5 MiB for ~3 KB JSON rows. On a cache
miss that block was read into a fresh buffer and copied into the segment cache. Under snmalloc, the
default allocator, both fresh buffers are faulted in page by page: on Linux about 2,000 minor faults
per lookup, most of the lookup's CPU.
Every Cayenne write path now caps a string or binary block at 4 MiB of referenced bytes
(`DATA_BLOCK_MAX_BYTES`): the full tier, light delta encodings, compaction and the cold tier. Over
the cap a block is split into a power-of-two number of rows, so block boundaries stay aligned with
the 8,192-row zones and with other columns. A table whose blocks are all under the cap is written
byte for byte as before. There is no new parameter.
This needs two Vortex fork changes, pinned here at a temporary branch:
- byte-capped blocks, with encodings chosen once per 8,192 rows and replayed on the split blocks,
since searching each small block on its own made writes 3-4x slower;
- chunked reads skip the blocks a resolved row selection excludes, without which the key-block point
read fetched every split block of its 8,192-row range and read more than with no cap.
`a_point_read_of_a_wide_column_reads_one_capped_block` guards both: one row costs 810,699 bytes
read instead of 5,724,035. Adds `wide_column_point_lookup` benches under the system allocator,
jemalloc and snmalloc. Refs spiceai#15323.
* docs(fork_patches): link the vortex fork PR the temporary pin waits on
* fix(cayenne): pin the vortex fork commit that chooses a block's encodings on a sample of the whole block
* bench(cayenne): measure write and full-scan costs of the wide column alongside its lookups
* style(cayenne): rustfmt the wide-column bench
* fix(cayenne): compile the partition conflict-detection setter only with partition-table-provider
Its only caller is behind that feature, so a build without it failed -D dead-code.
* fix(cayenne): pin the vortex fork commit with the review fixes to byte-capped blocks
* fix(cayenne): pin the vortex fork commit that keeps coalesced blocks under the byte maximum
* fix(cayenne): pin the vortex fork commit whose split pieces hold only their own bytes
* fix(cayenne): pin the vortex fork's merged byte-capped string blocks
spiceai/vortex#106 merged as e5566746 on spiceai-55, so the pin leaves the PR branch and the
ledger drops its TEMPORARY marker.
* bench(cayenne): take a backtrace credit only while one is left
Two allocating threads past the credit check could both decrement the last credit and wrap
the count to u64::MAX, so capture never stopped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WHAT
Two patches for spiceai/spiceai#15323: a point lookup on a wide string column read a whole ~12.5 MiB block to return one row.
WriteStrategyBuilder::with_data_block_max_bytes, defaultNone;SPICE_PATCHES.mdrow 17).MaskFuture::known; row 18).WHY
effective_block_lensizes data blocks by bytes only for fixed-width types, so a Utf8/Binary column always gets 8,192-row blocks whatever its rows weigh. A block is one segment, and a segment is the smallest unit the reader fetches, so a one-row read of a ~3 KB/row JSON column reads ~12.5 MiB.Splitting blocks alone is not enough:
FlatReaderfetches its segment before it looks at the mask. With split blocks, this made a lookup read more bytes than with no cap.HOW
RepartitionStrategy::with_block_size_maximumsplits a canonical VarBinView chunk whose referenced bytes exceed the maximum. It cuts the chunk into row blocks, then halves any block still over the maximum until each piece fits or holds one row, so a cluster of wide rows can't leave a block oversized.CompressionPlan:CascadingCompressor::plan_forchooses the schemes for a whole row block on the same stratified samplecompress()would draw from the unsplit block, recording each decision keyed by cascade path;compress_with_planapplies them to every split chunk.LIKECPU.compressed::tests::a_group_is_encoded_as_its_unsplit_data_would_becovers this, and fails on the first-chunk version (FSST vs OnPair).CompressingStrategy::with_group_rowscompresses consecutive chunks smaller than N rows as one CPU task. It applies only to Utf8/Binary chunks, andWriteStrategyBuilderenables it only when a maximum is set. Other small chunks, such as a wide fixed-size list's blocks, stay parallel.CompressorPlugin::compress_chunksdefaults to one call per chunk and is forwarded explicitly throughArc<dyn CompressorPlugin>.ChunkedReaderskips chunks whose slice of a known mask is all-false, in both projection and filter evaluation.Evidence
All runs are on Apple M5 Max, macOS, release profile, base
806e48da2.Write CPU,
vortex-btrblocks/benches/small_chunk_compress.rs(98,304 rows of ~3 KB JSON, 1,024-row chunks, thread CPU, median of 5):Point reads through the builder (same data, through Spice's bench via a local
[patch]):Every row read back byte-identical.
Read path with the skip, narrow columns (
vortex-file/benches/scan_mask_paths.rs, in this PR; 4M rows; files byte-identical with and without the skip): full projection, filtered range, exclude-plus-filter and point-read CPU are equal within run-to-run noise.Tests:
vortex-compressor,vortex-btrblocks(golden snapshots included),vortex-layoutandvortex-filepass. The new tests:Negative controls: each of these turned its target tests red, and the sources were then restored:
split_rowsreturningNone;Lint:
cargo clippy -p vortex-compressor -p vortex-btrblocks -p vortex-layout -p vortex-file --all-targets --all-features -- -D warningsexits 0.CI at
877e96db6(re-running at the new head): all Rust lint, docs, MSRV, miri, asan, tsan, musl, Windows, wasm and golden-corpus jobs pass. The base branch's last PR (#102) fails the same other checks, for the following reasons, none of them in this PR's code:vortex-ffi/cinclude/vortex.hout of date, a file this PR does not touch;lint-toml, minimal-dependency and advisory checks fail.The changelog-label check needs a label.
Review fixes (
ed8b9e422..dce56c6ea, one commit per review thread, each with a regression test that failed first):MaskFuture's slices still run its callback;The measurements below were taken at
877e96db6. For uniform-width strings the split and the encodings are unchanged, and the 3 KB JSON dataset gives the same file size and segment count.Cases that could be worse
vortex-file/benches/wide_string_cases.rs(in this PR) writes each dataset with and without a 4 MiB maximum in one process. It then reads both arms, interleaved A B B A, through an in-memory reader that coalesces like an object store (1 MiB gap, 16 MiB max) and counts what it is asked for. The values each arm returns are compared and equal.877e96db6; Apple M5 Max; load average 4.9–10.6.pread.LIKECPU / wallWhere it is worse:
LIKEuses 40% more CPU, spread over more parallel blocks, so its wall time is 5x lower.LIKE75 → 32 ms, file 60.7 → 54.7 MiB.Spice consumer: spiceai/spiceai#15434, pinned to this branch's head
dce56c6ea8b95eebb92633fd65ae6ea271ff1002.