Add by-lex sorted-set coverage: ZRANGEBYLEX / ZREVRANGEBYLEX / ZLEXCOUNT / ZRANGESTORE BYLEX - #554
Add by-lex sorted-set coverage: ZRANGEBYLEX / ZREVRANGEBYLEX / ZLEXCOUNT / ZRANGESTORE BYLEX#554neoLsH wants to merge 2 commits into
Conversation
…UNT / ZRANGESTORE BYLEX Five specs mirroring the existing by-score shapes, as proposed in redis#548: - 1key-zset-100-elements-zrangebylex-all-elements (listpack lex scan) - 1key-zset-100-elements-zrevrangebylex-all-elements (reverse lex walk) - 1key-zset-1M-elements-zrangebylex-100-elements-pipeline-10 (skiplist seek + slice) - 1key-zset-1M-elements-zlexcount-pipeline-10 (count-only, no reply materialisation) - 1key-zset-600K-elements-zrangestore-300K-elements-bylex (by-lex mirror of the by-score hero test) All specs use constant-score datasets, which BYLEX range semantics require: with equal scores the (score,member) ordering is purely lexicographical. The 1M specs reuse the existing 1key-zset-1M-elements-const-score dataset from the zmscore family; the 100-elements and 600K specs add const-score siblings of their by-score datasets. Slice sizes are made exact with LIMIT (unpadded integer members have no lexicographic halfway value, so a bare range cannot express 'first 300K'). Fixes redis#548
|
🤖 Automated first-pass review — a human maintainer's review is still required before merge. This is a single read-only pass over the PR description, diff, and the sibling specs already on Thanks for this — the by-lex half of the range family genuinely had no coverage, and the dataset hygiene is the right call: the two 1M specs reuse the The Separately, not blockers:
Four of the five specs look sound to me as written; it's really just the ZLEXCOUNT range form I'd want settled before this merges. |
…he 1M ZRANGEBYLEX spec Two corrections from the first-pass review, both verified against t_zset.c: 1. ZLEXCOUNT description no longer claims it 'walks the same lex-ordered iteration path' as ZRANGEBYLEX. At skiplist encoding zlexcountCommand does NOT iterate: it resolves both endpoints via zslNthInLexRange (nth=0 / -1, each an O(log N) seek that yields its rank) and derives the count by rank arithmetic -- matching the O(log(N)) complexity in commands.json. Only the listpack branch iterates, and this spec is 1M/skiplist. So it is not a traversal-vs-materialisation counterpart of the slice spec; the description now says what it actually covers (lex-range parse + both *InLexRange seeks + zslGetRank) and notes the Ops/sec is largely dispatch overhead at this cost. 2. The 1M ZRANGEBYLEX spec switched from '- + LIMIT 0 100' to a random lex start '[__key__ + LIMIT 0 100' (R key-pattern, --key-prefix "" to keep the integer lex domain clean), renamed to ...zrangebylex-random-position-pipeline-10 to mirror the existing zrangebyscore-random-position spec. With the '-' min sentinel zslLexValueGteMin is unconditionally true, so zslFirstInLexRange short-circuits to the leftmost node and the old form scanned the same head-of-skiplist 100 elements every call (cache-hot, cardinality-independent) -- a degenerate seek, not the large-set seek the issue asked for. Verified live: '- +' always returns '1','10','100'; '[50000 + LIMIT 0 100' seeks to '50000' and returns a full 100-slice; near-tail starts return short slices as expected for a uniform random start. Refs redis#548
|
Both points confirmed against ZLEXCOUNT does not traverse at skiplist encoding. Verified: Adopted the random lex start. The 1M ZRANGEBYLEX spec now uses Re-ran the local smoke (now 19 checks incl. the random-position seek, the |
Implements the five-spec starter set proposed in #548 — the lexicographical half of the sorted-set range family (
zslParseLexRange/zslNextValidElement/sdscmplencomparisons,-/+sentinels) had zero coverage, so any zset-line change (skiplist iteration, encoding conversion, B-tree POCs) could regress the by-lex path unnoticed.What's added
1key-zset-100-elements-zrangebylex-all-elements- +scanzrangebyscore-all-elementsbaseline1key-zset-100-elements-zrevrangebylex-all-elements+ -, max first)1key-zset-1M-elements-zrangebylex-100-elements-pipeline-10LIMIT 0 100, pipeline 101key-zset-1M-elements-zlexcount-pipeline-101key-zset-600K-elements-zrangestore-300K-elements-bylex- + BYLEX LIMIT 0 300000zrangestore-300Khero testAll in
oss-standalone,sorted-setgroup, same build variants / priorities as their by-score counterparts.Design notes
Constant-score datasets. BYLEX semantics are only meaningful when all members share one score (with distinct scores, Redis filters in
(score,member)order and the "lexicographical" result order is an artifact). So:1key-zset-1M-elements-const-scoredataset (from the zmscore family — single-__key__preload, verified exact 1,000,000, no Shared zset preload (ZADD lb __key__ __key__) undercounts cardinality due to per-occurrence key draws #508 dual-draw shortfall);1key-zset-100-elements-const-score(same member list as the by-score100-elements-floatdataset, constant score 0);1key-zset-600K-elements-const-score(members 1..600001, constant score 0 — exact cardinality for the same single-draw reason).Exact slices via LIMIT. The proposed
600K → 300Kcopy cannot be expressed as a lex value range: unpadded integer members have no lexicographic halfway point ("3..."covers far more than half the domain).LIMIT— valid precisely withBYSCORE/BYLEX— makes every slice exact:LIMIT 0 100for the 1M read spec,LIMIT 0 300000for the store spec.No dependency on #542: all four commands are already in the current
commands.json.Verification
redis-server v=8.9.241 sha=80920a7d): for each spec, parsed the yml and executed itsinit_commands/preload semantics + client command — keyspacelen=1, listpack encoding at 100 elements, both full scans return all 100 members in ascending/descending lex order,LIMIT 0 100returns exactly 100,ZRANGESTORE ... BYLEX LIMIT 0 300000copies exactly 300K elements forming a lex prefix of the source,ZLEXCOUNT - +equalsZCARD. 14/14 checks pass.redis-benchmarks-spec-cli --tool stats --fail-on-required-diffexits 0; ERROR count identical to the main baseline (pre-existingarray-*spec noise, untouched).Note
Low Risk
Adds benchmark specification YAML only; no Redis server or client code changes.
Overview
Adds five new memtier_benchmark YAML specs so lexicographical sorted-set commands (
ZRANGEBYLEX,ZREVRANGEBYLEX,ZLEXCOUNT,ZRANGESTORE ... BYLEX) are exercised in CI—parallel to existing by-score coverage, which previously had no BYLEX counterpart.The 100-member pair uses inline
ZADDinit with constant score 0 (1key-zset-100-elements-const-score) and full- +/+ -scans on listpack encoding. 1M-member specs reuse1key-zset-1M-elements-const-score: one runsZLEXCOUNT lb - +with pipeline 10 (skiplist rank path), the otherZRANGEBYLEXwith a random lex lower bound andLIMIT 0 100so each call does a real seek instead of always hitting the skiplist head. The 600K spec preloads1key-zset-600K-elements-const-scoreand benchmarksZRANGESTORE zset1 zset - + BYLEX LIMIT 0 300000as the by-lex mirror of the existing by-scorezrangestorehero test.All specs target
oss-standalone,sorted-set, and the same GCC/dockerhub build variants and priorities as their by-score siblings.Reviewed by Cursor Bugbot for commit 7d68991. Bugbot is set up for automated code reviews on this repo. Configure here.