release(v0.8.9): a command must not write what it cannot propagate - #829
Merged
Conversation
Patch release rolling up 2 merged PRs closing 3 issues since v0.8.8.
One invariant. Propagation is gated on the reply not being an error
(`!matches!(resp, Frame::Error(_))`), so any command that mutates and THEN
returns an error leaves a write applied on the master, never appended to the
AOF and never sent to a replica: silent data loss across restart, permanent
replica divergence.
CORRECTNESS
- `redis.call`/`redis.pcall` turned a Lua nil, boolean or table argument into a
frame shape no wire client can produce, which HSET, HMSET, LPUSH, RPUSH,
LPUSHX, RPUSHX, ZREM, MSET and MSETNX each discovered from INSIDE their
mutation loop. Measured: `EVAL "return redis.pcall('LPUSH','mylist','a','b',
true)" 0` answered an arity error with `LLEN mylist = 2` resident, and 0
after restart. Fixed at the boundary, as Redis 8.6.1 refuses it, AND by
hoisting validation above the mutation window in all nine commands — defence
in depth, because the raw wire reaches the same code with no Lua anywhere
(#823, PR #824).
- XADD called `get_or_create_stream` before parsing the ID, so all five of
`bogus`, `0-0`, `1-1-1`, `abc-1` and `-5` created a charged, DBSIZE-visible
stream that was never logged and vanished on restart. 3,000 rejected XADDs
cost 1.46 MB nothing credits back (#823, PR #824).
- Blocking pops propagated NOTHING outside MULTI. BLPOP, BRPOP, BLMOVE,
BRPOPLPUSH, BZPOPMIN, BZPOPMAX, BLMPOP and BZMPOP mutated, acked the client
and fed neither plane — sixteen cases across all eight commands on both the
immediate and the parked-then-woken path, sixteen losses. For a queue
consumer on BLPOP this redelivered every message already consumed after a
master restart or on failover (#827, PR #828).
The blocking path is an INTERCEPT: it short-circuits the dispatch exit where
every other write meets the AOF and the replication stream. That is the same
structural gap moon#644 closed for tracking invalidation on the same path in
v0.8.7. The new record is the SYNTHESISED non-blocking sibling — a replica
applying a literal BLPOP would park its apply loop — derived from the REPLY,
since a blocking pop takes many keys and only the reply says which one served.
KNOWN DIVERGENCE RIDING THIS RELEASE
moon#825: SPOP and `XADD *` propagate their literal bytes, so a replica or AOF
replay produces a different result. Deferred to v0.8.10; both blocked fix seams
are documented on the issue.
VALIDATION
GCE Linux gate green on both release heads, all four legs each (monoio with
io_uring LIVE 644s, tokio 620s, client-compat vs real redis-server 234s, MSRV
1.94); hosted dispatch matrix green on both. The #827 fix was mutation-tested:
disabling only the sharded wiring turns all sixteen cases red again, so both
runtime sites are genuinely exercised. Over-propagation was checked against the
AOF bytes — timeouts, WRONGTYPE and misses reach neither plane; `BLMPOP … COUNT
10` that popped 3 logs `LPOP mp 3`; multi-key `BLPOP first second` served by
`second` logs `LPOP second` — and moon#539's phantom-key guard is unaffected.
Also corrects two artefacts of v0.8.8's cut, which was squashed into #785: the
README version table had no v0.8.8 row and still marked v0.8.7 current, and
CHANGELOG carried a duplicate truncated #788 bullet.
Refs #823, #827
author: Tin Dang
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
|
Warning Review limit reachedNext included review available in 20 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patch release rolling up 2 merged PRs closing 3 issues since v0.8.8.
One invariant. Propagation is gated on the reply not being an error
(
!matches!(resp, Frame::Error(_))), so any command that mutates and THENreturns an error leaves a write applied on the master, never appended to the
AOF and never sent to a replica: silent data loss across restart, permanent
replica divergence.
CORRECTNESS
redis.call/redis.pcallturned a Lua nil, boolean or table argument into aframe shape no wire client can produce, which HSET, HMSET, LPUSH, RPUSH,
LPUSHX, RPUSHX, ZREM, MSET and MSETNX each discovered from INSIDE their
mutation loop. Measured:
EVAL "return redis.pcall('LPUSH','mylist','a','b', true)" 0answered an arity error withLLEN mylist = 2resident, and 0after restart. Fixed at the boundary, as Redis 8.6.1 refuses it, AND by
hoisting validation above the mutation window in all nine commands — defence
in depth, because the raw wire reaches the same code with no Lua anywhere
(A non-string Lua argument reaches command argv: nine commands write part of a command, then refuse it — data loss across restart and replica divergence #823, PR fix(scripting): refuse a non-string Lua argument, and validate before mutating in nine commands #824).
get_or_create_streambefore parsing the ID, so all five ofbogus,0-0,1-1-1,abc-1and-5created a charged, DBSIZE-visiblestream that was never logged and vanished on restart. 3,000 rejected XADDs
cost 1.46 MB nothing credits back (A non-string Lua argument reaches command argv: nine commands write part of a command, then refuse it — data loss across restart and replica divergence #823, PR fix(scripting): refuse a non-string Lua argument, and validate before mutating in nine commands #824).
BRPOPLPUSH, BZPOPMIN, BZPOPMAX, BLMPOP and BZMPOP mutated, acked the client
and fed neither plane — sixteen cases across all eight commands on both the
immediate and the parked-then-woken path, sixteen losses. For a queue
consumer on BLPOP this redelivered every message already consumed after a
master restart or on failover (Blocking pops propagate NOTHING outside MULTI: acked BLPOP/BRPOP/BLMOVE/BZPOPMIN are undone by restart and never reach a replica #827, PR fix(server): propagate blocking pops outside MULTI (#827) #828).
The blocking path is an INTERCEPT: it short-circuits the dispatch exit where
every other write meets the AOF and the replication stream. That is the same
structural gap moon#644 closed for tracking invalidation on the same path in
v0.8.7. The new record is the SYNTHESISED non-blocking sibling — a replica
applying a literal BLPOP would park its apply loop — derived from the REPLY,
since a blocking pop takes many keys and only the reply says which one served.
KNOWN DIVERGENCE RIDING THIS RELEASE
moon#825: SPOP and
XADD *propagate their literal bytes, so a replica or AOFreplay produces a different result. Deferred to v0.8.10; both blocked fix seams
are documented on the issue.
VALIDATION
GCE Linux gate green on both release heads, all four legs each (monoio with
io_uring LIVE 644s, tokio 620s, client-compat vs real redis-server 234s, MSRV
1.94); hosted dispatch matrix green on both. The #827 fix was mutation-tested:
disabling only the sharded wiring turns all sixteen cases red again, so both
runtime sites are genuinely exercised. Over-propagation was checked against the
AOF bytes — timeouts, WRONGTYPE and misses reach neither plane;
BLMPOP … COUNT 10that popped 3 logsLPOP mp 3; multi-keyBLPOP first secondserved bysecondlogsLPOP second— and moon#539's phantom-key guard is unaffected.Also corrects two artefacts of v0.8.8's cut, which was squashed into #785: the
README version table had no v0.8.8 row and still marked v0.8.7 current, and
CHANGELOG carried a duplicate truncated #788 bullet.
Refs #823, #827