Skip to content

fix(redis): ask INFO, not CLUSTER INFO, whether this is a cluster - #2352

Merged
giokur merged 2 commits into
mainfrom
test/redis-probe-uses-info
Sep 23, 2026
Merged

giokur merged 2 commits into
mainfrom
test/redis-probe-uses-info

Conversation

@giokur

@giokur giokur commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

cluster_enabled lives in INFO's Cluster section. CLUSTER INFO reports the state of a
cluster — cluster_state, cluster_slots_assigned, cluster_known_nodes — and never carries
cluster_enabled. So the probe added in #2348

jedis.clusterInfo().contains("cluster_enabled:1")   // false against every server alive

always answered "not a cluster".

Why it got through

The wrong answer is the right answer on a single node. dev and qa are single-node Memorystore, so the
probe looked correct there and the password-reset lookup kept working.

Production is a six-node cluster. There, clusterMode() returned false, getResetToken took the
standalone branch, SCAN ran against one seed and the first MGET came back:

JedisMovedDataException: MOVED 4478 redis-cluster-4.internal.openframe.ai:6379
    at redis.clients.jedis.UnifiedJedis.mget(UnifiedJedis.java:912)
    at com.openframe.test.data.redis.Redis.findToken(Redis.java:100)
    at com.openframe.test.data.redis.Redis.getResetToken(Redis.java:82)   ← standalone branch

46 times across the rotator's 90-second poll, which surfaces as No password-reset token for test@flamingo.cx appeared within 90s (46 scans). The production tenant report failed its 16:00 run on
2026-09-23 — the first after 6.36.10 rolled out — and every later run would have failed the same way.
Production keeps no fallback password for that account by design, so a failed rotation stops the report
entirely rather than degrading it.

The prod log line that names the bug outright:

Redis at redis-cluster-0.redis-cluster-headless.datasources.svc.cluster.local:6379
  reports cluster mode disabled

against a server that answers MOVED. That is the probe calling a live cluster a single node.

The fix

jedis.info("cluster"), whose Cluster section is where cluster_enabled actually is.

Everything else is unchanged: a probe that cannot connect still assumes a cluster for that one lookup
and remembers nothing, and RedisConfig.getConfiguredCluster() still wins.

Compiles.

Ships with saas-shared, which pins test.redis.cluster: true in configs/prod so production is
restored at its next config refresh rather than waiting on this release, the pin bump and a rollout.
That pin can come out once this is deployed.


🤖 Generated with Claude Code

cluster_enabled lives in INFO's Cluster section. CLUSTER INFO reports the state
of a cluster — cluster_state, cluster_slots_*, cluster_known_nodes — and never
carries cluster_enabled, so `clusterInfo().contains("cluster_enabled:1")` is
false against every server there is.

The failure is one-sided and silent. On dev and qa, single-node Memorystore, the
wrong answer is the right one and everything passed. On production's six-node
cluster the probe reported "cluster mode disabled", getResetToken took the
standalone branch, and the first MGET came back

    JedisMovedDataException: MOVED 4478 redis-cluster-4.internal.openframe.ai:6379

46 times over the 90s poll, which the rotator reports as a missing token. That
took the production tenant report down on the 16:00 run of 2026-09-23 — the
first run after 6.36.10 rolled out — and it would have taken every later run
with it.

Behaviour where the probe cannot connect is unchanged: assume a cluster for that
one lookup, remember nothing. RedisConfig.getConfiguredCluster() still wins, and
configs/prod pins cluster: true so production does not wait on this release.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🦩 Flamingo Code Review

No findings on the current head.

Mode: advisory


Need another pass? Commits pushed after this review are not reviewed automatically.

  • Review the new commits — the commits added since this review
  • Review the whole diff again — ignoring what was already reviewed

Prefer typing? Comment @flamingo-review, or @flamingo-review full. To review every push on this pull request, add the flamingo-review-always label.

React 👍/👎 on inline comments to teach the reviewer.

Started 2026-09-23 16:41 UTC · updated 2026-09-23 16:41 UTC · workflow run

@giokur
giokur enabled auto-merge (squash) September 23, 2026 16:43
@giokur
giokur merged commit 7265bdb into main Sep 23, 2026
12 of 13 checks passed
@giokur
giokur deleted the test/redis-probe-uses-info branch September 23, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants