Skip to content

multinode: random pick within tier for PriorityLevel selection - #129

Merged
ilija42 merged 3 commits into
mainfrom
feat/priority-level-random-pick
Nov 18, 2026
Merged

ilija42 merged 3 commits into
mainfrom
feat/priority-level-random-pick

Conversation

@Krish-vemula

Copy link
Copy Markdown
Contributor

Problem

Every pod builds its own PriorityLevel selector, and each selector's round-robin counter starts at 0. So every pod
picks the same RPC first, and after a failover they all move to the same next RPC together. When that RPC fails, all
pods are affected at once.

Change

PriorityLevel now picks a random alive node within the highest-priority tier instead of using round-robin.

  • Tiers are unchanged: a lower tier is used only when no node in a higher tier is alive.
  • Stickiness is unchanged: a pod keeps its selected RPC until it is no longer Alive, or until a lease tick when
    LeaseDuration > 0.
  • The per-tier round-robin counters and the now unused nrOfPriorityTiers are removed.

The pick uses math/rand/v2, the same as the RandomRPC selector. The randomness only spreads load, so
cryptographic randomness isn't needed (#nosec G404).

RandomRPC already exists, but it picks a new RPC on every request. This change keeps PriorityLevel sticky per pod and
only randomizes which RPC each pod starts on, so pods spread across RPCs without each pod's reads hopping between them.

Tests

  • Same-tier cases now check that every alive node in the top tier is picked over many draws and that no other node
    ever is. A new case covers a top tier with only some nodes alive.
  • TestPriorityLevelNodeSelector_FirstPickSpreadsAcrossSelectors builds fresh selectors, as pods would, and checks
    that their first picks spread across the tier. It fails on the old round-robin code.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

📊 API Diff Results

No changes detected for module github.com/smartcontractkit/chainlink-framework/multinode

View full report

@Krish-vemula
Krish-vemula marked this pull request as ready for review November 17, 2026 00:08
@Krish-vemula
Krish-vemula requested a review from a team as a code owner November 17, 2026 00:08
@ilija42
ilija42 merged commit 1195bbc into main Nov 18, 2026
25 checks passed
@ilija42
ilija42 deleted the feat/priority-level-random-pick branch November 18, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants