Repository navigation
workflows/syncer: adaptive, configurable WorkflowRegistry page size - #24231
Open
prashantkumar1982 wants to merge 5 commits into
Open
prashantkumar1982 wants to merge 5 commits into
prashantkumar1982 wants to merge 5 commits into
Conversation
The syncer paged WorkflowRegistry reads with a hardcoded 1000 items. On staging Zone-A getWorkflowListByDON at that size needs ~17M gas, over the RPC node's eth_call cap (16,777,216), so every reconciliation tick failed. - Page size now comes from the cresettings limit WorkflowRegistryMaxResultsPerQuery (default 300), re-read each call. - On any gas error the page size is halved (floor 10) and the same range retried; the reduction is kept across ticks and dropped if the configured value changes. - Applies to getWorkflowListByDON and getActiveAllowlistedRequestsReverse. Requires the chainlink-common change adding the limit. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
|
I see you updated files related to
|
Contributor
|
✅ No conflicts with other open PRs targeting |
When a registry read still hits the RPC gas cap at the minimum page size, adaptive paging has run out of options. Log at critical level so it can be alerted on, for both getWorkflowListByDON and getActiveAllowlistedRequestsReverse. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Picks up the WorkflowRegistryMaxResultsPerQuery cresettings limit (chainlink-common#2560) used by the syncer's adaptive page size. Ran make gomodtidy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Resolved go.mod/go.sum conflicts by taking develop and re-applying the chainlink-common bump to d1731bf4579a, then ran make gomodtidy. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fixes prealloc lint findings. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
prashantkumar1982
marked this pull request as ready for review
November 18, 2026 19:20
product-security-plaid-production
Bot
requested review from
DylanTinianov,
george-dorin,
ilija42 and
skudasov
November 18, 2026 19:20
george-dorin
approved these changes
Nov 18, 2026
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Paginated reads can combine different block states and activate paused workflows, while invalid defaults and misclassified transport errors can disrupt or slow synchronization.
3 open issues
What changed in this PR
Makes WorkflowRegistry pagination configurable and adaptive so workflow and allowlist synchronization can stay within RPC gas limits.
Changes:
- Replaces the hardcoded 1,000-item page size with a runtime setting defaulting to 300.
- Halves pages after gas errors, down to 10, and remembers reductions until configuration changes.
- Adds pagination tests and updates
chainlink-commonacross modules.
| File | Description |
|---|---|
system-tests/tests/go.sum |
Updates dependency checksums. |
system-tests/tests/go.mod |
Updates chainlink-common. |
system-tests/lib/go.sum |
Updates dependency checksums. |
system-tests/lib/go.mod |
Updates chainlink-common. |
integration-tests/load/go.sum |
Updates dependency checksums. |
integration-tests/load/go.mod |
Updates chainlink-common. |
integration-tests/go.sum |
Updates dependency checksums. |
integration-tests/go.mod |
Updates chainlink-common. |
go.sum |
Updates root dependency checksums. |
go.mod |
Adds the dependency revision providing the new setting. |
deployment/go.sum |
Updates dependency checksums. |
deployment/go.mod |
Updates chainlink-common. |
core/services/workflows/syncer/v2/workflow_syncer_v2_test.go |
Adjusts pagination setup to the configured default. |
core/services/workflows/syncer/v2/workflow_registry.go |
Wires settings and adaptive allowlist pagination. |
core/services/workflows/syncer/v2/page_size.go |
Adds shared page-size configuration and reduction logic. |
core/services/workflows/syncer/v2/contract_workflow_source.go |
Applies adaptive pagination to workflow reads. |
core/services/workflows/syncer/v2/contract_workflow_source_test.go |
Tests workflow pagination, configuration, and error handling. |
core/services/workflows/syncer/v2/allowlist_page_size_test.go |
Tests allowlist page reduction and minimum-size failures. |
core/scripts/go.sum |
Updates dependency checksums. |
core/scripts/go.mod |
Updates chainlink-common. |
Review effort: Balanced
Comment on lines
+93
to
+94
| pageSize := c.pageSizer.current(ctx, c.lggr) | ||
| params.Limit = big.NewInt(pageSize) |
| lggr.Warnw("Failed to read WorkflowRegistryMaxResultsPerQuery, using default", "err", err) | ||
| } | ||
| if v < 1 { | ||
| v = setting.DefaultValue |
| next := max(failed/2, minRegistryPageSize) | ||
| p.mu.Lock() | ||
| defer p.mu.Unlock() | ||
| p.reduced = next |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Requires
WorkflowRegistryMaxResultsPerQuerylimit). This PR does not compile until that merges andgo.modis bumped to it, hence draft.Summary
WorkflowRegistryreads now comes from the cresettings limitWorkflowRegistryMaxResultsPerQuery(default 300, was a hardcoded 1000), re-read on every call.getWorkflowListByDON(ContractWorkflowSource) andgetActiveAllowlistedRequestsReverse(allowlist sync). The latter returns fixed-size structs so it has more headroom, but shared the same constant.Why
On CRE mainline staging Zone-A,
getWorkflowListByDON(zone-a, 0, 1000)needs ~17M gas, above the RPC node'seth_callcap of 16,777,216. Every reconciliation tick failed and the whole contract source was skipped.