Repository navigation
Conversation
Contributor
|
✅ No conflicts with other open PRs targeting |
Contributor
|
I see you updated files related to
|
pkcll
force-pushed
the
incident-2673-pmai-chipingress-retry-config
branch
from
October 6, 2026 03:33
8823d06 to
da18bd8
Compare
|
pkcll
force-pushed
the
incident-2673-pmai-chipingress-retry-config
branch
from
October 6, 2026 03:54
da18bd8 to
3c21acb
Compare
Contributor
CORA - Pending Reviewers
Legend: ✅ Approved | ❌ Changes Requested | 💬 Commented | 🚫 Dismissed | ⏳ Pending | ❓ Unknown For more details, see the full review summary. |
Enables node operators to turn on gRPC-level retries for the node's chip ingress client (and LOOP plugins' own clients) via [Telemetry] TOML, wiring the pass-through added by chainlink-common#2441 on top of the retry support from chainlink-common#2373. - [Telemetry] ChipIngressRetry* keys: Enabled (off by default) plus optional MaxAttempts / InitialBackoff / MaxBackoff / BackoffMultiplier / RetryableStatusCodes overrides; unset knobs fall back to chipingress.DefaultRetryPolicy values. - core/cmd/shell.go newBeholderClient resolves them into beholder.Config.ChipIngressRetryPolicy for the node's shared chip ingress client (custom events + telemetry/heartbeat path). - plugins/loop_registry.go forwards the resolved values to LOOP plugin subprocesses over the existing CL_CHIP_INGRESS_* env channel. - Validation: MaxAttempts >= 2 (gRPC silently installs no retry policy below 2), positive backoffs/multiplier; unknown status-code names fail node startup via chipingress.ParseStatusCodes. - docs: core.toml keys, regenerated CONFIG.md, new docs/CHIP_INGRESS_RETRY.md operator page. Follows the wiring pattern of #23491 (OTel metric export batching). Retries remain opt-in: replayed publishes can duplicate events unless producers set an idempotency key; the durable emitter path is intentionally excluded (its retransmit loop already guarantees delivery). Jira: INFOPLAT-19422
pkcll
force-pushed
the
incident-2673-pmai-chipingress-retry-config
branch
from
October 6, 2026 03:59
3c21acb to
02f4dfe
Compare
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Summary
Wires the chip ingress gRPC retry policy (chainlink-common #2373, pass-through via chainlink-common #2441) into the node's
[Telemetry]TOML config, following the pattern of #23491 (OTel metric export batching).[Telemetry] ChipIngressRetry*keys:ChipIngressRetryEnabled(off by default) plus optionalMaxAttempts,InitialBackoff,MaxBackoff,BackoffMultiplier,RetryableStatusCodesoverrides. Unset knobs fall back tochipingress.DefaultRetryPolicyvalues (3 attempts, 100ms–1s backoff ×2,Unavailable+ResourceExhausted).core/cmd/shell.gonewBeholderClientresolves them intobeholder.Config.ChipIngressRetryPolicyfor the node's shared chip ingress client — covers both the custom-event emitter and the telemetry/heartbeat path (telemetry/manager.goreusesbeholder.GetClient().Chip).plugins/loop_registry.goforwards the resolved values to LOOP plugin subprocesses over the existingCL_CHIP_INGRESS_*env channel (CL_CHIP_INGRESS_RETRY_*).MaxAttempts≥ 2 (gRPC silently installs no retry policy below 2), positive backoffs/multiplier; unknown status-code names fail startup viachipingress.ParseStatusCodes.core.tomlkeys, regeneratedCONFIG.md, newdocs/CHIP_INGRESS_RETRY.mdoperator page (duplicate-Publish caveat, throttling pairing, env overrides, LOOP passthrough).chainlink-common+ nestedpkg/chipingresspins to the Bump github.com/gin-contrib/cors from 1.3.0 to 1.3.1 #2441 branch head (91c6604e5); re-pins to its merge commit before ready-for-review. Note: subsumes the common bump in deps: bump chainlink-common to latest main #23910 once Bump github.com/gin-contrib/cors from 1.3.0 to 1.3.1 #2441 merges.Retries stay opt-in: replayed publishes can duplicate events unless producers set an idempotency key; the durable emitter path is intentionally excluded (its retransmit loop already guarantees delivery). Incident context: INCIDENT-2673 (Beholder heartbeats not ingested) — without retries, every transient
UNAVAILABLE/RESOURCE_EXHAUSTEDchip publish is a permanent drop.Validation
go test ./core/services/chainlink/config suites (marshal + effective-config goldens + new getter tests, incl. defaults-mirror-DefaultRetryPolicycheck) — greengo test ./plugins/(LOOP env assertions extended to the retry fields) — greengo test ./core/config/docs/(TOML↔CONFIG.md sync) — greengo buildof all touched packages,gofmtclean on touched filesJira: INFOPLAT-19422
Known CI note (stacked on chainlink-common#2441)
Validate go.mod dependenciesis expected to fail while #2441 is unmerged: the go.mod pins point at the #2441 branch head (84edf0848) so this PR compiles against the retry pass-through, and that commit is by definition not on chainlink-commonmainyet. As soon as #2441 merges, the pins are re-pointed at its merge commit and the check goes green. All other checks are unaffected.