feat(lifecycle): alert when index creation fails#2753
Conversation
Hello delthas,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Request integration branchesWaiting for integration branch creation to be requested by the user. To request integration branches, please comment on this pull request with the following command: Alternatively, the |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
@@ Coverage Diff @@
## development/9.4 #2753 +/- ##
================================================
Coverage 74.84% 74.85%
================================================
Files 199 199
Lines 13657 13657
================================================
+ Hits 10222 10223 +1
+ Misses 3425 3424 -1
Partials 10 10
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
b65c73e to
29521bd
Compare
Add LifecycleIndexCreationFailed Prometheus rule wired to the `putBucketIndexesFailed` status tag emitted by the conductor (BB-783). The producer falls back to v1 (full-scan) processing on failure; the alert surfaces the failure so operators can investigate the underlying MongoDB error (disk-fill, IndexBuildMinAvailableDiskSpaceMB, etc.). Issue: BB-784
29521bd to
5514c80
Compare
|
LGTM — alert rule follows the existing |
Summary
LifecycleIndexCreationFailedPrometheus alert (monitoring/lifecycle/alerts.yaml) wired to thes3_lifecycle_legacy_tasks_total{status="putBucketIndexesFailed"}counter introduced by BB-783 (feat(lifecycle): emit putBucketIndexesFailed metric on index creation errors #2751).warning— mirrors theCircuitBreakerErrorshape: the conductor falls back to v1 full-scan, so this is recoverable but always warrants investigation (typically a disk-fill /indexBuildMinAvailableDiskSpaceMBrefusal).alerts.test.yaml.Part of the Empty bucket feature epic. A symmetric MongoDB-side createIndexes-failed alert lives in the Zenko chart under ZENKO-5285.
Issue: BB-784