Skip to content

Suppress non-actionable engineering-loop Discord summaries#39

Draft
Svaag wants to merge 1 commit into
mainfrom
fix/discord-alert-routing
Draft

Suppress non-actionable engineering-loop Discord summaries#39
Svaag wants to merge 1 commit into
mainfrom
fix/discord-alert-routing

Conversation

@Svaag

@Svaag Svaag commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • stop posting idle, lock, budget, triage, and failure run summaries directly to Discord
  • leave failure delivery to the persistent Icinga-to-NOC-Agent AI case path, avoiding duplicate messages
  • keep the passive over-budget heartbeat healthy because reaching the configured daily budget is expected steady state
  • retain a direct message only when the loop publishes a PR

Validation

  • uv run pytest -q — 265 passed
  • uv run ruff check src tests
  • uv run --group dev mypy --strict src

Rollout

Keep this draft until the engineering-loop Discord secret is moved from the legacy NOC webhook to the AI route.

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🏅 Score: 92
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Autonomy-boundary erosion

Changing over_budget exit status from 1 (critical) to 0 (OK) in ICINGA_EXIT_STATUS weakens the passive monitoring signal. If the engineering-loop reaches its daily budget, Icinga will now report OK instead of a warning, removing the visibility that the loop is at capacity. This is a tier/capability-envelope weakening: the operator loses awareness that the loop is consuming its full budget, which is a prerequisite for approving higher-tier auto-approval paths that depend on success-history evidence. The change should retain a non-zero exit for over_budget to preserve the monitoring invariant.

"over_budget": 0,

@github-actions

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Review budget exit status change

Changing over_budget exit status from 1 to 0 may cause Icinga to treat budget
exhaustion as a success, potentially masking operational issues. Verify this aligns
with the monitoring strategy for budget-related outcomes.

src/hyrule_engineering_loop/daemon.py [331-339]

+ICINGA_EXIT_STATUS: dict[str, int] = {
+    "published": 0,
+    "idle": 0,
+    "needs_triage": 1,
+    "over_budget": 0,
+    "locked": 0,
+    "refused_ci": 2,
+    "error": 2,
+}
 
-
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies a potentially impactful change in the PR (changing over_budget exit status from 1 to 0). This could mask operational issues if not aligned with monitoring strategy, making it a relevant and important observation.

Medium
General
Add debug logging for skipped notifications

The early return for non-published outcomes will skip the webhook environment
variable check, which may be intentional but could mask configuration issues.
Consider logging a debug message when skipping to aid troubleshooting.

src/hyrule_engineering_loop/daemon.py [314-315]

 if report.outcome != "published":
+    logger.debug("Skipping Discord notification for non-published outcome: %s", report.outcome)
     return False
Suggestion importance[1-10]: 4

__

Why: The suggestion is valid and could aid debugging, but it's a minor improvement. The early return is intentional per the PR's logic, and adding logging is not critical for correctness.

Low

@Svaag

Svaag commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

The over_budget OK status is intentional and matches the selected alerting policy:

  • the daily run/cost cap is a configured safety envelope and is expected to be reached during normal operation;
  • Icinga freshness still proves the hourly timer ran, and the passive-check output plus daemon journal retain the over_budget outcome and detail;
  • needs_triage remains warning, while refused_ci and error remain critical.

I also left out an extra debug line for each skipped Discord notification because the run outcome is already journaled and an hourly duplicate log would add another noise stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant