Webhook payload is Slack-legacy attachments only, with no text/content/summary, yet docs say Teams and Discord accept it directly (CR-A08-002) #112

Wave 0 · theme D0-8: Packaging and delivery promises · highest severity High · 19 finding(s) · effort small

Summary

An operator follows the manual and points webhook.url at a Discord or Teams incoming webhook. Every POST is rejected with a 4xx. Only a 'Webhook HTTP 400 ... response body suppressed' WARNING appears in stderr, and nobody is paged on failures or awaiting-approval events.

Primary finding

CR-A08-002 — Webhook payload is Slack-legacy attachments only, with no text/content/summary, yet docs say Teams and Discord accept it directly

severity High · reachability default · intent contract-violation · confidence confirmed · independently confirmed · reproduced (plumbing only)

What happens. WebhookNotifier._send builds one generic body {event, run_name, status, metrics, reason, model_path, attachments:[{title,text,color}]} (webhook.py:297-306/316), with no top-level text/content/summary, no MessageCard/Adaptive Card and no Discord embed. webhooks.md, README.md:159, site/features.html:324-325 and config.py:1466 say Slack/Teams/Discord accept it directly. A receiver's 4xx is only logged at WARNING.

Where.

• forgelm/webhook.py:293-312 (payload built 293-306, extras merged 307-312)
• docs/usermanuals/en/operations/webhooks.md:86
• forgelm/config.py:1464
• README.md:159 (says only 'Slack / Teams webhooks', no Discord)
• site/features.html:325 (the claim sits on the

  at line 325, not 324)

• forgelm/webhook.py:297-316
• docs/usermanuals/en/operations/webhooks.md:20,53-55,72-73,83-86
• docs/usermanuals/en/operations/webhooks.md:53
• docs/usermanuals/en/operations/webhooks.md:72-86
• forgelm/webhook.py

Evidence.

• scr/offwh.py loopback receiver got keys ['attachments','event','metrics','model_path','reason','run_name','status']; there is no top-level text, content or summary.
• Static payload construction at webhook.py:297-316; the receiver schemas need external confirmation (see research_questions).
• Payload inspected on the loopback sink. Receiver behaviour is not testable offline.

Recommended fix. Add a top-level 'text' (Slack/Teams fallback) and 'content' (Discord), or a provider-aware formatter. Otherwise correct the docs and README/site claims to require a relay for Teams and Discord.

Other findings with the same root cause

These close with the same fix; update them in the same pull request.

• TECH-T09-001 (technology, Now) — Experiment tracking, webhooks and monitoring: Microsoft Teams webhook payload after the Office 365 Connectors retirement (forgelm/webhook.py:297-306) (incorrect-claim)
  severity Now
  Item. Microsoft Teams webhook payload after the Office 365 Connectors retirement (forgelm/webhook.py:297-306) · verdict: incorrect-claim · urgency: Now
  State in ForgeLM. forgelm/webhook.py:297-306 sends one JSON body {event, run_name, status, metrics, reason, model_path, attachments:[{title,text,color}]}. It has no type:'message', no attachments[].contentType/content (Adaptive Card) and no @​type/summary (MessageCard). forgelm/webhook.py:230 treats only a non-2xx response as a failure, so any 2xx counts as delivered. docs/usermanuals/en/operations/webhooks.md:20-2…
  State of the ecosystem on 2026-10-01. Retired. The Microsoft 365 Developer Blog post (update 4/14/2026) says the rollout began May 18, 2026 and completed May 22, 2026: 'After these dates, Office 365 Connectors will no longer function. All connector-based webhooks must be migrated to Power Automate-based workflows'. office365itpros (2026-09-18): 'On May 22, 2026, Microsoft completed the retirement of Office 365 Connectors from Teams'.…
  Recommendation. Fix the docs now: Teams needs a Workflows URL plus a relay that wraps the payload in an Adaptive Card or MessageCard. Remove the 'Teams connector token' control. Later, add an opt-in webhook.format selector (generic | teams_adaptive | discord) that builds attachments[0] = {contentType: application/vnd.microsoft.card.adaptive, content: AdaptiveCard with TextBlock + FactSet}. Document that a 2xx fr…
  Sources.

  • Retirement of Office 365 connectors within Microsoft Teams (update 4/14/2026) (2026-04-14)
  • Create an Incoming Webhook - Teams (Microsoft Learn) (2026-08-03)

• TECH-T09-002 (technology, Now) — Experiment tracking, webhooks and monitoring: Discord webhook payload (content/embeds requirement) (incorrect-claim)
  severity Now
  Item. Discord webhook payload (content/embeds requirement) · verdict: incorrect-claim · urgency: Now
  State in ForgeLM. The payload carries no content, embeds, components, file or poll. Its 'attachments' entries are {title,text,color}. docs/usermanuals/en/operations/webhooks.md:86 says 'Discord — accepts the JSON directly via the bot/webhook URL'. Lines 53-55 and 72-73 say the same, as do TR :53-55, :72-73 and :87. forgelm/config.py:1466 lists Discord URLs.
  State of the ecosystem on 2026-10-01. Discord API docs, webhook.mdx:212 (file last committed 2026-06-25): 'when sending a message, you must provide a value for at least one of content, embeds, components, file, or poll'. In Discord, 'attachments' is file metadata whose id 'must match the n in files[n]' (message.mdx:681). The documented errors are 50006 'Cannot send an empty message' and 50035 'Invalid form body' (opcodes-and-status-c…
  Recommendation. Fix the docs now: Discord users must append /slack to the webhook URL (the Slack-compatible endpoint renders the attachments) or use a relay. Verify the /slack form on a live webhook before documenting it. Optionally add a 'discord' format with top-level content and/or embeds:[{title, description, color:int}].
  Sources.

  • Discord API docs source: Webhook resource (Execute Webhook; Execute Slack-Compatible Webhook) (2026-06-25 (last commit to file))
  • Discord API docs source: Opcodes and Status Codes (50006, 50035) (retrieved 2026-10-01 (main))

• TECH-T09-011 (technology, Watch) — Experiment tracking, webhooks and monitoring: Slack legacy secondary attachments (title/text/color) in the generic payload (forgelm/webhook.py:305) (superseded-method)
  severity Watch
  Item. Slack legacy secondary attachments (title/text/color) in the generic payload (forgelm/webhook.py:305) · verdict: superseded-method · urgency: Watch
  State in ForgeLM. Slack receives attachments:[{title,text,color}] (hex colors such as #​36a64f) with no top-level text and no blocks. Custom top-level keys (event, run_name, status, metrics) are not Slack message fields. webhooks.md:78-79 (TR :80-81) says 'Slack renders the JSON's top-level fields'.
  State of the ecosystem on 2026-10-01. Slack still accepts and renders secondary attachments from incoming webhooks. Its docs label them 'a legacy part of messaging functionality for Slack apps. We recommend you stick with layout blocks', and say the legacy fields 'may be subject to reductions in visibility or functionality'. color is a current field. Without blocks, 'one of either fallback or text are required', and ForgeLM supplies…
  Recommendation. Add a top-level text (for example '<title>: ') as the notification fallback. Consider sending blocks (header, section, fields) inside the attachment to keep the color bar, behind a slack format option. Correct webhooks.md:78-79 in EN and TR.
  Sources.

  • Slack: Legacy secondary message attachments (retrieved 2026-10-01 (page undated))
  • Slack: Sending messages using incoming webhooks (retrieved 2026-10-01 (page undated))

• DOC-BFIELD-018 (docs, Medium) — forgelm/config.py: wrong detail — Env var name carrying the webhook URL (overrides url when set).
  severity Medium
  Where. forgelm/config.py:1468-1470 (WebhookConfig.url_env, F55) (Turkish mirror: docs/usermanuals/tr/operations/webhooks.md:102) · type: wrong detail
  The document says. Env var name carrying the webhook URL (overrides url when set).
  What the code does. webhook.py:80-83 returns url when it is set and reads url_env only when url is empty. Reproduced: with url='https://literal-hook.example/y' and url_env pointing at 'https://env-hook.example/x', _resolve_url() returned the literal url.
  Code references. forgelm/webhook.py:69-83
  Proposed correction. 'Env var name carrying the webhook URL; used only when url is unset.' Alternatively, change the precedence in code to match the docs.

• DOC-BLEGAL-055 (docs, Medium) — README.md: false claim — … Slack / Teams webhooks …
  severity Medium
  Where. README.md:159 · type: false claim
  The document says. … Slack / Teams webhooks …
  What the code does. Microsoft retired Office 365 Connectors in Teams, with the rollout completed on 2026-05-22, and connector webhooks no longer function. The replacement Workflows webhook requires an Adaptive Card envelope (type 'message', attachments[].contentType 'application/vnd.microsoft.card.adaptive') or a valid MessageCard. ForgeLM's payload (forgelm/webhook.py:297-306) is neither, so the claim holds for Sla…
  Proposed correction. 'Slack webhooks (Microsoft Teams Workflows webhooks are not supported yet)', or ship an Adaptive Card payload.

• DOC-BLEGAL-139 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: false claim — Slack and Teams ingest it directly ... Slack / Teams / Discord all accept it directly via incoming-webhook endpoints ... Slack renders the JS…
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:20,53-55,72-86 (Turkish mirror: docs/usermanuals/tr/operations/webhooks.md:20,53,72-87) · type: false claim
  The document says. Slack and Teams ingest it directly ... Slack / Teams / Discord all accept it directly via incoming-webhook endpoints ... Slack renders the JSON's top-level fields ... Teams renders incoming JSON natively ... Discord — accepts the JSON directly via the bot/webhook URL.
  What the code does. Slack accepts the payload, but renders only the legacy attachments[] entry. Top-level event/run_name/status/metrics are ignored, and Slack labels secondary attachments legacy. Teams Office 365 Connectors were retired on 22 May 2026. The Workflows trigger needs an Adaptive Card or MessageCard envelope, so the payload is rejected. A native Discord webhook needs one of content/embeds/components/file…
  Proposed correction. Slack: the payload's attachments entry renders (top-level fields are not shown). Teams: Office 365 connectors are retired; Workflows webhooks need an Adaptive Card, so route through a relay. Discord: use the <webhook-url>/slack endpoint or a relay; the native endpoint rejects the payload.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-140 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: false claim — [IS-T09-03] 'The single generic JSON payload shown above is what Slack, Teams, and Discord all expect on their incoming-webhook endpoints.'
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:72-73 · type: false claim
  The document says. [IS-T09-03] 'The single generic JSON payload shown above is what Slack, Teams, and Discord all expect on their incoming-webhook endpoints.'
  What the code does. Office 365 Connectors in Teams were retired (rollout completed 22 May 2026). The replacement Workflows webhook expects an Adaptive Card envelope or a MessageCard. A plain Discord webhook URL requires at least one of content, embeds, components, file or poll and rejects the payload. Only the /slack-suffixed Discord endpoint accepts Slack-style attachments.
  Proposed correction. The generic JSON payload is accepted by Slack incoming webhooks (only its attachments[] entry renders) and by Discord's /slack-suffixed webhook endpoint. Teams Workflows webhooks and plain Discord webhook URLs do not accept it; use a relay that converts it to an Adaptive Card or Discord content/embeds.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-141 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: false claim — [IS-T09-04] 'Slack renders the JSON's top-level fields.'
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:78-79 · type: false claim
  The document says. [IS-T09-04] 'Slack renders the JSON's top-level fields.'
  What the code does. Slack renders text/blocks/attachments. ForgeLM's custom top-level keys (event, run_name, status, metrics) are ignored; only the legacy secondary attachment renders. Mobile notifications use message.text, which ForgeLM does not send.
  Proposed correction. Slack renders only the payload's legacy attachments[] entry (title/text/color); the top-level fields are not displayed.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-142 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: false claim — [IS-T09-05] 'Teams renders incoming JSON natively but the visual is plain'
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:83-85 · type: false claim
  The document says. [IS-T09-05] 'Teams renders incoming JSON natively but the visual is plain'
  What the code does. The Office 365 Connector path is retired (May 2026). Workflows ('When a Teams webhook request is received') renders Adaptive Card or MessageCard payloads; no Microsoft source says it renders arbitrary JSON.
  Proposed correction. Microsoft Teams — Office 365 Connectors were retired in May 2026; Teams Workflows webhooks require an Adaptive Card (or MessageCard) envelope, which ForgeLM does not emit. Run a relay that converts the payload.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-143 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: false claim — [IS-T09-06] 'Discord — accepts the JSON directly via the bot/webhook URL.'
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:86 · type: false claim
  The document says. [IS-T09-06] 'Discord — accepts the JSON directly via the bot/webhook URL.'
  What the code does. Discord webhook.mdx:212: 'you must provide a value for at least one of content, embeds, components, file, or poll'. Otherwise Discord returns 50006/50035. POST /webhooks/{id}/{token}/slack accepts Slack format.
  Proposed correction. Discord — append /slack to the webhook URL (Discord's Slack-compatible endpoint); the plain webhook URL rejects the payload.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BLEGAL-149 (docs, Medium) — docs/usermanuals/tr/operations/webhooks.md: false claim — Teams O365 Connectors are retired, and Workflows needs an Adaptive Card or MessageCard. A plain Discord URL needs content/embeds; only /slack…
  severity Medium
  Where. docs/usermanuals/tr/operations/webhooks.md:20-21, 53-55, 72-73, 79-81, 84-86, 87 (Turkish mirror: docs/usermanuals/en/operations/webhooks.md:72-86) · type: false claim
  The document says. [IS-T09-07] "Slack ve Teams bunu incoming-webhook uçlarından doğrudan kabul eder"; "Slack / Teams / Discord hepsi bunu ... doğrudan kabul eder"; "Slack JSON'un top-level alanlarını render eder"; "Teams gelen JSON'u natively render eder"; "Discord — JSON'u doğrudan bot/webhook URL üzerinden kabul ed…
  What the code does. Teams O365 Connectors are retired, and Workflows needs an Adaptive Card or MessageCard. A plain Discord URL needs content/embeds; only /slack accepts the payload. Slack renders only the attachment.
  Proposed correction. Generic JSON payload'u Slack incoming webhook'ları kabul eder (yalnızca attachments[] girdisi render edilir) ve Discord'un /slack sonekli endpoint'i kabul eder. Teams (Office 365 Connector'ları Mayıs 2026'da kaldırıldı; Workflows webhook'u Adaptive Card veya MessageCard bekler) ve düz Discord webhook URL'i (content/embeds zorunlu) bu payload'u…

• DOC-BLEGAL-156 (docs, Medium) — site/features.html: false claim — "Slack, Teams, Discord, or any HTTP webhook receiver alerts on training events."
  severity Medium
  Where. site/features.html:325 (EN key features.ent.webhook.body, site/js/translations.js:1002) (Turkish mirror: site/js/translations.js:1107 (TR features.ent.webhook.body)) · type: false claim
  The document says. "Slack, Teams, Discord, or any HTTP webhook receiver alerts on training events."
  What the code does. ForgeLM posts a generic JSON body with Slack-style attachments and no top-level text/content (forgelm/webhook.py:296-306). Discord's Execute Webhook requires at least one of content, embeds, components, file or poll and answers 50006/50035 (HTTP 400). Only the '/slack' suffixed endpoint accepts Slack format. Microsoft retired Office 365 Connector webhooks in Teams between 18 and 22 May 2026, and…
  Proposed correction. 'Slack (and Discord via its /slack-compatible URL) or any HTTP receiver that accepts ForgeLM's JSON payload; Microsoft Teams needs a Power Automate Workflow that maps the payload to an Adaptive Card.'

• DOC-BLEGAL-157 (docs, Medium) — site/features.html: misleading — [IS-T09-11] 'Slack, Teams, Discord, or any HTTP webhook receiver alerts on training events.' (also site/js/translations.js:1002 and locale copies)
  severity Medium
  Where. site/features.html:325 · type: misleading
  The document says. [IS-T09-11] 'Slack, Teams, Discord, or any HTTP webhook receiver alerts on training events.' (also site/js/translations.js:1002 and locale copies)
  What the code does. Teams Workflows and plain Discord webhook URLs do not accept or render ForgeLM's generic payload.
  Proposed correction. Slack (and Discord's Slack-compatible endpoint) or any HTTP receiver — alerts on training events; Teams needs a relay to Adaptive Cards.

• DOC-D1B06-013 (docs, Medium) — docs/qms/access_control.md: stale — The webhook wire-format event vocabulary has 5 events: training.start, training.success, training.failure, training.reverted, approval.required
  severity Medium
  Where. docs/qms/access_control.md:237-243 (Turkish mirror: docs/qms/access_control-tr.md:239-244) · type: stale
  The document says. The webhook wire-format event vocabulary has 5 events: training.start, training.success, training.failure, training.reverted, approval.required
  What the code does. WebhookNotifier emits 8 events. The other 3 are pipeline.started, pipeline.completed and pipeline.stage_reverted, from notify_pipeline_started/_completed/reverted.
  Code references. forgelm/webhook.py:432,444,475,504,559,593,634,661
  Proposed correction. List all 8 events and the 8 notify* methods.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B02-043 (docs, Medium) — docs/usermanuals/en/operations/webhooks.md: misleading — The single generic JSON payload is accepted directly by Slack, Teams and Discord ('Discord — accepts the JSON directly via the bot/webhook URL…
  severity Medium
  Where. docs/usermanuals/en/operations/webhooks.md:20,53,72-86 (Turkish mirror: docs/usermanuals/tr/operations/webhooks.md) · type: misleading
  The document says. The single generic JSON payload is accepted directly by Slack, Teams and Discord ('Discord — accepts the JSON directly via the bot/webhook URL'); Teams 'renders incoming JSON natively'
  What the code does. The payload (webhook.py:293-310) has event, run_name, status, metrics, reason, model_path and a Slack-style attachments list. It has no Discord content or embeds key, and there is no Discord code path (the name appears only in comments and redaction notes, _http.py:382,417 and webhook.py:91,464). Discord's webhook API is externally documented to reject a body with neither field, unless the UR…
  Code references. forgelm/webhook.py:293-310, forgelm/_http.py:382,417
  Proposed correction. State that only Slack-style incoming webhooks accept the payload as-is. For Discord (use a '/slack' endpoint, if confirmed) and for Teams, use a relay or Workflows adapter. Remove the 'accepts directly' wording until verified.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-BFIELD-034 (docs, Low) — forgelm/config.py: wrong detail — POST a notify_start / notify_success / notify_failure event; failure: when training fails (any non-zero exit)
  severity Low
  Where. forgelm/config.py:1471-1477 (WebhookConfig.notify_on_start/notify_on_success/notify_on_failure, F56 F57 F58) · type: wrong detail
  The document says. POST a notify_start / notify_success / notify_failure event; failure: when training fails (any non-zero exit)
  What the code does. The wire event names are training.start, training.success and training.failure (webhook.py:429-510); the quoted names are method names. The flags also gate pipeline.started, approval.required, pipeline.completed and training.reverted (webhook.py:469-472, 555, 590, 621-627, 657). notify_failure fires only on an exception inside ForgeTrainer.train (trainer.py:1706) and on reject (_approve.py:736).…
  Code references. forgelm/webhook.py:429-510, forgelm/trainer.py:1686-1710
  Proposed correction. Name the real event strings and list the extra events each flag controls; drop 'any non-zero exit'.

• DOC-BNUM-006 (docs, Low) — docs/qms/access_control.md: false claim — The webhook wire-format event vocabulary (5 events) ... dispatch to those five wire events
  severity Low
  Where. docs/qms/access_control.md:237,243; docs/qms/access_control-tr.md:239,244 · type: false claim
  The document says. The webhook wire-format event vocabulary (5 events) ... dispatch to those five wire events
  What the code does. There are exactly eight webhook events. docs/reference/webhook_schema.md:31 and audit_event_catalog.md:152 say eight, the guard counts webhook_events=8, and forgelm/webhook.py also has notify_pipeline_* methods.
  Code references. docs/reference/webhook_schema.md:31, docs/reference/audit_event_catalog.md:152, forgelm/webhook.py
  Proposed correction. Change to eight and list all eight events, or link to webhook_schema.md.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

• DOC-D3B02-066 (docs, Low) — docs/usermanuals/en/operations/webhooks.md: wrong detail — The payload is {event, run_name, status, reason}, with the example reason "S5 hate-speech +0.18 over baseline". "Payload keys vary by even…
  severity Low
  Where. docs/usermanuals/en/operations/webhooks.md:57-66 (Turkish mirror: docs/usermanuals/tr/operations/webhooks.md:57-66) · type: wrong detail
  The document says. The payload is {event, run_name, status, reason}, with the example reason "S5 hate-speech +0.18 over baseline". "Payload keys vary by event."
  What the code does. Every payload has event, run_name, status, metrics, reason, model_path and an attachments array (title, text, color). Pipeline events add allowlisted extras. The safety gate is an absolute ceiling, not a delta over baseline.
  Code references. forgelm/webhook.py:299-318, forgelm/webhook.py:38-45
  Proposed correction. Show the full base shape including metrics, model_path and attachments; use a reason string the safety gate actually emits.
  Turkish mirror. the Turkish page has the same error; fix both in the same PR

Acceptance criteria

• A regression test reproduces the failure scenario and asserts the corrected behaviour.
• Documentation listed above is corrected in English and in its Turkish mirror in the same PR.
• Every finding in this issue is resolved or explicitly closed with a reason.
• The release note names the contract this fix restores.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}