Skip to content

feat(slack-agent): tune prompt for concise, Slack-native replies - #280

Open
JeremyFunk wants to merge 2 commits into
mainfrom
feat/slack-agent-concise-replies
Open

feat(slack-agent): tune prompt for concise, Slack-native replies#280
JeremyFunk wants to merge 2 commits into
mainfrom
feat/slack-agent-concise-replies

Conversation

@JeremyFunk

@JeremyFunk JeremyFunk commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

The Slack bot writes "paragraphs on paragraphs" — headers, preamble, a recap of
its own investigation, and an unasked-for next-steps list. That reads badly in a
chat surface. This tunes the prompt's tone and length only; capabilities, tool
guidance, approval rules, and the "Linking into Maple" section are untouched.

Levers used

agent/instructions.md — "Response style"

  • An explicit, numeric length budget: default to 2–5 short sentences, or one
    top-line sentence plus 3–5 bullets
    , with long-form gated behind an explicit
    request ("write it up", "full report", "deep dive").
  • Lead with the answer; no preamble, no restating the question, no recap, no
    closing summary.
  • Many findings → top-line finding + tight bullet list + one offer to
    expand, rather than pre-emptively dumping everything.
  • Prefer a Maple deep link over pasted data; never paste raw tool output or JSON.
  • Corrected the formatting guidance to Slack mrkdwn: the old text told the
    model to use markdown tables and headers, neither of which Slack renders.
    Now *bold* for key numbers, backticks for ids/service names, - bullets.
  • Added a "Length calibration" block with bad-vs-good micro-examples — models
    copy a demonstrated shape far more reliably than they follow adjectives.

agent/skills/incident-investigation/SKILL.md — this was the main source of
the long-form output: it mandated a six-section report (Summary,
Suspected cause, Affected scope, Evidence, Suggested actions,
Confidence). Replaced with a ~6-line Slack shape — one line on what broke,
2–4 evidence bullets, one line on the first action — plus an instruction to hold
the timeline / ruled-out hypotheses / secondary evidence and offer to expand.
The section headers are now explicitly opt-in. Investigation depth, tool
selection, the evidence and no-invented-identifiers rules, and the linking
requirements are unchanged
— it investigates just as hard, it just reports
shorter.

agent/skills/dashboard-builder/SKILL.md — already mostly concise; capped
per-widget commentary at a sentence or two and stopped it restating widget
config in prose (the proposal card already shows it).

Testing

  • apps/slack-agent parity.test.ts 5/5 pass — it asserts the prompt files keep
    the maple__ tool prefix and the not-linked fallback guidance, all preserved.
  • Full apps/slack-agent suite: 165 pass, 2 fail. Both failures are pre-existing
    Cannot find module 'eve/tools' / 'eve/channels/slack' resolution errors,
    unrelated to these markdown changes.
  • Markdown-only change, so no bun typecheck needed.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Follow-up fix (98ff18c): the formatting guidance originally said "Slack mrkdwn, not markdown"
with *bold*. That's wrong for this transport — eve's Slack binding turns the reply string into
{markdown: text}markdownText → Slack's chat.postMessage markdown_text field, which
takes standard markdown (and eve never calls its gfmToSlackMrkdwn converter on the outbound path,
while the inbound path runs slackMrkdwnToGfm — the design is symmetric). Under markdown_text
*text* renders as italic, so every emphasised metric would have come out italic. Guidance and
examples now use **bold**; the <url|text> link syntax is unchanged.

The bot was writing paragraphs on paragraphs, which reads badly in Slack.
Tighten the prompt's tone/length levers without touching capabilities, tool
guidance, or the linking rules.

- instructions.md: give "Response style" an explicit length budget (2-5 short
  sentences, or a top line plus 3-5 bullets), ban preamble/recap/next-step
  menus, require Slack mrkdwn (no `#` headers, no tables - Slack renders
  neither), prefer linking into Maple over pasting data, and add bad-vs-good
  micro-examples so the model has a shape to copy.
- incident-investigation: replace the mandated six-section report (Summary /
  Suspected cause / Scope / Evidence / Actions / Confidence) with a ~6-line
  Slack shape plus an offer to expand; the long form is now opt-in when the
  engineer asks for a write-up. Investigation depth and evidence rules are
  unchanged.
- dashboard-builder: cap the per-widget commentary at a sentence or two and
  stop it restating widget config in prose.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pullfrog

pullfrog Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Your Pullfrog Router balance is exhausted.

You have a payment method on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.

Top up balance → · Enable auto-reload →

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Rerun failed job ➔View workflow run | via Pullfrog | Using Claude Opus𝕏

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

🍁 Maple PR preview

The preview is ready:

Service Preview URL
Web app-pr-280.maple.dev
API api-pr-280.maple.dev
Chat chat-pr-280.maple.dev
Electric Sync sync-pr-280.maple.dev

Deployed from 98ff18c · View workflow run

The agent's reply string is posted through eve's Slack binding, which sets
`markdownText` -> Slack's `chat.postMessage` `markdown_text` field. That field
takes standard markdown, not Slack mrkdwn, so `*text*` renders as italic. Every
emphasised metric would have come out italic instead of bold.

Corrects the formatting guidance and the examples this PR added to use
`**bold**`, and notes why inline so it isn't "corrected" back. Slack
angle-bracket link syntax is left alone (pre-existing, passes through).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pullfrog

pullfrog Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Your Pullfrog Router balance is exhausted.

You have a payment method on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.

Top up balance → · Enable auto-reload →

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Rerun failed job ➔View workflow run | via Pullfrog | Using Claude Opus𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant