Repository navigation
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
dcramer
added a commit
that referenced
this pull request
Oct 9, 2026
…result (#2052) Scheduled automation and Event automation runs used the chat Turn contract. The runtime delivered the final assistant text, the model had to emit `[[NO_REPLY]]` to stay silent, and the interactive rules told an unattended run to ask questions. This is the root cause of most of #2014, #1741, and part of #554. Automation runs now have their own mode, and each run ends with one declared result. Only a declared message is posted. **What changes** - **One switch.** `buildDispatchRoutingContext` sets `dispatch.declaresResult` for an automation Source. The prompt, the tools, the result check, and Slack Delivery read that field. Watches and other plugin dispatches keep the chat Turn contract. - **Automation prompt.** `<automation-run>` rules replace the task, conversation, and Slack action rules. The stored instruction is the job. Nobody can answer a question during the run. The run checks every condition in the instruction before an action with side effects. - **Declared result.** `finishAutomationRun` is registered only for these runs. It must be the only tool call in its message, and the run stops after it. - `send_message`: the runtime posts the declared message to the stored outcomes. The tool does not offer this result when the automation has no message outcome. - `no_action`: the dispatch completes and nothing is posted. - `misconfigured`: the dispatch is recorded as blocked with the declared reason. - A message that is only `[[NO_REPLY]]` is saved as `no_action`, so instructions written before this change stay silent. - **Who receives the message.** The run context lists each stored outcome and says if it is the creator's direct message or a channel. - **Who is mentioned.** The task input line for the creator now says that "me" and "my" mean the creator, to mention them in the message, and not to mention them otherwise. - **A run without a result.** The model gets one reminder. A second stop without a result fails the dispatch. - **A result is final.** A resumed slice returns a saved result and does not call the model again. **Behavior changes to accept** - A failed run posts nothing to its outcomes. Before, the channel got a failure reply. The failure shows on the dispatch, in the execution history, and as the last run status on the dashboard. - The model can end a run as `misconfigured`. For a Scheduled automation, the heartbeat already blocks the automation when its dispatch is blocked. For an Event automation, only that dispatch is blocked, and the automation runs again on the next event. - A person who replies in the thread of a posted message continues the run's Conversation with a normal chat Turn. That Turn sees the history of the run. **Not in this PR** - A `blocked` status for Event automations (#2095), and a direct message that tells the creator about a block (#2096). These were in the first version of this PR. They do not depend on this PR. - Tools such as `sendFiles` still post to the automation's Destination, not to its outcomes. - Repeated run failures do not notify the creator. **Review focus** - `automation-result.ts` owns the result contract. `finishedRunReply` decides what a finished run posts, for first runs in `turn.ts` and resumed runs in `resume.ts`. - The reminder loop and the early return for a saved result are in `agent/index.ts`. Unit tests cover their rules. No end-to-end test covers the reminder, because a real model cannot be made to skip the tool in a reliable way. - The agent test fixture now takes a person's reply in the thread of an automation's first post. One eval uses it to check the follow-up reply. The Slack mock now keeps a top-level post as the root of its thread. - The creator line changes the same text as #2033. The new prompt and tool change the model requests of the automation evals, so CI records them again. The chat prompt and the chat tools do not change. Refs #2051 <!-- junior-request-attribution:start --> via **David Cramer**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0B595QDZLL%3A1791321334.446739 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AC0B595QDZLL%3A1791321334.446739) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0B595QDZLL%3A1791321334.446739/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: David Cramer <david@sentry.io> Co-authored-by: David Cramer <dcramer@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
dcramer
force-pushed
the
feat/event-automation-blocked
branch
from
October 9, 2026 18:32
82c6bdb to
4e362a6
Compare
dcramer
force-pushed
the
feat/automation-blocked-notice
branch
from
October 9, 2026 18:38
412f641 to
ddac3d4
Compare
A blocked Scheduled automation or Event automation stopped in silence. The reason showed only on the dashboard and in the automation tools. The creator now gets one direct message with the reason and a resume link. The notice goes only after the automation is stored as blocked: at dispatch block for an Event automation, and at heartbeat reconcile for a Scheduled automation. A redelivered block sends nothing, and a schedule that the creator changed during the run stays active and sends nothing. The notice is best-effort and logs automation.blocked_notice.failed. A missing-authorization reason now names the fix. A run with system credentials cannot use a connected account, so its reason says to switch to creator credentials. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The heartbeat also blocks a Scheduled automation when it cannot build the dispatch metadata or create the dispatch. That path sent no notice. It now sends the same notice as a blocked dispatch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dcramer
marked this pull request as ready for review
October 10, 2026 01:58
dcramer
force-pushed
the
feat/automation-blocked-notice
branch
from
October 10, 2026 02:02
ddac3d4 to
48d150e
Compare
github-actions
Bot
added
risk: medium
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 48d150e. Configure here.
A run that blocks while its automation is paused only stores the reason, and the automation stays paused. The creator still got the notice that says it is blocked, while the dashboard showed paused. The notice now goes only when the automation becomes blocked. A paused automation returns to blocked on resume, and its creator sees the reason then. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For a job that needs a provider Junior does not have, an Event automation run ended as no_action in 7 of 8 samples. Its reason said that the result could not be verified, which the no_action guidance named. The automation then never became blocked, and its creator got no notice. The guidance now says that no_action is for a run that worked with nothing to post, or for a temporary failure. A missing provider, tool, account, or access is misconfigured. The same case then ended as misconfigured in 8 of 8 samples. One eval covers the chain: the run ends as misconfigured, the only post is the blocked notice, and it goes to the creator's direct message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
1 screenshot change — 1 changed · 0 added · 0 removed Review screenshots in Frameshift
|
The integration evals recorded these model responses in https://github.com/getsentry/junior/actions/runs/38066806951. The evals of this commit replay them in strict mode. Eval-Suite: integration
Sentry Evalsjunior-behavioral
Base commit junior-guardian
Base commit junior-integration
Base commit junior-router
Base commit |
The behavioral evals recorded these model responses in https://github.com/getsentry/junior/actions/runs/38066806919. The evals of this commit replay them in strict mode. Eval-Suite: behavioral
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A blocked Scheduled automation or Event automation stops in silence. The reason shows only on the dashboard and in the automation tools. The creator now gets one direct message with the reason and a link to resume the automation.
What changes
automations/blocked-notice.tssends the notice. It goes only after the automation is stored as blocked, so the dashboard matches the message:blockEventAutomationstores a new block. A redelivered block sends nothing.advanceScheduledAutomationAfterRunnow returns whether it did). If the creator changed the schedule during the run, the automation stays active and no notice goes out.automation.blocked_notice.failed.<provider>account." A run on system credentials cannot use a connected account, so its reason says to switch to creator credentials and connect one.openSlackDirectMessage.The result guidance of
finishAutomationRunchanges tooFor a job that needs a provider Junior does not have, an Event automation run ended as
no_actionin 7 of 8 samples. Its reason said that the result "could not be verified", and theno_actionguidance named exactly that. The automation then never became blocked, and no notice went out. The guidance now says thatno_actionis for a run that worked with nothing to post, or for a temporary failure, and that a missing provider, tool, account, or access ismisconfigured. The same case then ended asmisconfiguredin 8 of 8 samples. One integration eval covers the chain: the run ends asmisconfigured, the only post is the blocked notice, and it is in the creator's direct message.This changes the tool description of every automation run, so CI records the automation evals again.
Review focus
This is the last of three parts of the first version of #2052. The other two, #2052 and #2095, are merged.
Refs #2051
🤖 Generated with Claude Code