Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/system-specs/modules/app-notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,8 @@ Settings import writes the file through two more paths. `parse_imported_settings

A note fires at most once per published version. `announced-app-updates.json` under the data home records the version last announced per app, and only a different version announces again. It is deliberately NOT under `cache/`, because dropping a cache is a supported repair and dropping this would re-announce the whole backlog. An entry is dropped only on evidence it is finished with — the app's row says it is no longer installed, or that the installed version has reached the one recorded. An entry whose app is merely ABSENT from a listing is kept, because absence says nothing about the app and forgetting it would announce the same version again when the row returns. A **missing** ledger is a first observation: the pending backlog is adopted silently, because a fresh install with updates waiting would otherwise open with a note per app about versions that have been out for weeks, and the badge already states that standing count. A ledger that is unreadable or of the wrong shape reads as empty instead, which fails toward one duplicate note rather than toward permanent silence. Persist before you publish: the ledger is written BEFORE any note goes out, and a ledger that cannot be written sends nothing — otherwise every later listing would read the stale ledger and announce again. A failed delivery is therefore lost rather than repeated, the same ordering and the same trade as `issue_radar`'s watcher. The whole read-modify-write runs under one exclusive lock (`announced-app-updates.lock`, beside the ledger): the store page and the shell badge query list concurrently, and two unserialized passes reading the same ledger would each write back only their own view, so the second would erase the entry the first committed and that version would announce again. A lock that cannot be taken fails closed: no note, nothing written. Only `lifecycle: "gateway"` apps are announced: an `app`- or `locked`-lifecycle app updates itself, so a note would point at an Update button the store cannot press. The note is titled `<name> <version> available`, version first so the feed's truncation takes the word rather than the version, using the app's validated name — never its display name, which on an external row is content that row's author controls and which a bell note would present as the gateway speaking. For the same reason a published version is repeated only when it is a strict semver of bounded length (`apps.update_status.publishable_version`); `9.0.0-SECURITY ALERT: …` produces no note at all. `test_app_update_notify.py` pins the once-per-version property, the first-observation adoption, the kept entry on an absent row, the unwritable-ledger silence, the locked read-modify-write and its fail-closed refusal, the catalog-only trigger, delivery on the loop thread, the skipped lifecycles and the delivery-failure ordering.

`system.turn` is a system channel for "a chat turn stopped with an error the operator must act on". A turn that dies at a terminal, non-retry arm of `_run_chat` — a provider 5xx/throttle or process give-up, a content-filter refusal, a transport death, a signed-out agent — sets `turn.died_notice` to the one-line body; the turn's `finally` fires `dashboard/chat_turn/turn_reply_seams.py::_notify_turn_died` ONCE from that single choke point, so every terminal arm (stop-reason and `except` alike) rings through one site and a future arm is covered by setting the flag. A retry arm, a branch reached only on a user Stop, and the nested-turn (`_prompt_depth > 0`) arm whose parent may absorb the death all leave `died_notice` unset and stay silent; an app-owned chat (`slot._app`) is skipped, since an app owns its chat's continuation. The note is built with `payload_from_legacy(channel="system.turn", ...)` and pushed on `state.notification_bus` directly — the way every other legacy producer stacks a note (`member_message_notes`, `terminal_lifecycle`, `member_goals`), since `group_key` is a payload field the bus drops from `meta` and a second way to pass it through the adapter would be a duplicate mechanism. The legacy `kind` is `agent` — NOT `turn`, which is the frontend's conversation-ready sound category (`presetForKind`), so the channel's own mute/priority govern the sound rather than the hand-back chime. It carries `url=/chat?sid=<slot>` (the deep link), `meta={"slot": <slot>}` so the banner gate scopes to the dying chat, `group_key=turn-died:<session>` so one session's repeated deaths stack into one row, and `publisher_session=<slot key>` so a muted session's alert is delivered silenced — the honest form of "only when it is not the session in focus", without the backend tracking per-client focus. It keys on the slot, not the effective session key, because `session_is_muted` resolves the publisher through `caller_slot_key`, which matches a slot's key but never an unbound channel-born slot's `dashboard:`-prefixed session key. Default priority, not protected; Settings → Notifications labels it "Chat errors" (`CHANNEL_LABELS`), because the bare id "turn" reads as the conversation-handoff sound and "stopped" alone does not say the chat failed. `test_turn_died_notify.py` pins the channel, the schema-field `group_key` through the real bus, the slot meta, the deep link, the app-owned skip, the swallowed delivery failure and the blank-session case.

Dashboard-user settings routes (in `dashboard/messaging_api/notifications.py`, beside the feed routes) expose the union of registered channels and stored settings through `api_notification_channels`; `api_notification_channel_settings` accepts mute and priority updates, clears an override for `priority: null`, and broadcasts `notification_channel_settings`.

## Agent notifications and expiration
Expand Down Expand Up @@ -589,7 +591,13 @@ active chat on a chat route, or whose `url` path is the current route. A
mate's message (`system.member`) instead counts as on screen only when its
`slot` is the active chat on a chat route or the thread the Members page
registered in `lib/viewedThread`, read at arrival: its `url` is the whole
Members page, which shows one thread at a time. Opening
Members page, which shows one thread at a time. A stopped-chat alert
(`system.turn`) is on screen only when its `slot` is the active chat on a chat
route: its `url` is `/chat?sid=<dying slot>`, so the generic `url`-path branch
would strip the query and match the bare `/chat` route, suppressing the banner
whenever the operator is on ANY chat page — including a different one, the case
it exists for — so the channel short-circuits to not-on-screen there instead.
Opening
the popover, landing on the inbox page, or switching the preference off also
retires every pending card.

Expand Down
11 changes: 10 additions & 1 deletion src/kiro_crew/dashboard/chat_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,7 @@
_clear_local_turn_marker,
_gateway_shutdown_requested,
)
from kiro_crew.dashboard.chat_turn.turn_reply_seams import _flush_thinking_stream
from kiro_crew.dashboard.chat_turn.turn_reply_seams import _flush_thinking_stream, _notify_turn_died
from kiro_crew.dashboard.chat_turn.turn_spec_hooks import _turn_stop_hook
from kiro_crew.dashboard.chat_turn.turn_stats import ( # noqa: F401
_MAX_MODEL_ID_LEN,
Expand Down Expand Up @@ -1261,6 +1261,15 @@ async def _run_chat(
slot._git_repo_touch_overflow = False
await _turn_record_outcome(turn)
await _turn_close_crew_log(turn)
# Dead-turn bell, fired at this ONE choke point so every terminal arm --
# stop-reason and exception alike -- rings through the same site, and a
# future terminal arm is covered by setting `turn.died_notice` rather
# than by remembering a scattered call. The producer is best-effort (it
# swallows its own errors), and skips app-owned chats and muted
# sessions; a retry arm leaves `died_notice` None, so an auto-recovering
# turn is silent.
if turn.died_notice:
_notify_turn_died(turn, turn.died_notice)
await _turn_release_handles(turn)
_held_flush_cancel = await _turn_flush_file_changes(turn)
await _turn_settle_debts(turn)
Expand Down
16 changes: 16 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/acp_recovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -365,13 +365,20 @@ async def _on_process_died(turn: _ChatTurn, exc) -> None:
"msg msg-err",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_SESSION_STUCK),
)
turn.died_notice = (
"The connection to the agent kept dropping. Open the chat to start a new one."
)
else:
slot.append(
"error",
gn.GATEWAY_SHARED_RUNTIME_RESTARTING_TEXT,
"msg msg-err",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_SHARED_RUNTIME_RESTARTING),
)
turn.died_notice = (
"The agent process kept restarting and the turn did not finish. "
"Open the chat to retry."
)
else:
slot.append(
"error",
Expand Down Expand Up @@ -464,13 +471,22 @@ async def _on_acp_retryable(turn: _ChatTurn, exc) -> None:
"msg msg-err",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_SESSION_STUCK),
)
turn.died_notice = (
"The connection to the agent kept dropping. Open the chat to start a new one."
if _is_pipe_death
else "The chat stopped and could not recover. Open the chat to start a new one."
)
else:
slot.append(
"error",
gn.GATEWAY_SHARED_RUNTIME_RESTARTING_TEXT,
"msg msg-err",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_SHARED_RUNTIME_RESTARTING),
)
turn.died_notice = (
"The agent process kept restarting and the turn did not finish. "
"Open the chat to retry."
)
elif _prompt_depth == 0:
# Single emit (see the _on_process_died note): slot.append persists +
# broadcasts one chat_message via _on_message; no explicit broadcast_ws.
Expand Down
8 changes: 8 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/recovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -804,6 +804,7 @@ async def _requeue_after_prompt_busy(
_is_synthetic: bool,
_queue_recovery: Callable[..., str],
_stop_pressed: Callable[[], bool],
turn: Any = None,
) -> None:
"""Re-queue a turn whose provider was reset after prompt-busy retries ran out.

Expand Down Expand Up @@ -851,6 +852,13 @@ async def _requeue_after_prompt_busy(
"msg msg-err",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_SESSION_STUCK),
)
# Terminal give-up the operator must act on: ring the dead-turn bell via
# the finally choke point. Past the Stop guard above, so
# a user cancel never reaches here.
if turn is not None:
turn.died_notice = (
"The chat stayed busy and could not recover. " "Open the chat to start a new one."
)
else:
# depth>0 with budget remaining: no re-queue (mirrors AcpProcessDied),
# but still surface feedback so the nested turn doesn't fail silently.
Expand Down
7 changes: 7 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/segment_flush.py
Original file line number Diff line number Diff line change
Expand Up @@ -1167,6 +1167,10 @@ async def _turn_settle_answer(turn: _ChatTurn) -> None:
"msg msg-info",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_REFUSAL_FALLBACK_SAME_MODEL),
)
turn.died_notice = (
"The model declined to answer and the chat stopped. "
"Open the chat to see why."
)
elif _stop_reason == STOP_REASON_REFUSAL:
# Model-side content refusal with no accompanying text: Anthropic's
# bare `refusal` stop reason (passed through by every harness), or
Expand Down Expand Up @@ -1243,6 +1247,9 @@ async def _turn_settle_answer(turn: _ChatTurn) -> None:
"msg msg-info",
meta=gn.gateway_notice_meta(gn.GATEWAY_NOTICE_REFUSAL_FALLBACK_SAME_MODEL),
)
turn.died_notice = (
"The model declined to answer and the chat stopped. " "Open the chat to see why."
)
elif not turn._terminal_directive_applied and should_continue_after_compaction(
# A turn that ended on an applied terminal directive is answered by
# that directive, whatever compaction happened before it: the empty
Expand Down
18 changes: 18 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/stop_recovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,9 @@ async def _turn_stop_recovery(turn: _ChatTurn) -> Literal[_Flow.END] | None:
_emit_recovery_outcome("stale_recover", "exhausted", slot._stale_recovery_retries)
slot._stale_recovery_exhausted_emitted = True
_emit_stale(turn, gn.GATEWAY_SESSION_STUCK_TEXT, notice=gn.GATEWAY_NOTICE_SESSION_STUCK)
turn.died_notice = (
"The chat stalled and could not recover. Open the chat to start a new one."
)
else:
# depth>0 (nested turn) with budget remaining: reset the session
# but don't re-queue (mirrors the pipe-death depth>0 handling);
Expand Down Expand Up @@ -196,6 +199,9 @@ async def _turn_stop_recovery(turn: _ChatTurn) -> Literal[_Flow.END] | None:
_emit_recovery_outcome("tool_stall", "exhausted", slot._tool_stall_retries)
slot._tool_stall_exhausted_emitted = True
_emit_stall(turn, gn.GATEWAY_SESSION_STUCK_TEXT, notice=gn.GATEWAY_NOTICE_SESSION_STUCK)
turn.died_notice = (
"The chat stalled and could not recover. Open the chat to start a new one."
)
else:
_emit_stall(
turn,
Expand Down Expand Up @@ -356,6 +362,10 @@ async def _turn_stop_recovery(turn: _ChatTurn) -> Literal[_Flow.END] | None:
notice=gn.GATEWAY_NOTICE_SESSION_STUCK,
**_rc_args,
)
turn.died_notice = (
"The connection to the agent kept dropping. "
"Open the chat to start a new one."
)
else:
_emit_error(
turn,
Expand All @@ -364,13 +374,21 @@ async def _turn_stop_recovery(turn: _ChatTurn) -> Literal[_Flow.END] | None:
notice=gn.GATEWAY_NOTICE_SHARED_RUNTIME_RESTARTING,
**_rc_args,
)
turn.died_notice = (
"The agent process kept restarting and the turn did not finish. "
"Open the chat to retry."
)
else:
_emit_error(
turn,
f"⟳ Connection lost{_rc_suffix} — please retry.",
notice=gn.GATEWAY_NOTICE_CONNECTION_LOST_GIVE_UP,
**_rc_args,
)
# No bell here: this arm runs only for a NESTED turn
# (_prompt_depth > 0) whose parent turn is still live and may
# absorb the death, so ringing would report a stop the operator
# does not yet own.
return _Flow.END
return None

Expand Down
8 changes: 8 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/turn_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,14 @@ async def _turn_open(turn: _ChatTurn) -> _Flow | None:
_crew_log_error = ""
turn._crew_log_error = _crew_log_error

#: The body of the dead-turn bell entry, set by a terminal
#: non-retry error arm (stop-reason or exception) to the one-line text for
#: the operator; None when the turn did not end in a death that needs them.
#: The finally closer reads it and fires the note ONCE, so the bell is wired
#: at a single choke point and a future terminal arm is covered by setting
#: this flag rather than by remembering a scattered call.
turn.died_notice = None

# Time-to-first-token clock: starts when the user's message reaches the
# runner, stops at the first visible model output (text OR thinking chunk).
# This is the end-to-end latency eager spawn / warm pooling exist to cut —
Expand Down
17 changes: 17 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/turn_failures.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ async def _on_auth_required(turn: _ChatTurn, exc) -> None:
# store the row stays a plain error. The prose is unchanged.
slot.append("error", _auth_msg, "msg msg-err", meta=_terminal_error_meta(exc))
_mark_kiro_signed_out(state)
turn.died_notice = "The chat stopped because the agent is signed out. Open the chat to sign in."
await _deliver_linked_slack_message(state, slot, sessions, session_key, _auth_msg)


Expand All @@ -167,6 +168,7 @@ async def _on_prompt_busy(turn: _ChatTurn) -> None:
_is_synthetic=_is_synthetic,
_queue_recovery=functools.partial(_queue_recovery, turn),
_stop_pressed=functools.partial(_stop_pressed, turn),
turn=turn,
)


Expand Down Expand Up @@ -410,6 +412,11 @@ async def _on_acp_error_card(turn: _ChatTurn, exc) -> None:
"msg msg-err",
meta=_unentitled_meta,
)
# The turn ended on a terminal provider/runtime error the user
# must act on (a 5xx/throttle timeout, an unentitled model, a
# malformed-request rejection). Ring the bell so an operator in
# another session is not left to find the inline card.
turn.died_notice = "The chat stopped with an error. Open the chat to see what happened."
# Record a STRUCTURAL terminal outcome for this slot's last turn,
# but ONLY for a self-driven nudge fire (``_directive_self_wake``).
# A malformed-request rejection is deterministic in the payload's
Expand Down Expand Up @@ -612,6 +619,16 @@ async def _on_error(turn: _ChatTurn, exc) -> None:
# this arm carries none of the tags and keeps its plain row.
_err_meta = _terminal_error_meta(exc)
slot.append("error", _err_text, "msg msg-err", meta=_err_meta)
# Ring the dead-turn bell for a GENUINE runtime death that lands here -- a
# shared-runtime prompt timeout / session-start timeout (AcpRequestTimeout /
# AcpSessionStartTimeout, which descend from AcpRuntimeError, not AcpError,
# so they skip the arms above) or a plain terminal Exception. NOT for a
# deterministic config refusal: a memory-unavailable end and a
# capability/session-start refusal (`_err_meta["code"]`) carry their own
# guided-fix card and a retry repeats them unchanged, so they are not the
# "work silently stopped" case the bell exists for.
if not (isinstance(_err_meta, dict) and _err_meta.get("code")):
turn.died_notice = "The chat stopped with an error. Open the chat to see what happened."
# The SIBLING of the tagged-start report in the AcpError branch (_on_acp_error_card).
# A session start on the SHARED runtime raises AcpRequestTimeout /
# AcpSessionStartTimeout, which descend from AcpRuntimeError and not from
Expand Down
6 changes: 6 additions & 0 deletions src/kiro_crew/dashboard/chat_turn/turn_ledgers.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,12 @@ def _queue_recovery(
getattr(state, "sessions", None), _refusal_key or session_key
),
)
# A queued continuation means the turn is NOT a terminal dead end: it will
# run on this same slot. Retract any dead-turn bell a terminal arm set
# earlier this turn (a refusal arm whose Stop-hook continuation is queued
# afterward, a transient give-up that is then re-queued), so the finally
# choke point does not ring "a chat stopped" while the chat continues.
turn.died_notice = None
return _recovery_qid


Expand Down
1 change: 1 addition & 0 deletions src/kiro_crew/dashboard/chat_turn/turn_locals.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ class _ChatTurn:
_current_replay_message: "dict[Any, Any] | None"
_declined_grant_notices: "dict[str, str]"
_deferred_cut_text: "str"
died_notice: "str | None"
_deferred_options_expiry: "bool"
_dir_consumed_out: "dict[str, str]"
_dispatch_lock: "Lock"
Expand Down
Loading
Loading