Skip to content

Commit 556e4a5

Browse files
committed
fix(cli): expose list-page truncation in the payload
An oversize structured list page is emitted as its leading rows, but the reduction was announced only on stderr while total / has_next_page / search_after_ctx kept describing the page the server returned — so a script that discards stderr and stops on has_next_page silently loses every withheld row. A reduced list envelope now carries truncated: true plus emitted_rows: N when rows were withheld (the case paging can repair); when long values inside rows were clipped instead, truncated rides alone, because re-requesting cannot restore them. Bare top-level arrays have nowhere to carry the marker and keep the stderr note as their only signal. Docs: README / README_zh bounded-pages section; a page-level note in the flashduty skill so agents recognise the two keys.
1 parent 95bac72 commit 556e4a5

5 files changed

Lines changed: 99 additions & 4 deletions

File tree

‎README.md‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -321,18 +321,25 @@ inc_def456 High memory usage Warning Processing Staging 2026
321321
Showing 2 results (page 1, total 2).
322322
```
323323

324-
**JSON (`--json` / `--output-format json`):** Machine-parseable, full data, no truncation.
324+
**JSON (`--json` / `--output-format json`):** Machine-parseable output for `jq` and scripts.
325325

326326
```bash
327327
flashduty incident list --json | jq '.[].title'
328328
```
329329

330-
**TOON (`--output-format toon`):** Token-Oriented Object Notation — full data, no truncation, but drops the per-row repeated keys that JSON emits for uniform arrays, so list output costs materially fewer tokens. Preferred for LLM/agent consumption. Not directly `jq`-able; use `--json` when you need to pipe into `jq`.
330+
**TOON (`--output-format toon`):** Token-Oriented Object Notation — drops the per-row repeated keys that JSON emits for uniform arrays, so list output costs materially fewer tokens. Preferred for LLM/agent consumption. Not directly `jq`-able; use `--json` when you need to pipe into `jq`.
331331

332332
```bash
333333
flashduty incident list --output-format toon
334334
```
335335

336+
**Bounded list pages.** Every structured list page is capped at 16 KiB: an oversize page is emitted as the leading rows that fit, and the reduction is announced on stderr. A reduced **list envelope says so in the payload** too — scripts routinely discard stderr — and the marker's shape tells you which reduction happened:
337+
338+
- `"truncated": true` **with** `"emitted_rows": N` — the page carries its first N rows and withheld the rest. The envelope's `total` / `has_next_page` / `search_after_ctx` still describe the page as the server returned it, so a page cut to 7 of 100 rows reads as complete. To collect everything, re-request with a `--limit` no larger than the rows you received (or, where the command documents its cursor as a row id, pass the last received row's id back as `--search-after-ctx`) and repeat until the rows you hold reach `total`; stopping on `has_next_page=false` alone silently drops the withheld rows.
339+
- `"truncated": true` **alone** — every row was emitted, but long values inside them were clipped (stderr names the fields). Paging cannot restore them; narrow `--fields` and re-request.
340+
341+
A bare top-level array has nowhere to carry the marker, so it announces a reduction only on stderr — page it with a lower `--limit`, or switch to a page-envelope command (`alert event-list`, `insight incident-list`) when a script needs completeness.
342+
336343
**No truncation (`--no-trunc`):** Table with full field content.
337344

338345
---

‎README_zh.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -317,12 +317,14 @@ inc_def456 High memory usage Warning Processing Staging 2026
317317
Showing 2 results (page 1, total 2).
318318
```
319319

320-
**JSON(`--json`):** 机器可解析,完整数据,不截断。
320+
**JSON(`--json`):** 机器可解析,可直接管道给 `jq`。
321321

322322
```bash
323323
flashduty incident list --json | jq '.[].title'
324324
```
325325

326+
**列表页有 16 KiB 上限。** 结构化列表的一页超出上限时,只输出能装下的前若干行,并在 stderr 说明。如果该页是分页信封(形如 `{items, total, has_next_page, …}`),载荷内也会带上标记:`"truncated": true` 与 `"emitted_rows": N`(保留了前 N 行,其余被丢弃)。`total` / `has_next_page` / `search_after_ctx` 仍是服务端原值,因此一页被裁到"100 行里只发 7 行"时,看起来与完整页无异。脚本要完整翻页时,请从**实际收到的最后一行**之后继续(用不大于已收到行数的 `--limit` 重新请求,再跟随该响应的游标),不要只依赖 `has_next_page`;若只有 `"truncated": true` 而没有 `emitted_rows`,说明行内长值被裁剪——翻页无法恢复,应收窄 `--fields` 后重新请求。
327+
326328
**不截断(`--no-trunc`):** 表格显示完整字段内容。
327329

328330
---

‎internal/cli/gen_support.go‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,8 @@ func printGenericResult(ctx *RunContext, data any) error {
359359
// byte-identical output. Only an over-cap list payload detours through the
360360
// bounding machinery, and only there does the output change (fewer rows; a
361361
// rebuilt envelope, so key order is no longer the struct's field order).
362+
// A reduced list envelope also carries truncated/emitted_rows in the payload
363+
// itself, so a machine consumer sees the reduction without reading stderr.
362364
func printBoundedGenericResult(ctx *RunContext, data any) error {
363365
encoded, err := marshalStructured(data)
364366
if err != nil || len(encoded)+1 < compactListOutputLimit {
@@ -382,6 +384,8 @@ func printBoundedGenericResult(ctx *RunContext, data any) error {
382384
if err != nil {
383385
return err
384386
}
387+
// A bare array has nowhere to carry the marker the envelope branch
388+
// adds; the stderr note is its only signal.
385389
noteProjectionBound(ctx.Cmd.ErrOrStderr(), note)
386390
return ctx.Printer.Print(bounded, nil)
387391
case map[string]any:
@@ -407,6 +411,19 @@ func printBoundedGenericResult(ctx *RunContext, data any) error {
407411
return err
408412
}
409413
value[key] = bounded
414+
if note != "" {
415+
// In-payload, not just on stderr: scripts discard stderr, and
416+
// the pagination siblings keep describing the server page, so a
417+
// reduced page would otherwise read as complete. emitted_rows
418+
// is set only when rows were WITHHELD (a prefix was dropped),
419+
// which is the case paging can repair; when instead long values
420+
// were clipped, truncated rides alone — re-requesting cannot
421+
// restore them, narrowing --fields can.
422+
value["truncated"] = true
423+
if len(bounded) < len(rows) {
424+
value["emitted_rows"] = len(bounded)
425+
}
426+
}
410427
out, err := marshalStructured(value)
411428
if err != nil {
412429
return err

‎internal/cli/gen_support_test.go‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,14 +90,36 @@ func TestPrintGenericResultBoundsListEnvelope(t *testing.T) {
9090
t.Errorf("bounded %s output lost envelope key %q:\n%s", format, key, out)
9191
}
9292
}
93+
// The reduced page says so in the payload, not only on stderr —
94+
// with the row-withheld shape: emitted_rows present alongside
95+
// truncated. A values-clipped reduction carries truncated alone.
96+
if format == "toon" {
97+
// TOON renders the marker as envelope-level lines beside the
98+
// items block.
99+
if !strings.Contains(out, "truncated: true") || !strings.Contains(out, "emitted_rows:") {
100+
t.Errorf("bounded toon output lost the in-payload truncation marker:\n%s", out)
101+
}
102+
}
93103
if format == "json" {
94104
var envelope map[string]any
95105
if err := json.Unmarshal([]byte(strings.TrimSpace(out)), &envelope); err != nil {
96106
t.Fatalf("bounded json is not an object: %v", err)
97107
}
98-
if _, ok := envelope["items"].([]any); !ok {
108+
items, ok := envelope["items"].([]any)
109+
if !ok {
99110
t.Fatalf("bounded json lost the items array: %v", envelope)
100111
}
112+
if envelope["truncated"] != true {
113+
t.Errorf("bounded json truncated = %v, want true", envelope["truncated"])
114+
}
115+
emitted, ok := envelope["emitted_rows"].(float64)
116+
if !ok {
117+
t.Fatalf("bounded json lost emitted_rows: %v", envelope)
118+
}
119+
if int(emitted) != len(items) || int(emitted) >= len(rows) {
120+
t.Errorf("bounded json emitted_rows = %v, want len(items)=%d and < %d requested rows",
121+
envelope["emitted_rows"], len(items), len(rows))
122+
}
101123
}
102124
})
103125
}
@@ -227,4 +249,49 @@ func TestPrintGenericResultShortenedRowStaysUTF8(t *testing.T) {
227249
if !strings.Contains(stderrText, "were shortened to fit") {
228250
t.Errorf("shortened row should announce the clipped fields on stderr, got:\n%s", stderrText)
229251
}
252+
// Clipped values are data loss too: the marker appears even though every
253+
// row was emitted — and it rides WITHOUT emitted_rows, because paging
254+
// cannot restore a clipped value (only a narrower --fields can), so the
255+
// withheld-rows continuation must not be read into this payload.
256+
var envelope map[string]any
257+
if err := json.Unmarshal([]byte(strings.TrimSpace(out)), &envelope); err != nil {
258+
t.Fatalf("shortened single row is not a JSON object: %v", err)
259+
}
260+
if envelope["truncated"] != true {
261+
t.Errorf("shortened single row lost the in-payload truncation marker: %v", envelope)
262+
}
263+
if _, ok := envelope["emitted_rows"]; ok {
264+
t.Errorf("a values-clipped page must not carry emitted_rows (it misreads as withheld rows): %v", envelope)
265+
}
266+
if items, _ := envelope["items"].([]any); len(items) != 1 {
267+
t.Errorf("clipping must not drop rows, got %d items", len(items))
268+
}
269+
}
270+
271+
// TestPrintGenericResultCompleteEnvelopeUnmarked guards the marker's negative
272+
// case: a page that fits carries no truncated/emitted_rows keys — the marker
273+
// means "this page was reduced", not "this command supports reduction".
274+
func TestPrintGenericResultCompleteEnvelopeUnmarked(t *testing.T) {
275+
saveAndResetGlobals(t)
276+
stub := newGFStub(t)
277+
stub.data = map[string]any{
278+
"items": []any{
279+
map[string]any{"incident_id": "inc-1", "title": "small", "severity": "Info"},
280+
map[string]any{"incident_id": "inc-2", "title": "smaller", "severity": "Info"},
281+
},
282+
"total": 2,
283+
"has_next_page": false,
284+
}
285+
286+
out, stderrText, err := execCommandSplit("insight", "incident-list",
287+
"--start-time", "7d", "--end-time", "now", "--output-format", "json")
288+
if err != nil {
289+
t.Fatalf("execCommandSplit: %v", err)
290+
}
291+
if strings.Contains(out, "truncated") || strings.Contains(out, "emitted_rows") {
292+
t.Errorf("within-budget envelope must not carry the truncation marker:\n%s", out)
293+
}
294+
if strings.Contains(stderrText, "note: emitted") {
295+
t.Errorf("within-budget envelope must not announce a reduction, got:\n%s", stderrText)
296+
}
230297
}

‎skills/flashduty/SKILL.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ Append `--output-format toon` to read commands: it drops the per-row repeated ke
2929

3030
**Shape the payload before you fetch it.** For ID scans, counts, or "find the matching row" tasks, prefer `--fields` projections and compact list verbs over full detail dumps. Huge raw JSON dumps are a last resort, not a default.
3131

32+
**A structured list page is capped at 16 KiB.** An oversize page is emitted as its leading rows, and the envelope carries `"truncated": true` with `"emitted_rows": N`; `total` / `has_next_page` / `search_after_ctx` still describe the page as the server returned it, so a reduced page reads as complete unless you check the marker. `truncated` WITHOUT `emitted_rows` means long values inside the rows were clipped — narrow `--fields` instead of paging. When a walk must cover everything (a script, a full export), resume after the last row you actually received: re-request with a `--limit` no larger than the rows you got, then follow that response's cursor.
33+
3234
**Empty result = authoritative not-found.** A filter returning `[]` means no such entity in scope — report it (optionally the 1–2 closest names) and stop. Do **not** brute-force (no shifted-keyword re-queries, no widening past caps, no full-dump grep). Never infer "feature not enabled" from an empty list, and never fabricate data absent from tool output.
3335

3436
**A result you did not fetch is "unknown", never "empty" — and "fetched" means the same scope, not just the same verb.** You may report a command's result for a given window, entity, or aspect — including "returned empty" or any count/list/finding — **only if a call covering that exact scope appears in your tool-call history this turn**. A wider or different time window, or a sibling entity's result, does not transfer: extrapolating from what you did fetch is the same fabrication as skipping the fetch. If the scope wasn't queried, the honest answer is "未查询 — 可运行 <command>", not a filled-in number or a generalized claim.

0 commit comments

Comments
 (0)