Skip to content

docs: say stdout keeps a partial JSON result after a failure - #1116

Merged
ragnorc merged 1 commit into
ModernRelay:mainfrom
ragnorc:docs/stdout-after-partial-json
Oct 21, 2026
Merged

ragnorc merged 1 commit into
ModernRelay:mainfrom
ragnorc:docs/stdout-after-partial-json

Conversation

@ragnorc

@ragnorc ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

What

docs/user/operations/troubleshooting.md said that when part of a JSON result already reached stdout, the failure goes to stderr "so stdout holds one document". That is not what a streamed command leaves behind. diff --json and changes poll --json write one open object, and a later page error leaves that prefix on stdout. The sentence now says stdout keeps the bytes already written, which can end inside an unfinished object, and that the failure document is on stderr.

Why

Optional finding 1 of the review of #1078 at e3f98130, which merged before it could be addressed there. A script that reads stdout after a failed streamed command must not assume it parses.

Verification

python3 scripts/check-docs.py: Documentation OK. typos on the changed file: clean. Documentation only; no behavior changes.

ragnorc commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Why

The page said that a failure after a streamed page still leaves one JSON document on stdout. That is false. A script that parses stdout after a failed streamed command can read a prefix that is not JSON.

What

The new text says stdout keeps the bytes already written. Those bytes can stop inside an object that is not closed. The examples are diff --json and changes poll --json. The failure document goes to stderr.

How

This matches the CLI on current main (277478d8), which this branch merges onto cleanly.

Auto-pagination writes one open object and flushes each page. finish writes the closing bytes only after the last page. A later page error returns before finish. StartedOutput records that stdout received bytes. The failure path then writes the failure with write_json to stderr.

The four auto-paginated JSON commands are:

  • diff --json and branch diff --json use DiffJsonStream.
  • commit changes --json uses CommitChangesJsonStream.
  • changes poll --json uses ChangeFeedJsonStream.

changes poll always uses that stream. The other three use it only when they follow every page. With --page-token, they call write_json on one complete page.

A command that fails before any result byte still writes one pretty JSON failure on stdout. write_json builds the whole value first. That is the path for snapshot, query, mutate, load, one-page history commands, and changes baseline.

These history commands are reads. A read failure has effects: none and exits 1, including when stdout already holds a prefix. Exit 1 means the command changed nothing. It does not mean stdout parses.

Human mode still prints the disposition line on stderr. A target refusal still exits 1 with action: fix, and in JSON mode that refusal is written to stdout. Target resolution happens before any result bytes.

Impact

A caller must not parse stdout after a non-zero streamed command. The failure document on stderr is one pretty JSON value and includes disposition. A partial changes poll also has no cursor. finish writes cursor and caught_up only on the terminal page. docs/user/branching/changes.md already says the CLI prints the cursor only after that page.

Blocking

None.

Optional

  1. The examples name two of the four open-object commands. commit changes --json and branch diff --json leave the same prefix. See the inline note. This does not block the merge. The general sentence is already true.

Inline note on docs/user/operations/troubleshooting.md line 107:

Optional. commit changes --json and branch diff --json use the same open object when they follow every page (CommitChangesJsonStream, DiffJsonStream). finish writes the closing bytes. A later page error returns before that, so stdout keeps the same kind of prefix. With --page-token, diff, branch diff, and commit changes write one complete page through write_json. The two names in this sentence are examples, and the general sentence is already true. Add the other two names if you want the examples to list the whole class. This does not block the merge.

Tradeoff and liability

Correctness comes first. Each page is flushed so a long result does not stay resident. The cost is that a failed run can leave stdout that is not valid JSON. Closing the object on the failure path would hide a short read. The old sentence did that in the docs.

This change removes a false operator contract. It adds no API, state, or storage rule. The code behavior is unchanged. The sentence has to stay true for the three JSON stream writers. The comments in app.rs still say that stdout keeps one result. That phrase is how the old sentence got onto this page. A later edit can copy it again. Updating those two comments is optional and is not required to make this page true.

docs/user/cli/reference.md still says --json keeps structured errors on stdout, and it links here. That remains the common case: no result byte yet, so the failure is one document on stdout. This section is the exception. export streams JSONL and blob get streams bytes. A failure after the first byte also leaves the prefix and puts a JSON failure on stderr. The new text does not call those prefixes one open object. That is correct.

No other open pull request edits this sentence. #1115 still uses CommitChangesJsonStream and does not change this failure routing. This contract is the CLI process boundary. Lance does not own it.

Earlier finding

Optional finding 1 on #1078 at e3f98130 asked for this correction. This pull request does that. It also says the prefix can be an unfinished object. That matches the writers.

Validation

Reviewed head ec64c3f489fc115f7ff3e61a8c376a04545d2f39. Main 277478d84cf19159f43cb21f202b49a044f0185e. Merge base 269c6189fb3ddba2355d72829c3d10fc0fab2b4c. git merge-tree --write-tree 277478d8 ec64c3f4 exits 0. The three-dot diff is only docs/user/operations/troubleshooting.md (+4 / -2).

GQT does not apply. GQT checks rows, counts, column types, and query errors. It does not see which stdio stream holds a failure, or a prefix that is not a document. This pull request does not change behavior.

Temporary probe on main, then removed. Not pushed. One page from each JSON stream, with no finish:

cargo test -p omnigraph-cli --lib --locked probe_open_json_page_is_not_a_document -- --test-threads=1

Passed. Each prefix starts with { and does not parse as JSON. The feed prefix does not contain "cursor":.

Existing tests on that same tree:

cargo test -p omnigraph-cli --lib --locked -- a_failure_after_part_of_a_json_result_goes_to_stderr diff_streams_keep_one_header feed_json_stream_stitches commit_json_stream_preserves history_stream_pages_propagate --test-threads=8

5 passed. The routing test checks both sides. With no result byte, the failure JSON is on stdout, stderr is empty, exit is 1, and effects is none. With a result byte already written, the failure JSON is on stderr and stdout is left unchanged.

python3 scripts/check-docs.py on a detached checkout of ec64c3f4: Documentation OK (366 Markdown files). typos 1.50.1 on docs/user/operations/troubleshooting.md: clean.

I did not run a live multi-page diff or changes poll in a process. The stream unit tests and the failure-routing test cover the two layers the sentence describes.

Recommendation

Merge. The optional names can land here or in a follow-up.

@ragnorc
ragnorc added this pull request to the merge queue Oct 21, 2026
Merged via the queue into ModernRelay:main with commit e466a12 Oct 21, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant