Support --format json for commands creating a run - #921
Merged
Conversation
--format json now prints the created run as {runId, runUrl, runPublicUrl}
for `start`, `run` without a command, `run --remote` and `run <command>`.
Any other format value keeps printing the bare run id, so existing
`--format id` scripts are unaffected, and `run --filter-list --format json`
keeps printing the array of matching test ids.
`run` without a command previously printed nothing to stdout even with
--format id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Errors were still reported as [TESTOMATIO] prefixed text while the run itself was printed as JSON. The logger now renders every message as one JSON object per line when TESTOMATIO_LOG_JSON=1, which the CLI sets for --format json. Failed requests to Testomat.io add their data as fields (status, method, url, error, response, request) built from the same values as the text message, with the API token hidden as before. Error paths of the Testomat.io pipe went through console.log/console.warn directly, so they bypassed the logger and printed to stdout: the failed request details and the "create an issue" hint polluted the output captured from --format commands. They are logged with log.error/log.warn now, which prints them to stderr. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…created The token was masked only where the failed request is formatted, so any log call which received a raw request error printed it: util.format inspects the error and prints its config, including the request body. Masking moved to src/utils/hide_token.js and is applied by the logger to every message and to the JSON fields, so it covers callers which don't mask themselves. A bare `tstmt_...` is now masked too, not only the quoted forms. `run` without a command exited 0 when the run was not created, which with --format json is indistinguishable from success: empty stdout, exit 0. It exits 1 as `start` does, and both read the run id from the pipe store. A cut request body is not valid JSON, so the JSON log marks it with requestTruncated instead of silently turning `request` into a string. Report and artifact status lines used console.log directly, so they went to stdout and polluted the output captured from --format commands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DenysKuchma
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--format jsonpreviously only worked forrun --filter-list, where it prints the matching test IDs. For the commands that create a run the value was ignored —startalways printed the bare run id, andrunprinted nothing at all to stdout.Now
--format jsonprints the created run on stdout:{"runId":"a1b2c3d4","runUrl":"https://app.testomat.io/projects/demo/runs/a1b2c3d4","runPublicUrl":"https://app.testomat.io/p/a1b2c3d4"}RUN_URL=$(npx @testomatio/reporter start --format json | jq -r .runUrl)Covered paths:
startrunwith no command (previously printed nothing to stdout, even with--format id)run --remote <profile>run "<command>"— printed before the runner is spawned, so it is the first stdout linerunUrl/runPublicUrlare omitted when Testomat.io did not return them.JSON logs
With
--format jsonthe logs are machine-readable too — the CLI setsTESTOMATIO_LOG_JSON=1and the logger prints one JSON object per line on stderr instead of[TESTOMATIO]prefixed text, so a failing run can be diagnosed without scraping text:{"status":403,"method":"POST","url":"https://app.testomat.io/api/reporter","error":"Project API Token is invalid","response":{"message":"Project API Token is invalid"},"request":{"api_key":"<hidden>","kind":"manual","status":"scheduled"},"level":"error","message":"⚠️ Request to Testomat.io failed: ..."} {"level":"error","message":"Error creating Testomat.io report (see details above), please check if your API key is valid. Skipping report"} {"level":"error","message":"Failed to create run on Testomat.io."}The JSON fields are built from the same values as the text message (no output is parsed), and the API token stays hidden.
TESTOMATIO_LOG_JSON=1works outside the CLI as well, for reporters running inside a test framework.Token masking is done by the logger
hideTestomatioTokenmoved tosrc/utils/hide_token.jsand is applied inside the logger — to the rendered message and to the serialized JSON fields — instead of per call site. Any log call that receives a raw request error is covered:util.formatinspects the error and prints itsconfig, request body included. A baretstmt_…is masked as well, not only the quoted forms.Other fixes in this branch
runwithout a command exited 0 when the run was not created — with--format jsonthat is empty stdout plus a success code, indistinguishable from success. It exits 1 likestartdoes, and both read the run id from the pipe store rather than from env vars."requestTruncated": trueinstead of silently makingrequesta string.client.js,pipe/html.jsandpipe/csv.jsusedconsole.logdirectly, so they landed on stdout and polluted captured output. They go throughlog.info/log.warn/log.errornow — unchanged for normal runs (INFO level, stdout), suppressed in machine-readable mode.Behavior change to note
The error paths of the Testomat.io pipe used⚠️ Request to Testomat.io failed" block and the "create an issue" hint went to stdout. They now go through
console.log/console.warndirectly, bypassing the logger: the "log.error/log.warn, so they are printed to stderr with the[TESTOMATIO]prefix in text mode as well. This also fixes the failed-request blob leaking intoRUN_ID=$(reporter start --format id)on a 403.Backward compatibility
--formatvalue other thanjsonstill prints the bare run id, soRUN_ID=$(reporter start --format id)is unchanged.run --filter-list --format jsonkeeps its current meaning (JSON array of test IDs) — no run is created there.Notes
run "<command>"the runner inherits stdout, so the docs recommend capturing the whole output and slicing the first line instead of piping intohead(piping closes stdout under the running tests).--format jsonrequest/response fields:responseis an object,requestis an object unless it was truncated (then a string plusrequestTruncated: true).Tests
formatRunOutputunit tests intests/unit/pipe_utils_test.jstests/unit/logger_test.jstests/unit/cli_start_remote_test.jshideTestomatioTokentests intests/unit/hide_token_test.js, including the escaped-JSON form printed by inspected errorsnpm run lint,npm run buildand the full unit suite (578 passing) are greenDocs updated in
docs/cli.md(including the--formatreference section, which claimed the format value was "not significant" forstartand that--formatonly applied together with--filter-list) anddocs/configuration.mdforTESTOMATIO_LOG_JSON.🤖 Generated with Claude Code