Support --format json for commands creating a run

docs/cli.md

@@ -29,7 +29,7 @@ npx @testomatio/reporter <command> [options]
 
 Starts a new test run and returns its ID. This requires an API key to be set in the `TESTOMATIO` environment variable.
 
-With `--format id` (or any `--format`), `start` prints **only the run id to `stdout`** (the banner and progress logs go to `stderr`), so it is safe to capture: `RUN_ID=$(npx @testomatio/reporter start --format id)`. It exits non-zero if the run could not be created.
+With `--format id`, `start` prints **only the run id to `stdout`** (the banner and progress logs go to `stderr`), so it is safe to capture: `RUN_ID=$(npx @testomatio/reporter start --format id)`. Use `--format json` to get the run id together with its URLs as a JSON object. It exits non-zero if the run could not be created.
 
 **Usage:**
 
@@ -56,7 +56,7 @@ npx @testomatio/reporter start --filter "testomatio:tag-name=smoke"
 - `--env-file <envfile>`: Load environment variables from a specific env file. If none specified, it will look for `.env` file.
 - `--kind <type>`: Specify run type: `automated`, `manual`, `mixed`, or `detect`. Determines how the test run is categorized in Testomat.io. See [Detecting the run kind](#11-detecting-the-run-kind).
 - `--filter <filter>`: Scope the prepared run to the tests matching the filter (same syntax as [`run --filter`](#31-filter-pipes)). The run is created with that test list but **not** executed — useful to prepare a run and launch it later on CI (see [Prepare a run, then launch it on CI](#34-prepare-a-run-then-launch-it-on-ci)).
-- `--format <format>`: Print **only the run id** to `stdout` (banner and logs go to `stderr`) so it can be captured: `RUN_ID=$(npx @testomatio/reporter start --format id)`.
+- `--format <format>`: Machine-readable output on `stdout` (banner and logs go to `stderr`): `id` prints the bare run id so it can be captured — `RUN_ID=$(npx @testomatio/reporter start --format id)` — and `json` prints `{"runId", "runUrl", "runPublicUrl"}`.
 - `--warn`: Exit `0` instead of `1` when the filter matches no tests — the warning is still printed. Use in pipelines where an empty scope is a normal outcome (e.g. a PR touching no mapped files).
 
 The run is created as **scheduled**, not running: nothing has been executed yet. Testomat.io promotes it to *running* as soon as the first test result is reported, or when you launch it by hand.
@@ -128,7 +128,7 @@ Alias for this command – `test`, e.g. `npx @testomatio/reporter test [options]
 - `-c, --command <cmd>`: Test runner command (required).
 - `--filter <filter>`: [Filter executed tests](./pipes/testomatio.md#filter-tests) by tag, label, jira, plan.
 - `--filter-list <filter>`: Print the list of tests matching the filter without running them. Useful for inspecting which tests would run, or for piping IDs into another command. See [Coverage Pipe](./pipes/coverage.md#machine-readable-output-with---format) for examples.
-- `--format <format>`: Machine-readable output format for `--filter-list`. Supported values: `grep`, `json`, `newline`, `ids`. When set, the CLI banner is suppressed and informational logs go to `stderr` so `stdout` stays clean for piping.
+- `--format <format>`: Machine-readable output on `stdout`. With `--filter-list` it selects how the matched test IDs are encoded (`grep`, `json`, `newline`, `ids`); otherwise it prints the run the command creates — `id` for the bare run id, `json` for `{"runId", "runUrl", "runPublicUrl"}`. When set, the CLI banner is suppressed and informational logs go to `stderr` so `stdout` stays clean for piping.
 - `--env-file <envfile>`: Load environment variables from a specific env file.
 - `--kind <type>`: Specify run type: `automated`, `manual`, `mixed`, or `detect`. Determines how the test run is categorized in Testomat.io. `detect` needs a run scoped to a list of tests to resolve from, which `run --filter` does not create — see [Detecting the run kind](#11-detecting-the-run-kind).
 - `--remote <profile>`: Trigger the run on a CI profile configured on the Testomat.io project (e.g. `github`, `gitlab`, `jenkins`) instead of executing tests locally. The CLI creates the run on Testomat.io, asks the backend to dispatch the named CI workflow, and exits. Equivalent to setting [`TESTOMATIO_CI_PROFILE`](./configuration.md#testomatio_ci_profile).
@@ -394,7 +394,7 @@ For more details about debug files, see the [Debug Pipe documentation](pipes/deb
 
 ## The `--format` flag
 
-`--format` switches a command into **machine-readable mode**: `stdout` carries only the requested data so it can be captured or piped, while the banner and progress logs are routed to `stderr`. Two commands support it — [`run --filter-list`](#3-run) and [`start`](#1-start) — and machine-readable mode behaves the same way for both.
+`--format` switches a command into **machine-readable mode**: `stdout` carries only the requested data so it can be captured or piped, while the banner and progress logs are routed to `stderr`. It is supported by [`start`](#1-start) and [`run`](#3-run), and machine-readable mode behaves the same way for both.
 
 **What machine-readable mode does (regardless of command or value):**
 
@@ -405,9 +405,27 @@ For more details about debug files, see the [Debug Pipe documentation](pipes/deb
 
 This is what makes `$( … )` capture and `|` piping reliable — without `--format`, the banner and `[TESTOMATIO]` logs are interleaved on `stdout`.
 
+**With `--format json` the logs are machine-readable too.** Every reporter message is printed to `stderr` as one JSON object per line instead of prefixed text, so a failing run can be diagnosed without scraping text:
+
+```bash
+npx @testomatio/reporter start --format json 2>errors.log
+```
+
+```json
+{"level":"error","message":"Error creating Testomat.io report (see details above), please check if your API key is valid. Skipping report"}
+```
+
+A failed request to Testomat.io adds its data as fields — `status`, `method`, `url`, `error`, `response` and `request`. The API token is hidden in every log, whichever field or message it appears in. A request body longer than 1000 characters is cut, which makes `request` a string instead of an object and adds `"requestTruncated": true`:
+
+```json
+{"status":403,"method":"POST","url":"https://app.testomat.io/api/reporter","error":"Project API Token is invalid","level":"error","message":"⚠️ Request to Testomat.io failed: ..."}
+```
+
+The same output is enabled outside the CLI with [`TESTOMATIO_LOG_JSON=1`](./configuration.md#testomatio_log_json), which the CLI also passes to the test runner it spawns.
+
 ### With `run --filter-list`
 
-Prints the IDs of the tests matching the filter **without running them**. `--format` only takes effect together with `--filter-list`; the value selects the encoding:
+Prints the IDs of the tests matching the filter **without running them**. The value selects the encoding:
 
 | Value     | Output                        | Example              |
 | --------- | ----------------------------- | -------------------- |
@@ -427,30 +445,64 @@ GREP=$(npx @testomatio/reporter run --filter-list "coverage:file=coverage.yml" -
 npx @testomatio/reporter run --filter-list "coverage:file=coverage.yml" --format json > affected-tests.json
 ```
 
+With `--filter-list` no run is created, so `--format json` prints the array of matching test IDs — not the run object described below.
+
 Only the `testomatio:` and `coverage:` filter pipes are supported (see [3.1 Filter pipes](#31-filter-pipes)). The [Coverage Pipe docs](./pipes/coverage.md#machine-readable-output-with---format) cover the formats in more detail.
 
-### With `start`
+### With a command that creates a run
+
+`start` always creates a run, and so does `run` when it is not listing tests with `--filter-list`. For those invocations `--format` selects how the created run is printed to `stdout`:
 
-Prints **only the new run id** to `stdout`, so it can be captured directly:
+| Value  | Output                                                     |
+| ------ | ---------------------------------------------------------- |
+| `id`   | the bare run id (default for any value other than `json`)  |
+| `json` | a JSON object with the run id and its URLs                 |
 
 ```bash
 RUN_ID=$(npx @testomatio/reporter start --format id)
 echo "$RUN_ID"   # e.g. a1b2c3d4
+
+npx @testomatio/reporter start --format json
+# {"runId":"a1b2c3d4","runUrl":"https://app.testomat.io/projects/demo/runs/a1b2c3d4","runPublicUrl":"https://app.testomat.io/p/a1b2c3d4"}
+```
+
+`runUrl` and `runPublicUrl` are omitted when Testomat.io did not return them. Pick single fields with any JSON tool:
+
+```bash
+RUN_URL=$(npx @testomatio/reporter start --format json | jq -r .runUrl)
 ```
 
-`start` emits a single value (the run id), so for `start` the format **value is not significant** — `--format id` is the conventional choice, but any value turns on machine-readable mode. `start` exits non-zero if the run could not be created, so `RUN_ID` is set only on success. It combines with `--kind` and `--filter`:
+Every command listed here exits non-zero and prints nothing to `stdout` if the run could not be created, so a captured variable is set only on success. `start` combines with `--kind` and `--filter`:
 
 ```bash
 RUN_ID=$(npx @testomatio/reporter start --kind manual --format id)
 RUN_ID=$(npx @testomatio/reporter start --filter "testomatio:tag-name=smoke" --format id)
 ```
 
+The same output is printed by every `run` invocation that creates a run:
+
+```bash
+# create a run without executing tests
+RUN_ID=$(npx @testomatio/reporter run --format id)
+
+# trigger a run on a CI profile
+npx @testomatio/reporter run --remote github --format json
+
+# executing tests: the runner inherits stdout, so the run data is the FIRST line.
+# Capture it all, then slice — piping into `head` closes stdout under the running tests.
+OUT=$(npx @testomatio/reporter run "npx playwright test" --format json)
+RUN=$(printf '%s\n' "$OUT" | head -1)
+```
+
 ### Quick reference
 
-| Command             | Accepted `--format` values       | `stdout` contains      | Needs                |
-| ------------------- | -------------------------------- | ---------------------- | -------------------- |
-| `run --filter-list` | `ids`, `grep`, `json`, `newline` | the matching test IDs  | `--filter-list`      |
-| `start`             | any (use `id`)                   | the new run id         | —                    |
+| Command                    | Accepted `--format` values       | `stdout` contains                       |
+| -------------------------- | -------------------------------- | --------------------------------------- |
+| `run --filter-list`        | `ids`, `grep`, `json`, `newline` | the matching test IDs, no run created   |
+| `start`                    | `id`, `json`                     | the new run                             |
+| `run` (no command)         | `id`, `json`                     | the new run                             |
+| `run --remote <profile>`   | `id`, `json`                     | the run triggered on CI                 |
+| `run "<command>"`          | `id`, `json`                     | the new run on the first line, then the runner output (capture, do not pipe) |
 
 ## Environment Variables
 

docs/configuration.md

@@ -338,6 +338,22 @@ TESTOMATIO_LOG_LEVEL=ERROR npm test
 > 📖 See [Log Level Control](./log-level.md) for more details.
 > 🐛 For detailed debugging, use the `DEBUG` environment variable with the debug package specified.
 
+#### `TESTOMATIO_LOG_JSON`
+
+Print every `[TESTOMATIO]` message as a JSON object instead of prefixed text, one object per line:
+
+```json
+{"level":"error","message":"Error creating Testomat.io report ..."}
+```
+
+Failed API requests add their data as fields (`status`, `method`, `url`, `error`, `response`, `request`). The API token is hidden in every message and field, in JSON as well as in text output.
+
+```
+TESTOMATIO_LOG_JSON=1 npm test
+```
+
+The CLI sets this automatically for [`--format json`](./cli.md#the---format-flag), so both the output and the logs of a run are machine-readable.
+
 #### `TESTOMATIO_UPDATE_CODE`
 
 Sends the `code` of your tests to Testomat.io on each run. (If not enabled (default) assumes the code is pushed using [check-tests](https://github.com/testomatio/check-tests#cli)).

src/bin/cli.js

@@ -16,7 +16,7 @@ import { filesize as prettyBytes } from 'filesize';
 import dotenv from 'dotenv';
 import Replay from '../replay.js';
 import { log } from '../utils/log.js';
-import { formatFilterListIds } from '../utils/pipe_utils.js';
+import { formatFilterListIds, formatRunOutput } from '../utils/pipe_utils.js';
 import fs from 'fs';
 import path from 'path';
 
@@ -43,6 +43,8 @@ program
     if (subOpts.filterList || subOpts.format) {
       process.env.TESTOMATIO_LOG_STDERR = '1';
       process.env.TESTOMATIO_LOG_LEVEL ||= 'WARN';
+      // with --format json the logs are machine-readable too: one JSON object per line on stderr
+      if (subOpts.format === 'json') process.env.TESTOMATIO_LOG_JSON = '1';
     } else {
       console.log(pc.cyan(pc.bold(` 🤩 Testomat.io Reporter v${version}`)));
     }
@@ -53,7 +55,7 @@ program
   .description('Start a new run and return its ID')
   .option('--kind <type>', 'Specify run type: automated, manual, mixed, or detect')
   .option('--filter <filter>', 'Scope the prepared run to tests matching the filter (no execution)')
-  .option('--format <format>', 'Machine-readable output: print only the run id to stdout (e.g. --format id)')
+  .option('--format <format>', 'Machine-readable output: the run id (--format id) or run details (--format json)')
   .option('--warn', 'Exit 0 instead of 1 when the filter matches no tests (warn only)')
   .action(async opts => {
     cleanLatestRunId();
@@ -82,7 +84,7 @@ program
 
     await client.createRun(createRunParams);
 
-    const runId = client.pipeStore.runId || process.env.runId;
+    const runId = client.pipeStore.runId;
     if (!runId) {
       log.error(pc.red('Failed to create run on Testomat.io.'));
       process.exit(1);
@@ -93,8 +95,8 @@ program
     const plannedTests = (client.pipeStore.preparedTestIds || []).map(id => ({ test_id: id, title: id }));
     await client.updateRunStatus('pending', { tests: plannedTests });
 
-    // stdout carries ONLY the run id so it can be captured: RUN_ID=$(reporter start)
-    console.log(runId);
+    // stdout carries ONLY the run data so it can be captured: RUN_ID=$(reporter start)
+    console.log(formatRunOutput(client.pipeStore, opts.format));
     process.exit(0);
   });
 
@@ -129,7 +131,10 @@ program
   .argument('[command]', 'Test runner command')
   .option('--filter <filter>', 'Additional execution filter')
   .option('--filter-list <filter>', 'Get a list of all tests by filter before running')
-  .option('--format <format>', 'Machine-readable output format for --filter-list (grep, json, newline, ids)')
+  .option(
+    '--format <format>',
+    'Machine-readable output: test ids for --filter-list (grep, json, newline, ids), or the run created (id, json)',
+  )
   .option('--kind <type>', 'Specify run type: automated, manual, mixed, or detect')
   .option('--remote <profile>', 'Trigger run on the named Testomat.io CI profile instead of executing locally')
   .option(
@@ -230,6 +235,8 @@ program
 
       log.info(`🚀 CI build triggered on profile ${pc.cyan(opts.remote)}`);
       log.info(`📊 Report URL: ${pc.magenta(client.pipeStore.runUrl)}`);
+      const remoteOutput = formatRunOutput(client.pipeStore, opts.format);
+      if (opts.format && remoteOutput) console.log(remoteOutput);
       return process.exit(0);
     }
 
@@ -245,13 +252,20 @@ program
 
       if (apiKey) {
         await client.createRun(createRunParams);
-        const runId = process.env.TESTOMATIO_RUN || process.env.runId;
+
+        const runId = client.pipeStore.runId;
+        if (!runId) {
+          log.error(pc.red('Failed to create run on Testomat.io.'));
+          process.exit(1);
+        }
+
         if (client.pipeStore.runUrl) log.info( `📊 Report URL: ${pc.magenta(client.pipeStore.runUrl)}`);
 
         if (opts.kind !== 'manual') {
           log.info( `No command passed, so you need to run tests yourself:`);
           log.info( `TESTOMATIO_RUN=${runId} <command>`);
         }
+        if (opts.format) console.log(formatRunOutput(client.pipeStore, opts.format));
       } else {
         log.info( '⚠️  No API key provided. Cannot create run without TESTOMATIO key.');
         process.exit(1);
@@ -288,7 +302,11 @@ program
     }
 
     if (apiKey) {
-      await client.createRun(createRunParams).then(runTests);
+      await client.createRun(createRunParams);
+      // the runner inherits stdout, so the run data is printed first, on its own line
+      const createdOutput = formatRunOutput(client.pipeStore, opts.format);
+      if (opts.format && createdOutput) console.log(createdOutput);
+      await runTests();
     } else {
       await runTests();
     }

src/client.js

@@ -187,7 +187,7 @@ class Client {
       }
 
     } catch (err) {
-      console.error(APP_PREFIX, 'Error in uploadStepArtifacts for testRid', testRid, ':', err);
+      log.error('Error in uploadStepArtifacts for testRid', testRid, ':', err.message || err);
       throw err;
     }
   }
@@ -225,7 +225,7 @@ class Client {
     try {
       await this.uploadStepArtifacts(steps, rid);
     } catch (err) {
-      console.log(APP_PREFIX, 'Failed to upload step artifacts:', err);
+      log.error('Failed to upload step artifacts:', err.message || err);
     }
 
     const uploadedFiles = [];
@@ -423,7 +423,7 @@ class Client {
           const pathPadding = Math.max(...failedUploads.map(upload => upload.relativePath.length)) + 1;
 
           failedUploads.forEach(upload => {
-            console.log(
+            log.info(
               `  ${pc.gray('|')} 🔴 ${upload.relativePath.padEnd(pathPadding)} ${pc.gray(
                 `| ${upload.sizePretty.padStart(filesizeStrMaxLength)} |`,
               )}`,
@@ -439,7 +439,7 @@ class Client {
           }));
           const pathPadding = Math.max(...skippedUploads.map(upload => upload.relativePath.length)) + 1;
           skippedUploads.forEach(upload => {
-            console.log(
+            log.info(
               `  ${pc.gray('|')} 🟡 ${upload.relativePath.padEnd(pathPadding)} ${pc.gray(
                 `| ${upload.sizePretty.padStart(filesizeStrMaxLength)} |`,
               )}`,

src/pipe/csv.js

@@ -6,6 +6,7 @@ import pc from 'picocolors';
 import merge from 'lodash.merge';
 import { isSameTest, getCurrentDateTime, ansiRegExp } from '../utils/utils.js';
 import { CSV_HEADERS } from '../constants.js';
+import { log } from '../utils/log.js';
 
 const debug = createDebugMessages('@testomatio/reporter:pipe:csv');
 /**
@@ -76,11 +77,11 @@ class CsvPipe {
     this.checkExportDir();
 
     if (!this.outputFile) {
-      console.log(pc.yellow(`⚠️  CSV file is not set, ignoring`));
+      log.warn(pc.yellow(`⚠️  CSV file is not set, ignoring`));
       return;
     }
 
-    console.log(pc.yellow(`⏳ The test results will be added to the csv. It will take some time...`));
+    log.info(pc.yellow(`⏳ The test results will be added to the csv. It will take some time...`));
 
     try {
       // Create csv writer object
@@ -91,7 +92,7 @@ class CsvPipe {
       // Save csv file based on the current data
       return await writer.writeRecords(data);
     } catch (e) {
-      console.log('Unknown csv error: ', e);
+      log.error('Unknown csv error: ', e);
     }
   }
 
@@ -135,7 +136,7 @@ class CsvPipe {
     // Save results based on the default headers
     if (this.isEnabled) {
       await this.saveToCsv(this.results, CSV_HEADERS);
-      console.log(pc.green(`🗃️  Recording completed! You can check the result in file = ${this.outputFile}`));
+      log.info(pc.green(`🗃️  Recording completed! You can check the result in file = ${this.outputFile}`));
     }
   }