Releases: TestSprite/testsprite-cli
Releases · TestSprite/testsprite-cli
Release list
v0.3.0
[0.3.0] - 2026-07-09
Added
testsprite doctor— one-command environment diagnostic that checks your Node version, credentials, endpoint reachability, and installed agent skills, and reports what's misconfigured.test scaffold— emit a schema-correct starter plan (frontend) or a backend test skeleton to bootstrap a new test without hand-writing the JSON.test lint— offline validator for plan / steps files; catches malformed test definitions before they are sent to the server.test diff <run-a> <run-b>— compare two runs of the same test to isolate what changed between a passing and a failing run.test flaky <test-id>— repeat-run flaky-test detector. Replays a test N times (--runs, default 5), aggregates the outcomes, and reports a stability verdict (stable/flaky/failing) plus therunIdandfailureKindof every attempt that did not pass. Replays run with auto-heal off (strict verbatim) so a nondeterministic pass/fail can't be masked. Exit code is 0 only when every attempt passed, so CI can gate a merge on flakiness. Flags:--runs(1–10),--until-fail,--timeout,--output json.- JUnit XML report export for batch runs.
test run --alland batchtest rerunaccept--report junit --report-file <path>to write a CI-friendly XML sidecar after--waitpolling completes. The report is written even when the batch exits non-zero;--output jsonis unchanged;--dry-runwrites a canned sample without network calls. test waitis now variadic — pass several run ids to attach to and poll multiple runs in a single invocation.agent status— report which TestSprite skills are installed for each agent target and whether they are current; installed skills are now stamped with a version/hash marker.- New
agent installtargets: GitHub Copilot, Windsurf, and Kiro (experimental), alongside the existing Claude / Cursor / Cline / Codex / Antigravity targets. project credential/project auto-auth— configure a project's backend credentials (static credential, free) or a recurring auto-auth token (Pro) from the CLI, with surfaced auth warnings and managed-credential guidance.- Proxy support — the CLI now honors
HTTPS_PROXY/HTTP_PROXY/NO_PROXYfor use behind corporate and CI proxies. NO_COLORsupport — colored output is suppressed whenNO_COLORis set, per no-color.org.- "New version available" notice — a non-blocking, 24h-cached npm version check prints an upgrade hint on stderr. Opt out with the documented env var; automatically silenced in CI and under
--output json.
Changed
- Node.js 20.19+, 22.13+, or 24+ is now the minimum supported runtime. The CLI checks the running Node version at startup and exits with a clear message on an unsupported version; builds and CI run against Node 20 and 22.
- Graceful shutdown — the CLI handles termination signals cleanly and guards against broken-pipe (
EPIPE) errors when its output is piped to a closing consumer (e.g.| head). - Interactive prompts and preamble now go to stderr, keeping stdout pure for machine consumers even in interactive mode.
- Empty environment variables are treated as unset when resolving config, so
TESTSPRITE_API_URL=no longer overrides the built-in default with an empty string. agent installdefaults--targettoclaudein non-interactive / CI contexts (matchingsetup).- The
usagecommand no longer implies backend test runs are free. setup's "Next steps" guidance no longer suggeststest listbefore any project exists.
Fixed
- Timeouts & polling:
RequestTimeoutErroris now classified as a timeout in the--all --waitfan-out; per-attempt timeout timers are cleared so they can't fire late;run --all --waitno longer polls still-queued runs past the shared deadline; a partial result is emitted on stdout whenrun --wait/test waittimes out (so a redirected file is never zero-byte). - Batch rerun: the exit code is preserved and auth errors escalate correctly; explicit ids combined with
--all— or--status/--skip-terminalwithout--all— are rejected with a clear validation error; auto-minted idempotency keys are surfaced under--output json. - HTTP: non-JSON
200responses map to a typed error envelope instead of crashing the parser. - Failure bundles / artifacts: artifact downloads retry on transient errors and guard the default run-id path; the
--outdirectory no longer sweeps unrelated pre-existing files (data-loss fix); run-scoped per-step error text and step type are surfaced. - Input validation (fail fast, before any network call): malformed API keys, invalid
--request-timeout, directory--code-file/--outpaths, blank or whitespace-only--name(test and project create/update), blank inline project passwords, fractional pagination flags / page sizes, and--sinceoverflow are all rejected up front withVALIDATION_ERRORrather than crashing or failing late server-side.--outputis validated uniformly across all command groups. - Setup / auth: the endpoint is validated before the key check; the typed API-error envelope is preserved when key verification fails; the per-request timeout is honored during
configure. - Misc: cursor pagination no longer drops empty pages; trailing-dot hostnames are treated as loopback by the local-target guard; buffered input is preserved between interactive prompts; the Codex managed-section skill check requires a complete section;
code getstrips a leading BOM and rejects an empty--out.
Security
- INI injection: CR/LF characters are stripped from credential values before they are written to
~/.testsprite/credentials. - Symlink fail-close: the own-file
agent installpath applies its symlink containment guard under--dry-runas well, so a planted symlink cannot place or clobber files outside--dir.
v0.2.0
[0.2.0] - 2026-06-29
Added
- Seed-suite onboarding skill.
agent installnow installs a second skill by default —testsprite-onboard— which guides your coding agent to create a first test suite in a repository that doesn't have one yet (alongside the existingtestsprite-verifyskill). Useagent install --skill <name>to install only a specific subset.
Changed
test resultnow reports the test verdict and execution status as separate fields. The latest-result output gains averdictfield (the run's pass / fail / blocked judgement) and anexecutionStatusfield (how the run terminated);summaryis now a human-readable string.- Breaking change to
--output json:summarywas previously an object ({ passed, failed, skipped }). Scripts that readsummary.passed/.failed/.skippedmust move to the newverdict/executionStatusfields. The legacystatusfield is unchanged.
- Breaking change to
project create,project update, andtest run --allnow print the[dry-run] sample response — not from the serverbanner under--dry-run, consistent with every other command.- The CLI no longer suggests TypeScript/JavaScript test code is supported — TestSprite runs all test code as Python.
Fixed
- Security (failure-bundle writer):
test failure getandtest artifact getnow validate the response's step index and evidence kind before composing file paths, so a malformed or hostile API response can no longer write files outside the chosen--outdirectory. test create --type backendandtest code putnow reject a non-Python--code-fileimmediately with a clear validation error, instead of failing late server-side.test failure get --outis validated before the network call;test code get --outnow writes atomically (no truncation if the fetch fails); artifact downloads retry on transient transport errors.- Clearer validation messages for malformed
--endpoint-urlvalues and profile names; argument-parse errors now emit a structured JSON envelope under--output json. - Batch rerun dispatch is de-duplicated and serialized.
v0.1.2
v0.1.2
testsprite setup— onboarding is consolidated into one command (formerlyinit). The granular commands stay as hidden, deprecated aliases (auth configure→setup,auth whoami→auth status,auth logout→auth remove).- The CLI now reports its version in the
User-Agentheader. - README: the launch video no longer renders as a bare URL on the npm page.
v0.1.1 — README launch-video patch
Docs-only patch release: the npm 0.1.0 package was published before the README launch video was updated, so the npm page embedded the old video. npm READMEs are immutable per version — this release refreshes it. No code changes; the package file list is identical to 0.1.0.
Changed
- README: point the launch video at the updated public asset
- Prettier-clean the README video block (fixes
format:checkonmain)
Note: this version was published to npm manually (
npm publishfrom a maintainer machine, same as0.1.0). Thereleaseworkflow run attached to this tag is expected to fail at the publish step — the repo has noNPM_TOKENsecret configured yet. Full changelog:CHANGELOG.md.