fix(cli,minimize): validate --mode upfront and reject invalid modes (exit 2)

README.md

@@ -31,7 +31,7 @@ If you're writing evals, Stepfork complements them: evals measure model quality
 </p>
 
 <p align="center">
-  Install the alpha: <code>pip install --pre stepfork</code>
+  Install the current release: <code>pip install stepfork</code>
 </p>
 
 Product goal:
@@ -127,11 +127,10 @@ pip install --pre stepfork
 uv pip install --pre stepfork
 ```
 
-The `--pre` flag is required while Stepfork is a pre-release. You can also
-install from the tagged GitHub repository:
+You can also install from source or a tagged release:
 
 ```bash
-pip install "git+https://github.com/utsab345/stepfork.git@v0.1.0a6"
+pip install "git+https://github.com/utsab345/stepfork.git@v0.2.0"
 ```
 
 Requires Python 3.11, 3.12, or 3.13. This installs the `stepfork` CLI and the
@@ -325,9 +324,12 @@ business outcome - it lives in `expected.json`, defined by a human.
 |---|---|
 | `stepfork validate [PATH] [--partial] [--verify-integrity]` | Check a `.sftrace` bundle structurally (and its integrity). |
 | `stepfork inspect [PATH] [--json] [--events] [--errors-only] [--step N]` | Review a bundle locally. |
-| `stepfork replay [PATH] --entrypoint MODULE:FUNCTION [--mode M]` | Rerun the entrypoint with dependencies answered from the trace. |
+| `stepfork replay [PATH] --entrypoint MODULE:FUNCTION [--mode M] [--allow-live-tool NAME] [--verbose]` | Rerun the entrypoint with dependencies answered from the trace. |
+| `stepfork minimize [PATH] --entrypoint MODULE:FUNCTION [--expect-output FILE] [--mode M] [--output PATH] [--overwrite] [--timeout SECONDS] [--attempts N]` | Minimize a failing trace to the smallest reproducible subset. |
+| `stepfork fork [PATH] [--at-step N \| --at-event ID] [--output PATH] [--overwrite]` | Fork a trace at a specific step (Python API for overrides). |
+| `stepfork score [TEST] [--expect-output PATH] [--bundle PATH] [--json]` | Score an exported expectation against a mutant battery. |
 | `stepfork diff [BASELINE] [CANDIDATE] [--json]` | Compare the behavior of two runs. |
-| `stepfork export [PATH] --pytest --entrypoint MODULE:FUNCTION [--expect-output JSON] [--expect-trajectory JSON] [--output FILE] [--overwrite]` | Generate an executable pytest regression test. |
+| `stepfork export [PATH] --pytest --entrypoint MODULE:FUNCTION [--expect-output JSON] [--expect-trajectory JSON] [--output FILE] [--overwrite] [--mode M] [--allow-live-tool NAME]` | Generate an executable pytest regression test. |
 
 The full reference (arguments, examples, expected output, and common errors)
 is in [docs/cli.md](docs/cli.md).
@@ -379,16 +381,16 @@ sources live in `docs/`.
 - [Release checklist](docs/release-checklist.md)
 - [PyPI publishing checklist](docs/pypi-publishing.md)
 - [Early adopter guide](docs/community/early-adopter-guide.md)
-- [Latest release notes (v0.1.0a6)](docs/releases/v0.1.0a6.md)
+- [Latest release notes (v0.2.0)](docs/releases/v0.2.0.md)
 - [Changelog](CHANGELOG.md)
 
 ## Security and Limitations
 
 Stepfork is experimental. The API, CLI, and `.sftrace` trace format may change
 before v1.0.
 
-- **Alpha on PyPI.** `pip install --pre stepfork` installs the current
-  pre-release; the API and trace format may still change.
+- **Beta status.** `pip install stepfork` installs the current release; the API
+  and trace format may still change before v1.0.
 - **Limited integrations.** Optional adapters cover the OpenAI Python SDK
   (synchronous and asynchronous, non-streaming `chat.completions.create`) and
   LangGraph (synchronous and asynchronous `BaseChatModel` generation and

docs/cli.md

@@ -1,7 +1,7 @@
 # CLI Reference
 
 The `stepfork` CLI manages `.sftrace` bundles. This page documents every
-command as of v0.1.0a6. Run `stepfork --help` for an up-to-date summary.
+command as of v0.2.0. Run `stepfork --help` for an up-to-date summary.
 
 ```text
 Usage: stepfork [OPTIONS] COMMAND [ARGS]...
@@ -22,6 +22,8 @@ Commands:
 - [inspect](#inspect)
 - [replay](#replay)
 - [minimize](#minimize)
+- [fork](#fork)
+- [score](#score)
 - [diff](#diff)
 - [export](#export)
 
@@ -261,7 +263,7 @@ Options:
 | Option | Description |
 |---|---|
 | `--expect-output FILE` | JSON file with expected output for comparison. |
-| `--mode`, `-m` | Replay mode: `frozen`, `live`, or `hybrid`. Default: `frozen`. |
+| `--mode`, `-m` | Replay mode: `frozen`, `live`, `forbidden`, `manual`, or `derived`. Default: `frozen`. |
 | `--output`, `-o` | Path for the minimized `.sftrace` bundle. Default: `<path>-minimized.sftrace`. |
 | `--overwrite` | Overwrite the output bundle if it exists. Default: `false`. |
 | `--timeout SECONDS` | Timeout in seconds for each replay attempt. Default: `30.0`. |

docs/community/early-adopter-guide.md

@@ -32,14 +32,14 @@ Stepfork is on PyPI. Install the current alpha:
 ```bash
 python -m venv .venv
 source .venv/bin/activate
-pip install --pre stepfork
+pip install stepfork
 pip install pytest
 stepfork --version
 ```
 
 The `--pre` flag is required while Stepfork is a pre-release. Alternatively,
 install the tagged repository build:
-`pip install "git+https://github.com/utsab345/stepfork.git@v0.1.0a6"`.
+`pip install "git+https://github.com/utsab345/stepfork.git@v0.2.0"`.
 
 Requires Python 3.11, 3.12, or 3.13.
 

docs/getting-started.md

@@ -9,14 +9,14 @@ This guide installs Stepfork and walks through the full workflow with a
 Install the current alpha from PyPI:
 
 ```bash
-pip install --pre stepfork
+pip install stepfork
 ```
 
 The `--pre` flag is required while Stepfork is a pre-release. To install from
 the tagged GitHub repository instead:
 
 ```bash
-pip install "git+https://github.com/utsab345/stepfork.git@v0.1.0a6"
+pip install "git+https://github.com/utsab345/stepfork.git@v0.2.0"
 ```
 
 Requires Python 3.11, 3.12, or 3.13. This installs the `stepfork` CLI and the

docs/index.md

@@ -73,4 +73,4 @@ in this repository.
 
 Stepfork is experimental. See the
 [changelog](https://github.com/utsab345/stepfork/blob/main/CHANGELOG.md) and
-the [v0.1.0a6 release notes](releases/v0.1.0a6.md) for what shipped.
+the [v0.2.0 release notes](releases/v0.2.0.md) for what shipped.

docs/minimize.md

@@ -85,7 +85,7 @@ stepfork minimize <trace> --entrypoint MODULE:FUNCTION [OPTIONS]
 | Option | Description | Default |
 |--------|-------------|---------|
 | `--expect-output FILE` | JSON file with expected output for comparison | None |
-| `--mode`, `-m` | Replay mode: `frozen`, `live`, or `hybrid` | `frozen` |
+| `--mode`, `-m` | Replay mode: `frozen`, `live`, `forbidden`, `manual`, or `derived` | `frozen` |
 | `--output`, `-o` | Path for the minimized `.sftrace` bundle | `<trace>-minimized.sftrace` |
 | `--overwrite` | Overwrite the output bundle if it exists | `false` |
 | `--timeout SECONDS` | Timeout for each replay attempt | `30.0` |

docs/pytest-export.md

@@ -9,6 +9,32 @@ A trace records what the agent *did*, including the bug. If you generate a test
 that asserts the recorded output, it passes against the very code that produced
 the recording, which is not a regression test.
 
+## Frozen-replay contract and limitations
+
+Stepfork's pytest export uses frozen replay: the generated test executes your
+trusted entrypoint with instrumented tool and LLM calls answered from the recorded
+trace. The contract is strict to preserve determinism and prevent silent
+acceptance of unexpected behavior.
+
+**Supported when:** the fixed implementation makes the same instrumented calls in
+the same order with the same inputs as recorded in the failure trace. The
+regression test can then assert the corrected output against the trace.
+
+**Not supported when:** the fix legitimately changes later tool arguments,
+invokes a tool not present in the recorded trace, or omits/moves recorded calls.
+In these cases, frozen replay will raise a divergence (e.g., `ReplayMismatchError`
+for changed inputs, or `ReplayExhaustedError`/unmatched calls for sequence
+changes). The test generated from the failure trace **cannot pass** on the fixed
+code because replay cannot satisfy the recorded sequence.
+
+When a fix changes the call sequence or arguments, record the *fixed* run and
+export a test from that trace instead. The exported test will fail on the buggy
+code (by replay divergence) and pass on the fixed code. `stepfork score` can help
+assess whether an expectation is claim-pinned (rejects the recorded run) or
+run-pinned (accepts it).
+
+---
+
 The regression test needs a separate **expected outcome**: the behavior you want
 after the bug is fixed. Pass it with `--expect-output`.
 

docs/releases/v0.2.0.md

@@ -0,0 +1,46 @@
+# Stepfork v0.2.0 - Behavioral Replay
+
+This current release adds trajectory assertions, async instrumented boundaries, and an
+opt-in hybrid mode for selected live LLM calls. Existing `.sftrace` bundles
+remain compatible; frozen replay remains the default.
+
+## Changes
+
+- Assert tool calls, absence, counts, JSON arguments, order, sequences, and a
+  maximum dependency-step count against the execution that actually replayed.
+  Generated pytest tests can include a reviewed JSON trajectory expectation.
+- Record and frozen-replay async tools, LLM requests, and agent entrypoints.
+  Supported OpenAI and LangGraph adapters now include async paths.
+- Re-execute selected LLMs in hybrid replay while recorded tools stay frozen.
+  The [offline model-decision example](https://github.com/utsab345/stepfork/tree/v0.2.0/examples/model_decision)
+  uses a deterministic fake provider and shows a failing regression before a
+  prompt fix and a passing one afterward. It does not establish real-model
+  quality improvement.
+- Live tool execution now requires `allow_live_tools={"name"}` or
+  `--allow-live-tool name`. Previously, `mode="live"` ran every matching tool.
+  See [migration guidance](../replay.md#modes).
+
+Hybrid replay is **not deterministic**. Selected live models can incur network
+calls, token costs, and latency. The tool allowlist controls instrumented tool
+bodies by name; it does not sandbox arbitrary code or uninstrumented side
+effects. Concurrent async calls still match one strict invocation sequence,
+while task scheduling remains outside Stepfork's control.
+
+## Installation
+
+```bash
+pip install stepfork
+```
+
+Requires Python 3.11, 3.12, or 3.13.
+
+## Verification
+
+- 444 tests passed on Python 3.11, 3.12, and 3.13. Python 3.12 combined
+  statement/branch coverage was 92%.
+- Ruff, MyPy, strict documentation builds, six offline demos, source and wheel
+  builds, and an installed-wheel smoke of the new APIs passed locally.
+
+## Feedback
+
+https://github.com/utsab345/stepfork/issues

src/stepfork/cli/minimize.py

@@ -49,7 +49,7 @@ def minimize_command(
         typer.Option(
             "--mode",
             "-m",
-            help="Replay mode: frozen (default), live, or hybrid",
+            help="Replay mode: frozen (default), live, forbidden, manual, or derived",
         ),
     ] = "frozen",
     output: Annotated[
@@ -126,6 +126,13 @@ def minimize_command(
 
     # Run minimization
     console.print("[yellow]Detecting failure signature...[/yellow]")
+    try:
+        from stepfork.trace.replay_policy import ReplayPolicy
+
+        ReplayPolicy(mode)
+    except ValueError:
+        console.print(f"[red]Invalid replay mode {mode!r}.[/red]")
+        raise typer.Exit(2) from None
     engine = MinimizationEngine(
         trace,
         entrypoint,

src/stepfork/minimize/engine.py

@@ -88,6 +88,20 @@ def minimize(self) -> MinimizationResult:
         """
         # Step 1: Detect the original failure signature
         try:
+            try:
+                from stepfork.trace.replay_policy import ReplayPolicy
+
+                ReplayPolicy(self.mode)
+            except ValueError as exc:
+                return MinimizationResult(
+                    success=False,
+                    original_event_count=len(self.trace.events),
+                    minimized_event_count=None,
+                    replay_attempts=self.total_replay_attempts,
+                    preserved_failure=False,
+                    is_minimal=False,
+                    error_message=f"Invalid replay mode {self.mode!r}: {exc}",
+                )
             original_signature = detect_failure_signature(
                 self.trace,
                 self.entrypoint,

tests/unit/test_minimize_mode_validation.py

@@ -0,0 +1,92 @@
+"""Unit tests for minimize mode validation."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from typer.testing import CliRunner
+
+from stepfork import record, trace_tool
+from stepfork.cli.main import app
+
+runner = CliRunner()
+
+
+@trace_tool(name="simple_tool")
+def simple_tool(x: int) -> int:
+    """Simple test tool."""
+    return x * 2
+
+
+def simple_agent() -> dict[str, int]:
+    """Simple test agent."""
+    result = simple_tool(5)
+    return {"value": result}
+
+
+def test_minimize_rejects_hybrid_mode(tmp_path: Path) -> None:
+    """Minimize rejects unsupported 'hybrid' mode."""
+    trace_path = tmp_path / "test.sftrace"
+    with record("cli-test-agent", output=str(trace_path)) as session:
+        result = simple_agent()
+        session.set_output(result)
+
+    result = runner.invoke(
+        app,
+        [
+            "minimize",
+            str(trace_path),
+            "--entrypoint",
+            "test_minimize_mode_validation:simple_agent",
+            "--mode",
+            "hybrid",
+        ],
+    )
+    assert result.exit_code == 2
+    assert "Invalid replay mode" in result.stdout
+
+
+def test_minimize_rejects_bogus_mode(tmp_path: Path) -> None:
+    """Minimize rejects invalid mode values."""
+    trace_path = tmp_path / "test.sftrace"
+    with record("cli-test-agent", output=str(trace_path)) as session:
+        result = simple_agent()
+        session.set_output(result)
+
+    result = runner.invoke(
+        app,
+        [
+            "minimize",
+            str(trace_path),
+            "--entrypoint",
+            "test_minimize_mode_validation:simple_agent",
+            "--mode",
+            "bogus",
+        ],
+    )
+    assert result.exit_code == 2
+    assert "Invalid replay mode" in result.stdout
+
+
+def test_minimize_accepts_valid_modes(tmp_path: Path) -> None:
+    """Valid modes are accepted by minimize."""
+    trace_path = tmp_path / "test.sftrace"
+    with record("cli-test-agent", output=str(trace_path)) as session:
+        result = simple_agent()
+        session.set_output(result)
+
+    for mode in ("frozen", "live", "forbidden", "manual", "derived"):
+        result = runner.invoke(
+            app,
+            [
+                "minimize",
+                str(trace_path),
+                "--entrypoint",
+                "test_minimize_mode_validation:simple_agent",
+                "--mode",
+                mode,
+            ],
+        )
+        # Valid modes may still fail minimization if trace doesn't fail as expected,
+        # but should not be rejected with exit 2 for invalid mode
+        assert result.exit_code != 2, f"Mode {mode} was incorrectly rejected"