[rushilbhat]

Summary

• add a versioned JSONL trajectory manifest that preserves each trajectory's ordered prompt/output shape, delays, and explicit reset boundaries
• add FIFO rolling admission with a configurable active-agent cap and reusable routing slots for stable DP-rank balance
• generate exact-length, non-special synthetic content and expose prompt-framing calibration knobs
• record manifest identity, admission/drain metrics, and per-trajectory lifecycle JSONL
• retain the existing independently sampled workload mode and dry-run field prefix

Why

The existing sampler draws initial prompt, output, environment growth, and invocation count independently. Empirical replay preserves their within-trajectory correlations and lets us run a complete queued workload at a fixed active concurrency.

Validation

• cargo fmt --check
• cargo test (43 tests)
• cargo check --all-targets
• cargo clippy --all-targets with warnings denied (excluding two pre-existing repository lints)
• release build of batchbench-agent
• replay and legacy synthetic dry-run smokes

Accuracy boundary

This replays token shape, not original customer content. Live usage.prompt_tokens remains authoritative because chat-template/tool framing is backend-specific; the new initial/turn overhead flags allow calibration. Exact output targets likewise depend on the backend honoring min/max generation constraints.

No GPU or Modal benchmark was launched as part of this PR.

[Copilot]

Pull request overview

Adds a new trajectory replay workload mode to batchbench-agent, enabling empirical token-shape replay from a versioned JSONL manifest, plus FIFO rolling admission to cap active concurrency while preserving stable DP-rank routing balance. This extends the Rust agent benchmarker and CLI to support replay-specific calibration knobs, lifecycle reporting, and manifest identity tracking.

Changes:

• Introduces JSONL trajectory-plan parsing/validation and compiles replay specs into agent plans with explicit reset boundaries and overhead calibration.
• Implements rolling admission (active-agent cap + reusable routing slots) and records per-trajectory lifecycle metrics in the benchmark report.
• Improves synthetic text generation to target exact token counts while avoiding special tokens; adds CLI outputs/CSV fields/events JSONL to persist replay metadata.

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
rust/src/lib.rs	Re-exports new lifecycle record type from the agent module.
rust/src/agent.rs	Core implementation: trajectory JSONL replay, rolling admission, lifecycle reporting, and exact-length synthetic text generation.
rust/src/agent_cli.rs	CLI wiring for replay mode, max-active admission, manifest hashing, CSV extensions, and lifecycle JSONL output.
rust/README.md	Documents replay mode usage and rolling admission at the Rust CLI level.
rust/Cargo.toml	Adds sha2 dependency for manifest hashing.
rust/Cargo.lock	Locks new sha2 (and transitive) crates.
README.md	Adds end-to-end docs for trajectory replay, overhead calibration, rolling admission, and events output.
examples/trajectory-replay/plans.jsonl	Provides an example replay manifest with reset and delay semantics.
.gitignore	Re-includes the trajectory-replay example directory/file under examples/.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.