Skip to content

Add sanitized fixture record and replay - #77

Merged
clifton merged 1 commit into
mainfrom
codex/record-sanitize-replay-fixtures
Aug 4, 2026
Merged

clifton merged 1 commit into
mainfrom
codex/record-sanitize-replay-fixtures

Conversation

@clifton

@clifton clifton commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a versioned Fixture format and an LLMClient-compatible FixtureRecorder<C> for non-streaming calls
  • require a caller-supplied sanitizer before any request or response is retained, structurally redact common credential fields, and never persist inline media bytes
  • add strict ordered ReplayClient matching for operation, sanitized prompt, target schema, and media metadata
  • preserve token usage, complete extraction reports, typed errors, HTTP status, and request IDs across record/replay
  • add a runnable key-free example and a checked-in real-world Apple 2023 10-K fixture

Behavior

Replay mismatches report only the differing field and leave the interaction unconsumed. ReplayClient::assert_finished() catches tests that skipped recorded calls. Fixture JSON is pretty, deterministic, version-checked, and available through both file and string APIs.

The mock feature remains dependency-light and owns the fixture API; provider HTTP dependencies are not required for replay.

Verification

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features --target-dir tmp/pr4-all-target -- -D warnings
  • cargo check --no-default-features --features derive,mock --target-dir tmp/pr4-min-target
  • all-feature library suite: 312 passed
  • fixture integration suite: 2 passed, including canonical real-world replay and schema drift
  • deterministic all-feature integration matrix (credentialed live-provider modules skipped)
  • doctests: 84 passed
  • commit-hook default-feature unit suite: 219 passed

@clifton
clifton force-pushed the codex/observable-transport-telemetry branch from bc5be71 to 31e8628 Compare August 4, 2026 18:10
@clifton
clifton force-pushed the codex/record-sanitize-replay-fixtures branch from 90a315f to 3b50e19 Compare August 4, 2026 18:11
@clifton
clifton force-pushed the codex/observable-transport-telemetry branch from 31e8628 to abbb17d Compare August 4, 2026 18:30
Base automatically changed from codex/observable-transport-telemetry to main August 4, 2026 18:33
@clifton
clifton marked this pull request as ready for review August 4, 2026 18:35
@clifton
clifton force-pushed the codex/record-sanitize-replay-fixtures branch from 3b50e19 to 07c6c6d Compare August 4, 2026 18:35
@clifton
clifton merged commit d49ee50 into main Aug 4, 2026
9 checks passed
@clifton
clifton deleted the codex/record-sanitize-replay-fixtures branch August 4, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant