Skip to content

Add retrieval telemetry imports for evaluations#341

Merged
placerda merged 2 commits into
developfrom
feature/retrieval-telemetry-contract
Jun 26, 2026
Merged

Add retrieval telemetry imports for evaluations#341
placerda merged 2 commits into
developfrom
feature/retrieval-telemetry-contract

Conversation

@placerda

Copy link
Copy Markdown
Contributor

Summary

  • Adds the telemetry_imports configuration contract and agentops telemetry validate, preview, and import commands.
  • Supports dataset-backed response reuse with response_source: dataset.
  • Adds grey-box HTTP response field mapping from $response.context for retrieval telemetry.
  • Updates evaluation documentation and changelog coverage for the retrieval telemetry import workflow.

Validation

  • git diff --check
  • PYTHONPATH=src python -m pytest tests/unit/test_agentops_config.py tests/unit/test_cli_commands.py tests/unit/test_http_response_fields.py tests/unit/test_runtime_dataset_response_source.py tests/unit/test_telemetry_import.py -q (77 passed)

Docs

  • Updated README, concepts, evaluation docs, Foundry evaluator guide, and CHANGELOG.md.

Refs #145

placerda and others added 2 commits June 26, 2026 13:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@placerda placerda merged commit 41dce82 into develop Jun 26, 2026
12 checks passed
@placerda placerda deleted the feature/retrieval-telemetry-contract branch June 26, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant