Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: "Test an eval"
description: "Try an eval on sample or real data and read its verdict"
---

Every eval has a **Test Evaluation** panel on its page, so you can run it on a bit of data and read the result and the reason before you attach it to anything. Open an eval from the playground, and the panel sits on the right of its **Eval Details** tab.

## Four ways to feed it data

The panel gives the eval its inputs one of four ways, each its own tab:

- **[Dataset](/docs/dataset):** pull a row from one of your datasets
- **[Tracing](/docs/observe):** pull a span or trace from a project's traces, so you test on real production data
- **[Simulation](/docs/simulation):** pull a call from a simulation
- **Custom:** paste your own test data as JSON, one value per variable the eval needs

<img src="/images/docs/evaluation/explore-playground/test-an-eval/custom-test.png" alt="The eval's Test Evaluation panel in Custom mode, with a JSON test-data editor and the input, output, and context field mapping" style={{ borderRadius: '5px' }} />
Comment thread
YashMohan23 marked this conversation as resolved.
*The Test Evaluation panel in Custom mode*

## Map the eval's inputs

An eval declares the variables it needs, and this hallucination eval needs `input`, `output`, and `context`. In **Custom** those variables are the JSON keys, so they are already mapped and you fill in the values directly. For a real data source like a **trace**, you map each variable to a field in your data, so `context` might map to a span attribute like `input.value`, because the attribute names depend on how your app is instrumented.

## Read the result

Click **Test Evaluation** and the panel returns the eval's **Result**, a pass or fail here, and an **Explanation** of why. That is your check that the eval behaves the way you meant before you wire it in anywhere.

<img src="/images/docs/evaluation/explore-playground/test-an-eval/tracing-result.png" alt="Testing the eval on a trace, showing the mapped span attributes, a Pass result, and the explanation of the verdict" style={{ borderRadius: '5px' }} />
Comment thread
YashMohan23 marked this conversation as resolved.
*Testing on a trace: the eval returns a pass or fail and explains its reasoning*

## Dive deeper

<CardGroup cols={2}>
<Card title="Running evaluations" icon="play" href="/docs/evaluation/guides/running-evaluations">
Attach an eval to your data and run it for real
</Card>
<Card title="Create a custom eval" icon="wand-magic-sparkles" href="/docs/evaluation/guides/custom-evals">
Write your own eval to test and use
</Card>
</CardGroup>