-
Notifications
You must be signed in to change notification settings - Fork 2
docs(evaluation): Usage & analytics #754
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
khushalsonawat
merged 5 commits into
docs/evaluation-revamp
from
docs/eval-explore-playground-usage
Jul 16, 2026
+39
−0
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
c337a16
docs(evaluation): Usage & analytics
b624486
docs(evaluation): address review on Usage & analytics
a8d16e2
docs(evaluation): recapture Usage tab screenshot at matching size
99f7eab
Merge branch 'docs/evaluation-revamp' into docs/eval-explore-playgrou…
khushalsonawat 90d8e90
Fixing the analytics page
khushalsonawat File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file added
BIN
+584 KB
public/images/docs/evaluation/explore-playground/usage-analytics/usage-tab.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
39 changes: 39 additions & 0 deletions
39
src/pages/docs/evaluation/guides/explore-playground/usage-analytics.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| --- | ||
| title: "Usage & analytics" | ||
| description: "See how an eval has performed over time, and dig into any single run" | ||
| --- | ||
|
|
||
| The **Usage** tab on an eval's page shows how the eval has performed over time, and lets you open any single run. A summary and chart sit at the top, and the full evaluation log runs below. | ||
|
|
||
| ## Open the eval | ||
|
|
||
| From the **Evaluations** list, open the eval you want to inspect and switch to its **Usage** tab. | ||
|
|
||
| <img src="/images/docs/evaluation/explore-playground/test-an-eval/select-eval.png" alt="The Evaluations list, searched for hallucination, with a prompt to select the eval you want to open" style={{ borderRadius: '5px' }} /> | ||
| *From the Evaluations list, open an eval to reach its Usage tab* | ||
|
|
||
| ## Pick a time range | ||
|
|
||
| A row of ranges at the top scopes everything below it: **30 mins**, **6 hrs**, **Today**, **Yesterday**, **7D**, **30D**, **3M**, **6M**, and **12M**, or **Custom** for your own dates. Narrow it to a spike you are investigating, or widen it to watch the long trend. | ||
|
|
||
| <img src="/images/docs/evaluation/explore-playground/usage-analytics/usage-tab.png" alt="The eval's Usage tab with a time-range selector, summary stats, a volume and completion-rate chart, and searchable evaluation logs below" style={{ borderRadius: '5px' }} /> | ||
| *The Usage tab: pick a time range, read the summary and chart, then dig into any run in the evaluation logs* | ||
|
|
||
| ## The summary and chart | ||
|
|
||
| For the chosen range you get the headline numbers, **Runs**, **Success**, **Errors**, and **Task Completion Rate**, alongside a chart that plots **Volume**, how many runs, against the **Task Completion Rate** over time. A rate that dips or a volume that drops is your cue to look closer. | ||
|
|
||
| ## Evaluation logs | ||
|
|
||
| Below the chart, **Evaluation Logs** lists every run the eval scored, one row each: its **Score** and **Result** (passed or failed), the **Input** it saw, the **Reason** for the verdict, the **Source** it ran under (**Playground**, **Dataset**, or **Tracer**, or the name of the project), and when it **Ran**. Use the **Search** box to find a run by a free-text match across its id, input, result, and reason. | ||
|
|
||
| ## Dive deeper | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Built-in evals" icon="tags" href="/docs/evaluation/builtin"> | ||
| The library of evals you can browse and open | ||
| </Card> | ||
| <Card title="Running evaluations" icon="play" href="/docs/evaluation/guides/running-evaluations"> | ||
| Attach an eval to your data so it starts recording usage | ||
| </Card> | ||
| </CardGroup> |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.