Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions src/pages/docs/simulation/concepts/optimization.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: "Optimization"
description: "Let an algorithm search for a better prompt when hand-fixing isn't enough"
---

## What optimization is

**Optimization** improves your agent's prompt and configuration automatically, scored by the same evals your simulation runs. Rather than hand-editing the prompt and rerunning, you let an algorithm propose and test many variations and hand you the best one. It's what Fix My Agent is built on: the diagnosis names what is wrong, and an optimization run does the rewriting.

## How a run works

An optimization run is a search for a better prompt, judged by your evals.

<Mermaid chart={`
%%{init: {"flowchart": {"curve": "linear"}}}%%
flowchart LR
IN["Simulation results<br/>+ eval scores"] --> ALG["Algorithm<br/>proposes candidates"]
ALG --> TR["Trials<br/>each candidate run"]
TR --> SC["Scores"]
SC --> BEST["Best configuration"]
`} />

Say your refund agent keeps failing a resolution eval. You feed those failing calls in, the algorithm generates candidate prompts, each one is trialed against your scenarios and scored on that same eval, and the best-performing prompt surfaces for you to review and apply. Because the score is your own eval, the winner is the prompt that best satisfies the bar you set.

## The algorithms

You pick the search strategy to match the task and budget:

- **Random Search**, quick variations for a fast baseline
- **Bayesian**, intelligent selection of few-shot examples and configurations
- **Meta-Prompt**, analyses failures and rewrites the whole prompt through deeper reasoning
- **ProTeGi**, critiques failures and applies targeted fixes with beam search
- **PromptWizard**, mutates across thinking styles, then critiques and refines the top ones
- **GEPA**, an evolutionary search for production-grade results at the highest cost

The [Optimization](/docs/optimization) product docs cover each algorithm in depth.

## Optimization and Fix My Agent

The two work together. Fix My Agent reads a run and hands you a prioritised list of issues with recommended fixes you apply by hand. An optimization run goes further and executes the improvement automatically, generating and scoring prompts until it finds a better one. Start with Fix My Agent's suggestions, and reach for optimization when manual fixes aren't enough.

## Keep exploring

<CardGroup cols={2}>
<Card title="Runs & results" icon="chart-mixed" href="/docs/simulation/concepts/runs-and-results">
The results an optimization run learns from
</Card>
<Card title="Agent definitions & versions" icon="robot" href="/docs/simulation/concepts/agent-definitions">
Save the improved prompt as a new version
</Card>
</CardGroup>