diff --git a/src/pages/docs/simulation/concepts/optimization.mdx b/src/pages/docs/simulation/concepts/optimization.mdx new file mode 100644 index 00000000..8c32e006 --- /dev/null +++ b/src/pages/docs/simulation/concepts/optimization.mdx @@ -0,0 +1,51 @@ +--- +title: "Optimization" +description: "Let an algorithm search for a better prompt when hand-fixing isn't enough" +--- + +## What optimization is + +**Optimization** improves your agent's prompt and configuration automatically, scored by the same evals your simulation runs. Rather than hand-editing the prompt and rerunning, you let an algorithm propose and test many variations and hand you the best one. It's what Fix My Agent is built on: the diagnosis names what is wrong, and an optimization run does the rewriting. + +## How a run works + +An optimization run is a search for a better prompt, judged by your evals. + ++ eval scores"] --> ALG["Algorithm
proposes candidates"] + ALG --> TR["Trials
each candidate run"] + TR --> SC["Scores"] + SC --> BEST["Best configuration"] +`} /> + +Say your refund agent keeps failing a resolution eval. You feed those failing calls in, the algorithm generates candidate prompts, each one is trialed against your scenarios and scored on that same eval, and the best-performing prompt surfaces for you to review and apply. Because the score is your own eval, the winner is the prompt that best satisfies the bar you set. + +## The algorithms + +You pick the search strategy to match the task and budget: + +- **Random Search**, quick variations for a fast baseline +- **Bayesian**, intelligent selection of few-shot examples and configurations +- **Meta-Prompt**, analyses failures and rewrites the whole prompt through deeper reasoning +- **ProTeGi**, critiques failures and applies targeted fixes with beam search +- **PromptWizard**, mutates across thinking styles, then critiques and refines the top ones +- **GEPA**, an evolutionary search for production-grade results at the highest cost + +The [Optimization](/docs/optimization) product docs cover each algorithm in depth. + +## Optimization and Fix My Agent + +The two work together. Fix My Agent reads a run and hands you a prioritised list of issues with recommended fixes you apply by hand. An optimization run goes further and executes the improvement automatically, generating and scoring prompts until it finds a better one. Start with Fix My Agent's suggestions, and reach for optimization when manual fixes aren't enough. + +## Keep exploring + + + + The results an optimization run learns from + + + Save the improved prompt as a new version + +