docs: add MkDocs docs site, comparison page, and SEO metadata

.github/workflows/docs.yml

@@ -0,0 +1,48 @@
+name: Docs
+
+# Build the MkDocs Material site and publish it to the `gh-pages` branch via
+# `mkdocs gh-deploy` — the same mechanism UiPath/uipath-python uses. This workflow
+# only pushes a branch (needs `contents: write`); it never calls the Pages API, so
+# it succeeds even before Pages is switched on.
+#
+# One-time setup (org owner): the UiPath org blocks Pages *creation*, so enable
+# Pages for this repo once — Settings → Pages → Build and deployment →
+# Source: "Deploy from a branch" → branch `gh-pages` / `/ (root)`. That's the same
+# (legacy, branch-based) configuration uipath-python already runs on.
+on:
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+      - ".github/workflows/docs.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: docs-gh-pages
+  cancel-in-progress: true
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+      - name: Install social-card system libraries
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends \
+            libcairo2 libfreetype6 libjpeg-turbo8 libpng16-16 pngquant
+      - name: Install MkDocs Material
+        run: pip install "mkdocs-material[imaging]>=9.5,<10"
+      - name: Configure git identity for gh-pages commits
+        run: |
+          git config user.name "coder-eval"
+          git config user.email "coder-eval@uipath.com"
+      - name: Build strictly and publish to gh-pages
+        run: mkdocs gh-deploy --force --strict

.gitignore

@@ -1,3 +1,7 @@
+# MkDocs build output
+site/
+.cache/
+
 # Python-generated files
 __pycache__/
 *.py[oc]

README.md

@@ -1,25 +1,33 @@
-# coder_eval — evaluate AI coding agents & their skills
+# Coder Eval — evaluate & benchmark AI coding agents and Claude Code skills
 
 [![PyPI](https://img.shields.io/pypi/v/coder-eval.svg)](https://pypi.org/project/coder-eval/)
+[![Docs](https://img.shields.io/badge/docs-uipath.github.io%2Fcoder__eval-1f6feb.svg)](https://uipath.github.io/coder_eval/)
 [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
 [![Python 3.13+](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://www.python.org/downloads/)
 [![CI](https://github.com/UiPath/coder_eval/actions/workflows/pr-checks.yml/badge.svg)](https://github.com/UiPath/coder_eval/actions/workflows/pr-checks.yml)
 [![Code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 
-A framework for evaluating AI coding agents **and their skills** — built for CLI
+**Coder Eval** (`pip install coder-eval` / `uv tool install coder-eval`) is an open-source framework for
+**evaluating and benchmarking AI coding agents and their skills** — built for CLI
 and skill builders — with sandboxing, reproducibility, and data-driven analysis.
-Not an "agentic coding" benchmark: it measures how effective your CLI and skills
-are when used by coding agents.
+It runs a real agent (**Claude Code**, **Codex**, or **Google Antigravity /
+Gemini**) in a sandbox against declarative YAML tasks, then scores the files and
+commands it actually produced. Not an "agentic coding" benchmark: it measures how
+effective your CLI and skills are when used by coding agents.
+
+Reach for it when you want to **test whether a Claude Code skill triggers**,
+**A/B-test Claude Code vs. Codex vs. Gemini** (or model vs. model, prompt vs.
+prompt), or **gate CI on coding-agent quality**. Unlike fixed datasets (SWE-bench,
+SkillsBench) that rank models on a shared leaderboard, Coder Eval evaluates the
+tasks, skills, and workflows *you* ship — with weighted 0.0–1.0 criteria, a
+`skill_triggered` activation check, an A/B experiment layer, and per-tool cost
+telemetry. See [How it compares](https://uipath.github.io/coder_eval/comparison/).
+📚 **Full docs:** **[uipath.github.io/coder_eval](https://uipath.github.io/coder_eval/)**.
 
 <p align="center">
-  <img src="docs/assets/hero.gif" alt="coder_eval running the hello_date task: a sandboxed agent writes and runs a script from a YAML task, then the scored result is browsed in evalboard" width="100%">
+  <img src="docs/assets/hero.gif" alt="Coder Eval running the hello_date task: a sandboxed agent writes and runs a script from a YAML task, then the scored result is browsed in evalboard" width="100%">
 </p>
 
-> **The Coding Agents Gym.** A sandboxed, reproducible framework to evaluate,
-> benchmark, and A/B-test AI coding agents — Claude Code, Codex, and Google
-> Antigravity (Gemini) today, any agent via a plugin SPI — with declarative
-> YAML tasks and weighted scoring.
-
 - **Declarative YAML tasks** with pinned dependencies and clear success criteria
 - **Sandboxed execution** in isolated environments with resource limits
 - **Weighted, continuous scoring** (0.0–1.0) with fractional credit and thresholds
@@ -30,14 +38,14 @@ are when used by coding agents.
 
 ## What you can do with it
 
-- **Benchmark coding agents** — score an agent across a suite of tasks with weighted, pass/fail thresholds
+- **Benchmark coding agents** — score an agent across a suite of tasks with weighted scoring and pass/fail thresholds
 - **Compare models & configs** — A/B-test Claude vs. Codex vs. Gemini, model vs. model, tool-on vs. tool-off, prompt vs. prompt
 - **Evaluate skills** — verify an agent actually engages a target skill (`skill_triggered`) and score skill-driven suites (SkillsBench-style)
 - **Keep skills up to date in CI** — re-validate your skills on every change or on a schedule; catch silent regressions when models, prompts, or the skills themselves drift
 - **Gate CI on agent quality** — run the suite in GitHub Actions and fail the build on regressions
 - **Bring your own dataset** — fan one task out over many rows for larger benchmark suites
 
-> **Keeping skills fresh?** Run coder_eval as a scheduled GitHub Actions job so your
+> **Keeping skills fresh?** Run Coder Eval as a scheduled GitHub Actions job so your
 > skills are continuously re-evaluated against the latest model — a skill that quietly
 > stops triggering surfaces as a failing criterion before your users hit it. See
 > **[Tutorial 02 — Running coder_eval in CI](docs/tutorials/02-ci-pipeline.md)**.
@@ -53,7 +61,9 @@ git clone https://github.com/UiPath/coder_eval.git
 cd coder_eval
 
 uv sync --extra dev          # install core + dev tools
-cp .env.example .env         # then set ANTHROPIC_API_KEY
+cp .env.example .env         # then set ANTHROPIC_API_KEY — or skip that: an
+                             # existing Claude Code login (`claude login`) is
+                             # picked up automatically
 
 uv run coder-eval plan tasks/hello_date.yaml   # validate (no tokens spent)
 uv run coder-eval run  tasks/hello_date.yaml   # run your first evaluation
@@ -67,11 +77,23 @@ The optional `[uipath]` extra (`uv sync --extra dev --extra uipath`) adds the in
 required). Without it the framework runs end-to-end; uipath-dependent features fail
 at dispatch with a clear hint.
 
-> **Using coder_eval in CI or another project?** Install the published package:
-> `pip install coder-eval` (or `uv add coder-eval`; extras install the same way —
-> `pip install "coder-eval[codex,antigravity]"`). In a real CI gate, pin to a
-> specific released version so a harness upgrade can't silently move your results.
-> See [Tutorial 02 — Running coder_eval in CI](docs/tutorials/02-ci-pipeline.md) for the full setup.
+**Using Coder Eval in CI or another project?** Install the published package
+instead of cloning:
+
+```bash
+uv tool install coder-eval    # puts the `coder-eval` CLI on your PATH,
+                              # in its own isolated environment
+
+uv tool install "coder-eval[codex,antigravity]"   # same, with agent extras
+coder-eval --version                              # verify the install
+```
+
+To add it as a project dependency instead: `uv add coder-eval` or
+`pip install coder-eval`. In a real CI gate, pin to a specific released version
+so a harness upgrade can't silently move your results. (The example `tasks/`
+live in this repo — clone it or point the CLI at your own task files.) See
+[Tutorial 02 — Running coder_eval in CI](docs/tutorials/02-ci-pipeline.md) for
+the full setup.
 
 ## Telemetry
 
@@ -99,17 +121,18 @@ at dispatch with a clear hint.
 
 ## How it compares
 
-- **vs. SWE-bench and fixed benchmarks** — SWE-bench is a fixed dataset of GitHub
-  issues; coder_eval is a *framework* for authoring your own tasks in declarative
-  YAML, so you evaluate the skills and workflows you care about (and can still wrap
-  a fixed dataset via [Bring Your Own Dataset](docs/BYOD.md)).
-- **vs. LLM-output eval harnesses (e.g. OpenAI Evals)** — those grade a model's text;
-  coder_eval runs a full **agent** in a **sandbox** with real tool use and multi-turn
-  dialog, then scores the files and commands it actually produced (continuous
-  0.0–1.0) — not just a judge over a string.
+- **vs. fixed benchmarks (SWE-bench, SkillsBench)** — they score a canonical dataset;
+  Coder Eval scores *your* tasks with continuous 0.0–1.0 weighted criteria (and can
+  still wrap a fixed dataset via [Bring Your Own Dataset](docs/BYOD.md)).
+- **vs. large-scale / RL harnesses (Harbor)** — Harbor targets scale and RL rollouts;
+  Coder Eval targets weighted, skill-aware suites gated in CI.
+- **vs. model-output eval tools (OpenAI Evals)** — they grade model text; Coder Eval
+  runs a full agent in a sandbox and scores the files and commands it produced.
 - **vs. hand-rolled scripts** — reproducible sandboxes, weighted criteria,
   cost/token telemetry, A/B experiments, and CI-ready pass/fail gates out of the box.
 
+See the full [comparison — with sources](https://uipath.github.io/coder_eval/comparison/).
+
 ## Task Definition
 
 A task is a YAML file: a prompt, the agent config, a sandbox, and success criteria.
@@ -159,12 +182,12 @@ extension points (new criteria, new agents).
 
 ## Known limits & non-goals
 
-- **Not a fixed benchmark or leaderboard** — coder_eval scores *your* tasks and ships
+- **Not a fixed benchmark or leaderboard** — Coder Eval scores *your* tasks and ships
   example tasks, not a canonical scored dataset.
 - **Tasks execute real code** — run untrusted tasks only under the container driver
   (see [Docker Isolation](docs/DOCKER_ISOLATION.md)); the `tempdir` driver is not a
   security boundary.
-- **Bring your own model credentials** — Anthropic, Bedrock, or Gemini keys; coder_eval
+- **Bring your own model credentials** — Anthropic, Bedrock, or Gemini keys; Coder Eval
   does not proxy or supply model access.
 - **Python 3.13+ only.**
 

docs/AB_EXPERIMENTS.md

@@ -1,3 +1,10 @@
+---
+description: >-
+  A/B-test AI coding agents with coder_eval's experiment layer — Claude Code vs.
+  Codex vs. Gemini, model vs. model, skill on vs. off, prompt vs. prompt — on
+  identical tasks.
+---
+
 # A/B Experiment Guide
 
 How to run the same tasks across multiple configuration variants ("arms") and

docs/BYOD.md

@@ -1,3 +1,10 @@
+---
+description: >-
+  Use a custom Docker image with coder_eval — extend the base coder-eval-agent
+  image with your own dependencies and tools, then point task configuration at
+  it.
+---
+
 # Bring Your Own Docker (BYOD)
 
 The BYOD feature allows customers to use custom Docker images that extend the base `coder-eval-agent` image, enabling them to add custom dependencies and tools while maintaining the latest coder-eval codebase.

docs/CODEX_AGENT_GUIDE.md

@@ -1,8 +1,15 @@
-# Codex Agent Implementation
+---
+description: >-
+  Run OpenAI Codex as the agent under evaluation in coder_eval — installation,
+  authentication, task configuration, and how Codex telemetry maps to sandboxed,
+  weighted scoring.
+---
+
+# Running OpenAI Codex in coder_eval
 
 ## Overview
 
-A new `CodexAgent` has been added to coder_eval that integrates OpenAI's Codex SDK. The implementation mirrors the structure of `ClaudeCodeAgent` and provides seamless integration with the evaluation framework.
+coder_eval can run OpenAI's Codex as the agent under evaluation, via the official Codex SDK. The `CodexAgent` mirrors the structure of `ClaudeCodeAgent` and plugs into the same sandbox, scoring, and telemetry pipeline — set `agent.type: codex` in a task and the rest of the framework works unchanged.
 
 ## Setup
 

docs/DOCKER_ISOLATION.md

@@ -1,9 +1,14 @@
+---
+description: >-
+  Run each coder_eval task in its own fresh Docker container — strong host
+  isolation, a pinned reproducible agent runtime, and custom images for
+  task-specific dependencies.
+---
+
 # Docker Isolation
 
 Run each evaluation task inside its own fresh container. Strong host isolation and a pinned, reproducible agent runtime.
 
-> Supersedes the agent-side FS perimeter flag from #199 (reverted in 9fe4320). The container boundary subsumes what that flag tried to do at the agent level.
-
 ## When to use
 
 Set `sandbox.driver: docker` on a task (or pass `--driver docker` on the CLI —

docs/TASK_DEFINITION_GUIDE.md

@@ -1,3 +1,10 @@
+---
+description: >-
+  Full schema reference for coder_eval task YAML — agent config, sandboxes, run
+  limits, dataset fan-out, and all 14 success criterion types with weighted
+  0.0–1.0 scoring.
+---
+
 # Task Definition Guide
 
 Complete reference for defining evaluation tasks in coder_eval.
@@ -128,6 +135,8 @@ an error.
 - `codex` — OpenAI Codex agent (requires `[codex]` extra; set `CODEX_API_KEY` and optional `CODEX_BASE_URL` environment variables).
 - `none` — No-op agent: no coding agent runs and no model API call is made. See [No-op / System Tasks](#no-op--system-tasks-type-none) below.
 
+<a id="no-op--system-tasks-type-none"></a>
+
 ### No-op / System Tasks (`type: none`)
 
 Set `agent: {type: none}` to run a task with **no coding agent** — "coder-eval
@@ -159,7 +168,7 @@ Contract (enforced at load): a `type: none` task must declare no `initial_prompt
 every criterion must be agent-independent — criteria that inspect the agent
 trajectory (`command_executed`, `skill_triggered`, `reference_comparison`,
 `commands_efficiency`) are rejected. A worked example lives at
-[`tasks/agentless_smoke_test.yaml`](../tasks/agentless_smoke_test.yaml).
+[`tasks/agentless_smoke_test.yaml`](https://github.com/UiPath/coder_eval/blob/main/tasks/agentless_smoke_test.yaml).
 
 ### `max_turns`, `task_timeout`, `turn_timeout` location
 
@@ -590,7 +599,7 @@ Checks whether the agent executed specific tools/commands during evaluation. Ins
 
 Evaluates a UiPath agent against a named evaluation set. **Fractional scoring:** metrics passed / total metrics.
 
-> The `uipath` CLI must be available **inside the sandbox** (typically declared in the task's own Python deps). This is independent of the host's optional `coder-eval[uipath]` extra — see the install matrix in [README.md](../README.md#installation).
+> The `uipath` CLI must be available **inside the sandbox** (typically declared in the task's own Python deps). This is independent of the host's optional `coder-eval[uipath]` extra — see the install matrix in [README.md](https://github.com/UiPath/coder_eval/blob/main/README.md#quick-start).
 
 ```yaml
 - type: "uipath_eval"

docs/USER_GUIDE.md

@@ -1,3 +1,10 @@
+---
+description: >-
+  Complete coder_eval reference — every CLI command and flag, configuration
+  layers and -D overrides, environment variables, run outputs, and reports for
+  evaluating AI coding agents.
+---
+
 # coder_eval User Guide
 
 The full command, configuration, and output reference. For a gentle introduction
@@ -81,6 +88,8 @@ in `.claude/commands/`, available when using Claude Code in this repository:
 | `/coder-eval-run-analysis <path>` | Analyze evaluation runs and suggest improvements to tasks, config, and prompts. Works at task, variant, or run scope. |
 | `/coder-eval-task-create` | Create evaluation task YAML files from a natural language description. |
 
+<a id="api-routing--benchmarking"></a>
+
 ## API Routing & Benchmarking
 
 `coder-eval` supports two API routing modes, selected via `--backend` or the