diff --git a/.claude/skills/ingest-run/SKILL.md b/.claude/skills/ingest-run/SKILL.md new file mode 100644 index 000000000..c91ae2379 --- /dev/null +++ b/.claude/skills/ingest-run/SKILL.md @@ -0,0 +1,117 @@ +--- +name: ingest-run +description: Ingest one vngcloud/InferenceX GitHub Actions benchmark run into this fork's self-hosted prod DB, over SSH into the deploy host. Use whenever asked to "ingest this run", given a vngcloud/InferenceX actions/runs/ URL, or asked to get benchmark data from a workflow run into the dashboard. NOT for the upstream SemiAnalysisAI/InferenceX-app Neon-DB flow (a different agent/env covers that). +--- + +# Ingest one benchmark run (this fork, self-hosted host) + +This fork does **not** use Neon. Prod is a self-hosted docker-compose stack on a +single runner host, with its own dockerized Postgres. That changes the ingest +mechanics vs. the upstream project's docs — follow this skill, not generic +`admin:db:ingest:run` examples you find elsewhere in the repo/docs. + +## Environment (fixed facts about this deployment) + +- **Deploy host**: `ssh -p 234 hoanq333@61.28.228.19` +- **Prod compose project**: `/opt/docker-compose/docker-compose.yml`, services + `inferencex-app-1`, `inferencex-postgres-1` (local Postgres, NOT Neon), `inferencex-nginx-1`, + `inferencex-certbot-1`. +- **Source repo for runs**: `vngcloud/InferenceX` (not `SemiAnalysisAI/InferenceX`). +- **`DATABASE_WRITE_URL`** on this host points at the local `postgres` compose service. + It does **not** speak TLS — every `bun run admin:db:*` command below **MUST** pass + `--no-ssl`, or you get a confusing `Client network socket disconnected before secure TLS +connection was established` / `ECONNRESET` error that looks like network flakiness but + isn't. `deploy.yml`'s own migration step already does this (`admin:db:migrate --yes --no-ssl`). +- Dev stack (`inferencex-dev`, port `:8080`) is a separate, isolated compose project on the + same host — this skill is about **prod**. + +## Step 0 — confirm the run is actually finished + +```bash +gh api repos/vngcloud/InferenceX/actions/runs/ --jq '{name,status,conclusion}' +``` + +If `status` isn't `completed`, wait — do not attempt to ingest a partial run. Re-check later +(a `ScheduleWakeup`/poll loop, not tight retries). + +## Step 1 — check whether the model/GPU/precision/framework is already registered + +Read the run's job names and artifact names (`gh run view --repo vngcloud/InferenceX`, +not `--log`) and check whether the model prefix, hardware key, precision, and framework already +resolve in the codebase (`packages/constants/src/models.ts` `DB_MODEL_TO_DISPLAY`, +`packages/constants/src/gpu-keys.ts` `HW_REGISTRY`, `packages/db/src/etl/normalizers.ts` +`PRECISION_ALIASES`/`MODEL_TO_KEY`). **If anything is new, follow +`docs/adding-entities.md` first** — ingesting before the mapping lands means the DB silently +skips those rows (no error, no partial write — the ETL just drops what it can't resolve). Commit, +push to `master`, and wait for `deploy.yml` to finish (`gh run watch `) before proceeding — +the new mapping has to actually be in the container's code for ingest to pick it up. + +## Step 2 — download artifacts once, persistently, inside the container + +The artifact-download step and the DB write step both go over flaky links from this host +(GitHub's artifact CDN, and TLS-handshake resets from the app container to the local Postgres +under load). Don't repeat `--download` end-to-end on every retry — it wastes time re-fetching +30+ artifacts just to retry the DB half. Split it: + +```bash +ssh -p 234 hoanq333@61.28.228.19 \ + "cd /opt/docker-compose && docker compose exec -T app sh -c \ + 'mkdir -p /tmp/ingest- && cd /tmp/ingest- && gh run download -R vngcloud/InferenceX && ls | wc -l'" +``` + +Confirm the artifact count matches what `gh run view` listed before moving on. + +## Step 3 — ingest in CI mode against the local artifacts (retry this, not the download) + +```bash +ssh -p 234 hoanq333@61.28.228.19 \ + "cd /opt/docker-compose && docker compose exec -T \ + -e INGEST_RUN_ID= -e INGEST_RUN_ATTEMPT=1 \ + -e INGEST_ARTIFACTS_PATH=/tmp/ingest- -e INGEST_REPO=vngcloud/InferenceX \ + -e UNMAPPED_ENTITIES_OUTPUT=/tmp/unmapped.json \ + app bun run admin:db:ingest:ci --no-ssl" +``` + +If it fails with the TLS/ECONNRESET error above, that's the local-Postgres connection blip, not +a real problem with the data — just retry this exact command a few times. Do NOT switch to +`admin:db:ingest:run vngcloud/InferenceX` (the one-shot download+ingest command) as your +retry loop; that re-downloads everything each time. + +Note the positional-arg gotcha if you ever do use the one-shot form: it's +`admin:db:ingest:run ` — **positional**, not `--repo `. Passing +`--repo` makes the script try to parse `--repo` itself as a run ID and fail immediately. + +Watch the summary block at the end for `+N new, M duplicate` counts per data type (benchmark +results, run stats, eval results, eval samples, changelog entries). `0 new, N duplicate` for a +run you know was already ingested is expected, not an error. + +## Step 4 — overrides, verify, invalidate + +```bash +ssh -p 234 hoanq333@61.28.228.19 "cd /opt/docker-compose && docker compose exec -T app bun run admin:db:apply-overrides --yes --no-ssl" +ssh -p 234 hoanq333@61.28.228.19 "cd /opt/docker-compose && docker compose exec -T app bun run admin:db:verify --no-ssl" +ssh -p 234 hoanq333@61.28.228.19 "cd /opt/docker-compose && docker compose exec -T app bun run admin:cache:invalidate http://localhost:3000" +``` + +`apply-overrides` reporting everything as "not in DB, skipping" is normal — most registry entries +in `run-overrides.ts` target other runs. `db:verify`'s "Sample Rows" section is the fastest way to +eyeball that the new config/benchmark_results rows look right (correct model key, precision, +hardware, throughput numbers in a sane range). + +## Cleanup + +```bash +ssh -p 234 hoanq333@61.28.228.19 "cd /opt/docker-compose && docker compose exec -T app rm -rf /tmp/ingest- /tmp/unmapped.json" +``` + +## Don't forget + +- If Step 1 required a code change, that same PR/commit should have updated + `packages/app/src/lib/api-route-catalog.ts`'s digest for `models.ts` (there's a vitest guard + that fails the build otherwise — `bun run test:unit` will tell you which digest to recompute: + `python3 -c "import hashlib; print(hashlib.sha256(open('','rb').read()).hexdigest())"`). +- If the model/GPU should be selectable in the dashboard UI (not just stored), that's a + **separate, optional** follow-up: `Model`/GPU registry entries in + `packages/app/src/lib/data-mappings.ts`, `compare-slug.ts`, `compare-ssr.ts` — see + `docs/adding-entities.md`. Ask the user whether they want that now or the DB write is enough + for this pass. diff --git a/.claude/skills/review-zh-copy/SKILL.md b/.claude/skills/review-zh-copy/SKILL.md new file mode 100644 index 000000000..80a2473d9 --- /dev/null +++ b/.claude/skills/review-zh-copy/SKILL.md @@ -0,0 +1,113 @@ +--- +name: review-zh-copy +description: Use when reviewing PRs or diffs that add or modify user-visible Simplified Chinese in InferenceX, including refactors and files whose names do not contain zh. +--- + +# Review Simplified Chinese copy + +## Operating mode: advisory + +Every finding produced by this skill is non-blocking. Do not label a Chinese-copy finding +`BLOCKING`, submit a request-changes review for it, or otherwise hold another contributor's merge. +Do not mention `@edwingao28` for a clean review, routine coverage, or an ordinary wording or +naturalness suggestion. Report an ordinary finding without a CC. Mention `@edwingao28` only for a +high-confidence semantic or factual error, changed attribution or speaker voice, or high-impact +ambiguity that cannot be resolved from the source and context. Label that finding +`Needs Chinese maintainer confirmation` and ask for review. It remains a suggestion while the +Chinese maintainer decides whether any follow-up is needed. + +The Chinese maintainer makes the final wording decision and manually reviews every changed +Chinese passage. + +## Editorial source of truth + +Read [docs/chinese-copy.md](../../../docs/chinese-copy.md) completely before reviewing. Apply its +audience, surface-specific register, context-aware terminology, rewrite method, and checklist. + +Semantic fidelity and natural Chinese are independent gates. Every changed passage must pass +both: + +- **Fidelity:** facts, scope, modality, metrics, numbers, links, product names, and attribution + retain the English source's meaning. +- **Naturalness:** the result reads as original Chinese technical writing in the register of its + surface, without preserving English clause order or noun stacking. + +A fluent mistranslation fails fidelity. An accurate sentence shaped like English fails +naturalness. + +## Find the complete review scope + +Review every added or modified user-visible Chinese passage in the complete diff against its +actual base. Do not infer scope from filenames. Chinese may appear in: + +- `zh:` dictionaries inside otherwise English TypeScript or TSX files; +- locale-aware ternaries and registry labels; +- `/zh` routes, Chinese MDX, JSON, metadata, schema.org content, alt text, and captions; +- tooltips, modals, controls, option labels, empty/loading/error states, and feature-gated UI. + +Include the full UI element or paragraph, not only the changed word or source line. When an +English/Chinese dictionary entry changes, inspect both siblings even if only one side is in the +diff. Do not report awkward pre-existing Chinese that the diff did not touch; record it separately +if it matters to a later rollout. + +Exclude identifiers, test data that is not rendered, DB-stored benchmark content, logs, and code +syntax unless the changed text is presented to a user. + +## Build context before judging + +For each passage, collect: + +1. the complete English source; +2. the route, component, and UI surface; +3. the surrounding heading, control, paragraph, or attributed quotation; +4. the intended meaning in plain language; +5. facts, qualifications, modality, metrics, links, names, and attribution that cannot change; +6. rendered desktop and mobile context when width, hierarchy, or interaction affects wording. + +If no English source exists, say so. Infer intent from adjacent UI and product behavior, and lower +confidence rather than inventing a source claim. + +## Review workflow + +For every changed passage: + +1. State the source intent without copying the English sentence structure. +2. Evaluate fidelity. Reject additions, omissions, changed scope, stronger certainty, altered + metric definitions, or a changed speaker voice. +3. Hide the old Chinese and draft from the stated intent. +4. Evaluate naturalness without looking at the English. Check information order, sentence + structure, register, and established Chinese ML infrastructure usage. +5. Compare the proposed rewrite with the source again. +6. Check the complete rendered element or paragraph. A locally natural word can still be wrong in + its button, tooltip, metadata, marketing, technical-prose, or quotation context. + +Do not use fixed substitutions for contextual questions. In particular: + +- sentence length alone is not evidence of translationese; +- technical English is not wrong merely because a Chinese equivalent exists; +- one concept may need different wording in a UI action, prose, a unit, or an attributed quote; +- established English technical terms and phrases that Chinese ML infrastructure engineers + normally use in English should remain English in first-party UI and technical prose; judge this + from real industry usage and the surface, not from a closed list. `warmup`, `seed`, and `offload` + are examples, not the whole category. Chinese-first explanations such as `预热(warmup)` remain + valid for a broader audience, and attributed quotations preserve the speaker's wording; +- `您` is normally omitted in controls and tooltips but may be intentional in a respectful CTA; +- #819 and other bulk-rewrite outputs are not automatic editorial ground truth; only their + separately verified mechanical cases may seed deterministic CI fixtures. + +## Report high-signal findings + +For each finding provide: + +- `file:line` and route or surface; +- the exact English source and a one-sentence statement of its intent; +- which gate failed: fidelity, naturalness, or both; +- the specific defect and its user impact; +- one complete suggested replacement, not an isolated word swap; +- confidence and advisory severity. + +Do not report only “this reads awkwardly.” Stay silent when the existing wording is accurate, +natural, and appropriate for its surface, even if another wording is also possible. + +End with coverage: list the changed Chinese surfaces reviewed and any surface that could not be +rendered or whose English source could not be identified. diff --git a/.claude/skills/write-inferencex-blog/iso_interactivity.py b/.claude/skills/write-inferencex-blog/iso_interactivity.py index 64058e8ba..fef08a88f 100644 --- a/.claude/skills/write-inferencex-blog/iso_interactivity.py +++ b/.claude/skills/write-inferencex-blog/iso_interactivity.py @@ -7,7 +7,8 @@ packages/app/src/components/inference/hooks/useInterpolatedTrendData.ts. Pipeline: - 1. Build the upper-left Pareto frontier on (interactivity, throughput). + 1. Build the upper-left Pareto frontier on (interactivity, throughput). For a + proportional metric, use the total/output/input throughput it scales. 2. Sort the frontier by interactivity ascending. 3. Build monotone cubic Hermite slopes (Steffen 1990 method, same as d3.curveMonotoneX) over the frontier knots for the metric of interest. @@ -18,12 +19,18 @@ 6. Clamp the interpolated metric value to [min(ys), max(ys)] of the frontier to prevent cubic-spline overshoots above/below the data. -Reciprocal metrics ($/M tok, J/token) are a special case: they are a per-chip -constant divided by a throughput, so they are NOT splined independently. Two -independent splines need not preserve `metric * throughput = constant` between +Reciprocal metrics ($/M tok in blog tables, J/token) are a special case: they are +a per-chip constant divided by a throughput, so they are NOT splined independently. +Two independent splines need not preserve `metric * throughput = constant` between measured knots. Pass `reciprocal_of='throughput'` (or the matching output/input -throughput key) to spline that throughput and re-derive the metric, which is what -the dashboard does. See docs/tco-calculator.md for a reproducible measurement. +throughput key) to spline that throughput and re-derive the metric. The live +inference charts also expose tokens/$ and normalized token revenue as separate +metrics; both are directly proportional to throughput. OpenRouter-priced revenue +is proportional only when the input/output mix stays constant; otherwise spline +the already-priced metric directly. Pass +`proportional_to='throughput'` (or the matching output/input throughput key) so +they use the same Pareto knots and spline as that throughput before applying the +recovered multiplier. See docs/tco-calculator.md for a reproducible measurement. Usage as a module: from iso_interactivity import interpolate_metric, pareto_front_upper_left @@ -182,6 +189,25 @@ def recover_reciprocal_numerator( return numerator +def recover_proportional_multiplier( + values: list[Optional[float]], throughputs: list[Optional[float]] +) -> Optional[float]: + """Recover the constant `k` of a metric defined as `throughput * k`.""" + relative_tolerance = 1e-3 + multiplier: Optional[float] = None + for value, throughput in zip(values, throughputs): + if value is None or throughput is None: + continue + if not value > 0 or not throughput > 0: + continue + candidate = value / throughput + if multiplier is None: + multiplier = candidate + elif abs(candidate - multiplier) > abs(multiplier) * relative_tolerance: + return None + return multiplier + + def interpolate_metric( points: list[dict], target_iv: float, @@ -189,6 +215,7 @@ def interpolate_metric( iv_key: str = 'interactivity', tput_key: str = 'throughput', reciprocal_of: Optional[str] = None, + proportional_to: Optional[str] = None, ) -> Optional[float]: """Interpolate `metric_key` at `target_iv` using the chart's algorithm. @@ -196,16 +223,23 @@ def interpolate_metric( the chart code does not extrapolate. Blog tables should render this as `_unreachable_` in that row. - The Pareto frontier is ALWAYS built on (interactivity, throughput) regardless - of which metric you're interpolating — this matches the chart, where the - frontier is defined by the upper-left throughput envelope and other metrics - (cost, energy, TPOT, ...) are derived values at frontier points. + The Pareto frontier is built on (interactivity, throughput). For tokens/$ or + normalized token revenue (and revenue with a constant token mix), + `proportional_to` names the total/output/input + throughput it scales; that throughput also supplies the frontier knots so + both curves stay aligned. `reciprocal_of` names the throughput key a metric divides, for metrics of the form `constant / throughput` ($/M tok, J/token). Set it for those metrics: that throughput is splined and the metric re-derived, matching the dashboard. Leave it None for metrics splined directly (throughput itself, tok/s/MW, latency, and measured energy — whose numerator varies per point). + + `proportional_to` names the throughput key for metrics of the form + `throughput * constant` (tokens/$, normalized token revenue, and revenue + with a constant input/output mix). That + throughput is splined and the recovered multiplier applied, matching the + dashboard. """ if not points: return None @@ -213,7 +247,7 @@ def interpolate_metric( frontier = pareto_front_upper_left( points, get_x=lambda p: p[iv_key], - get_y=lambda p: p[tput_key], + get_y=lambda p: p[proportional_to or tput_key], ) if not frontier: return None @@ -241,6 +275,19 @@ def interpolate_metric( return None ys.append(value) + if proportional_to is not None: + tputs: list[float] = [] + for point in sorted_front: + throughput = point.get(proportional_to) + if throughput is None: + return None + tputs.append(throughput) + multiplier = recover_proportional_multiplier(ys, tputs) + if multiplier is not None: + tput_slopes = monotone_slopes(xs, tputs) + tput = hermite_interpolate(xs, tputs, tput_slopes, target_iv) + return max(0.0, tput) * multiplier + if reciprocal_of is not None: tputs: list[float] = [] for point in sorted_front: @@ -276,7 +323,8 @@ def interpolate_metric( def _cli() -> None: """Stdin: {"points": [...], "target_iv": N, "metric_key": "...", - "reciprocal_of": "throughput" for $/M tok and J/token} + "reciprocal_of": "throughput" for $/M tok and J/token, + "proportional_to": "throughput" for tokens/$ or revenue with a constant mix} Stdout: {"value": N or null}""" req = json.loads(sys.stdin.read()) value = interpolate_metric( @@ -286,6 +334,7 @@ def _cli() -> None: iv_key=req.get('iv_key', 'interactivity'), tput_key=req.get('tput_key', 'throughput'), reciprocal_of=req.get('reciprocal_of'), + proportional_to=req.get('proportional_to'), ) json.dump({'value': value}, sys.stdout) sys.stdout.write('\n') diff --git a/.github/claude/review-prompt.md b/.github/claude/review-prompt.md index bc3e2665c..2eb0e5104 100644 --- a/.github/claude/review-prompt.md +++ b/.github/claude/review-prompt.md @@ -91,6 +91,30 @@ PostHog autocapture tracks all interactions automatically. Named `track()` calls **Important**: Only flag NEW or MODIFIED interactive elements in the PR diff. Do NOT flag existing code that was not changed in this PR. +## 💡 NON-BLOCKING: Simplified Chinese copy quality + +The `/zh` side is hand-authored. **If the complete diff touches any user-visible Chinese string, load `.claude/skills/review-zh-copy/SKILL.md` and follow it.** Search the whole diff rather than relying on `zh` in a filename: Chinese also lives in component-local dictionaries, locale ternaries, metadata, option registries, tooltips, states, and feature-gated UI. + +Read `docs/chinese-copy.md` completely. Review the full UI element or paragraph together with its English source, intended meaning, surface, and rendered context. Evaluate semantic fidelity and natural Chinese as independent gates; fluent Chinese that changes the source is still wrong. + +**Chinese-copy findings are always non-blocking while the skill says `Operating mode: advisory`.** Do not label them `🔴 BLOCKING`, submit a request-changes review for them, or otherwise hold another contributor's merge. Do not mention `@edwingao28` for a clean review, routine coverage, or an ordinary wording or naturalness suggestion. Mention `@edwingao28` only for a high-confidence semantic or factual error, changed attribution or speaker voice, or high-impact ambiguity that cannot be resolved from the source and context. Label that finding `Needs Chinese maintainer confirmation` and ask for review. It remains a suggestion; the Chinese maintainer decides whether any follow-up is needed. + +Worth a suggestion: + +- Chinese that follows English clause order or modifier chains closely enough to sound translated. +- A rendering that changes or loses the source meaning — fluent Chinese can still be a mistranslation, and that is the more expensive error. +- Register wrong for the surface: marketing voice in a UI label, invented internet slang, or an attributed quote rewritten into a voice its speaker would not use. + +Not worth a comment: + +- Wording you would have phrased differently when the existing rendering is accurate and natural. +- Long Chinese sentences per se. Length is not evidence of translationese. +- Established English technical terms and phrases that Chinese ML infrastructure engineers normally use in English. Keep them English in first-party UI and technical prose; decide from real industry usage and the surface, not from a closed list or capitalization. `warmup`, `seed`, and `offload` are examples, not the whole category. Chinese-first explanations may be appropriate for a broader audience, and attributed quotations preserve the speaker's wording. +- `您` solely because it appears. Controls normally omit the pronoun, but a respectful CTA may use it intentionally. +- Awkward Chinese the diff did not touch. Mention it separately from the review. + +For each finding, quote the English source, state its intended meaning and failed gate, and give a complete replacement. Never comment "this reads awkwardly" on its own. End with the changed Chinese surfaces covered and any context that could not be rendered or verified. + ## 🔴 BLOCKING: Test coverage enforcement When reviewing a PR diff, check if new code was added WITHOUT corresponding tests: diff --git a/.github/workflows/apply-run-overrides.yml b/.github/workflows/apply-run-overrides.yml index 4ec9fcf26..a0fa1189d 100644 --- a/.github/workflows/apply-run-overrides.yml +++ b/.github/workflows/apply-run-overrides.yml @@ -22,7 +22,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -30,14 +30,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies diff --git a/.github/workflows/cache-refresh.yml b/.github/workflows/cache-refresh.yml index 06cd7d680..61b6c2e8d 100644 --- a/.github/workflows/cache-refresh.yml +++ b/.github/workflows/cache-refresh.yml @@ -17,7 +17,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -25,14 +25,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index cb3159300..a4dffca59 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -24,17 +24,17 @@ jobs: route: name: Claude (route) if: >- - ((github.event_name == 'issue_comment' || github.event_name == 'pull_request_review_comment') - && contains(github.event.comment.body || '', '@claude') - && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "BOT"]'), github.event.comment.author_association)) - || - (github.event_name == 'issues' - && (contains(github.event.issue.body || '', '@claude') || contains(github.event.issue.title || '', '@claude')) - && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "BOT"]'), github.event.issue.author_association)) - || - (github.event_name == 'pull_request' - && !github.event.pull_request.draft - && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "BOT"]'), github.event.pull_request.author_association)) + ((github.event_name == 'issue_comment' || github.event_name == + 'pull_request_review_comment') && contains(github.event.comment.body || + '', '@claude') && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", + "BOT"]'), github.event.comment.author_association)) || (github.event_name + == 'issues' && (contains(github.event.issue.body || '', '@claude') || + contains(github.event.issue.title || '', '@claude')) && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "BOT"]'), + github.event.issue.author_association)) || (github.event_name == + 'pull_request' && !github.event.pull_request.draft && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR", "BOT"]'), + github.event.pull_request.author_association)) runs-on: ubuntu-latest # No repo write/PR access needed — this job only calls the Anthropic API. permissions: @@ -42,7 +42,8 @@ jobs: # Latest event wins: cancel an in-flight route for the same PR/issue when a newer # event arrives, so rapid pushes don't fan out into parallel downstream reviews. concurrency: - group: claude-route-${{ github.event.pull_request.number || github.event.issue.number }} + group: claude-route-${{ github.event.pull_request.number || + github.event.issue.number }} cancel-in-progress: true outputs: profile: ${{ steps.route.outputs.profile }} @@ -58,7 +59,9 @@ jobs: ROUTER_MODEL: ${{ env.CLAUDE_ROUTER_MODEL }} BODY: ${{ github.event.comment.body || github.event.issue.body || '' }} TITLE: ${{ github.event.issue.title || '' }} - IS_PR: ${{ github.event_name == 'pull_request' || github.event.issue.pull_request != null || github.event.pull_request != null }} + IS_PR: ${{ github.event_name == 'pull_request' || + github.event.issue.pull_request != null || github.event.pull_request + != null }} run: | set -euo pipefail @@ -147,7 +150,8 @@ jobs: implement: name: Claude (implement) needs: route - if: contains(fromJson('["ui", "code", "docs", "question"]'), needs.route.outputs.profile) + if: contains(fromJson('["ui", "code", "docs", "question"]'), + needs.route.outputs.profile) runs-on: ubuntu-latest env: GITHUB_TOKEN: ${{ secrets.PAT }} @@ -165,12 +169,13 @@ jobs: # Serialize @claude runs on the same issue/PR (incl. reviews — shared group with the # review job) so back-to-back triggers don't clobber each other. concurrency: - group: claude-${{ github.event.pull_request.number || github.event.issue.number }} + group: claude-${{ github.event.pull_request.number || github.event.issue.number + }} cancel-in-progress: false steps: - name: Checkout repository - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 token: ${{ secrets.PAT }} @@ -184,14 +189,15 @@ jobs: bun-version-file: package.json - name: Setup Node.js - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- @@ -239,7 +245,8 @@ jobs: fi - name: Install Cypress binary - if: needs.route.outputs.profile == 'ui' && steps.cypress-cache.outputs.cache-hit != 'true' + if: needs.route.outputs.profile == 'ui' && steps.cypress-cache.outputs.cache-hit + != 'true' working-directory: packages/app run: bunx cypress install @@ -324,7 +331,7 @@ jobs: # so a dev server that didn't come up does not block this step. - name: Run Claude Code id: claude - uses: anthropics/claude-code-action@536f2c32a39763739000b0e1ac69ca2647d97ce9 # v1.0.170 + uses: anthropics/claude-code-action@459ad358ae43fea66bfefd0a1f8d840b4b9791fb # v1.0.194 env: GH_TOKEN: ${{ secrets.PAT }} GITHUB_TOKEN: ${{ secrets.PAT }} @@ -361,10 +368,9 @@ jobs: # author/draft gate, so this is plain success()-gated: run when route said "review" # and there's a PR to review. if: >- - needs.route.outputs.profile == 'review' - && (github.event_name == 'pull_request' - || github.event.issue.pull_request - || github.event.pull_request) + needs.route.outputs.profile == 'review' && (github.event_name == + 'pull_request' || github.event.issue.pull_request || + github.event.pull_request) runs-on: ubuntu-latest env: DATABASE_READONLY_URL: ${{ secrets.DATABASE_READONLY_URL }} @@ -376,7 +382,8 @@ jobs: actions: read # Inspect workflow runs and checks id-token: write # Authenticate the Claude Code action with OIDC concurrency: - group: claude-${{ github.event.pull_request.number || github.event.issue.number }} + group: claude-${{ github.event.pull_request.number || github.event.issue.number + }} cancel-in-progress: false steps: @@ -399,7 +406,7 @@ jobs: echo "Pinned PR #${PR_NUMBER} head sha: $SHA" - name: Checkout PR head - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: ref: ${{ steps.prsha.outputs.sha }} fetch-depth: 0 @@ -414,14 +421,15 @@ jobs: bun-version-file: package.json - name: Setup Node.js - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- @@ -500,7 +508,7 @@ jobs: - name: Run Claude Code id: claude - uses: anthropics/claude-code-action@536f2c32a39763739000b0e1ac69ca2647d97ce9 # v1.0.170 + uses: anthropics/claude-code-action@459ad358ae43fea66bfefd0a1f8d840b4b9791fb # v1.0.194 env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/db-backup.yml b/.github/workflows/db-backup.yml index 9e623d796..7da94437a 100644 --- a/.github/workflows/db-backup.yml +++ b/.github/workflows/db-backup.yml @@ -48,8 +48,8 @@ jobs: dump: name: Dump and compress needs: preflight - timeout-minutes: 180 - runs-on: ubuntu-latest + timeout-minutes: 360 + runs-on: blacksmith-32vcpu-ubuntu-2404 steps: - name: Dump, compress, and split database env: @@ -86,7 +86,7 @@ jobs: cat SHA256SUMS - name: Upload dump parts - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 with: name: db-dump-parts path: | @@ -105,7 +105,7 @@ jobs: contents: write # Create the database dump release steps: - name: Download dump parts - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: name: db-dump-parts diff --git a/.github/workflows/ingest-agentic-results.yml b/.github/workflows/ingest-agentic-results.yml index 398376b9e..e9e273777 100644 --- a/.github/workflows/ingest-agentic-results.yml +++ b/.github/workflows/ingest-agentic-results.yml @@ -87,7 +87,8 @@ on: permissions: {} concurrency: - group: ${{ github.workflow }}-${{ github.event.client_payload.source-run-id || github.event.client_payload.run-id || inputs.run-id }} + group: ${{ github.workflow }}-${{ github.event.client_payload.source-run-id || + github.event.client_payload.run-id || inputs.run-id }} cancel-in-progress: false jobs: @@ -102,19 +103,19 @@ jobs: permissions: contents: read env: - REQUESTED_DATABASE_TARGET: ${{ github.event.client_payload.database-target || inputs.database-target || 'production' }} + REQUESTED_DATABASE_TARGET: ${{ github.event.client_payload.database-target || + inputs.database-target || 'production' }} steps: - name: Wait for source run to finish if: >- - github.event_name == 'repository_dispatch' && - github.event.action == 'ingest-agentic-results' && - ( + github.event_name == 'repository_dispatch' && github.event.action == + 'ingest-agentic-results' && ( github.event.client_payload.source-run-id == '' || github.event.client_payload.source-run-id == github.event.client_payload.merge-run-id ) run: sleep 300 - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -122,14 +123,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies @@ -144,7 +146,8 @@ jobs: DATABASE_WRITE_URL_STAGING: ${{ secrets.DATABASE_STAGING_WRITE_URL }} INVALIDATE_SECRET_DEFAULT: ${{ secrets.VERCEL_INVALIDATE_SECRET }} PROTECTION_BYPASS_SECRET_STAGING: ${{ secrets.VERCEL_STAGING_BYPASS_SECRET }} - STAGING_SITE_URL: ${{ vars.STAGING_SITE_URL || 'https://inferencemax-app-git-staging-semianalysisai.vercel.app' }} + STAGING_SITE_URL: ${{ vars.STAGING_SITE_URL || + 'https://inferencemax-app-git-staging-semianalysisai.vercel.app' }} run: | case "$REQUESTED_DATABASE_TARGET" in production) @@ -202,8 +205,10 @@ jobs: id: artifacts env: GH_TOKEN: ${{ secrets.INFX_MAIN_PAT }} - SOURCE_RUN_ID: ${{ github.event.client_payload.source-run-id || github.event.client_payload.run-id || inputs.run-id }} - MERGE_RUN_ID: ${{ github.event.client_payload.merge-run-id || github.event.client_payload.run-id || inputs.run-id }} + SOURCE_RUN_ID: ${{ github.event.client_payload.source-run-id || + github.event.client_payload.run-id || inputs.run-id }} + MERGE_RUN_ID: ${{ github.event.client_payload.merge-run-id || + github.event.client_payload.run-id || inputs.run-id }} ARTIFACTS_PATH: ${{ github.workspace }}/artifacts INGEST_REPO: SemiAnalysisAI/InferenceX run: bun run admin:db:prepare:ci @@ -263,8 +268,9 @@ jobs: fi - name: Notify Slack on unmapped entities - if: steps.unmapped.outputs.found == 'true' && env.REQUESTED_DATABASE_TARGET != 'staging' - uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3 + if: steps.unmapped.outputs.found == 'true' && env.REQUESTED_DATABASE_TARGET != + 'staging' + uses: slackapi/slack-github-action@dcb1066f776dd043e64d0e8ba94ca15cc7e1875d # v4.0.0 with: webhook: ${{ secrets.SLACK_WEBHOOK_URL }} webhook-type: incoming-webhook @@ -275,7 +281,7 @@ jobs: - name: Notify Slack on failure if: failure() && env.REQUESTED_DATABASE_TARGET != 'staging' - uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3 + uses: slackapi/slack-github-action@dcb1066f776dd043e64d0e8ba94ca15cc7e1875d # v4.0.0 with: webhook: ${{ secrets.SLACK_WEBHOOK_URL }} webhook-type: incoming-webhook diff --git a/.github/workflows/ingest-results.yml b/.github/workflows/ingest-results.yml index be2ed5abd..238d2a7b7 100644 --- a/.github/workflows/ingest-results.yml +++ b/.github/workflows/ingest-results.yml @@ -7,7 +7,8 @@ on: permissions: {} concurrency: - group: ${{ github.workflow }}-${{ github.event.client_payload.source-run-id || github.event.client_payload.run-id }} + group: ${{ github.workflow }}-${{ github.event.client_payload.source-run-id || + github.event.client_payload.run-id }} cancel-in-progress: false jobs: @@ -21,10 +22,11 @@ jobs: - name: Wait for source run to finish if: >- github.event.client_payload.source-run-id == '' || - github.event.client_payload.source-run-id == github.event.client_payload.merge-run-id + github.event.client_payload.source-run-id == + github.event.client_payload.merge-run-id run: sleep 300 - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -32,14 +34,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies @@ -51,8 +54,10 @@ jobs: id: artifacts env: GH_TOKEN: ${{ secrets.INFX_MAIN_PAT }} - SOURCE_RUN_ID: ${{ github.event.client_payload.source-run-id || github.event.client_payload.run-id }} - MERGE_RUN_ID: ${{ github.event.client_payload.merge-run-id || github.event.client_payload.run-id }} + SOURCE_RUN_ID: ${{ github.event.client_payload.source-run-id || + github.event.client_payload.run-id }} + MERGE_RUN_ID: ${{ github.event.client_payload.merge-run-id || + github.event.client_payload.run-id }} ARTIFACTS_PATH: ${{ github.workspace }}/artifacts INGEST_REPO: SemiAnalysisAI/InferenceX run: bun run admin:db:prepare:ci @@ -115,7 +120,7 @@ jobs: - name: Notify Slack on unmapped entities if: steps.unmapped.outputs.found == 'true' - uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3 + uses: slackapi/slack-github-action@dcb1066f776dd043e64d0e8ba94ca15cc7e1875d # v4.0.0 with: webhook: ${{ secrets.SLACK_WEBHOOK_URL }} webhook-type: incoming-webhook @@ -126,7 +131,7 @@ jobs: - name: Notify Slack on failure if: failure() - uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3 + uses: slackapi/slack-github-action@dcb1066f776dd043e64d0e8ba94ca15cc7e1875d # v4.0.0 with: webhook: ${{ secrets.SLACK_WEBHOOK_URL }} webhook-type: incoming-webhook diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index e44856e89..69a43a32b 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -20,7 +20,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -28,14 +28,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies @@ -46,3 +47,5 @@ jobs: run: bun run lint - name: Format check run: bun run fmt + - name: Typography gate + run: bun run check:typography diff --git a/.github/workflows/stage-results.yml b/.github/workflows/stage-results.yml index 74155047f..9f91a765c 100644 --- a/.github/workflows/stage-results.yml +++ b/.github/workflows/stage-results.yml @@ -40,6 +40,10 @@ on: permissions: {} +concurrency: + group: stage-results + cancel-in-progress: false + jobs: validate: name: Validate staging request diff --git a/.github/workflows/tests-e2e.yml b/.github/workflows/tests-e2e.yml index 0ab6b69b4..48f292e70 100644 --- a/.github/workflows/tests-e2e.yml +++ b/.github/workflows/tests-e2e.yml @@ -13,6 +13,7 @@ on: - 'bunfig.toml' - 'bun.lock' - 'tsconfig.json' + - 'tsconfig.base.json' permissions: {} @@ -29,7 +30,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -37,7 +38,7 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Restore Bun package cache @@ -45,7 +46,8 @@ jobs: uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies @@ -56,7 +58,8 @@ jobs: uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.cache/Cypress - key: cypress-binary-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('bun.lock') }} + key: cypress-binary-${{ runner.os }}-${{ runner.arch }}-${{ + hashFiles('bun.lock') }} restore-keys: | cypress-binary-${{ runner.os }}-${{ runner.arch }}- - name: Install Cypress binary if missing @@ -84,7 +87,7 @@ jobs: # duplicated fixed cost dominates and yields sharply diminishing returns. shard: [1, 2, 3, 4] steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -92,7 +95,7 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Restore Bun package cache @@ -100,7 +103,8 @@ jobs: uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - uses: browser-actions/setup-firefox@0bc507ddf224827e3b1af68e014d5e42ab93e795 # v1.7.2 @@ -120,7 +124,9 @@ jobs: # New cache whenever packages or source files change; fall back to # the newest cache for the same lockfile so changed sources still # start from a warm compiler cache. - key: ${{ runner.os }}-nextjs-${{ hashFiles('bun.lock') }}-${{ hashFiles('packages/**/*.js', 'packages/**/*.jsx', 'packages/**/*.ts', 'packages/**/*.tsx') }} + key: ${{ runner.os }}-nextjs-${{ hashFiles('bun.lock') }}-${{ + hashFiles('packages/**/*.js', 'packages/**/*.jsx', + 'packages/**/*.ts', 'packages/**/*.tsx') }} restore-keys: | ${{ runner.os }}-nextjs-${{ hashFiles('bun.lock') }}- ${{ runner.os }}-nextjs- @@ -133,14 +139,16 @@ jobs: uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.cache/Cypress - key: cypress-binary-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('bun.lock') }} + key: cypress-binary-${{ runner.os }}-${{ runner.arch }}-${{ + hashFiles('bun.lock') }} restore-keys: | cypress-binary-${{ runner.os }}-${{ runner.arch }}- - name: Install Cypress binary if missing working-directory: packages/app run: bunx cypress verify || bunx cypress install - name: Save Cypress binary cache - if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && steps.cypress-cache.outputs.cache-hit != 'true' }} + if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && + steps.cypress-cache.outputs.cache-hit != 'true' }} uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.cache/Cypress @@ -148,7 +156,8 @@ jobs: # The component job only restores this cache. One E2E shard writes it, avoiding # concurrent cache reservations across the nine parallel test jobs. - name: Save Bun package cache - if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && steps.bun-cache.outputs.cache-hit != 'true' }} + if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && + steps.bun-cache.outputs.cache-hit != 'true' }} uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache @@ -158,13 +167,14 @@ jobs: env: E2E_FIXTURES: '1' - name: Save Next.js build cache - if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && steps.nextjs-cache.outputs.cache-hit != 'true' }} + if: ${{ matrix.browser == 'chrome' && matrix.shard == 1 && + steps.nextjs-cache.outputs.cache-hit != 'true' }} uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: packages/app/.next/cache key: ${{ steps.nextjs-cache.outputs.cache-primary-key }} - name: Run integration tests - uses: cypress-io/github-action@fa4a118725a8f001170d49631ea89e5d66fee626 # v7.4.1 + uses: cypress-io/github-action@c32f12761482a282d24ca0fd7466d8ae86f54ba8 # v7.4.2 with: working-directory: packages/app install: false diff --git a/.github/workflows/tests-unit.yml b/.github/workflows/tests-unit.yml index abdfb1041..76f003e6c 100644 --- a/.github/workflows/tests-unit.yml +++ b/.github/workflows/tests-unit.yml @@ -10,10 +10,12 @@ on: - 'packages/app/**' - 'packages/constants/**' - 'packages/db/**' + - 'packages/mcp/**' - 'bunfig.toml' - 'bun.lock' - 'scripts/run-workspace-script.ts' - 'tsconfig.json' + - 'tsconfig.base.json' permissions: {} @@ -30,7 +32,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Setup Bun @@ -38,14 +40,15 @@ jobs: uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 with: bun-version-file: package.json - - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0 + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: node-version: '24' - name: Cache Bun packages uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 with: path: ~/.bun/install/cache - key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} + key: ${{ runner.os }}-bun-${{ runner.arch }}-${{ + steps.setup-bun.outputs.bun-version }}-${{ hashFiles('bun.lock') }} restore-keys: | ${{ runner.os }}-bun-${{ runner.arch }}-${{ steps.setup-bun.outputs.bun-version }}- - name: Install dependencies diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index 1ce7240ca..389777957 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -27,10 +27,11 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: persist-credentials: false - name: Install uv - uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 + uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1 - name: Audit workflows - run: uvx --exclude-newer P1D zizmor@latest --strict-collection --pedantic --no-ignores . + run: uvx --exclude-newer P1D zizmor@latest --strict-collection --pedantic + --no-ignores . diff --git a/.oxlintrc.json b/.oxlintrc.json index 5a03a5a0f..8a2e56158 100644 --- a/.oxlintrc.json +++ b/.oxlintrc.json @@ -7,7 +7,9 @@ "nursery": "deny" }, "rules": { - "capitalized-comments": "off", + "eslint/capitalized-comments": "off", + "eslint/one-var": "off", + "one-var": "off", "curly": "off", "func-names": "off", "func-style": "off", @@ -38,6 +40,7 @@ "unicorn/filename-case": "off", "unicorn/max-nested-calls": "off", "unicorn/no-array-callback-reference": "off", + "unicorn/no-array-sort": "off", "unicorn/no-nested-ternary": "off", "unicorn/no-null": "off", "unicorn/no-useless-undefined": "off", diff --git a/AGENTS.md b/AGENTS.md index a50342f68..1f943f877 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,9 @@ For detailed subsystem docs, see [docs/index.md](./docs/index.md). > **PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English.** Title format: ` / <中文标题>` (keep bracket prefixes at the front untranslated). In the PR/issue body, follow the English content with a `## 中文说明` section mirroring the summary; don't translate code blocks, logs, or stack traces — summarize around them. **Commit messages must include a Chinese translation too**: keep the subject line in English (conventional-commit style) and include the Chinese translation of the subject and key points in the commit body (e.g. a trailing `中文:` paragraph); squash-merge commits inherit the bilingual PR title, which satisfies the subject requirement automatically. -> **Translation quality bar:** write natural technical Chinese, not word-for-word machine translation (style reference: [`vllm-project/vllm-ascend` `README.zh.md`](https://github.com/vllm-project/vllm-ascend/blob/main/README.zh.md)). Preserve product names, hardware SKUs, framework/library names (Next.js, React Query, D3.js, Tailwind ...), flags, and code identifiers in English. Use parenthetical English clarification for acronyms on first use. Preferred terms: benchmark 基准测试, dashboard 仪表板, chart 图表, config 配置, throughput 吞吐量, latency 延迟, single-node/multi-node 单节点/多节点, evaluation 评估, artifact 产物. +> **Translation quality bar:** write natural technical Chinese, not word-for-word machine translation (style reference: [`vllm-project/vllm-ascend` `README.zh.md`](https://github.com/vllm-project/vllm-ascend/blob/main/README.zh.md)). Preserve product names, hardware SKUs, framework/library names (Next.js, React Query, D3.js, Tailwind ...), flags, and code identifiers in English. Use parenthetical English clarification for acronyms on first use. Preferred terms: benchmark 基准测试, dashboard 仪表板, chart 图表, config 配置, throughput 吞吐量, latency 延迟, single-node/multi-node 单节点/多节点, evaluation 评估, artifact 产物. In first-party ML infrastructure UI and technical prose, keep established English technical terms and phrases that Chinese engineers normally use in English; decide from real industry usage and the surface, not from a closed list. `warmup`, `seed`, and `offload` are examples, not the whole category. Follow `docs/chinese-copy.md` for explanatory and quotation exceptions. + +> **Chinese editorial decisions are context-aware.** Follow [docs/chinese-copy.md](./docs/chinese-copy.md) for audience, surface-specific register, terminology exceptions, the two independent fidelity/naturalness gates, the Claude-plus-manual-review workflow, and the PR checklist. Do not turn a preferred wording or pronoun choice into a global mechanical rule unless it is correct in every supported context. > **The website itself is bilingual too — every indexable page must ship a Simplified Chinese sibling under `/zh`.** See [Chinese Website Pages](#chinese-website-pages-zh--mandatory-for-all-indexable-surfaces) below; a new page, tab, or blog post without its `/zh` version is 🔴 BLOCKING on PR review. @@ -27,6 +29,7 @@ InferenceX App — Next.js 16 dashboard for ML inference benchmark data. DB-back ```bash bun install # Install dependencies bun run dev # Dev server with Turbopack (http://localhost:3000) +bun run mcp # MCP server exposing read-only benchmark tools bun run build # Production build bun run typecheck # TypeScript type checking (all packages) bun run lint # Lint with oxlint @@ -52,7 +55,8 @@ packages/ │ ├── hooks/api/ # React Query hooks (use-benchmarks, use-availability, etc.) │ └── lib/ # Utilities, constants, d3-chart/, chart-utils, blog, data-mappings ├── constants/ # Shared constants (GPU keys, model mappings, SEO) -└── db/ # DB layer, ETL, migrations, queries, ingest scripts +├── db/ # DB layer, ETL, migrations, queries, ingest scripts +└── mcp/ # MCP server exposing read-only benchmark tools ``` **Path alias**: `@/*` → `packages/app/src/` @@ -123,6 +127,7 @@ Static content routes (no DB): - **Linter**: oxlint — `bun run lint` / `bun run lint:fix` - **Formatter**: oxfmt — `bun run fmt` / `bun run fmt:fix` - **Type checking**: `bun run typecheck` (tsc --noEmit, strict mode) +- **Typography gate**: `bun run check:typography` (CI + pre-commit) — no new arbitrary font sizes (`text-[11px]`) or letter-spacing (`tracking-[0.16em]`) in class strings, and no quoted font-size literals in `src/lib/d3-chart/`. Use `text-2xs`/`text-3xs`, `tracking-eyebrow`/`tracking-eyebrow-wide`/`tracking-heading`, the ``/`` components, or `CHART_TYPE` from `@/lib/d3-chart/typography` (chart font sizes must stay in TS — CSS variables don't survive PNG export). Existing offenders live in `packages/app/scripts/typography-allowlist.json` and migrate on touch. See [Typography](./docs/typography.md). - **Node**: 24.x ## Environment Variables @@ -150,7 +155,7 @@ All interactive elements should have `track()` from `@/lib/analytics` (autocaptu ## Tab Structure -Order: `inference` → `evaluation` → `historical` → `calculator` → `reliability` → `gpu-specs` (defined in `page-content.tsx` `VALID_TABS`). Tab value = URL hash. +Dashboard route keys, paths, primary/feature-gated/footer-only navigation, indexability, provider capabilities, `/zh` mirroring, and share-parameter scopes are defined once in `packages/app/src/lib/dashboard-routes.ts`. `TabNav`, `DashboardShell`, metadata, i18n, share URLs, and the sitemap derive from that registry; do not add a second route array. ## Unofficial Run Support — Mandatory for Inference / Evaluation Features @@ -172,14 +177,18 @@ The site ships a hand-authored Simplified Chinese sibling for every indexable pa **Every new indexable page, dashboard tab, or blog post MUST ship its Chinese version in the same PR:** -1. **New page** → create `packages/app/src/app/zh//page.tsx` with fully translated content and metadata. Metadata: `alternates: zhAlternates('')` plus `openGraph.locale: ZH_OG_LOCALE`. Switch the English page's `alternates` to `enAlternates('')` so both sides carry bidirectional hreflang. Register the route in `ZH_MIRRORED_ROUTES` (`src/lib/i18n.ts`) so the header nav and EN↔中文 toggle link to it, and add it to the sitemap via `localizedPair()` in `src/app/sitemap.ts`. -2. **New dashboard tab** → add the tab to `ZH_TAB_KEYS`, `TAB_META_ZH`, `TAB_INTRO_ZH`, and `TAB_LABELS_ZH` in `src/lib/tab-meta-zh.ts`, then create `src/app/zh/(dashboard)//page.tsx` mirroring the English page with `tabMetadataZh('')` and a `` block above the chart; the chart's own UI strings must follow rule 5. `tab-meta-zh.test.ts` enforces dictionary completeness. +1. **New page** → create `packages/app/src/app/zh//page.tsx` with fully translated content and metadata. Metadata: `alternates: zhAlternates('')` plus `openGraph.locale: ZH_OG_LOCALE`. Switch the English page's `alternates` to `enAlternates('')` so both sides carry bidirectional hreflang. Register non-dashboard routes in `ZH_MIRRORED_ROUTES` (`src/lib/i18n.ts`) so the header nav and EN↔中文 toggle link to them, and add them to the sitemap via `localizedPair()`. +2. **New dashboard tab** → add one entry to `src/lib/dashboard-routes.ts`, then add exact entries to `TAB_META_ZH`, `TAB_INTRO_ZH`, and `TAB_LABELS_ZH` in `src/lib/tab-meta-zh.ts`. Create `src/app/zh/(dashboard)//page.tsx` mirroring the English page with `tabMetadataZh('')` and a `` block above the chart; the registry automatically supplies `/zh` route matching and indexable sitemap entries. The chart's own UI strings must follow rule 5. Registry and metadata tests enforce route parity and dictionary completeness. 3. **New blog post** → the translation `packages/app/content/blog/zh/.mdx` is REQUIRED in the same PR. Translate frontmatter `title`/`subtitle` and the body; keep `date`, `publishDate`, `modifiedDate`, `tags`, and the filename/slug identical (English and Chinese posts pair by filename; visibility gating always follows the English post's `publishDate`). Rewrite internal `/blog/` links to `/zh/blog/`; never alter numbers, code blocks, or `
`/`` structure. The `/zh/blog` listing, hreflang, and sitemap pick the file up automatically. -4. **Editing an existing English page or post** → update its Chinese sibling in the same PR. Content drift between languages is a 🔴 BLOCKING review issue. +4. **Editing an existing English page or post** → update its Chinese sibling in the same PR. Omitting the required Chinese sibling update is a 🔴 BLOCKING review issue. Fidelity or wording issues inside an updated Chinese sibling follow the advisory workflow below; serious findings require Chinese maintainer confirmation. 5. **ALL user-visible UI strings MUST have a Chinese equivalent** — no carve-outs for "chart internals" or "option labels". This includes: headers/footers, card titles/descriptions, control and filter labels, buttons, toggles (Log Scale, Optimal Only, …), nudges, dropdown OPTION display names (Y-axis metric names, token types, scale modes), searchable-select placeholders ("Search…"), table column headers and action buttons ("Prompts"), modal/drawer chrome, legend footnotes, and empty/loading/error messages. Mechanism: client components call `useLocale()` (`src/lib/use-locale.ts`) and read from a component-local `STRINGS = { en, zh }` dict; server components take an optional `locale` prop passed from the /zh page; registry-defined display names (e.g. `Y_AXIS_METRICS`, legend toggle configs) carry a `labelZh` field resolved through a locale-aware label helper at render time. The `en` values must keep the exact original strings so English pages stay byte-identical. -6. **What stays English** (only these): brand/product names, hardware SKUs, model/framework/precision names, units (tok/s/user, GB/s, $/M tok), code identifiers and flags — per the translation quality bar — plus DB-stored _content_ (benchmark rows, dataset conversation text, run logs), which is data, not UI. +6. **What stays English**: brand/product names, hardware SKUs, model/framework/precision names, units (tok/s/user, GB/s, $/M tok), code identifiers and flags — per the translation quality bar — plus established English technical terms and phrases that Chinese ML infrastructure engineers normally use in English, as documented contextually in `docs/chinese-copy.md` (`warmup`, `seed`, and `offload` are non-exhaustive examples), and DB-stored _content_ (benchmark rows, dataset conversation text, run logs), which is data, not UI. 7. **Compare slug narrative sync**: the per-slug compare pages are mirrored at `/zh/compare/[slug]` and `/zh/compare-per-dollar/[slug]`; their Chinese prose templates live in `src/lib/compare-ssr-zh.ts`, a 1:1 port of the English templates in `compare-ssr.ts`. The variant compare pages (`/zh/compare-precision/[slug]` and `/zh/compare-spec-decode/[slug]`) have their Chinese templates in `src/lib/compare-variant-ssr-zh.ts`, porting `compare-variant-ssr.ts`. Any PR that changes the English narrative templates MUST update the zh port in the same commit. -8. **Every route gets a /zh sibling — including hidden/feature-gated ones** (`/datasets`, `/ai-chart`, `/current-inferencex-image`, `/feedback`, agentic detail pages). Noindex routes keep their noindex on both sides. The only exceptions: `feed.xml`/`llms.txt` (single-language machine feeds) and per-post OG images (Chinese posts reuse the English post's OG image — the OG renderer's font has no CJK glyphs). +8. **Every route gets a /zh sibling, including hidden or feature-gated ones** (`/agentx`, `/ai-chart`, `/current-inferencex-image`, `/feedback`, agentic detail pages). Noindex routes keep their noindex on both sides. The only exceptions are `feed.xml` and `llms.txt` (single-language machine feeds) plus per-post OG images. Chinese posts reuse the English post's OG image because the renderer font has no CJK glyphs. + +> **Claude reviews every changed user-visible Chinese string, but the Chinese maintainer makes the final decision.** Load `review-zh-copy` before opening or reviewing any PR that touches user-visible Chinese text, including refactors whose filenames do not contain `zh`. The skill evaluates semantic fidelity and natural Chinese as separate gates and follows [docs/chinese-copy.md](./docs/chinese-copy.md). Its findings never block another contributor's merge. Do not mention `@edwingao28` for a clean review, routine coverage, or an ordinary wording or naturalness suggestion. Mention `@edwingao28` only for a high-confidence semantic or factual error, changed attribution or speaker voice, or unresolved high-impact ambiguity, and ask for confirmation. The Chinese maintainer makes the final editorial decision. + +> **Chinese copy has a narrow mechanical CI guard.** `packages/app/src/lib/zh-copy.test.ts` checks objective regressions: the standalone English common noun `Chip` in Chinese text (with explicit identifier and unit exemptions), prose-like English `` labels that bypass an available `zh:` dictionary, and redundant English-first double translations such as `warmup 预热`. It does not globally ban `预热`, `随机种子`, or `卸载`, and deliberately does **not** enforce pronouns, preferred contextual terminology, punctuation style, sentence structure, fluency, tone, or register; those require page context and human review. Every mechanical rule must use the same matcher for the source scan and a hand-checked case in `packages/app/src/lib/zh-copy-mechanical-regressions.jsonl`, including an exemption case whenever the rule has one. ## Chart Interpolation — TS and Python Helpers MUST Stay in Sync @@ -217,6 +226,7 @@ Authoritative total / active parameter counts for every model in the dashboard. | Kimi-K2.6 | 1T | 32B | `moonshotai/Kimi-K2.6` | HF model card | | Kimi-K2.7-Code | 1T | 32B | `moonshotai/Kimi-K2.7-Code` | HF model card | | Qwen3.5-397B-A17B | 397B | 17B | `Qwen/Qwen3.5-397B-A17B` | HF model card | +| Qwen3.8-Flash-Next | 176B | 6B | `Qwen/Qwen3.8-Flash-Next-FP8` | HF model card | | GLM-5 | 744B | 40B | `zai-org/GLM-5` | HF model card | | GLM-5.1 | 744B | 40B | `zai-org/GLM-5.1-FP8` | HF model card (same base as GLM-5) | | MiniMax-M2.5 | 230B | 10B | `MiniMaxAI/MiniMax-M2.5` | HF model card | @@ -226,6 +236,7 @@ Authoritative total / active parameter counts for every model in the dashboard. **Common mislabel traps** (have all bitten this repo at least once — do not repeat): +- **Qwen3.8-Flash-Next is 176B, not 125B.** The model card leads with "125B with 6B activated", but that is the main model only; the 51B n-gram embedding table brings the total to 176B. The separate 4B MTP head sits outside both figures. It is a Qwen4-architecture preview (GatedDeltaNet + Qwen Sparse Attention, 512 experts, 10 routed + 1 shared), not a Qwen3.5 point release, so it gets its own DB bucket. - **GLM-5 ≠ 355B.** 355B is GLM-4.5. GLM-5 jumped to 744B / 40B active (256-expert MoE with DSA). - **MiniMax-M2.5/M2.7 ≠ 456B.** 456B is the older MiniMax-Text-01 / M1 (32 large experts). The M2 series is a different architecture: 230B / 10B active, 256 small experts. - **DeepSeek-R1 is 671B, not 685B.** HF metadata shows 685B because the bundled MTP head adds ~14B; the core MoE is 671B / 37B active. @@ -248,13 +259,12 @@ Authoritative total / active parameter counts for every model in the dashboard. ### Add/modify a metric -1. Register in `src/lib/chart-utils.ts`: `Y_AXIS_METRICS`, `calculateRoofline`, `computeAllRooflines`, `markRooflinePoints` -2. Add TS types: optional field in `InferenceData`, add to `YAxisMetricKey`, add `ChartDefinition` fields -3. Add chart config: `src/components/inference/inference-chart-config.json` -4. Add Y-axis dropdown: `ChartControls.tsx` -5. Add subtitle/disclaimer in `ChartDisplay.tsx` if metric depends on assumed constants -6. Add disagg caveat banner in `ChartDisplay.tsx` for per-GPU or per-MW metrics (animated amber `border-l-2` banner pattern) -7. Expose in UI state: `InferenceContext.tsx` +1. Register the field, bilingual labels/titles, polarity, and custom-source metadata in `src/components/inference/metric-registry.ts` (`METRIC_REGISTRY`). +2. Add or reuse the derived field in `buildDerivedChartFields` (`src/lib/chart-utils.ts`) and the corresponding optional `InferenceData` field. Historical trends request the same selective builder; do not duplicate formulas. +3. Add the metric to `METRIC_CONTROL_GROUPS` when it belongs in a selector group. Chart definitions and metric key types derive from the registry. +4. Add subtitle/disclaimer copy in `ChartDisplay.tsx` if the metric depends on assumed constants. +5. Add the disaggregated-config caveat banner for per-GPU or per-MW metrics. +6. Verify both official rows and `?unofficialrun=` overlays, including clipping, CSV/table behavior where applicable, zoom, labels, and Chinese UI. ### Add a new blog post @@ -282,11 +292,12 @@ See [Blog](./docs/blog.md) for content format, available MDX components, and des ### Adding a new tab -1. `page-content.tsx`: Add to `VALID_TABS`, add `TabsTrigger` (desktop), `SelectItem` (mobile), `TabsContent` -2. Create a per-section context provider (see `InferenceContext.tsx`, `EvaluationContext.tsx` for patterns) -3. Use `ChartLegend` with `variant="sidebar"`, sorted by `HW_REGISTRY` sort order, default expanded -4. Analytics: all interactive elements use `track()` with `{tabname}_` prefix -5. Create the Chinese sibling: extend `src/lib/tab-meta-zh.ts` dictionaries and add `src/app/zh/(dashboard)//page.tsx` (see [Chinese Website Pages](#chinese-website-pages-zh--mandatory-for-all-indexable-surfaces)) +1. Add the canonical route entry to `packages/app/src/lib/dashboard-routes.ts`, including navigation group, indexability, provider capabilities, locale mirroring, and share scopes. +2. Add explicit English metadata in `src/lib/tab-meta.ts` and Chinese metadata, intro, and label entries in `src/lib/tab-meta-zh.ts`. +3. Create the English and Chinese App Router pages. `dashboard-routes.test.ts` enforces structural parity. +4. Add the tab content and any section-specific provider only when the route actually needs one; declare dashboard-wide provider needs in the registry. +5. Use `ChartLegend` with `variant="sidebar"`, sorted by `HW_REGISTRY` sort order, default expanded. +6. Track interactive actions with the `{tabname}_` analytics prefix. ### Bumping dependencies @@ -303,7 +314,7 @@ Workflow for a periodic dep bump. Branch: `chore/bump-deps-YYYY-MM-DD`. Commit e Detailed design rationale (the "why" and "how", not the "what") lives in [docs/](./docs/index.md): - **[Index](./docs/index.md)** — index of all docs **MUST ALWAYS READ IN CASE OF RELEVANT INFORMATION** -- **[Architecture](./docs/architecture.md)** — Client-first design, hash routing, caching, color system +- **[Architecture](./docs/architecture.md)** — Client-first design, route navigation, URL state, caching, color system - **[D3 Charts](./docs/d3-charts.md)** — 4-effect architecture, zoom refs, tooltip lifecycle - **[Data Pipeline](./docs/data-pipeline.md)** — DB schema reasoning, ETL design, spline interpolation - **[Pitfalls](./docs/pitfalls.md)** — Token type bugs, schema evolution, stale closures, zoom loss diff --git a/README.md b/README.md index 43b6b968b..b50e3ed8d 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ Neon PostgreSQL → API routes (/api/v1/*) → React Query hooks → Context pro ``` The frontend fetches data from API routes backed by a Neon PostgreSQL read replica. All presentation logic lives in the frontend — API routes return raw DB data. +The MCP server exposes read-only benchmark tools through the same `db` and `constants` packages, without depending on frontend code. ### Monorepo Structure @@ -28,7 +29,8 @@ The frontend fetches data from API routes backed by a Neon PostgreSQL read repli packages/ ├── app/ # Next.js frontend ├── constants/ # Shared constants (GPU keys, model mappings) -└── db/ # DB layer, ETL, migrations, queries, ingest scripts +├── db/ # DB layer, ETL, migrations, queries, ingest scripts +└── mcp/ # MCP server exposing read-only benchmark tools ``` ## Prerequisites @@ -105,27 +107,29 @@ Open [http://localhost:3000](http://localhost:3000) with your browser. These are the main scripts you'll use during development. Admin scripts for database and cache management are listed separately below. Some of these may require additional setup or environment variables. -| Script | Description | -| ------------------------------ | ---------------------------------------------- | -| `bun run dev` | Start development server with Turbopack | -| `bun run build` | Production build | -| `bun run start` | Start production server | -| `bun run preview` | Build then start production server locally | -| `bun run typecheck` | TypeScript type checking (all packages) | -| `bun run lint` | Lint with oxlint | -| `bun run lint:fix` | Auto-fix lint issues | -| `bun run fmt` | Format check with oxfmt | -| `bun run fmt:fix` | Auto-fix formatting | -| `bun run security` | Security audit | -| `bun run test` | Run all tests (unit + E2E) | -| `bun run test:unit` | Vitest unit tests | -| `bun run test:unit:coverage` | Vitest unit tests with coverage | -| `bun run test:e2e` | Curated local Cypress smoke suite | -| `bun run test:e2e:full` | Complete Cypress component + integration suite | -| `bun run test:e2e:component` | Complete Cypress component suite only | -| `bun run test:e2e:integration` | Complete Cypress integration suite only | -| `bun run clean` | Remove build artifacts | -| `bun run clean:all` | Remove build artifacts + node_modules | +| Script | Description | +| ------------------------------ | ----------------------------------------------- | +| `bun run dev` | Start development server with Turbopack | +| `bun run mcp` | Start the InferenceX MCP server | +| `bun run build` | Production build + route bundle budget guard | +| `bun run check:bundle-budgets` | Check existing Next.js route bundle diagnostics | +| `bun run start` | Start production server | +| `bun run preview` | Build then start production server locally | +| `bun run typecheck` | TypeScript type checking (all packages) | +| `bun run lint` | Lint with oxlint | +| `bun run lint:fix` | Auto-fix lint issues | +| `bun run fmt` | Format check with oxfmt | +| `bun run fmt:fix` | Auto-fix formatting | +| `bun run security` | Security audit | +| `bun run test` | Run all tests (unit + E2E) | +| `bun run test:unit` | Vitest unit tests | +| `bun run test:unit:coverage` | Vitest unit tests with coverage | +| `bun run test:e2e` | Curated local Cypress smoke suite | +| `bun run test:e2e:full` | Complete Cypress component + integration suite | +| `bun run test:e2e:component` | Complete Cypress component suite only | +| `bun run test:e2e:integration` | Complete Cypress integration suite only | +| `bun run clean` | Remove build artifacts | +| `bun run clean:all` | Remove build artifacts + node_modules | This repository uses Vitest and Cypress. Use the `bun run test:*` scripts above; `bun test` invokes Bun's separate built-in test runner and is not supported. diff --git a/README_zh.md b/README_zh.md index 94c800ae5..502586663 100644 --- a/README_zh.md +++ b/README_zh.md @@ -21,6 +21,7 @@ Neon PostgreSQL → API routes (/api/v1/*) → React Query hooks → Context pro ``` 前端从由 Neon PostgreSQL 只读副本支撑的 API 路由获取数据。所有展示逻辑都在前端 — API 路由只返回原始数据库数据。 +MCP server 通过相同的 `db` 与 `constants` package 提供只读基准测试工具,不依赖前端代码。 ### Monorepo 结构 @@ -28,7 +29,8 @@ Neon PostgreSQL → API routes (/api/v1/*) → React Query hooks → Context pro packages/ ├── app/ # Next.js 前端 ├── constants/ # 共享常量(GPU key、模型映射) -└── db/ # 数据库层、ETL、迁移、查询、数据摄取脚本 +├── db/ # 数据库层、ETL、迁移、查询、数据摄取脚本 +└── mcp/ # 提供只读基准测试工具的 MCP server ``` ## 前置条件 @@ -105,27 +107,29 @@ bun run dev 以下是开发过程中的常用脚本。数据库与缓存管理的运维脚本单独列在下方。 部分脚本可能需要额外的配置或环境变量。 -| 脚本 | 说明 | -| ------------------------------ | --------------------------------- | -| `bun run dev` | 启动开发服务器(Turbopack) | -| `bun run build` | 生产构建 | -| `bun run start` | 启动生产服务器 | -| `bun run preview` | 本地构建并启动生产服务器 | -| `bun run typecheck` | TypeScript 类型检查(所有包) | -| `bun run lint` | 使用 oxlint 进行 lint | -| `bun run lint:fix` | 自动修复 lint 问题 | -| `bun run fmt` | 使用 oxfmt 检查格式 | -| `bun run fmt:fix` | 自动修复格式 | -| `bun run security` | 安全审计 | -| `bun run test` | 运行所有测试(单元 + E2E) | -| `bun run test:unit` | Vitest 单元测试 | -| `bun run test:unit:coverage` | Vitest 单元测试(含覆盖率) | -| `bun run test:e2e` | 精选的本地 Cypress smoke 测试套件 | -| `bun run test:e2e:full` | 完整 Cypress 组件与集成测试套件 | -| `bun run test:e2e:component` | 仅运行完整 Cypress 组件测试套件 | -| `bun run test:e2e:integration` | 仅运行完整 Cypress 集成测试套件 | -| `bun run clean` | 清除构建产物 | -| `bun run clean:all` | 清除构建产物 + node_modules | +| 脚本 | 说明 | +| ------------------------------ | ----------------------------------- | +| `bun run dev` | 启动开发服务器(Turbopack) | +| `bun run mcp` | 启动 InferenceX MCP server | +| `bun run build` | 生产构建并检查路由 bundle 预算 | +| `bun run check:bundle-budgets` | 检查已有的 Next.js 路由 bundle 诊断 | +| `bun run start` | 启动生产服务器 | +| `bun run preview` | 本地构建并启动生产服务器 | +| `bun run typecheck` | TypeScript 类型检查(所有包) | +| `bun run lint` | 使用 oxlint 进行 lint | +| `bun run lint:fix` | 自动修复 lint 问题 | +| `bun run fmt` | 使用 oxfmt 检查格式 | +| `bun run fmt:fix` | 自动修复格式 | +| `bun run security` | 安全审计 | +| `bun run test` | 运行所有测试(单元 + E2E) | +| `bun run test:unit` | Vitest 单元测试 | +| `bun run test:unit:coverage` | Vitest 单元测试(含覆盖率) | +| `bun run test:e2e` | 精选的本地 Cypress smoke 测试套件 | +| `bun run test:e2e:full` | 完整 Cypress 组件与集成测试套件 | +| `bun run test:e2e:component` | 仅运行完整 Cypress 组件测试套件 | +| `bun run test:e2e:integration` | 仅运行完整 Cypress 集成测试套件 | +| `bun run clean` | 清除构建产物 | +| `bun run clean:all` | 清除构建产物 + node_modules | 本项目使用 Vitest 和 Cypress。请使用上表中的 `bun run test:*` 脚本;`bun test` 会调用 Bun 自带的另一套测试运行器,本项目不支持该命令。 diff --git a/bun.lock b/bun.lock index 4bf79debf..654639de0 100644 --- a/bun.lock +++ b/bun.lock @@ -6,14 +6,14 @@ "name": "inferencex", "devDependencies": { "@babel/core": "^8.0.1", - "@types/node": "^26.1.1", + "@types/node": "^26.2.0", "is-ci": "^4.1.0", "lefthook": "^2.1.10", - "oxfmt": "^0.60.0", - "oxlint": "^1.75.0", + "oxfmt": "^0.63.0", + "oxlint": "^1.78.0", "rimraf": "^6.1.3", "typescript": "^6.0.3", - "vite": "^8.1.5", + "vite": "^8.2.1", }, }, "packages/app": { @@ -22,9 +22,8 @@ "dependencies": { "@chenglou/pretext": "^0.0.8", "@jpinsonneau/html-to-image": "^1.11.13", - "@noble/ciphers": "^2.2.0", - "@posthog/nextjs-config": "^1.9.68", - "@radix-ui/react-accordion": "^1.2.20", + "@noble/ciphers": "^2.3.0", + "@posthog/nextjs-config": "^1.9.69", "@radix-ui/react-dialog": "^1.1.23", "@radix-ui/react-label": "^2.1.15", "@radix-ui/react-popover": "^1.1.23", @@ -33,26 +32,26 @@ "@radix-ui/react-switch": "^1.3.7", "@radix-ui/react-tabs": "^1.1.21", "@radix-ui/react-tooltip": "^1.2.16", - "@react-three/fiber": "^9.6.1", + "@react-three/fiber": "^9.7.0", "@semianalysisai/inferencex-constants": "workspace:*", "@semianalysisai/inferencex-db": "workspace:*", - "@shikijs/rehype": "^4.3.1", + "@shikijs/rehype": "^4.4.3", "@tailwindcss/typography": "^0.5.20", "@tanstack/react-query": "^5.101.4", "@vercel/analytics": "^2.0.1", - "@vercel/blob": "^2.6.1", + "@vercel/blob": "^2.8.0", "@vercel/speed-insights": "^2.0.0", "adm-zip": "^0.6.0", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "d3": "^7.9.0", - "dompurify": "^3.4.12", + "dompurify": "^3.4.13", "gray-matter": "^4.0.3", "iwanthue": "^2.1.0", "katex": "^0.16.47", - "lucide-react": "^1.27.0", + "lucide-react": "^1.31.0", "mp4-muxer": "^5.2.2", - "next": "^16.2.12", + "next": "^16.3.1", "next-mdx-remote": "^6.0.0", "next-themes": "^0.4.6", "posthog-js": "^1.407.2", @@ -61,7 +60,7 @@ "rehype-katex": "^7.0.1", "remark-gfm": "^4.0.1", "remark-math": "^6.0.0", - "shiki": "^4.3.1", + "shiki": "^4.4.3", "tailwind-merge": "^3.6.0", "three": "^0.185.1", }, @@ -71,16 +70,17 @@ "@types/adm-zip": "^0.5.8", "@types/d3": "^7.4.3", "@types/mdast": "^4.0.4", - "@types/node": "^26.1.1", - "@types/react": "^19.2.17", - "@types/react-dom": "^19.2.3", - "@types/three": "^0.185.1", + "@types/node": "^26.2.0", + "@types/react": "19.2.17", + "@types/react-dom": "19.2.3", + "@types/three": "^0.185.4", "@vitest/coverage-v8": "^4.1.10", - "cypress": "^15.19.0", + "axe-core": "^4.13.0", + "cypress": "15.19.0", + "cypress-axe": "^1.7.0", "cypress-split": "^1.25.0", - "esbuild": "^0.28.1", - "fake-indexeddb": "^6.2.5", - "jsdom": "^29.1.1", + "esbuild": "^0.28.2", + "jsdom": "^30.0.1", "rimraf": "^6.1.3", "tailwindcss": "^4.3.3", "tw-animate-css": "^1.4.0", @@ -101,7 +101,7 @@ "version": "0.1.0", "dependencies": { "@neondatabase/serverless": "^1.1.0", - "@noble/ciphers": "^2.2.0", + "@noble/ciphers": "^2.3.0", "@semianalysisai/inferencex-constants": "workspace:*", "postgres": "^3.4.9", "stream-chain": "^4.2.5", @@ -109,7 +109,7 @@ }, "devDependencies": { "@types/adm-zip": "^0.5.8", - "@types/node": "^26.1.1", + "@types/node": "^26.2.0", "@types/stream-json": "^1.7.8", "@vitest/coverage-v8": "^4.1.10", "adm-zip": "^0.6.0", @@ -121,14 +121,14 @@ "name": "@semianalysisai/inferencex-mcp", "version": "1.0.0", "dependencies": { - "@modelcontextprotocol/sdk": "^1.29.0", + "@modelcontextprotocol/sdk": "^1.30.0", "@semianalysisai/inferencex-constants": "workspace:*", "@semianalysisai/inferencex-db": "workspace:*", "postgres": "^3.4.9", "zod": "^4.4.3", }, "devDependencies": { - "@types/node": "^26.1.1", + "@types/node": "^26.2.0", "typescript": "^6.0.3", "vitest": "^4.1.10", }, @@ -142,6 +142,17 @@ "lefthook", "cypress", ], + "overrides": { + "brace-expansion": "5.0.9", + "dompurify": "3.4.13", + "fast-uri": "4.1.2", + "hono": "4.13.2", + "ip-address": "10.5.0", + "js-yaml": "3.15.1", + "jsdom": "30.0.1", + "nanoid": "3.3.18", + "undici": "8.10.0", + }, "packages": { "@actions/core": ["@actions/core@2.0.3", "", { "dependencies": { "@actions/exec": "2.0.0", "@actions/http-client": "3.0.2" } }, "sha512-Od9Thc3T1mQJYddvVPM4QGiLUewdh+3txmDYHHxoNdkqysR1MbCT+rFOtNUxYAz+7+6RIsqipVahY2GJqGPyxA=="], @@ -153,13 +164,9 @@ "@alloc/quick-lru": ["@alloc/quick-lru@5.2.0", "", {}, "sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw=="], - "@asamuzakjp/css-color": ["@asamuzakjp/css-color@5.1.11", "", { "dependencies": { "@asamuzakjp/generational-cache": "1.0.1", "@csstools/css-calc": "3.2.1", "@csstools/css-color-parser": "4.1.9", "@csstools/css-parser-algorithms": "4.0.0", "@csstools/css-tokenizer": "4.0.0" } }, "sha512-KVw6qIiCTUQhByfTd78h2yD1/00waTmm9uy/R7Ck/ctUyAPj+AEDLkQIdJW0T8+qGgj3j5bpNKK7Q3G+LedJWg=="], - - "@asamuzakjp/dom-selector": ["@asamuzakjp/dom-selector@7.1.1", "", { "dependencies": { "@asamuzakjp/generational-cache": "1.0.1", "@asamuzakjp/nwsapi": "2.3.9", "bidi-js": "1.0.3", "css-tree": "3.2.1", "is-potential-custom-element-name": "1.0.1" } }, "sha512-67RZDnYRc8H/8MLDgQCDE//zoqVFwajkepHZgmXrbwybzXOEwOWGPYGmALYl9J2DOLfFPPs6kKCqmbzV895hTQ=="], + "@asamuzakjp/css-color": ["@asamuzakjp/css-color@6.0.7", "", { "dependencies": { "@csstools/css-calc": "^3.3.0", "@csstools/css-color-parser": "^4.1.10", "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0", "lru-cache": "^11.5.2" } }, "sha512-vC/bk1Lz7Tn/EfU9/apOTBk80/8dyGyWMowPoV1tJ52muDGsDqt2HPT2klrFUiY60MQmQv9q8yIht15JnBgDGw=="], - "@asamuzakjp/generational-cache": ["@asamuzakjp/generational-cache@1.0.1", "", {}, "sha512-wajfB8KqzMCN2KGNFdLkReeHncd0AslUSrvHVvvYWuU8ghncRJoA50kT3zP9MVL0+9g4/67H+cdvBskj9THPzg=="], - - "@asamuzakjp/nwsapi": ["@asamuzakjp/nwsapi@2.3.9", "", {}, "sha512-n8GuYSrI9bF7FFZ/SjhwevlHc8xaVlb/7HmHelnc/PZXBD2ZR49NnN9sMMuDdEGPeeRQ5d0hqlSlEpgCX3Wl0Q=="], + "@asamuzakjp/dom-selector": ["@asamuzakjp/dom-selector@8.3.2", "", { "dependencies": { "bidi-js": "^1.0.3", "css-tree": "^3.2.1", "is-potential-custom-element-name": "^1.0.1", "lru-cache": "^11.5.2" } }, "sha512-93Z1N+BQNXysodoicpOIyNh2drHfz/CTf9nnT0FEx72GJcIiwgydD7tGAr78j41LsYn3hlRn+LdGPuBLn1Bl8Q=="], "@babel/code-frame": ["@babel/code-frame@8.0.0", "", { "dependencies": { "@babel/helper-validator-identifier": "8.0.2", "js-tokens": "10.0.0" } }, "sha512-dYYg153EyN2Ekbqw2zAsbd6/JR+9N2SEoC7YV2GyyqMM7x9bLDTjBD6XBhSMLH0wtIVyJj03jWNriQhaN+eoCw=="], @@ -203,15 +210,15 @@ "@chenglou/pretext": ["@chenglou/pretext@0.0.8", "", {}, "sha512-yqm2GMxnPI7VHcHwe84P8ZF0JK/2d2DMKPqMN+s95jQhwDMYYXKVFVJUMEaVWckQStdsjdLav/0Vu+d9YbtGxA=="], - "@csstools/color-helpers": ["@csstools/color-helpers@6.1.0", "", {}, "sha512-064IFJdjTfUqnjpCVpMOdbr8FLQBhinbZj6yRv2An2E41O/pLEXqfFRWqGq/SxlE5PEUYTlvWsG2r8MswAVvkg=="], + "@csstools/color-helpers": ["@csstools/color-helpers@6.1.1", "", {}, "sha512-gLNsunvwf3mCi5u5o46/Z/JcJMnhbHSaZ69rkgPzNM3J4s8hWwpPUQB6/tt0EDFyCiWzxANlx+2LJwpYj4zS1w=="], - "@csstools/css-calc": ["@csstools/css-calc@3.2.1", "", { "peerDependencies": { "@csstools/css-parser-algorithms": "4.0.0", "@csstools/css-tokenizer": "4.0.0" } }, "sha512-DtdHlgXh5ZkA43cwBcAm+huzgJiwx3ZTWVjBs94kwz2xKqSimDA3lBgCjphYgwgVUMWatSM0pDd8TILB1yrVVg=="], + "@csstools/css-calc": ["@csstools/css-calc@3.3.0", "", { "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-c5ihYsPkdG6JCkU2zTMm4+k6r7RXuGxtWYhu5DHMIiF1FHzrfmHL5so11AoFpUv/tu61xfcmT4AmKoFfMPoqdQ=="], - "@csstools/css-color-parser": ["@csstools/css-color-parser@4.1.9", "", { "dependencies": { "@csstools/color-helpers": "6.1.0", "@csstools/css-calc": "3.2.1" }, "peerDependencies": { "@csstools/css-parser-algorithms": "4.0.0", "@csstools/css-tokenizer": "4.0.0" } }, "sha512-paQcIaOO53Rk5+YrBaBjm/SgrV4INImjo2BT1DtQRYr+XeTRbeAYlS+jxXp9drqvKmtFnWRJKIalDLhZZDu42A=="], + "@csstools/css-color-parser": ["@csstools/css-color-parser@4.2.0", "", { "dependencies": { "@csstools/color-helpers": "^6.1.1", "@csstools/css-calc": "^3.3.0" }, "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.0", "@csstools/css-tokenizer": "^4.0.0" } }, "sha512-5+5LEmFuY1AjXdYhmgjTJogtQnP1evJ1zrBZGUNZ0thkpwnnmKxcHdAMn/OtFjAb25zA+jKDVYVRl+5G7rjv1A=="], "@csstools/css-parser-algorithms": ["@csstools/css-parser-algorithms@4.0.0", "", { "peerDependencies": { "@csstools/css-tokenizer": "4.0.0" } }, "sha512-+B87qS7fIG3L5h3qwJ/IFbjoVoOe/bpOdh9hAjXbvx0o8ImEmUsGXN0inFOnk2ChCFgqkkGFQ+TpM5rbhkKe4w=="], - "@csstools/css-syntax-patches-for-csstree": ["@csstools/css-syntax-patches-for-csstree@1.1.6", "", { "optionalDependencies": { "css-tree": "3.2.1" } }, "sha512-TcJCWFbXLPpJYq6z7bfOyjWYJDiDg2/I4gyUC9pqPNqHFRIey0EB0q0L5cSnQDfWJg8Jd6VadakxdIez/3zkqQ=="], + "@csstools/css-syntax-patches-for-csstree": ["@csstools/css-syntax-patches-for-csstree@1.1.8", "", { "peerDependencies": { "css-tree": "^3.2.1" }, "optionalPeers": ["css-tree"] }, "sha512-CpMLjAvwQg3BL5S0IeqsZNMH7EQrEWi0kLKOC13ZBF0ZwERiLWlibNPJr8G1kdU3Ms/r2KiNrF81pUh2HwAHdg=="], "@csstools/css-tokenizer": ["@csstools/css-tokenizer@4.0.0", "", {}, "sha512-QxULHAm7cNu72w97JUNCBFODFaXpbDg+dP8b/oWFAZ2MTRppA3U00Y2L1HqaS4J6yBqxwa/Y3nMBaxVKbB/NsA=="], @@ -227,61 +234,61 @@ "@emnapi/core": ["@emnapi/core@1.11.1", "", { "dependencies": { "@emnapi/wasi-threads": "1.2.2", "tslib": "2.8.1" } }, "sha512-RSvbQmHzdKzNsLYa/wHrbc3KN4sYLKAdPZxqiM2HATqv/SBk2/ENSHpvXGaLOMcsAyz0poEGqkmmKYG3OWiJEQ=="], - "@emnapi/runtime": ["@emnapi/runtime@1.11.1", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw=="], + "@emnapi/runtime": ["@emnapi/runtime@1.11.2", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-kyOl3X0DuTiT1h2ft8r2fYO8JYtU9a9Xis/zBSiGArNaagCOWx90N1k2wxp18czFDH+OgcWGb5ZP/XMt3dcyPA=="], "@emnapi/wasi-threads": ["@emnapi/wasi-threads@1.2.2", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-c95qOXkHdydNKhscBTebqEC1CVAZpyqOfVfBzQ1qgzyl3gfeldUjIggDbIZgDKsHLgnsM+igH7TJ/eAasaVuMA=="], - "@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.1", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ=="], + "@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ=="], - "@esbuild/android-arm": ["@esbuild/android-arm@0.28.1", "", { "os": "android", "cpu": "arm" }, "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ=="], + "@esbuild/android-arm": ["@esbuild/android-arm@0.28.2", "", { "os": "android", "cpu": "arm" }, "sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg=="], - "@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.1", "", { "os": "android", "cpu": "arm64" }, "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg=="], + "@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.2", "", { "os": "android", "cpu": "arm64" }, "sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A=="], - "@esbuild/android-x64": ["@esbuild/android-x64@0.28.1", "", { "os": "android", "cpu": "x64" }, "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng=="], + "@esbuild/android-x64": ["@esbuild/android-x64@0.28.2", "", { "os": "android", "cpu": "x64" }, "sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q=="], - "@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q=="], + "@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw=="], - "@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ=="], + "@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.2", "", { "os": "darwin", "cpu": "x64" }, "sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw=="], - "@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw=="], + "@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.2", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw=="], - "@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ=="], + "@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.2", "", { "os": "freebsd", "cpu": "x64" }, "sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg=="], - "@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.1", "", { "os": "linux", "cpu": "arm" }, "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ=="], + "@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.2", "", { "os": "linux", "cpu": "arm" }, "sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w=="], - "@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g=="], + "@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.2", "", { "os": "linux", "cpu": "arm64" }, "sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug=="], - "@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.1", "", { "os": "linux", "cpu": "ia32" }, "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w=="], + "@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.2", "", { "os": "linux", "cpu": "ia32" }, "sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ=="], - "@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg=="], + "@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ=="], - "@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ=="], + "@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA=="], - "@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ=="], + "@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.2", "", { "os": "linux", "cpu": "ppc64" }, "sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ=="], - "@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ=="], + "@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.2", "", { "os": "linux", "cpu": "none" }, "sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA=="], - "@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag=="], + "@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.2", "", { "os": "linux", "cpu": "s390x" }, "sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg=="], - "@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.1", "", { "os": "linux", "cpu": "x64" }, "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA=="], + "@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.2", "", { "os": "linux", "cpu": "x64" }, "sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ=="], - "@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw=="], + "@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw=="], - "@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.1", "", { "os": "none", "cpu": "x64" }, "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg=="], + "@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.2", "", { "os": "none", "cpu": "x64" }, "sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw=="], - "@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.1", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q=="], + "@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.2", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ=="], - "@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw=="], + "@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.2", "", { "os": "openbsd", "cpu": "x64" }, "sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw=="], - "@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg=="], + "@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.2", "", { "os": "none", "cpu": "arm64" }, "sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q=="], - "@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.1", "", { "os": "sunos", "cpu": "x64" }, "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ=="], + "@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.2", "", { "os": "sunos", "cpu": "x64" }, "sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g=="], - "@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA=="], + "@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.2", "", { "os": "win32", "cpu": "arm64" }, "sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ=="], - "@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg=="], + "@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.2", "", { "os": "win32", "cpu": "ia32" }, "sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA=="], - "@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.1", "", { "os": "win32", "cpu": "x64" }, "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A=="], + "@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.2", "", { "os": "win32", "cpu": "x64" }, "sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g=="], "@exodus/bytes": ["@exodus/bytes@1.15.1", "", {}, "sha512-S6mL0yNB/Abt9Ei4tq8gDhcczc4S3+vQ4ra7vxnAf+YHC02srtqxKKZghx2Dq6p0e66THKwR6r8N6P95wEty7Q=="], @@ -367,31 +374,31 @@ "@mdx-js/react": ["@mdx-js/react@3.1.1", "", { "dependencies": { "@types/mdx": "2.0.13" }, "peerDependencies": { "@types/react": "19.2.17", "react": "19.2.8" } }, "sha512-f++rKLQgUVYDAtECQ6fn/is15GkEH9+nZPM3MS0RcxVqoTfawHvDlSCH7JbMhAM6uJ32v3eXLvLmLvjGu7PTQw=="], - "@modelcontextprotocol/sdk": ["@modelcontextprotocol/sdk@1.29.0", "", { "dependencies": { "@hono/node-server": "2.0.11", "ajv": "8.20.0", "ajv-formats": "3.0.1", "content-type": "1.0.5", "cors": "2.8.6", "cross-spawn": "7.0.6", "eventsource": "3.0.7", "eventsource-parser": "3.0.8", "express": "5.2.1", "express-rate-limit": "8.5.2", "hono": "4.12.32", "jose": "6.2.3", "json-schema-typed": "8.0.2", "pkce-challenge": "5.0.1", "raw-body": "3.0.2", "zod-to-json-schema": "3.25.2" }, "peerDependencies": { "zod": "4.4.3" } }, "sha512-zo37mZA9hJWpULgkRpowewez1y6ML5GsXJPY8FI0tBBCd77HEvza4jDqRKOXgHNn867PVGCyTdzqpz0izu5ZjQ=="], + "@modelcontextprotocol/sdk": ["@modelcontextprotocol/sdk@1.30.0", "", { "dependencies": { "@hono/node-server": "^1.19.9 || ^2.0.5", "ajv": "^8.17.1", "ajv-formats": "^3.0.1", "content-type": "^1.0.5", "cors": "^2.8.5", "cross-spawn": "^7.0.5", "eventsource": "^3.0.2", "eventsource-parser": "^3.0.0", "express": "^5.2.1", "express-rate-limit": "^8.2.1", "hono": "^4.11.4", "jose": "^6.1.3", "json-schema-typed": "^8.0.2", "pkce-challenge": "^5.0.0", "raw-body": "^3.0.0", "zod": "^3.25 || ^4.0", "zod-to-json-schema": "^3.25.1" }, "peerDependencies": { "@cfworker/json-schema": "^4.1.1" }, "optionalPeers": ["@cfworker/json-schema"] }, "sha512-xKd8OIzlqNzcqcNumGAa6g+PW2kjD5vrpcKOnfldAUPP3j7lnqMPwlTXQm8gF+UwH72z0lqaRbjr9hqGz0eITA=="], "@napi-rs/wasm-runtime": ["@napi-rs/wasm-runtime@1.1.6", "", { "dependencies": { "@tybys/wasm-util": "0.10.3" }, "peerDependencies": { "@emnapi/core": "1.11.1", "@emnapi/runtime": "1.11.1" } }, "sha512-ZLv/JdUfkvOy9eCnnBaGfiO+XimbjebAeO+MRQqD/B+FR1tnRN0tpKSJHRbE8sFfS6aqsXZ67TQjfwfsxULVbg=="], "@neondatabase/serverless": ["@neondatabase/serverless@1.1.0", "", {}, "sha512-r3ZZhRjEcfEdKIZnoB1RusNgvHuaBRqfCzV4Gi+5A9yUX0S4HTws/ASWqt13wL4y4I+0rqsWGdA2w7EQXHi3+Q=="], - "@next/env": ["@next/env@16.2.12", "", {}, "sha512-d0Z5Bc13Fa4nR8pFAKx2jay2yhJM16vlfHbTzYnUQAxlNb6B6lmn4hjt69lYNt4kRtyYP6gEM49lPRHNbIyneg=="], + "@next/env": ["@next/env@16.3.1", "", {}, "sha512-35G3xwkQUb2oETSDjFXGrVugknoayLFBh7vSE+yAcl9IP2zT9wyGwq7297AYHR11kJld807t5f8AJBs6WBzXsQ=="], - "@next/swc-darwin-arm64": ["@next/swc-darwin-arm64@16.2.12", "", { "os": "darwin", "cpu": "arm64" }, "sha512-0W1R0teHWJrqKX0FH20IzzIWAOuGtBxPGuObrxy1lE8hQvCFj49KE8a3WUg0D7sq6rn6zkM4c7YGUnhudBS6oA=="], + "@next/swc-darwin-arm64": ["@next/swc-darwin-arm64@16.3.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-ABMIu2zQ7cnNIHm5ivKGwZwUrm0pAai3yiJ/gK/rF1c1VP9UOnj7XECbMKFdVKp9I9eMYq9NoDs1WXOoowxzJw=="], - "@next/swc-darwin-x64": ["@next/swc-darwin-x64@16.2.12", "", { "os": "darwin", "cpu": "x64" }, "sha512-Hy5Ls099+aFUmOLmIgPfLqNi6iCwhL3uQCssz5rWk+5Nkc6TUKCE83DY5BbNylfm3+mfwcSFnLRfrZDJhVxdtw=="], + "@next/swc-darwin-x64": ["@next/swc-darwin-x64@16.3.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-gNG21e/UnrroeScbY/QndUEdl0mF1FRibW7BBeYUz/5ABCepjqDdEdgr592vpzMtCn/m7FTjYq3TN4TpyDnutw=="], - "@next/swc-linux-arm64-gnu": ["@next/swc-linux-arm64-gnu@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-+YqU2h1cQkHsGfvjAsrSmst8UIFBibBGm5x3Xgel8NLMiDQtNOM4sM2GOEMvG5YiOBNeN/Ykk8cQC2S0Xrqljg=="], + "@next/swc-linux-arm64-gnu": ["@next/swc-linux-arm64-gnu@16.3.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-6B6Lw016iwNUQuaJoraMMTLh6TwHzFUtxipSScD1F3YyymcrRWkobodRS2ftIOkF5vrs4zNlyUrTC5YZQ9Lz5w=="], - "@next/swc-linux-arm64-musl": ["@next/swc-linux-arm64-musl@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-0qjhiYBaKAqF63LA1ZWAAnKTzFUguAaZiRa5etMLGGPj/B6uEVjtIZldIzFEp3wHlB0koK6aTzqPtSdplTCjoA=="], + "@next/swc-linux-arm64-musl": ["@next/swc-linux-arm64-musl@16.3.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-JUiPXZKK9wOhjf4MgDiH29GZLxfqOesbLtHq2pDxwH/WwscTRV2ToymnOTh1egzaZf0ueUf8T2+CeYTGHjW0Iw=="], - "@next/swc-linux-x64-gnu": ["@next/swc-linux-x64-gnu@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-7A3q26W+h7gnA15uqBToNuDqBEFZZcqh0mW2mn4AJh/G5pdg2RVE3n4slzLEliASZFG3NmsbEzng/x2Sh09mBg=="], + "@next/swc-linux-x64-gnu": ["@next/swc-linux-x64-gnu@16.3.1", "", { "os": "linux", "cpu": "x64" }, "sha512-Uog9jsrmIRIL/lfvIp9htmskSNC7JcQsMVucXL2V2YY1y/D9IUN3LPEafqy0zRJ2cIU1SQ0V6F6TlffQ+pLAGg=="], - "@next/swc-linux-x64-musl": ["@next/swc-linux-x64-musl@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-qSjL/uppm+cbh21s72Ss8gkiOhQ4dExWHNGOWy6eZV7STj5WsKehgxT61beSsOj+YYQuTplL376lOCdMQU5T8w=="], + "@next/swc-linux-x64-musl": ["@next/swc-linux-x64-musl@16.3.1", "", { "os": "linux", "cpu": "x64" }, "sha512-6yy3FT13KgUFOj5H8bl8w/6nKiJwHIvbtwh1V+1acsu+7y4tJjnemSa6mhsh53BeoVrlozE+fMgZhXH46WmjMA=="], - "@next/swc-win32-arm64-msvc": ["@next/swc-win32-arm64-msvc@16.2.12", "", { "os": "win32", "cpu": "arm64" }, "sha512-X6hzsOUJac/e7AWSbn9gQ9nzHld1xWP5iyjHpYWvud8pufB679O1xg4JDyKr8Xd69Jvd+kM2Der6uftiZCmjYA=="], + "@next/swc-win32-arm64-msvc": ["@next/swc-win32-arm64-msvc@16.3.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-iOoN1QecUoGNZik536U/vtK43YwgyrCsGIkth52yIkl612n+0C9MjSnJbQAikISpb+WYRooBVhaDlUW7iZoKog=="], - "@next/swc-win32-x64-msvc": ["@next/swc-win32-x64-msvc@16.2.12", "", { "os": "win32", "cpu": "x64" }, "sha512-F6fakeHuFTLOPt0bslQJdf+xtT+WIP9DVn/m4y1w1mRnVPyh3D/cNvzlRkxM444xfm+IvvYNSOrKiA2CDJ0Uxw=="], + "@next/swc-win32-x64-msvc": ["@next/swc-win32-x64-msvc@16.3.1", "", { "os": "win32", "cpu": "x64" }, "sha512-d/k+PpAriUPaeMJJOG7HUSdqfEX46FEPWU1p3/nm2ACmXhj9hFEWdFODUBIpkuijXYkfL90qZzTqVPRp4BW/hw=="], - "@noble/ciphers": ["@noble/ciphers@2.2.0", "", {}, "sha512-Z6pjIZ/8IJcCGzb2S/0Px5J81yij85xASuk1teLNeg75bfT07MV3a/O2Mtn1I2se43k3lkVEcFaR10N4cgQcZA=="], + "@noble/ciphers": ["@noble/ciphers@2.3.0", "", {}, "sha512-Clu/xdfgVTf9o7ngLOURaxePwR0j8sjclKEtVij10/jGulwFsPWCvvRgG/XjUVf8Nei+jLG6uwyXzUTGY1DQrw=="], "@nodelib/fs.scandir": ["@nodelib/fs.scandir@2.1.5", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "run-parallel": "1.2.0" } }, "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g=="], @@ -401,108 +408,104 @@ "@opentelemetry/api": ["@opentelemetry/api@1.9.1", "", {}, "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q=="], - "@oxc-project/types": ["@oxc-project/types@0.139.0", "", {}, "sha512-r9gHphtCs+1M7J0pw6Sn/hh/Wpa/iQrOOkrNAlVLF/gHq+/CJmHIWKKUUhdWjcD6CIa8idarspCsASiXCXvFUw=="], + "@oxc-project/types": ["@oxc-project/types@0.144.0", "", {}, "sha512-nuhZIOLuI6TFQ32I/WnUx+SCPY7SdSKwgnFHydAuoS1+Z4BRcaP+RRJmGzl9lw+0OFF7UmaESf7KQRXaNLHypg=="], - "@oxfmt/binding-android-arm-eabi": ["@oxfmt/binding-android-arm-eabi@0.60.0", "", { "os": "android", "cpu": "arm" }, "sha512-1q4q4Jc8FlOMVojEisyFAVyl8h1yawNv6phjgmhGVEDeyeOdsSnSr9x0+D4mOnEKvpO5L4mxKZ/DP9X6U3A/Mw=="], + "@oxfmt/binding-android-arm-eabi": ["@oxfmt/binding-android-arm-eabi@0.63.0", "", { "os": "android", "cpu": "arm" }, "sha512-YmRth4ZPGgEXcgmkhvANbC9uD67dxmSobW7DQuyt5tOBOKvPnIpk5SVHBj88E+7wMNRI2FhqaDbOhQFBix+b8A=="], - "@oxfmt/binding-android-arm64": ["@oxfmt/binding-android-arm64@0.60.0", "", { "os": "android", "cpu": "arm64" }, "sha512-tD41I6nCt9k8SQXft0CSjjU9jg6SwG7uMu7PxodSEHXl+GDW0868oy6tTtoJkyUze8YKFgTpz/k5LuPUnFiGLw=="], + "@oxfmt/binding-android-arm64": ["@oxfmt/binding-android-arm64@0.63.0", "", { "os": "android", "cpu": "arm64" }, "sha512-icbahX8X2X3sRamOMecvdYeZXWjPDazRDIfvWfy7Ca1nc/ZDT2Y9k5Nt7s46EqFd7NQPdgk+CM3/SgIT5LPCaQ=="], - "@oxfmt/binding-darwin-arm64": ["@oxfmt/binding-darwin-arm64@0.60.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TTpzPug96Zxdyb46KvTyIUQDdsqbumXh2TKG9C23PCT0kF7JkW56Z/quPuG9rqOFKQIi1gpRNZ7DX18LwxXPnw=="], + "@oxfmt/binding-darwin-arm64": ["@oxfmt/binding-darwin-arm64@0.63.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-WV+Ze5v5gI2qoj8jpAovt8KBTW8pjEz/AiMXXjeTQS+Bmf/MmZXTS40S8xNPDszX+W8WDv2Bbk6qKrMTtUGu1A=="], - "@oxfmt/binding-darwin-x64": ["@oxfmt/binding-darwin-x64@0.60.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-CnOoWgQ7L+JL/YQaRJ+NyATciSfcftncm7y3kqyte1cGtFEGnStaCd1TAyrinkfQ7nRBfHrTs1/vTwUJr3WF2Q=="], + "@oxfmt/binding-darwin-x64": ["@oxfmt/binding-darwin-x64@0.63.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-CJGSBdDxXOWIpoFXHpverimCvz084KA7L483rqJ44c3jDtzv6d4qOSoR/V9ywSHfV+Ks1lwIj2P49BFhunLNAA=="], - "@oxfmt/binding-freebsd-x64": ["@oxfmt/binding-freebsd-x64@0.60.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-ychJo7S3hZxdO6eDZ9zM6F2lM9fpJS3EKS5CAUSWyprdLYxTu4gbaUKV/VBPTcMJwQa2Bpo+643y3OJ537pihA=="], + "@oxfmt/binding-freebsd-x64": ["@oxfmt/binding-freebsd-x64@0.63.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-BDfKY+KhL2078cgswBBFQPAYuxCy93bS/iC5frdSeSbTLcGrR6VC2hsuPTanoJmg84+wSyWl0wWC1eR+uTnkRg=="], - "@oxfmt/binding-linux-arm-gnueabihf": ["@oxfmt/binding-linux-arm-gnueabihf@0.60.0", "", { "os": "linux", "cpu": "arm" }, "sha512-36IH5o55T2Fx7E0feDttt+mifxN6yk9pWv4KfhAIsP0dFnUq27331OwbpOsZdoXF9soOLWm7mQUz5+UUmyec4g=="], + "@oxfmt/binding-linux-arm-gnueabihf": ["@oxfmt/binding-linux-arm-gnueabihf@0.63.0", "", { "os": "linux", "cpu": "arm" }, "sha512-Ov1cQEXT4mj7cojAokWSS1eoxkoyvbDfAbxNsGIKY2o36kvdAaFzPxRN6NxFRk9fD72B8oCoTTX/NuYTUWlpsg=="], - "@oxfmt/binding-linux-arm-musleabihf": ["@oxfmt/binding-linux-arm-musleabihf@0.60.0", "", { "os": "linux", "cpu": "arm" }, "sha512-G1Ve7lAa6sFBolVI2LWHfEAqy0YKh4vnioH8uYO9kAEdgM7mR40IksIx9/Zk4+vbYew/sGa4J9Q4tZ3n9gXDHA=="], + "@oxfmt/binding-linux-arm-musleabihf": ["@oxfmt/binding-linux-arm-musleabihf@0.63.0", "", { "os": "linux", "cpu": "arm" }, "sha512-0LE7ro3+6L79jcMANycAZfRaC7zxr9YZ2+vEL5uMD9QlEep+rS/r1kSJsnuLl991NXJZD60euh0PC1GHrR20vw=="], - "@oxfmt/binding-linux-arm64-gnu": ["@oxfmt/binding-linux-arm64-gnu@0.60.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-LTQdRBf6uzj/h7Xk6lKzbGD2hrF/fK4YI9LIN1c0509tPUn8wRa3mCmrFQpEWJPLYGFrLFFMTYW1Ljj6VqW2Hw=="], + "@oxfmt/binding-linux-arm64-gnu": ["@oxfmt/binding-linux-arm64-gnu@0.63.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-izPk+2Z4gjuZK32Fqh5qXoMpT/2NXzLh++ob57HiEiVSQZ1iYXu8EKMzb+K5AvWyIEXhdDIt7ADjGGtFhkT9Bw=="], - "@oxfmt/binding-linux-arm64-musl": ["@oxfmt/binding-linux-arm64-musl@0.60.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-2JMo3XPxMPx3hiqddSZYyaH+fKJm6cz0u8n1naYjP/CdOQOZW34i8lKBUfmbWiuFvd6KoYXLmhAyBuvojsYS7Q=="], + "@oxfmt/binding-linux-arm64-musl": ["@oxfmt/binding-linux-arm64-musl@0.63.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-alPmbOuWXFXiSo+lOtv6X71C7SYMEDW2WVvywOvf9BwKgEhSNGhMTLeFVSjKUMCamcjbbgVdsWF8GN1uy8xshg=="], - "@oxfmt/binding-linux-ppc64-gnu": ["@oxfmt/binding-linux-ppc64-gnu@0.60.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-L3C+nBD13lr306tr/PjM3RMll+BVqgFrIgUyoeHuai5oueJrRLgO3j+GO5/Cbhtkf5PSlHYTI1JY7iqBd1qa6A=="], + "@oxfmt/binding-linux-ppc64-gnu": ["@oxfmt/binding-linux-ppc64-gnu@0.63.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-BdzCPvolJc4AWZ+YMzgUDJcDzbQWrFjYuqBHoNHNqP1aCaluQRJNs4k3vNU5IG7vTpjf9zeD73D7MFM1TecZpg=="], - "@oxfmt/binding-linux-riscv64-gnu": ["@oxfmt/binding-linux-riscv64-gnu@0.60.0", "", { "os": "linux", "cpu": "none" }, "sha512-M4MsmvqlxFiPtSRGyBYQSZxchEf463AOyd+Dh4/9xDpjWBsRtDUTDMFN5EdHinjVK1/eDJQ8MLpcYjpYayaCnA=="], + "@oxfmt/binding-linux-riscv64-gnu": ["@oxfmt/binding-linux-riscv64-gnu@0.63.0", "", { "os": "linux", "cpu": "none" }, "sha512-7sIgfLzqtNKSkMGsGVyRpHwpjNezRg2XONvUOheFZs95TSZpM0JAuPpA8KrQFsWc4wPU95roX2O69JgH8igOgw=="], - "@oxfmt/binding-linux-riscv64-musl": ["@oxfmt/binding-linux-riscv64-musl@0.60.0", "", { "os": "linux", "cpu": "none" }, "sha512-OH+9UskYuxRB+GxqdGkVN8f5UpwhqG8YscNo1wl8+KJ62cd7wZdGga6iGLJIf8kibF1WBwvlfDUx3cez/VXwFg=="], + "@oxfmt/binding-linux-riscv64-musl": ["@oxfmt/binding-linux-riscv64-musl@0.63.0", "", { "os": "linux", "cpu": "none" }, "sha512-9Tcg0y0WcVa6Mm9AgcgFMseDS+VkFJZpKZ8We9SpDY4gg5jewSwln+0sO04QLcTS1BtfDl9MwR+NfID8L7PUTg=="], - "@oxfmt/binding-linux-s390x-gnu": ["@oxfmt/binding-linux-s390x-gnu@0.60.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-y7AAFutt9wFWBFOAn6+BHaV39usZmcr3YYH2385f+NHgPNpIF9HpqKp0jgUxPaUOCyG3oaX5VhJduL1Nw164rw=="], + "@oxfmt/binding-linux-s390x-gnu": ["@oxfmt/binding-linux-s390x-gnu@0.63.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-qWKC1pEOpx1qYhXaugPhHUeXwSfqEOk2wJH2LqVXGPV5iQYfdAZdt+d2XDiX4DTSWA2QDMUcFB+wEORh3Xn/sA=="], - "@oxfmt/binding-linux-x64-gnu": ["@oxfmt/binding-linux-x64-gnu@0.60.0", "", { "os": "linux", "cpu": "x64" }, "sha512-yKZ9+CXAI+1RO5nH/4Z/9M6DAsfOzd5bw/gtWk81KB4mpalMaRRSXfouc5/tHxazDmBek55HNPepNYBgaCew0Q=="], + "@oxfmt/binding-linux-x64-gnu": ["@oxfmt/binding-linux-x64-gnu@0.63.0", "", { "os": "linux", "cpu": "x64" }, "sha512-S9wXYOiGSqYGS4Fx/TFsY+xDd/7dE5s+rUgbA4TsHiVF9e8J3ZcKmP7dsP/7iqLI9Wz7Ic7TzEr3mdthRCTdrA=="], - "@oxfmt/binding-linux-x64-musl": ["@oxfmt/binding-linux-x64-musl@0.60.0", "", { "os": "linux", "cpu": "x64" }, "sha512-bCUGaF6hJOYnQzLJdHLZbvGsOd5oSvGAyJhPAKum2uyLYUuXmP8vqg690DWi2hqcnIoYpqSqCrjzE5aiUAgwQg=="], + "@oxfmt/binding-linux-x64-musl": ["@oxfmt/binding-linux-x64-musl@0.63.0", "", { "os": "linux", "cpu": "x64" }, "sha512-5eGyTJuMZNwBSHCivXt8Yuta6GeTYksOPXRk2MIhajiyFGQx7bjaHIwY+ZusAoFHhT157A9x6sktLjYo9D5oMQ=="], - "@oxfmt/binding-openharmony-arm64": ["@oxfmt/binding-openharmony-arm64@0.60.0", "", { "os": "none", "cpu": "arm64" }, "sha512-GrUeZOvzP30ExxfCuQiyofuUGI+OmvAgFwOO5w5p9mGPlxcyuqI+6Sy9fAKFFfLQrqKYWFgc5sYA2Unj/29nPg=="], + "@oxfmt/binding-openharmony-arm64": ["@oxfmt/binding-openharmony-arm64@0.63.0", "", { "os": "none", "cpu": "arm64" }, "sha512-Rz7hx+Dv3DoW/S6pwVAyjfFXp7/trdQ1zg+vNmsdsdDNlUccugp4XNqambSuEAeP0DaG9k72AtNyfDXCEg0AGw=="], - "@oxfmt/binding-win32-arm64-msvc": ["@oxfmt/binding-win32-arm64-msvc@0.60.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-WD4Q954kUl2TDJV/6q7UnE2rlKk047kXLJsr4bJ2mXRaAqNXcmV3nwKUsGCc3mz/jYDBnXtJEaBErJEybK8iQQ=="], + "@oxfmt/binding-win32-arm64-msvc": ["@oxfmt/binding-win32-arm64-msvc@0.63.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-T/IuizKN9mr4Xw6YYnptkXRNdLkyIlUZ7c8zfTOBpoytZyJ1BAsMUvsMDEx0X4YvSMpaivm+DR8112rQfzC25g=="], - "@oxfmt/binding-win32-ia32-msvc": ["@oxfmt/binding-win32-ia32-msvc@0.60.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-HqDekjr8JXzVDUP1YthDZ1Y3CBEcuZT4WX3B+1kaxj8CvZA8Y2YhcEsXqoSop3tVsgjACxjnFQFDkBo0r/jq1Q=="], + "@oxfmt/binding-win32-ia32-msvc": ["@oxfmt/binding-win32-ia32-msvc@0.63.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-XjrO5FJ5Wl9vsAxtCP1G/eaeT6y1K2s9CICUHGE42cEjou32/J6S+B1KnrOAboj6E7uhJnwPbRSvznWcxNdA0g=="], - "@oxfmt/binding-win32-x64-msvc": ["@oxfmt/binding-win32-x64-msvc@0.60.0", "", { "os": "win32", "cpu": "x64" }, "sha512-tz78yhmGPKboTMHCHSaUqXK8JrmoSejgDcWeqAtg2s07ZGKQ3rH5Jn8NuXPGNG33CDbY2e9NoQWXIVEmKO21Rw=="], + "@oxfmt/binding-win32-x64-msvc": ["@oxfmt/binding-win32-x64-msvc@0.63.0", "", { "os": "win32", "cpu": "x64" }, "sha512-sgsHCQy432OTQH4Ikk3tZptp3GqwnhwUDuY0loBH41zyHWfMZY9v8Dy78wsnSofHejvFozZGgJgBB1A0LQRwMQ=="], - "@oxlint/binding-android-arm-eabi": ["@oxlint/binding-android-arm-eabi@1.75.0", "", { "os": "android", "cpu": "arm" }, "sha512-lutovtFzJqlRaqpZrCqSSGaHZzl9nIxxpjLzhSRLunN6dCLylj0uzlCyQGaQDIys7rrv8kVXiFO+R4Zpn0bX7g=="], + "@oxlint/binding-android-arm-eabi": ["@oxlint/binding-android-arm-eabi@1.78.0", "", { "os": "android", "cpu": "arm" }, "sha512-Bu819lmAfZMUHErrpe0cEWj3iaefuUODHSU8+UbXy67V/r7/7f4K3FL0NmbD85E+wiFLDYuhP8Zlv0XnVeXshw=="], - "@oxlint/binding-android-arm64": ["@oxlint/binding-android-arm64@1.75.0", "", { "os": "android", "cpu": "arm64" }, "sha512-hXI0hDgHkw4w5nfru72aG7y+2iQJmC4waH/KV6H/hbgA6yAP5jYNx0P9yug15Hs0tWl/+mda3Jjn/2gmDT48tw=="], + "@oxlint/binding-android-arm64": ["@oxlint/binding-android-arm64@1.78.0", "", { "os": "android", "cpu": "arm64" }, "sha512-CDfxZgB61B7buRdY2FJoAYYPPXCZ1EoC1LKscnC5dg3kjobdxiconvAvvN1BmHyW4PyFT3jRLDag/BY/roSNBQ=="], - "@oxlint/binding-darwin-arm64": ["@oxlint/binding-darwin-arm64@1.75.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-D91BWbK/dMYfCcrghspPIuKs2D9LF4Z/OabVSQjw1AO6PWxArD7teDA48bm0ySFqWDaPVqmQRl5GMWNglTXyrQ=="], + "@oxlint/binding-darwin-arm64": ["@oxlint/binding-darwin-arm64@1.78.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-2Y2U9Ahrz+OO0Ej88f9SJYq51/jUBp1Mc7iZu0ukrbeeZ3gpRGfzIFnoqfHDY96xr0GEfNrPUBFEy0nN5aD7HA=="], - "@oxlint/binding-darwin-x64": ["@oxlint/binding-darwin-x64@1.75.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-02mpwzf12BonZ6PT0TuQoomvEh2kVl2WGBIKWezCyToIS+rYkQZ6GXnARBAl9A4Ovm2V+Xe7M4KretyqmmcnJQ=="], + "@oxlint/binding-darwin-x64": ["@oxlint/binding-darwin-x64@1.78.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-rpych6eJq6m9jDRypTEaPD1xysaEW5h9+xuxhGK/QhOg+/xaqPZrCrTNoIl/f3nEjuJeCEmstNDlrE9rJi/3/g=="], - "@oxlint/binding-freebsd-x64": ["@oxlint/binding-freebsd-x64@1.75.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-qZJgLnDaBsiL5YESx2t/TZ8eXkL9fEkKoXEdzegROhlz9A0lgyGnZ0dAzJrh7LJAHQl2K9RdRueN2s/9N7+odg=="], + "@oxlint/binding-freebsd-x64": ["@oxlint/binding-freebsd-x64@1.78.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-IcMGrQT3QizkOESUJd5et+rOhVqSkNDfNik1cvrKDqIbzqx9KMtRswpFgkCuNTSwylCFLKhGUu8KmqY1ZnC0Dg=="], - "@oxlint/binding-linux-arm-gnueabihf": ["@oxlint/binding-linux-arm-gnueabihf@1.75.0", "", { "os": "linux", "cpu": "arm" }, "sha512-7XlaWA5BJD3XpCfrEqjEe6Zseeb14S7QGa304XfwKignRaKQ+eIj775BQ7nIslggWickl4IsPUFqJ+/gAyNHVg=="], + "@oxlint/binding-linux-arm-gnueabihf": ["@oxlint/binding-linux-arm-gnueabihf@1.78.0", "", { "os": "linux", "cpu": "arm" }, "sha512-/uLdoJ0IXE6vo/0f0LKjinQAp+re+VMaCWaNT8ENIv2EOCkSsc8SGaflXAuW0Jua2dq5+GLVWm1NQK7P3UFSNQ=="], - "@oxlint/binding-linux-arm-musleabihf": ["@oxlint/binding-linux-arm-musleabihf@1.75.0", "", { "os": "linux", "cpu": "arm" }, "sha512-av6Tpv8yrcMMMOadOqENBhlsLRcGFXXwoQ0hzHhsmS9FJ4Wioy8we427GbcMe2XTxmL2e60T67H1Dyr3up+tAA=="], + "@oxlint/binding-linux-arm-musleabihf": ["@oxlint/binding-linux-arm-musleabihf@1.78.0", "", { "os": "linux", "cpu": "arm" }, "sha512-7xi4Wb/O8NRJhLoUXmDJMUVpNYvB5kefdhFU1Jb8rtae4QoXlTiLwI14X4YvAXVZLNZChP8m5qO9SQAlWQTbkQ=="], - "@oxlint/binding-linux-arm64-gnu": ["@oxlint/binding-linux-arm64-gnu@1.75.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-WcUhd8fHT5plrA14lANevl+hOl815mVI5t2hU21oFWrZKFXIVV/Sr4rWQV0NzSvzBupbMLNc5ErEA6Ehxh5jMg=="], + "@oxlint/binding-linux-arm64-gnu": ["@oxlint/binding-linux-arm64-gnu@1.78.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-4hFW0+fVXa3OIh1Y4A5SPkmvI4wuuBSrCVKzOyE7PTjhc7yEqZ1pmvEEeS5Lj/MaqvegFxXyF33N+6jkehxdyg=="], - "@oxlint/binding-linux-arm64-musl": ["@oxlint/binding-linux-arm64-musl@1.75.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UWzp5wRHFe/ESO3+eEaxXsTkYTGLYjnTsi/I5neEacXSItQ6WNleapfOAeA4x2b8nyhJ4uQxqvtv9pHv8kWJtQ=="], + "@oxlint/binding-linux-arm64-musl": ["@oxlint/binding-linux-arm64-musl@1.78.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-oC0mvsgBJjlMijSDEhx9KuvR9zYeHXceA9MjbuXB1F8NSR78Yj2unOBrstEvTVaq+pko+kuue6DajC00eqvTdg=="], - "@oxlint/binding-linux-ppc64-gnu": ["@oxlint/binding-linux-ppc64-gnu@1.75.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-XEVRwGMLKCUKrvhLAz4F6AIh8MJrQVdSZtAmPpRZt9tGPsUnamPOcl3dS/ZQzJnar/Ymgc//+xho0L60Emzuxg=="], + "@oxlint/binding-linux-ppc64-gnu": ["@oxlint/binding-linux-ppc64-gnu@1.78.0", "", { "os": "linux", "cpu": "ppc64" }, "sha512-XAllT5SUZS+ohjuZ3/5S0cwe0r7eboiuigeStCZ5DXRYx/2KVM2UvQXvAfyzXEimtQjAB7cDQ2YxDe2Zl2WNQQ=="], - "@oxlint/binding-linux-riscv64-gnu": ["@oxlint/binding-linux-riscv64-gnu@1.75.0", "", { "os": "linux", "cpu": "none" }, "sha512-mAG4DUXqfLC8cTjMD2kt3jDmVzFREYtDyeLNdLdsCcBc4Zbl2EMuiFektGBilQwkNjYnMvCqJs55U+Hyb+b+jw=="], + "@oxlint/binding-linux-riscv64-gnu": ["@oxlint/binding-linux-riscv64-gnu@1.78.0", "", { "os": "linux", "cpu": "none" }, "sha512-trucMER/0QtecoXvc1y/UVqE3kwJipDwrx4oHfj+nNm3dq2zjP44WT0CfHNDPM3G1DXIkx/gY6lAD21NSCZVhA=="], - "@oxlint/binding-linux-riscv64-musl": ["@oxlint/binding-linux-riscv64-musl@1.75.0", "", { "os": "linux", "cpu": "none" }, "sha512-95hrAvriAlI+pekSomTFIn0+bawMDlDwTNVmdjsFusTHyL2JWh7TWvRNG/Lkim72uN8OiCcO9wcaC6omLP5E3w=="], + "@oxlint/binding-linux-riscv64-musl": ["@oxlint/binding-linux-riscv64-musl@1.78.0", "", { "os": "linux", "cpu": "none" }, "sha512-cm3O4F/HQbdzOUX5mKHqG5KDL6E5w0pnlZ+fbBy2rmLryPOowkuLagFHTopQsEIpjcaZoPOrL+BmmAytAG9HFg=="], - "@oxlint/binding-linux-s390x-gnu": ["@oxlint/binding-linux-s390x-gnu@1.75.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-4b6f2+FrtruAESrCqIKcrarzfrSx+wk2QNcp+RT91/Prc+pMQMAfyZ1rG1c3tFQNl8Bc616tx40uNXyxNBRPbQ=="], + "@oxlint/binding-linux-s390x-gnu": ["@oxlint/binding-linux-s390x-gnu@1.78.0", "", { "os": "linux", "cpu": "s390x" }, "sha512-33wRf6HqGNsybJ3qX4cGaQN2ODPxNmc1rMa0mrTmx3eFq1VzOnvQooi9bIGVYakW8a/wmqVx1mgsUm8R2xfTiw=="], - "@oxlint/binding-linux-x64-gnu": ["@oxlint/binding-linux-x64-gnu@1.75.0", "", { "os": "linux", "cpu": "x64" }, "sha512-nshAhrUvXFUWOvqQ2soIw7HFNWvpvEV4o0cYSqPtzLiPF5gKyYTDOOTJ6Rn8g8K/iGvPIrbDA4v8+5MvnjJrrg=="], + "@oxlint/binding-linux-x64-gnu": ["@oxlint/binding-linux-x64-gnu@1.78.0", "", { "os": "linux", "cpu": "x64" }, "sha512-rRdISSYegj6VganMZ9tjRjijowfHJ09IZU01i0toBAqr6n5LEtwHq2IeS4FjW2RoskOHlb6efB26H5izYb3GEQ=="], - "@oxlint/binding-linux-x64-musl": ["@oxlint/binding-linux-x64-musl@1.75.0", "", { "os": "linux", "cpu": "x64" }, "sha512-e4jNxLKnxLC6sYBQRxrI2pgIIxnmMtF8U/VwNYcjTT/CLS+spH624cYVnj07bTKwaEWT37/e025isOs6j/0xqA=="], + "@oxlint/binding-linux-x64-musl": ["@oxlint/binding-linux-x64-musl@1.78.0", "", { "os": "linux", "cpu": "x64" }, "sha512-GmsP4rW0xTL6u5CVdcDsaN5Fbc7hBc382Wmar1kttbnwSEviM+rSINKOMQ+UQ6iH+AGwC+8gaAiwu134Tgh6Lg=="], - "@oxlint/binding-openharmony-arm64": ["@oxlint/binding-openharmony-arm64@1.75.0", "", { "os": "none", "cpu": "arm64" }, "sha512-hZ2lH+1qLf/DiEP9UWuQTK2JWj/BgvMB4jhIV4SmNU1wfEiYYX4TynQyAZXx0j9X4qRYizAL042SKaV+8ynh4w=="], + "@oxlint/binding-openharmony-arm64": ["@oxlint/binding-openharmony-arm64@1.78.0", "", { "os": "none", "cpu": "arm64" }, "sha512-sy9yeYuADc8a+n4TLBayzMCZiHPW78DcIFVpOXTmdKHWQeM9xe5uzkqIIZmi326D5hY9XVwacipEB1p7tQjPAg=="], - "@oxlint/binding-win32-arm64-msvc": ["@oxlint/binding-win32-arm64-msvc@1.75.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-Ilj6PNzGDS3bCU0MSJH7Msh0NhH+T/mRp2shwg+q+GHeVlPwP5LEboW96aW+3kVKFk6zYZy1Xi5pZkqZh6X8KQ=="], + "@oxlint/binding-win32-arm64-msvc": ["@oxlint/binding-win32-arm64-msvc@1.78.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-rjc2hF1KfMi8fZj1X/m3AmnHbdsF3rL0v6KQg0Uc880Yb2khjz+3U14sfdZ7jWTpRnN1m1NQa/TT7uU9lJWPrA=="], - "@oxlint/binding-win32-ia32-msvc": ["@oxlint/binding-win32-ia32-msvc@1.75.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-QVit2nOEOiPhkmsrksPSkoGCdnZRNkspt8fwoYyP09te1VEbnSj4LAxua4rc8FKTmWkySVe05j8iz9GXYfF1AQ=="], + "@oxlint/binding-win32-ia32-msvc": ["@oxlint/binding-win32-ia32-msvc@1.78.0", "", { "os": "win32", "cpu": "ia32" }, "sha512-zcuXFVrEFHIafRfkCQT8w/Xe41o07ozl/vwHq7p94vB29xVzsB0sZGYORU1jhcYKv3Lr0J3HbJ2T4fHH5rWmvA=="], - "@oxlint/binding-win32-x64-msvc": ["@oxlint/binding-win32-x64-msvc@1.75.0", "", { "os": "win32", "cpu": "x64" }, "sha512-DSxnNkBUAYARPwJtR12Ig3deWr8w0H997xP6jy33i+e0SyYJw8FKuz4+cZtpmPEhQmvlPJE3X/2vNxDmLkd/rA=="], + "@oxlint/binding-win32-x64-msvc": ["@oxlint/binding-win32-x64-msvc@1.78.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Sb5ocmLSuYeOuXd+CFOToGKp/gjXUEWDnvIGwhnh8aq8wY4TMmEnKnvbogSW7RdMZv77JSARduS7/gv+khYEjA=="], "@posthog/browser-common": ["@posthog/browser-common@0.2.1", "", { "dependencies": { "@posthog/core": "1.45.1", "@posthog/types": "1.398.0" } }, "sha512-FTVPsRw6GBKWvFwN26TNIQNNYPR9FsUj+B+CKhk/ht63IRzn4yYFtFOjcmW+bfXvNHuRADs6APbZ6Haz1kpoAw=="], - "@posthog/cli": ["@posthog/cli@0.7.24", "", { "dependencies": { "detect-libc": "2.1.2" }, "bin": { "posthog-cli": "run-posthog-cli.js" } }, "sha512-Jho5QDPYv21gDd8O1JDewPmA8ZEO2VbikszYTtD0rCem6nsDN528tpZnNUPGPYyeuAQHTQasoo7/g3IJVCBmDg=="], + "@posthog/cli": ["@posthog/cli@0.11.2", "", { "dependencies": { "detect-libc": "^2.1.2" }, "bin": { "posthog-cli": "run-posthog-cli.js" } }, "sha512-iqYIl/Q2FfE/+vftlHqGYFz4DRcDon7t28PU6kIefFFUMr0Z4bZrypnxU/Wl4xW/0uUw4fiAKs2tkWmwh3lSDA=="], "@posthog/core": ["@posthog/core@1.45.1", "", { "dependencies": { "@posthog/types": "1.398.0" } }, "sha512-tLtvzomavb2PPWdGYKsusyIzIeL2Px47v348Smibkay7sMy/83TyPk+Ptsp2NdeOgJsbuwSxWkR2+XA0aSCAaA=="], - "@posthog/nextjs-config": ["@posthog/nextjs-config@1.9.68", "", { "dependencies": { "@posthog/cli": "0.7.24", "@posthog/plugin-utils": "1.1.2", "@posthog/webpack-plugin": "1.5.24", "semver": "7.8.0" }, "peerDependencies": { "next": "16.2.12" } }, "sha512-MmzSrompIThnnrDArCKIOBZuD/b3RErH48ZJyzAuX+WfJTiHBjjTMvGWzr+qlIn8jCyJoglSygAwYVG8lJXybg=="], + "@posthog/nextjs-config": ["@posthog/nextjs-config@1.9.69", "", { "dependencies": { "@posthog/cli": "~0.11.1", "@posthog/plugin-utils": "^1.1.2", "@posthog/webpack-plugin": "^1.5.27", "semver": "^7.8.5" }, "peerDependencies": { "next": ">12.1.0" } }, "sha512-l1flhPrRGbJWXNwbbfFfmimQjO39d2pvha6MO9K9yTpHhJR9p8IbmyvsGFsY7l8xntaR+2gRsamto/fY8O086g=="], "@posthog/plugin-utils": ["@posthog/plugin-utils@1.1.2", "", { "dependencies": { "cross-spawn": "7.0.6" } }, "sha512-CS/14cMmny9NN/LHAb4QZdW3TfxhbKzsfyVYMCjsRdE0IKMTrXMr0cPRZxwczsN6VnWvVoy2Dd5wcG6RrIjlYA=="], "@posthog/types": ["@posthog/types@1.398.0", "", {}, "sha512-sJMkl4k+u8yS/0fjHsKqE9xTdsAh30a2WvgChiptellnVoE0e8QJKFgqOMD2sk8FaEArPdeFklAhXvmENAt3Sg=="], - "@posthog/webpack-plugin": ["@posthog/webpack-plugin@1.5.24", "", { "dependencies": { "@posthog/cli": "0.7.24", "@posthog/core": "1.40.1", "@posthog/plugin-utils": "1.1.2" }, "peerDependencies": { "webpack": "5.106.2" } }, "sha512-sdqedI8V/Iafy4+pT2g8glEMEhFSnE8PqW5rbE2qZH4TR57Xx3qfFrKFw6A9IJ29y9L0/kCdgmIWkxdTndY1+g=="], + "@posthog/webpack-plugin": ["@posthog/webpack-plugin@1.5.27", "", { "dependencies": { "@posthog/cli": "~0.11.1", "@posthog/core": "^1.48.0", "@posthog/plugin-utils": "^1.1.2" }, "peerDependencies": { "webpack": "^5" } }, "sha512-H5OWzFymU8UfLbFaZ4y6kEZHvgZJwSrTACosId34Gq1UnBymeRuCfOFQRIG5xVqPAB6+vN0YuPpFIzpJ82i28g=="], "@radix-ui/number": ["@radix-ui/number@1.1.3", "", {}, "sha512-Road2bidD0uu/1BGDOWNdPI06g0lIRy6IF9GZcIrDK2KGItfor8IQwQa+yM2ERgHM1MmHxaxpTzk0/Jp42lNfA=="], "@radix-ui/primitive": ["@radix-ui/primitive@1.1.7", "", {}, "sha512-rqWnm76nYT8HoNNqEjpgJ7Pw/DrBj5iBTrmEPo6HTX5+VJyBNOqTdv4g89G63HuR5g0AaENoAcH7Is5fF2kZ8Q=="], - "@radix-ui/react-accordion": ["@radix-ui/react-accordion@1.2.20", "", { "dependencies": { "@radix-ui/primitive": "1.1.7", "@radix-ui/react-collapsible": "1.1.20", "@radix-ui/react-collection": "1.1.15", "@radix-ui/react-compose-refs": "1.1.5", "@radix-ui/react-context": "1.2.2", "@radix-ui/react-direction": "1.1.4", "@radix-ui/react-id": "1.1.4", "@radix-ui/react-primitive": "2.1.10", "@radix-ui/react-use-controllable-state": "1.2.6" }, "optionalDependencies": { "@types/react": "19.2.17", "@types/react-dom": "19.2.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" } }, "sha512-jDhG9FvAEnlhnjrsINbNXcUa4G+L1KqSkJSunkbKEzFRcAb52jvM0PjPxPRvhe1HNc5F5yc0yzzWeeqlH4yBIg=="], - "@radix-ui/react-arrow": ["@radix-ui/react-arrow@1.1.15", "", { "dependencies": { "@radix-ui/react-primitive": "2.1.10" }, "optionalDependencies": { "@types/react": "19.2.17", "@types/react-dom": "19.2.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" } }, "sha512-v4zggRcjadnI+ClKDuijlQEW4tw3NoaeHc/PwpKnLoLLKNUG4InLegkstooLcRIUWCs+8L22dGURCVuFfOKfnA=="], - "@radix-ui/react-collapsible": ["@radix-ui/react-collapsible@1.1.20", "", { "dependencies": { "@radix-ui/primitive": "1.1.7", "@radix-ui/react-compose-refs": "1.1.5", "@radix-ui/react-context": "1.2.2", "@radix-ui/react-id": "1.1.4", "@radix-ui/react-presence": "1.1.10", "@radix-ui/react-primitive": "2.1.10", "@radix-ui/react-use-controllable-state": "1.2.6", "@radix-ui/react-use-layout-effect": "1.1.4" }, "optionalDependencies": { "@types/react": "19.2.17", "@types/react-dom": "19.2.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" } }, "sha512-mcGesGplBnzN2sbvJETzpCNfSMyPnb29q1GRLU+Ib7bJrpIG2ywmRoh2V5VbA2uNvKikKUlVbAPks7JDjz4A8Q=="], - "@radix-ui/react-collection": ["@radix-ui/react-collection@1.1.15", "", { "dependencies": { "@radix-ui/react-compose-refs": "1.1.5", "@radix-ui/react-context": "1.2.2", "@radix-ui/react-primitive": "2.1.10", "@radix-ui/react-slot": "1.3.3" }, "optionalDependencies": { "@types/react": "19.2.17", "@types/react-dom": "19.2.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" } }, "sha512-9W+B9NPF0NaaPh/1NJd3+KqsnlLqU9H7T2rvww+fp+T/evVXdNAyYcnfRQZFOjkR1ajQp3yORlqnI8soawLvNA=="], "@radix-ui/react-compose-refs": ["@radix-ui/react-compose-refs@1.1.5", "", { "optionalDependencies": { "@types/react": "19.2.17" }, "peerDependencies": { "react": "19.2.8" } }, "sha512-+48PbAAbq3didjJxa+OaWY2ZwgAKsNiRGyeHKszblZMQ+kcpd9pAaT11cMkGEie0vsOi3QdeTE6d5Fe3Gn61kA=="], @@ -565,37 +568,37 @@ "@radix-ui/rect": ["@radix-ui/rect@1.1.3", "", {}, "sha512-JtyZR+mqgBibTo8xea3B6ZRmzZiM/YeVBtUkas6zMuXjAlfIFIW2FgqeM9eLyvEaYX66vr6DJMK+4U6LV0KhNw=="], - "@react-three/fiber": ["@react-three/fiber@9.6.1", "", { "dependencies": { "@babel/runtime": "7.29.2", "@types/webxr": "0.5.24", "base64-js": "1.5.1", "buffer": "6.0.3", "its-fine": "2.0.0", "react-use-measure": "2.1.7", "scheduler": "0.27.0", "suspend-react": "0.1.3", "use-sync-external-store": "1.6.0", "zustand": "5.0.13" }, "optionalDependencies": { "react-dom": "19.2.8" }, "peerDependencies": { "react": "19.2.8", "three": "0.185.1" } }, "sha512-zF0rsKcVYpcJwbFEnv2HkHX9cvOEgsfQo/X8lwmR2dn13S4qEQJXir9fxf5js2LQFoXqxOY7MDkOkYx2uZ4gSg=="], + "@react-three/fiber": ["@react-three/fiber@9.7.0", "", { "dependencies": { "@babel/runtime": "^7.17.8", "@types/webxr": "*", "base64-js": "^1.5.1", "buffer": "^6.0.3", "its-fine": "^2.0.0", "react-use-measure": "^2.1.7", "scheduler": "^0.27.0", "suspend-react": "^0.1.3", "use-sync-external-store": "^1.4.0", "zustand": "^5.0.3" }, "peerDependencies": { "expo": ">=43.0", "expo-asset": ">=8.4", "expo-file-system": ">=11.0", "expo-gl": ">=11.0", "react": ">=19 <19.3", "react-dom": ">=19 <19.3", "react-native": ">=0.78", "three": ">=0.156" }, "optionalPeers": ["expo", "expo-asset", "expo-file-system", "expo-gl", "react-dom", "react-native"] }, "sha512-EWm9FwcaOZQu/ExFW5rggoCMM1NJet5YbxVxKaOE+KSncrjU0Wx7017qSyGFvupviK89nMYGCWU3BIK4dI1clw=="], - "@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.1.5", "", { "os": "android", "cpu": "arm64" }, "sha512-lZg8fqIv2v7FF237bwMgzGZEJvGL79/s5knJ/i6FmsGF4XXlzccZ4jb+TrFIxtSSxFtIpdsgrPZeMk1I9AFcyQ=="], + "@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.2.4", "", { "os": "android", "cpu": "arm64" }, "sha512-jHC2cnyKz5xU2fhECtFl8OZ83cYNt13GZQD+0uMJ/X3o+ijmd56okHhTUwxVSHPx1IRVIJEZ1/1pPzeLCU6XKA=="], - "@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.1.5", "", { "os": "darwin", "cpu": "arm64" }, "sha512-51Bnx9pNiMRKSUNtBfySkNJ9vMU9Hh3I1ozDd6gyPPYzaXCfnptUcEZxXGYFn+ul2dtcMUiqGR1Yai2K10uoTw=="], + "@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.2.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-Dc5mPD8F5F/FS8i01syd7FTF6yB2fVthH/TRkjwJkzUK6EpoxHtqvZQP5Zwq80/5z19TWYHIg1KOHboCgVx/aQ=="], - "@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.1.5", "", { "os": "darwin", "cpu": "x64" }, "sha512-Tm+gbfC0aHu1tBA/JvKQh32S0K6YgCHkiAF4/W6xX0K0RmNuc94VeK419dJoE65R5aRxmo+noZQSWrAMF6yb6g=="], + "@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.2.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-fpDm4oBo6SqLvWUYCmFhdde3U9KH2fRNNMeAnAPAIwxRL345xutL0EtEUcuoxsoazdJGv/MuDBQHlCDrtbvqOg=="], - "@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.1.5", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JMzDKCCXq93YccG5gz3hvOs1oXRKAf0XYpfOS88e+wZrC8Iugj6j68867vrYZkvpDDpKn/KoKORThmchMpF6TA=="], + "@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.2.4", "", { "os": "freebsd", "cpu": "x64" }, "sha512-rSJoreDE/HoIzoaib6MTp5jQtCTdMHKIvItAKT/ImS6Y6Ww76oUaeMyp4Vc/fAgd/ehji068IxetHXAnqUwN9A=="], - "@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.1.5", "", { "os": "linux", "cpu": "arm" }, "sha512-uML21j2K5TfPGutKxub+M+nLjZIrWjXQ5Grx4lCe/nimTj9B4L63zHpjXLl4y0L3mcm2htEQIb06oCG/szerNw=="], + "@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.2.4", "", { "os": "linux", "cpu": "arm" }, "sha512-/jm8OGHgn7oGaJu3i/qZI9spUGcJ+y/lk43ttQ/iO1tOd9NissG6o97bighBCiL+BKRngmcDuR6ikfwYdJmVuQ=="], - "@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.1.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-navSiuTMogvnQoZoM/v+l3ZWo50/NTwSHSzheABx/RCnmUPaKwq9qSo4Br2OYRs21+Fz8uFqITZM3H4opOB0/Q=="], + "@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.2.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-tIP06BeD9EqvECBrPZ+sqdPlYrT+aYaAiu1wYziVx5elRK/ftm33JxVDy2bXGbr6J0CrtirCkR87/X5a2euEng=="], - "@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.1.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-lAryqH7IteztmCXQXk0etKj4wBQ7Gx5S6LjKhsgp9zb8I5bsuvU/2llH1hDQcjsFeqIsovMVN339/8pUDDBXxA=="], + "@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.2.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-Ql1Q0EQqVThvn9VAVlwNzsUvbSFtCMGjLpRRi4pk5i7NZZ4n5ISiLMjHYtus4VQ2PvkSw24zyaCVsiS+sXPj1w=="], - "@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.1.5", "", { "os": "linux", "cpu": "ppc64" }, "sha512-fsK/sNBnxzBlL4O1JNrZakVQxPspqpED5dLtNsZS9oOKmtSpdNIzxH2kkol5HYTWJN47sE20ztMJPxfZ89qGOg=="], + "@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.2.4", "", { "os": "linux", "cpu": "ppc64" }, "sha512-GjbjXD4XXfN19D0LZNbmiCBUoDiRACsYHr0yaIbbn8aFsXjHZifcYqu/W5Er5X2X990WjHXFrxarn5chzItorQ=="], - "@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.1.5", "", { "os": "linux", "cpu": "s390x" }, "sha512-gLYb4BIadlfTOYT5gO503n8zQjXflgzpD0FcyKh0Mzx3rqCZKnHoJWV9xe1KXUJ5lx2JfcSHr/mhzS0PC/McAA=="], + "@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.2.4", "", { "os": "linux", "cpu": "s390x" }, "sha512-p5WR0NOwaRmJ/B1b6IjEFLLivwEsf3PrdBIhRbhTCQisbo2SvHHpG4ELB/+FgQNnB88LTOF86upmJmbvZdQ2lw=="], - "@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.1.5", "", { "os": "linux", "cpu": "x64" }, "sha512-FjcpEKUyJygHgs1o50VYNvkt5+7Le/VEdYt0AkRpkL33MnyQfwr8l5mXwMmfmTbyMPr5vJLC+8/Gd9gXnwU1QQ=="], + "@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.2.4", "", { "os": "linux", "cpu": "x64" }, "sha512-4/GyVjmhR+Tc6HLJvwc1sOhPqAZtySiSMesOZyX6JQ5XBxoTDEMKQzvo07NIK6nTon/SivlZqvhzvuVBNQhObQ=="], - "@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.1.5", "", { "os": "linux", "cpu": "x64" }, "sha512-Me+PfPI2TMeOQk0gYWfLQZtTktrmzbr8cDboqX83XKc7UrgAi55gF+2dUkWdxd19n55Essp2yeca+O9N5rBxHg=="], + "@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.2.4", "", { "os": "linux", "cpu": "x64" }, "sha512-l9eeLsCNvPpmSXUej0etw/J1eqV0Jj1D5G/xG6YTijmE6dkv6E2QezgWbTfQk63v952DPqrjOCoiqxq7Bw0YUQ=="], - "@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.1.5", "", { "os": "none", "cpu": "arm64" }, "sha512-yc5WrLzXks6zCQfn9Oxr8pORKyl/pF+QjHmW/Qx3qu0oyrrNC+y2JLTU1E2rcWYAmzlnqngWXHQjy51VzW70Vw=="], + "@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.2.4", "", { "os": "none", "cpu": "arm64" }, "sha512-e0F355MSTMm3+UOqtV3L24gFUp2N5m1f8L/7d56deik6va+AXdrt9F8LbzGpeWGWRbZEDq4m8NVnJDeBtf9DZg=="], "@rolldown/binding-wasm32-wasi": ["@rolldown/binding-wasm32-wasi@1.1.5", "", { "dependencies": { "@emnapi/core": "1.11.1", "@emnapi/runtime": "1.11.1", "@napi-rs/wasm-runtime": "1.1.6" }, "cpu": "none" }, "sha512-VbQGPX2b4r48TAMIM2cjgluIM1HYutm4pcTEJsle7iEP7sB1dFqtPLBVbdLAZCxy1txCcPxf4QFf4v8uvltPqA=="], - "@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.1.5", "", { "os": "win32", "cpu": "arm64" }, "sha512-gHv82k63z4qpV5+Q1y/12KrK0ltWBukVDI8nZcbT7Tt/ZlOIVwppazneq0F93oDxTo3IgAMEDIoQh3E2n6mVsw=="], + "@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.2.4", "", { "os": "win32", "cpu": "arm64" }, "sha512-AWLi0uBRYh6QlE7OKhiz+phZC0qwtij2QZmhmOdsLdFn64m7oMpooE9ICE3lhm9xMb4SpDo2WbHcxX1iFLFtqw=="], - "@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.1.5", "", { "os": "win32", "cpu": "x64" }, "sha512-tTZuDBPw85tEN5PQi1pnEBzDy0Z49HtScLAbD5t6hyeU92A95pRWaSMw1GZZi/RwgSgUIl0xrSlXIT/9QzvYSA=="], + "@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.2.4", "", { "os": "win32", "cpu": "x64" }, "sha512-UwSDJOg3dqCAejWdxclJjCsh3Qq4vLYMDxmyHqo1btz3stK2VqgwNd3mm5tuIwzSlGIQ/1H9Hr+Zn09mrezNqQ=="], "@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.1", "", {}, "sha512-2j9bGt5Jh8hj+vPtgzPtl72j0yRxHAyumoo6TNfAjsLB04UtpSvPbPcDcBMxz7n+9CYB0c1GxQFxYRg2jimqGw=="], @@ -607,27 +610,27 @@ "@semianalysisai/inferencex-mcp": ["@semianalysisai/inferencex-mcp@workspace:packages/mcp"], - "@shikijs/core": ["@shikijs/core@4.3.1", "", { "dependencies": { "@shikijs/primitive": "4.3.1", "@shikijs/types": "4.3.1", "@shikijs/vscode-textmate": "10.0.2", "@types/hast": "3.0.4", "hast-util-to-html": "9.0.5" } }, "sha512-ANMDxuaPsNMdDC1m4vfvhlDmJweMwkE5XitTwrq2rWHx5jM+dlm4MmHt2PP6t0uejfR77SuhrhJ0zEijIF/uhA=="], + "@shikijs/core": ["@shikijs/core@4.4.3", "", { "dependencies": { "@shikijs/primitive": "4.4.3", "@shikijs/types": "4.4.3", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.5", "hast-util-to-html": "^9.0.5" } }, "sha512-QCR4q2ZO/ILJEuwiBMel4wdcTDb1JGwfjKTxPDF6x8ixOaluPrVqIn06C99AcRPhmYlBR56d/Fb+GN58GzExpg=="], - "@shikijs/engine-javascript": ["@shikijs/engine-javascript@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1", "@shikijs/vscode-textmate": "10.0.2", "oniguruma-to-es": "4.3.6" } }, "sha512-JBItcnPuYq7jVJdZo/vMj94r+szT7XEjHFX+mvFDGSEIbVAXAGyHAHzhbWzpGOwYidCZrErJLLgn2PVeiokHnQ=="], + "@shikijs/engine-javascript": ["@shikijs/engine-javascript@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3", "@shikijs/vscode-textmate": "^10.0.2", "oniguruma-to-es": "^4.3.6" } }, "sha512-FbOjFJp9VLdo1Wevs10BBtVxiTWwNLqZh5Gkhjgda/ioL15YOgeSl9n+6XMa3qRlPQzfhFNe641SrynFHYG0nQ=="], - "@shikijs/engine-oniguruma": ["@shikijs/engine-oniguruma@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1", "@shikijs/vscode-textmate": "10.0.2" } }, "sha512-OXyNMzg0pews+msMj4cHeqT4xiYKKvbnn6VbdAXxfoFl3SSx4fJTc8FadECuc5/H9p3BzhNAoAUXKwAu9rWYhg=="], + "@shikijs/engine-oniguruma": ["@shikijs/engine-oniguruma@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3", "@shikijs/vscode-textmate": "^10.0.2" } }, "sha512-EcOQkxdxGQrc1Row/cC2c96/v1dbZqGnEVu1qTuT/MJmp6+cXCvQussowVmCv5Tqr3KuY3c7IbM6HTW3LJ1k9w=="], - "@shikijs/langs": ["@shikijs/langs@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1" } }, "sha512-m0l9nsDqgBHvbZbk7A0/kXz/impK3uB/c6rAn6Gpg/uPtdZRQ+alsN/17MU5thb68XTj/4DxkZAotrM0GGSpDQ=="], + "@shikijs/langs": ["@shikijs/langs@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3" } }, "sha512-ePic0yfAJGOF83D5wBHK/00EjK65oahBYxFk5epgq33WRv7X9UuxLEV8PtR0szC0z8dl7INIpIodB99JRFlR+A=="], - "@shikijs/primitive": ["@shikijs/primitive@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1", "@shikijs/vscode-textmate": "10.0.2", "@types/hast": "3.0.4" } }, "sha512-CXQRQOYy1leqQ8ceTeJdmXv/bsUY++6QyLpXJ94LZAAYj5X2SKRdc5ipguv4NPyGVKItB2PPwUpRNe0Sjh5S1A=="], + "@shikijs/primitive": ["@shikijs/primitive@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.5" } }, "sha512-m0wBeLDQDeIxRdUmrCPdQqfuUamDwRL5isCfYbguKD6NiaKpVbsv+3J81DyIKgNW5h4WAIIr8T4EkgQrBBxvaQ=="], - "@shikijs/rehype": ["@shikijs/rehype@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1", "@types/hast": "3.0.4", "hast-util-to-string": "3.0.1", "shiki": "4.3.1", "unified": "11.0.5", "unist-util-visit": "5.1.0" } }, "sha512-oshrlfUF3VPUJfnp5K1lLwsS/SRBKrIxONpdWebSKZXdBE3UsZnxgqpvRUA8UsofS7vmjFOCAHIT71ECbmOxTw=="], + "@shikijs/rehype": ["@shikijs/rehype@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3", "@types/hast": "^3.0.5", "hast-util-to-string": "^3.0.1", "shiki": "4.4.3", "unified": "^11.0.5", "unist-util-visit": "^5.1.0" } }, "sha512-vkG9jG1aRnrx05R31uAOKQHE8qpY7r1cBXE2sAZgFK2IaPnQHwaP4L1C6amQixmZ8thBsKwfbSsYxMDoo46UWQ=="], - "@shikijs/themes": ["@shikijs/themes@4.3.1", "", { "dependencies": { "@shikijs/types": "4.3.1" } }, "sha512-dgpoJ4WqNi2yTmizQHBJ5zcX6j2lE6icN/0yt4l1kkf16jrY/pwPLoTb1ETsWMz0OBLf9ZNvwmxft+cH+N9qSA=="], + "@shikijs/themes": ["@shikijs/themes@4.4.3", "", { "dependencies": { "@shikijs/types": "4.4.3" } }, "sha512-w8UHjeUnIR965KMWJHUPXOc2mNJUnK3vpVLYLvw5IYU2mnTTJ89E24OrJDBNiJDQ0qzb0tc4l7mrIXx5cFeIyw=="], - "@shikijs/types": ["@shikijs/types@4.3.1", "", { "dependencies": { "@shikijs/vscode-textmate": "10.0.2", "@types/hast": "3.0.4" } }, "sha512-CHFxE0jztBIZRHH6gxXE7DXUCFXjReEGxZ/j0rfSLGKZuwp2xBYycEP14875DSa9KLL/6700oxIq6oO6ef9K2g=="], + "@shikijs/types": ["@shikijs/types@4.4.3", "", { "dependencies": { "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.5" } }, "sha512-UEJxmRR++MAGR6hugn0vgVS2W/6lWAts84FFSrnlH9sP0LNol7E5+NQ792pH8liWUhyMyjhTgSUH3k7iD7tc5g=="], "@shikijs/vscode-textmate": ["@shikijs/vscode-textmate@10.0.2", "", {}, "sha512-83yeghZ2xxin3Nj8z1NMd/NCuca+gsYXswywDy5bHvwlWL8tpTQmzGeUuHd9FC3E/SBEMvzJRwWEOz5gGes9Qg=="], "@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], - "@swc/helpers": ["@swc/helpers@0.5.15", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-JQ5TuMi45Owi4/BIMAJBoSQoOJu12oOk/gADqlcUL9JEdHB8vyjUSsxqeNXnmXHjYKMi2WcYtezGEEhqUI/E2g=="], + "@swc/helpers": ["@swc/helpers@0.5.23", "", { "dependencies": { "tslib": "^2.8.0" } }, "sha512-5lSsMOTXURePglDfvuAQUqkGek9Hg2kksOYay2m0+XR++b2NWYL/4sWyuvVBIs8oKnJaxkdi9whaL/sqN13afw=="], "@tailwindcss/node": ["@tailwindcss/node@4.3.3", "", { "dependencies": { "@jridgewell/remapping": "2.3.5", "enhanced-resolve": "5.24.2", "jiti": "2.7.0", "lightningcss": "1.32.0", "magic-string": "0.30.21", "source-map-js": "1.2.1", "tailwindcss": "4.3.3" } }, "sha512-/T8IKEsf9VTU6tLjgC7+sv2mOPtQxzE2jMw7u4Tt40Tx+QSZxpzh95/H6cMKoja9XuW7iMdLJYBB0o9G1CaAgg=="], @@ -753,7 +756,7 @@ "@types/geojson": ["@types/geojson@7946.0.16", "", {}, "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg=="], - "@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "@types/hast": ["@types/hast@3.0.5", "", { "dependencies": { "@types/unist": "*" } }, "sha512-rp/ezSWaD1m44dPKICGhiskI13nVr7qTloFwDa/IYkhhf5nzwP+zIQcIJh3WIFSBOy/H1PzB40jPjMDksN4F+g=="], "@types/jsesc": ["@types/jsesc@2.5.1", "", {}, "sha512-9VN+6yxLOPLOav+7PwjZbxiID2bVaeq0ED4qSQmdQTdjnXJSaCVKTR58t15oqH1H5t8Ng2ZX1SabJVoN9Q34bw=="], @@ -767,7 +770,7 @@ "@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="], - "@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + "@types/node": ["@types/node@26.2.0", "", { "dependencies": { "undici-types": "~8.3.0" } }, "sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg=="], "@types/react": ["@types/react@19.2.17", "", { "dependencies": { "csstype": "3.2.3" } }, "sha512-MXfmqaVPEVgkBT/aY0aGCkRWWtByiYQXo3xdQ8r5RzuFrPiRn8Gar2tQdXSUQ2GKV3bkXckek89V8wQBY2Q/Aw=="], @@ -785,7 +788,7 @@ "@types/stream-json": ["@types/stream-json@1.7.8", "", { "dependencies": { "@types/node": "26.1.1", "@types/stream-chain": "2.1.0" } }, "sha512-MU1OB1eFLcYWd1LjwKXrxdoPtXSRzRmAnnxs4Js/ayB5O/NvHraWwuOaqMWIebpYwM6khFlsJOHEhI9xK/ab4Q=="], - "@types/three": ["@types/three@0.185.1", "", { "dependencies": { "@dimforge/rapier3d-compat": "0.12.0", "@tweenjs/tween.js": "23.1.3", "@types/stats.js": "0.17.4", "@types/webxr": "0.5.24", "fflate": "0.8.3", "meshoptimizer": "1.1.1" } }, "sha512-db1xTb+EgYF2didW+eudSvVPtn75zo+fGsY8ShQrJY/B5ZBmC2Fiaykv3aImHAlCNEGuMPkPGXBJGLwzu5mC7A=="], + "@types/three": ["@types/three@0.185.4", "", { "dependencies": { "@dimforge/rapier3d-compat": "~0.12.0", "@tweenjs/tween.js": "~23.1.3", "@types/stats.js": "*", "@types/webxr": ">=0.5.17", "fflate": "~0.8.2", "meshoptimizer": "~1.1.1" } }, "sha512-gAsBIC07NIFrxjbf7tH2t71c38uulFfk/RFoC7FNBSjMRAQ8J1x/RBvusX0N5PJouaYFJawXQqfCQ0RKUx/1nA=="], "@types/tmp": ["@types/tmp@0.2.6", "", {}, "sha512-chhaNf2oKHlRkDGt+tiKE2Z5aJ6qalm7Z9rlLdBwmOiAAf09YQvvoLXjWK4HWPF1xU/fqvMgfNfpVoBscA/tKA=="], @@ -811,7 +814,7 @@ "@vercel/analytics": ["@vercel/analytics@2.0.1", "", { "optionalDependencies": { "next": "16.2.12", "react": "19.2.8" } }, "sha512-MTQG6V9qQrt1tsDeF+2Uoo5aPjqbVPys1xvnIftXSJYG2SrwXRHnqEvVoYID7BTruDz4lCd2Z7rM1BdkUehk2g=="], - "@vercel/blob": ["@vercel/blob@2.6.1", "", { "dependencies": { "@vercel/oidc": "3.8.0", "async-retry": "1.3.3", "is-buffer": "2.0.5", "is-node-process": "1.2.0", "throttleit": "2.1.0", "undici": "6.27.0" } }, "sha512-KTJytw85j1XQBxjN5d6UXI7fIWNQe1jotn4nWN+0hePqLs+Qi1B3jHdQcSKFGF0m2rsy9uhPT6GOXMtHe3qNzg=="], + "@vercel/blob": ["@vercel/blob@2.8.0", "", { "dependencies": { "@vercel/oidc": "^3.6.1", "async-retry": "^1.3.3", "is-buffer": "^2.0.5", "is-node-process": "^1.2.0", "throttleit": "^2.1.0", "undici": "^6.23.0" } }, "sha512-Nu+HWKpkgovCh/ezlG7wCVwF7RErTzLzZMbGKFBdGBCbTKyK+s5VXPLl+0+TpNEQPH8AVaGzOpIsXUOtkqylCQ=="], "@vercel/cli-config": ["@vercel/cli-config@0.2.0", "", { "dependencies": { "xdg-app-paths": "5.5.1", "zod": "4.1.11" } }, "sha512-fJRRRB7734BDuXZ89yBEaA2ncYhH7bWX30mk04W80J6VAfQc+4iB8lyzAdaGpFV3/vNlkt9VZt+/uoQoWX6UsQ=="], @@ -943,6 +946,8 @@ "aws4": ["aws4@1.13.2", "", {}, "sha512-lHe62zvbTB5eEABUVi/AwVh0ZKY9rMMDhmm+eeyuuUQbQ3+J+fONVQOZyj+DdrvD4BY33uYniyRJ4UJIaSKAfw=="], + "axe-core": ["axe-core@4.13.0", "", {}, "sha512-UzGt8zg7Ny8djbYMhxl2zuEevVa7r2gJjYY5Lwr1xM7+XU2nd6CkIWFTVcCIbAP63vSz71NaVyyuSk9lHKcy0A=="], + "bail": ["bail@2.0.2", "", {}, "sha512-0xO6mYd7JB2YesxDKplafRpsiOzPt9V02ddPCLbY1xYGPOX24NTyN50qnUxgCPcSoYMhKpAuBTjQoRZCAkUDRw=="], "balanced-match": ["balanced-match@4.0.4", "", {}, "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA=="], @@ -961,7 +966,7 @@ "body-parser": ["body-parser@2.3.0", "", { "dependencies": { "bytes": "3.1.2", "content-type": "2.0.0", "debug": "4.4.3", "http-errors": "2.0.1", "iconv-lite": "0.7.2", "on-finished": "2.4.1", "qs": "6.15.3", "raw-body": "3.0.2", "type-is": "2.1.0" } }, "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw=="], - "brace-expansion": ["brace-expansion@5.0.8", "", { "dependencies": { "balanced-match": "4.0.4" } }, "sha512-JZyDyq3D4AUifKTPOB7DELf6XsB3WdPuNxCtob1vFXPsSXhdAiHBWJ/tJ8HAc9aH84BK+5JFZLNkJKx3G9kzQg=="], + "brace-expansion": ["brace-expansion@5.0.9", "", { "dependencies": { "balanced-match": "^4.0.2" } }, "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg=="], "braces": ["braces@3.0.3", "", { "dependencies": { "fill-range": "7.1.1" } }, "sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA=="], @@ -1061,7 +1066,9 @@ "csstype": ["csstype@3.2.3", "", {}, "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ=="], - "cypress": ["cypress@15.19.0", "", { "dependencies": { "@cypress/request": "4.0.1", "@cypress/xvfb": "1.2.4", "@types/sinonjs__fake-timers": "8.1.1", "@types/sizzle": "2.3.10", "@types/tmp": "0.2.6", "arch": "2.2.0", "blob-util": "2.0.2", "bluebird": "3.7.2", "buffer": "5.7.1", "cachedir": "2.4.0", "chalk": "4.1.2", "ci-info": "4.4.0", "cli-table3": "0.6.1", "commander": "6.2.1", "common-tags": "1.8.2", "dayjs": "1.11.21", "debug": "4.4.3", "eventemitter2": "6.4.7", "execa": "4.1.0", "executable": "4.1.1", "fs-extra": "9.1.0", "hasha": "5.2.2", "is-installed-globally": "0.4.0", "listr2": "9.0.5", "lodash": "4.18.1", "log-symbols": "4.1.0", "minimist": "1.2.8", "ospath": "1.2.2", "pretty-bytes": "5.6.0", "process": "0.11.10", "proxy-from-env": "1.0.0", "request-progress": "3.0.0", "supports-color": "8.1.1", "systeminformation": "5.31.15", "tmp": "0.2.7", "tree-kill": "1.2.2", "tslib": "1.14.1", "untildify": "4.0.0", "yauzl": "3.4.0" }, "bin": { "cypress": "bin/cypress" } }, "sha512-kjy1u3SWlMRWS5qffH3U+bXx1PqPP7qwhIzEY3UtERcW2r/8zy5uk6uSwa75pYXJyYFVP1SO8mAn/avZUvUbfg=="], + "cypress": ["cypress@15.19.0", "", { "dependencies": { "@cypress/request": "^4.0.0", "@cypress/xvfb": "^1.2.4", "@types/sinonjs__fake-timers": "8.1.1", "@types/sizzle": "^2.3.2", "@types/tmp": "^0.2.3", "arch": "^2.2.0", "blob-util": "^2.0.2", "bluebird": "^3.7.2", "buffer": "^5.7.1", "cachedir": "^2.4.0", "chalk": "^4.1.0", "ci-info": "^4.1.0", "cli-table3": "0.6.1", "commander": "^6.2.1", "common-tags": "^1.8.0", "dayjs": "^1.10.4", "debug": "^4.3.4", "eventemitter2": "6.4.7", "execa": "4.1.0", "executable": "^4.1.1", "fs-extra": "^9.1.0", "hasha": "5.2.2", "is-installed-globally": "~0.4.0", "listr2": "^9.0.5", "lodash": "^4.17.23", "log-symbols": "^4.0.0", "minimist": "^1.2.8", "ospath": "^1.2.2", "pretty-bytes": "^5.6.0", "process": "^0.11.10", "proxy-from-env": "1.0.0", "request-progress": "^3.0.0", "supports-color": "^8.1.1", "systeminformation": "^5.31.1", "tmp": "~0.2.4", "tree-kill": "1.2.2", "tslib": "1.14.1", "untildify": "^4.0.0", "yauzl": "^3.3.1" }, "bin": { "cypress": "bin/cypress" } }, "sha512-kjy1u3SWlMRWS5qffH3U+bXx1PqPP7qwhIzEY3UtERcW2r/8zy5uk6uSwa75pYXJyYFVP1SO8mAn/avZUvUbfg=="], + + "cypress-axe": ["cypress-axe@1.7.0", "", { "peerDependencies": { "axe-core": "^3 || ^4", "cypress": "^10 || ^11 || ^12 || ^13 || ^14 || ^15" } }, "sha512-zzJpvAAjauEB3GZl0KYXb8i3w6MztWAt2WM3czYTFyNVC30alDmqCm9E7GwZ4bgkldZJlmHakaVEyu73R5St4w=="], "cypress-split": ["cypress-split@1.25.0", "", { "dependencies": { "@actions/core": "2.0.3", "arg": "5.0.2", "console.table": "0.10.0", "debug": "4.4.3", "fast-shuffle": "6.1.1", "find-cypress-specs": "1.54.12", "globby": "11.1.0", "humanize-duration": "3.33.2" }, "bin": { "cypress-split-merge": "bin/merge.js", "cypress-split-preview": "bin/preview.js" } }, "sha512-/K8qQz3yeSc5GyHFyqJC6ipz3S47kwtFu7WfuZ+uSAjtfNlCxTBlPV5CTTn0v39zjWEVqrfpu2arZuzZegu/9g=="], @@ -1183,7 +1190,7 @@ "dir-glob": ["dir-glob@3.0.1", "", { "dependencies": { "path-type": "4.0.0" } }, "sha512-WkrWp9GR4KXfKGYzOLmTuGVi1UWFfws377n9cc55/tb6DuqyF6pcQ5AbiHEshaDpY9v6oaSr2XCDidGmMwdzIA=="], - "dompurify": ["dompurify@3.4.12", "", { "optionalDependencies": { "@types/trusted-types": "2.0.7" } }, "sha512-zQvGet8Z2sWbQhCmfFz/T5QWH2oBmjnqK3qvOjaqaNLrLEF912WamU+ohnTp0TCep/MFVHpdJuCZEdFOdTnEFg=="], + "dompurify": ["dompurify@3.4.13", "", { "optionalDependencies": { "@types/trusted-types": "^2.0.7" } }, "sha512-2vmYIoqjze2d+kakP8S/nS5shfsl587kzwEjcGlTdiksUVgFHnFCsLYDVj/JNqJVOQZGSYBTmuycv0PodwmnMQ=="], "dunder-proto": ["dunder-proto@1.0.1", "", { "dependencies": { "call-bind-apply-helpers": "1.0.2", "es-errors": "1.3.0", "gopd": "1.2.0" } }, "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A=="], @@ -1225,7 +1232,7 @@ "esast-util-from-js": ["esast-util-from-js@2.0.1", "", { "dependencies": { "@types/estree-jsx": "1.0.5", "acorn": "8.16.0", "esast-util-from-estree": "2.0.0", "vfile-message": "4.0.3" } }, "sha512-8Ja+rNJ0Lt56Pcf3TAmpBZjmx8ZcK5Ts4cAzIOjsjevg9oSXJnl6SUQ2EevU8tv3h6ZLWmoKL5H4fgWvdvfETw=="], - "esbuild": ["esbuild@0.28.1", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.1", "@esbuild/android-arm": "0.28.1", "@esbuild/android-arm64": "0.28.1", "@esbuild/android-x64": "0.28.1", "@esbuild/darwin-arm64": "0.28.1", "@esbuild/darwin-x64": "0.28.1", "@esbuild/freebsd-arm64": "0.28.1", "@esbuild/freebsd-x64": "0.28.1", "@esbuild/linux-arm": "0.28.1", "@esbuild/linux-arm64": "0.28.1", "@esbuild/linux-ia32": "0.28.1", "@esbuild/linux-loong64": "0.28.1", "@esbuild/linux-mips64el": "0.28.1", "@esbuild/linux-ppc64": "0.28.1", "@esbuild/linux-riscv64": "0.28.1", "@esbuild/linux-s390x": "0.28.1", "@esbuild/linux-x64": "0.28.1", "@esbuild/netbsd-arm64": "0.28.1", "@esbuild/netbsd-x64": "0.28.1", "@esbuild/openbsd-arm64": "0.28.1", "@esbuild/openbsd-x64": "0.28.1", "@esbuild/openharmony-arm64": "0.28.1", "@esbuild/sunos-x64": "0.28.1", "@esbuild/win32-arm64": "0.28.1", "@esbuild/win32-ia32": "0.28.1", "@esbuild/win32-x64": "0.28.1" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw=="], + "esbuild": ["esbuild@0.28.2", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.2", "@esbuild/android-arm": "0.28.2", "@esbuild/android-arm64": "0.28.2", "@esbuild/android-x64": "0.28.2", "@esbuild/darwin-arm64": "0.28.2", "@esbuild/darwin-x64": "0.28.2", "@esbuild/freebsd-arm64": "0.28.2", "@esbuild/freebsd-x64": "0.28.2", "@esbuild/linux-arm": "0.28.2", "@esbuild/linux-arm64": "0.28.2", "@esbuild/linux-ia32": "0.28.2", "@esbuild/linux-loong64": "0.28.2", "@esbuild/linux-mips64el": "0.28.2", "@esbuild/linux-ppc64": "0.28.2", "@esbuild/linux-riscv64": "0.28.2", "@esbuild/linux-s390x": "0.28.2", "@esbuild/linux-x64": "0.28.2", "@esbuild/netbsd-arm64": "0.28.2", "@esbuild/netbsd-x64": "0.28.2", "@esbuild/openbsd-arm64": "0.28.2", "@esbuild/openbsd-x64": "0.28.2", "@esbuild/openharmony-arm64": "0.28.2", "@esbuild/sunos-x64": "0.28.2", "@esbuild/win32-arm64": "0.28.2", "@esbuild/win32-ia32": "0.28.2", "@esbuild/win32-x64": "0.28.2" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA=="], "escalade": ["escalade@3.2.0", "", {}, "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA=="], @@ -1289,15 +1296,13 @@ "extsprintf": ["extsprintf@1.3.0", "", {}, "sha512-11Ndz7Nv+mvAC1j0ktTa7fAb0vLyGGX+rMHNBYQviQDGU0Hw7lhctJANqbPhu9nV9/izT/IntTgZ7Im/9LJs9g=="], - "fake-indexeddb": ["fake-indexeddb@6.2.5", "", {}, "sha512-CGnyrvbhPlWYMngksqrSSUT1BAVP49dZocrHuK0SvtR0D5TMs5wP0o3j7jexDJW01KSadjBp1M/71o/KR3nD1w=="], - "fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="], "fast-glob": ["fast-glob@3.3.3", "", { "dependencies": { "@nodelib/fs.stat": "2.0.5", "@nodelib/fs.walk": "1.2.8", "glob-parent": "5.1.2", "merge2": "1.4.1", "micromatch": "4.0.8" } }, "sha512-7MptL8U0cqcFdzIzwOTHoilX9x5BrNqye7Z/LuC7kCMRio1EMSyqRK3BEAUD7sXRq4iT4AzTVuZdhgQ2TCvYLg=="], "fast-shuffle": ["fast-shuffle@6.1.1", "", { "dependencies": { "pcg": "1.1.0" } }, "sha512-HPxFJxEi18KPmVQuK5Hi5l4KSl3u50jtaxseRrPqrxewqfvU+sTPTaUpP33Hj+NdJoLuJP5ipx3ybTr+fa6dEw=="], - "fast-uri": ["fast-uri@4.1.1", "", {}, "sha512-YPOs1zD5TG2+EZt+r88LwF6mclA7TPkpwMP7ZN3TO2HiHS8TXvq7QA/17iJsV9dubcLo/f8eEYqMBruyQV21hQ=="], + "fast-uri": ["fast-uri@4.1.2", "", {}, "sha512-TyGmBcbDTZXcb2cj5MV89DrF42DKvb3y5DDUNh95iO+IMeAzMkVSxK1PZRrRIpc9yg8U2GhGdbofNa0LS/a4Bw=="], "fastq": ["fastq@1.20.1", "", { "dependencies": { "reusify": "1.1.0" } }, "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw=="], @@ -1409,7 +1414,7 @@ "hastscript": ["hastscript@9.0.1", "", { "dependencies": { "@types/hast": "^3.0.0", "comma-separated-tokens": "^2.0.0", "hast-util-parse-selector": "^4.0.0", "property-information": "^7.0.0", "space-separated-tokens": "^2.0.0" } }, "sha512-g7df9rMFX/SPi34tyGCyUBREQoKkapwdY/T04Qn9TDWfHhAYt4/I0gMVirzK5wEzeUqIjEB+LXC/ypb7Aqno5w=="], - "hono": ["hono@4.12.32", "", {}, "sha512-XcuyW9qE2kJn07PkecMOBd5Vq/hMy7mmGw+idz1yblbg9N17ijJODrvPkn7/dwL3Kulj8LcRJ69DLOWf91dRUg=="], + "hono": ["hono@4.13.2", "", {}, "sha512-JydRilDRkYBQMt9qR9U92mXxmbGqsqSn/IKOrh4e7/gEbn+0zSr8igTu0obwJoNGN4sez28DIql7FBHWydoJpA=="], "html-encoding-sniffer": ["html-encoding-sniffer@6.0.0", "", { "dependencies": { "@exodus/bytes": "1.15.1" } }, "sha512-CV9TW3Y3f8/wT0BRFc1/KAVQ3TUHiXmaAb6VW9vtiMFf7SLoMd1PdAc4W3KFOFETBJUb90KatHqlsZMWV+R9Gg=="], @@ -1443,7 +1448,7 @@ "internmap": ["internmap@2.0.3", "", {}, "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg=="], - "ip-address": ["ip-address@10.2.0", "", {}, "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA=="], + "ip-address": ["ip-address@10.5.0", "", {}, "sha512-R5SnVLJmgYYvf2F2ZgwSBnelz5G4q5AxIC277GDfUaNbrZKNANcBC7RHqYYePlszf4kBolVkJauG0ZjHHFh55g=="], "ipaddr.js": ["ipaddr.js@1.9.1", "", {}, "sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g=="], @@ -1549,11 +1554,11 @@ "js-tokens": ["js-tokens@10.0.0", "", {}, "sha512-lM/UBzQmfJRo9ABXbPWemivdCW8V2G8FHaHdypQaIy523snUjog0W71ayWXTjiR+ixeMyVHN2XcpnTd/liPg/Q=="], - "js-yaml": ["js-yaml@3.15.0", "", { "dependencies": { "argparse": "1.0.10", "esprima": "4.0.1" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-ttBQIIQPDeLjpPOohtUdXuXUVoA2uIB6fEH9HyJ7234s5mBJ5wTx20njxplLZQgLaOfpmPQA7X2t5AX6tIPbog=="], + "js-yaml": ["js-yaml@3.15.1", "", { "dependencies": { "argparse": "^1.0.7", "esprima": "^4.0.0" }, "bin": { "js-yaml": "bin/js-yaml.js" } }, "sha512-S99WuO3HlhO3XN41EtYUNl9zzXjoJx7QvmipxsJVxtCBT0YHEFy+iOJhjSvrmV12nYhWpZaM8lPHkJm0yUMbag=="], "jsbn": ["jsbn@0.1.1", "", {}, "sha512-UVU9dibq2JcFWxQPA6KCqj5O42VOmAY3zQUfEKxU0KpTGXwNoCjkX1e13eHNvw/xPynt6pU0rZ1htjWTNTSXsg=="], - "jsdom": ["jsdom@29.1.1", "", { "dependencies": { "@asamuzakjp/css-color": "5.1.11", "@asamuzakjp/dom-selector": "7.1.1", "@bramus/specificity": "2.4.2", "@csstools/css-syntax-patches-for-csstree": "1.1.6", "@exodus/bytes": "1.15.1", "css-tree": "3.2.1", "data-urls": "7.0.0", "decimal.js": "10.6.0", "html-encoding-sniffer": "6.0.0", "is-potential-custom-element-name": "1.0.1", "lru-cache": "11.5.2", "parse5": "8.0.1", "saxes": "6.0.0", "symbol-tree": "3.2.4", "tough-cookie": "6.0.2", "undici": "7.28.0", "w3c-xmlserializer": "5.0.0", "webidl-conversions": "8.0.1", "whatwg-mimetype": "5.0.0", "whatwg-url": "16.0.1", "xml-name-validator": "5.0.0" } }, "sha512-ECi4Fi2f7BdJtUKTflYRTiaMxIB0O6zfR1fX0GXpUrf6flp8QIYn1UT20YQqdSOfk2dfkCwS8LAFoJDEppNK5Q=="], + "jsdom": ["jsdom@30.0.1", "", { "dependencies": { "@asamuzakjp/css-color": "^6.0.5", "@asamuzakjp/dom-selector": "^8.3.0", "@bramus/specificity": "^2.4.2", "@csstools/css-syntax-patches-for-csstree": "^1.1.7", "@exodus/bytes": "^1.15.1", "css-tree": "^3.2.1", "data-urls": "^7.0.0", "decimal.js": "^10.6.0", "html-encoding-sniffer": "^6.0.0", "is-potential-custom-element-name": "^1.0.1", "lru-cache": "^11.5.2", "parse5": "^8.0.1", "saxes": "^6.0.0", "symbol-tree": "^3.2.4", "tough-cookie": "^6.0.2", "undici": "^8.9.0", "w3c-xmlserializer": "^5.0.0", "webidl-conversions": "^8.0.1", "whatwg-mimetype": "^5.0.0", "whatwg-url": "^17.1.0", "xml-name-validator": "^5.0.0" }, "peerDependencies": { "canvas": "^3.2.3" }, "optionalPeers": ["canvas"] }, "sha512-52v7mUVUfNQVYYqE1lcdaymWL0njO7lTLUog6ZvW2U5KsbiLk/GnZlVJ+qx0xfNJZ6Gn+KSpPNE52vurbxZwrA=="], "jsesc": ["jsesc@3.1.0", "", { "bin": { "jsesc": "bin/jsesc" } }, "sha512-/sM3dO2FOzXjKQhJuo0Q173wf2KOo8t4I8vHy6lF9poUp7bKT0/NHE8fPX23PwfhnykfqnC2xRxOnVw5XuGIaA=="], @@ -1599,29 +1604,29 @@ "lefthook-windows-x64": ["lefthook-windows-x64@2.1.10", "", { "os": "win32", "cpu": "x64" }, "sha512-c/XH8YZtylG4XaxzqFfXluvq2LXq2W/p54Bnzn3+Z7E5X2Fk3JlFJAibulMbIt2+w8T7UI/r97ok5GqE4kGaeA=="], - "lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "2.1.2" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], + "lightningcss": ["lightningcss@1.33.0", "", { "dependencies": { "detect-libc": "^2.0.3" }, "optionalDependencies": { "lightningcss-android-arm64": "1.33.0", "lightningcss-darwin-arm64": "1.33.0", "lightningcss-darwin-x64": "1.33.0", "lightningcss-freebsd-x64": "1.33.0", "lightningcss-linux-arm-gnueabihf": "1.33.0", "lightningcss-linux-arm64-gnu": "1.33.0", "lightningcss-linux-arm64-musl": "1.33.0", "lightningcss-linux-x64-gnu": "1.33.0", "lightningcss-linux-x64-musl": "1.33.0", "lightningcss-win32-arm64-msvc": "1.33.0", "lightningcss-win32-x64-msvc": "1.33.0" } }, "sha512-WkUDrojuJs0xkgGf2udWxa3yGBRxPtxUkB79i6aCZLRgc7PM8fZe9TosfPDcvEpQZbuFASnHYmRLBLUbmLOIIA=="], - "lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], + "lightningcss-android-arm64": ["lightningcss-android-arm64@1.33.0", "", { "os": "android", "cpu": "arm64" }, "sha512-gEpRTalKdosp4Bb8qWtc2iOgE5SeIHlpS1up9bFq2wAyYhl1UdTObYiHe98zEM9SQvSoqQZ1IQD0JNpg3Ml5pg=="], - "lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.32.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ=="], + "lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.33.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-Sciaz8eenNTKn9b3t7+xr0ipTp9YxKQY4npwQ3mrRuL0BAVHBLyZxofhaKBAVtzmtRZ/zTyo0/to4B1uWG/Djg=="], - "lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.32.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w=="], + "lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.33.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-Z5UPAxzrjlWNNyGy6i65cJzzvgJ5D3T6wMvs+gWpY9d7qRhANrxqAp6LhxIgZhWEw18RfJTGcRxjuLIBr+m8XQ=="], - "lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.32.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig=="], + "lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.33.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-QQM/Ti/hQajJwCY+RiWuCZ9sdtI/XQk7nDK5vC8kkdwixezOlDgvDx7+RT+QjK6FcFT4MpsuoBnHIo/O3StRRg=="], - "lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw=="], + "lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.33.0", "", { "os": "linux", "cpu": "arm" }, "sha512-N7FVBe6iS24MlM6R/4RBTxGhQheZGs7tiQ9U32UtF75NzP5Q7xWPRqLBCKxlRQRk3rY1jCIPLzx7WzOhuUIRLQ=="], - "lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ=="], + "lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.33.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-j2v/itmy4HlNxlc6voKXYgBqNi0Ng2LShg4z7GufpEgs05P+2suBVyi9I6YHq5uoVFx9ETin3eCEhLVyXGQnKg=="], - "lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg=="], + "lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.33.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-yiO5ROMuYQgXbC60yjZU5CYSFZGKXL0HFATXt9mHJn1+zW55oCtMI9NfcVhYLMFDL7gV7oBPon/EmMMGg2OvtQ=="], - "lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA=="], + "lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.33.0", "", { "os": "linux", "cpu": "x64" }, "sha512-ar+Ju7LmcN0Jo4FpL4hpFybwNG9/3A/Br5KW2n2jyODg3MEZXaDYADdemoNS+BDNfMgKvylJLj4S5tyRActuAg=="], - "lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg=="], + "lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.33.0", "", { "os": "linux", "cpu": "x64" }, "sha512-RYiYbkokw0trfKqqzfF55lginwEPrD3OJDfTuJzFs1MK6iFnDenaz1fqLLtX4ITG3OktJQXOeTaw1awrBAlZPw=="], - "lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.32.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw=="], + "lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.33.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-1K+MPfLSFVpphzpdbfkhlWk6wBrTObBzS2T6db10PNOZgR9GoVsAWzwNyuhUYYbTp23j+4RrncfujZ4uAzXvwA=="], - "lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q=="], + "lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.33.0", "", { "os": "win32", "cpu": "x64" }, "sha512-OlEICDx/Xl0FqSp4bry8zFnCvGpig3Gl4gCquvYwHuqJKEC1+n9NgDniFvqHGmMv1ZkqDJrDqKKSykTDX+ehuA=="], "listr2": ["listr2@9.0.5", "", { "dependencies": { "cli-truncate": "5.2.0", "colorette": "2.0.20", "eventemitter3": "5.0.4", "log-update": "6.1.0", "rfdc": "1.4.1", "wrap-ansi": "9.0.2" } }, "sha512-ME4Fb83LgEgwNw96RKNvKV4VTLuXfoKudAmm2lP8Kk87KaMK0/Xrx/aAkMWmT8mDb+3MlFDspfbCs7adjRxA2g=="], @@ -1641,7 +1646,7 @@ "lru-cache": ["lru-cache@11.5.2", "", {}, "sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g=="], - "lucide-react": ["lucide-react@1.27.0", "", { "peerDependencies": { "react": "19.2.8" } }, "sha512-rJicGl/3Fly/E0rOH1YmPZ6e49JCnKknh1ox1vpHnkfjujAkKA6sqUZvH3MTAaXXjgexyUwgNwTJzTtYuAFYJw=="], + "lucide-react": ["lucide-react@1.31.0", "", { "peerDependencies": { "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-G8u2eEtoHUnUa9f8lbvqDhCiORMnYLdUEo06EEG9MQvHQrInKcX3Pa2TH39MM5qyzRcWETxB0+aOwAPI1g1kEg=="], "magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="], @@ -1797,13 +1802,13 @@ "ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="], - "nanoid": ["nanoid@3.3.16", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q=="], + "nanoid": ["nanoid@3.3.18", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w=="], "negotiator": ["negotiator@1.0.0", "", {}, "sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg=="], "neo-async": ["neo-async@2.6.2", "", {}, "sha512-Yd3UES5mWCSqR+qNT93S3UoYUkqAZ9lLg8a7g9rimsWmYGK8cVToA4/sF3RrshdyV3sAGMXVUmpMYOw+dLpOuw=="], - "next": ["next@16.2.12", "", { "dependencies": { "@next/env": "16.2.12", "@swc/helpers": "0.5.15", "baseline-browser-mapping": "2.10.42", "caniuse-lite": "1.0.30001803", "postcss": "8.5.23", "styled-jsx": "5.1.6" }, "optionalDependencies": { "@next/swc-darwin-arm64": "16.2.12", "@next/swc-darwin-x64": "16.2.12", "@next/swc-linux-arm64-gnu": "16.2.12", "@next/swc-linux-arm64-musl": "16.2.12", "@next/swc-linux-x64-gnu": "16.2.12", "@next/swc-linux-x64-musl": "16.2.12", "@next/swc-win32-arm64-msvc": "16.2.12", "@next/swc-win32-x64-msvc": "16.2.12", "@opentelemetry/api": "1.9.1", "sharp": "0.35.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" }, "bin": { "next": "dist/bin/next" } }, "sha512-iD59eYQWmbFcEbX7v/acG5DRym9iw1DdaPoD0WTA920naWsE25wShzJW4+UvAs8MK9EC2kBfIH6vtto1H1PHGw=="], + "next": ["next@16.3.1", "", { "dependencies": { "@next/env": "16.3.1", "@swc/helpers": "0.5.23", "baseline-browser-mapping": "^2.9.19", "caniuse-lite": "^1.0.30001579", "postcss": "8.5.23", "styled-jsx": "5.1.6" }, "optionalDependencies": { "@next/swc-darwin-arm64": "16.3.1", "@next/swc-darwin-x64": "16.3.1", "@next/swc-linux-arm64-gnu": "16.3.1", "@next/swc-linux-arm64-musl": "16.3.1", "@next/swc-linux-x64-gnu": "16.3.1", "@next/swc-linux-x64-musl": "16.3.1", "@next/swc-win32-arm64-msvc": "16.3.1", "@next/swc-win32-x64-msvc": "16.3.1", "sharp": "^0.35.3" }, "peerDependencies": { "@opentelemetry/api": "^1.1.0", "@playwright/test": "^1.51.1", "babel-plugin-react-compiler": "*", "react": "^18.2.0 || 19.0.0-rc-de68d2f4-20241204 || ^19.0.0", "react-dom": "^18.2.0 || 19.0.0-rc-de68d2f4-20241204 || ^19.0.0", "sass": "^1.3.0" }, "optionalPeers": ["@opentelemetry/api", "@playwright/test", "babel-plugin-react-compiler", "sass"], "bin": { "next": "dist/bin/next" } }, "sha512-hsAp0i7Rh+/dhe7DGIeN2YlpLM1DP4MNxti9EtDMtqcO612X81MvvEj388/oTce9U1EcEIOWDlGq0zRwrBKvuA=="], "next-mdx-remote": ["next-mdx-remote@6.0.0", "", { "dependencies": { "@babel/code-frame": "7.29.0", "@mdx-js/mdx": "3.1.1", "@mdx-js/react": "3.1.1", "unist-util-remove": "4.0.0", "unist-util-visit": "5.1.0", "vfile": "6.0.3", "vfile-matter": "5.0.1" }, "peerDependencies": { "react": "19.2.8" } }, "sha512-cJEpEZlgD6xGjB4jL8BnI8FaYdN9BzZM4NwadPe1YQr7pqoWjg9EBCMv3nXBkuHqMRfv2y33SzUsuyNh9LFAQQ=="], @@ -1843,9 +1848,9 @@ "ospath": ["ospath@1.2.2", "", {}, "sha512-o6E5qJV5zkAbIDNhGSIlyOhScKXgQrSRMilfph0clDfM0nEnBOlKlH4sWDmG95BW/CvwNz0vmm7dJVtU2KlMiA=="], - "oxfmt": ["oxfmt@0.60.0", "", { "dependencies": { "tinypool": "2.1.0" }, "optionalDependencies": { "@oxfmt/binding-android-arm-eabi": "0.60.0", "@oxfmt/binding-android-arm64": "0.60.0", "@oxfmt/binding-darwin-arm64": "0.60.0", "@oxfmt/binding-darwin-x64": "0.60.0", "@oxfmt/binding-freebsd-x64": "0.60.0", "@oxfmt/binding-linux-arm-gnueabihf": "0.60.0", "@oxfmt/binding-linux-arm-musleabihf": "0.60.0", "@oxfmt/binding-linux-arm64-gnu": "0.60.0", "@oxfmt/binding-linux-arm64-musl": "0.60.0", "@oxfmt/binding-linux-ppc64-gnu": "0.60.0", "@oxfmt/binding-linux-riscv64-gnu": "0.60.0", "@oxfmt/binding-linux-riscv64-musl": "0.60.0", "@oxfmt/binding-linux-s390x-gnu": "0.60.0", "@oxfmt/binding-linux-x64-gnu": "0.60.0", "@oxfmt/binding-linux-x64-musl": "0.60.0", "@oxfmt/binding-openharmony-arm64": "0.60.0", "@oxfmt/binding-win32-arm64-msvc": "0.60.0", "@oxfmt/binding-win32-ia32-msvc": "0.60.0", "@oxfmt/binding-win32-x64-msvc": "0.60.0" }, "bin": { "oxfmt": "bin/oxfmt" } }, "sha512-fViX6i+gJuZWY+jI/fnR6WRbRj70GZ9RlCd30MygJrHTUNc4DxvKHWw8vBjMjffv3PgU5qWDR0AzmojQByqaZA=="], + "oxfmt": ["oxfmt@0.63.0", "", { "dependencies": { "tinypool": "2.1.0" }, "optionalDependencies": { "@oxfmt/binding-android-arm-eabi": "0.63.0", "@oxfmt/binding-android-arm64": "0.63.0", "@oxfmt/binding-darwin-arm64": "0.63.0", "@oxfmt/binding-darwin-x64": "0.63.0", "@oxfmt/binding-freebsd-x64": "0.63.0", "@oxfmt/binding-linux-arm-gnueabihf": "0.63.0", "@oxfmt/binding-linux-arm-musleabihf": "0.63.0", "@oxfmt/binding-linux-arm64-gnu": "0.63.0", "@oxfmt/binding-linux-arm64-musl": "0.63.0", "@oxfmt/binding-linux-ppc64-gnu": "0.63.0", "@oxfmt/binding-linux-riscv64-gnu": "0.63.0", "@oxfmt/binding-linux-riscv64-musl": "0.63.0", "@oxfmt/binding-linux-s390x-gnu": "0.63.0", "@oxfmt/binding-linux-x64-gnu": "0.63.0", "@oxfmt/binding-linux-x64-musl": "0.63.0", "@oxfmt/binding-openharmony-arm64": "0.63.0", "@oxfmt/binding-win32-arm64-msvc": "0.63.0", "@oxfmt/binding-win32-ia32-msvc": "0.63.0", "@oxfmt/binding-win32-x64-msvc": "0.63.0" }, "peerDependencies": { "svelte": "^5.0.0", "vite-plus": "*" }, "optionalPeers": ["svelte", "vite-plus"], "bin": { "oxfmt": "bin/oxfmt" } }, "sha512-kgdDwv35wvVf6554U2Ab8Jnd0zTM+TsEQWwaB70RAjK3gICFAFGO+2Hd3Be27GMoXj3XRL9IKSNRVl7KBQL6iw=="], - "oxlint": ["oxlint@1.75.0", "", { "optionalDependencies": { "@oxlint/binding-android-arm-eabi": "1.75.0", "@oxlint/binding-android-arm64": "1.75.0", "@oxlint/binding-darwin-arm64": "1.75.0", "@oxlint/binding-darwin-x64": "1.75.0", "@oxlint/binding-freebsd-x64": "1.75.0", "@oxlint/binding-linux-arm-gnueabihf": "1.75.0", "@oxlint/binding-linux-arm-musleabihf": "1.75.0", "@oxlint/binding-linux-arm64-gnu": "1.75.0", "@oxlint/binding-linux-arm64-musl": "1.75.0", "@oxlint/binding-linux-ppc64-gnu": "1.75.0", "@oxlint/binding-linux-riscv64-gnu": "1.75.0", "@oxlint/binding-linux-riscv64-musl": "1.75.0", "@oxlint/binding-linux-s390x-gnu": "1.75.0", "@oxlint/binding-linux-x64-gnu": "1.75.0", "@oxlint/binding-linux-x64-musl": "1.75.0", "@oxlint/binding-openharmony-arm64": "1.75.0", "@oxlint/binding-win32-arm64-msvc": "1.75.0", "@oxlint/binding-win32-ia32-msvc": "1.75.0", "@oxlint/binding-win32-x64-msvc": "1.75.0" }, "bin": { "oxlint": "bin/oxlint" } }, "sha512-m9WzjRcRYA/uqIZDa9tclrieoPJ/ln1QYTKdFx6NUOs8uY5DiHlIwRQoCrHT6OM6O3ww3l2skY5gO7G7ZphE7g=="], + "oxlint": ["oxlint@1.78.0", "", { "optionalDependencies": { "@oxlint/binding-android-arm-eabi": "1.78.0", "@oxlint/binding-android-arm64": "1.78.0", "@oxlint/binding-darwin-arm64": "1.78.0", "@oxlint/binding-darwin-x64": "1.78.0", "@oxlint/binding-freebsd-x64": "1.78.0", "@oxlint/binding-linux-arm-gnueabihf": "1.78.0", "@oxlint/binding-linux-arm-musleabihf": "1.78.0", "@oxlint/binding-linux-arm64-gnu": "1.78.0", "@oxlint/binding-linux-arm64-musl": "1.78.0", "@oxlint/binding-linux-ppc64-gnu": "1.78.0", "@oxlint/binding-linux-riscv64-gnu": "1.78.0", "@oxlint/binding-linux-riscv64-musl": "1.78.0", "@oxlint/binding-linux-s390x-gnu": "1.78.0", "@oxlint/binding-linux-x64-gnu": "1.78.0", "@oxlint/binding-linux-x64-musl": "1.78.0", "@oxlint/binding-openharmony-arm64": "1.78.0", "@oxlint/binding-win32-arm64-msvc": "1.78.0", "@oxlint/binding-win32-ia32-msvc": "1.78.0", "@oxlint/binding-win32-x64-msvc": "1.78.0" }, "peerDependencies": { "oxlint-tsgolint": ">=7.0.2001", "vite-plus": "*" }, "optionalPeers": ["oxlint-tsgolint", "vite-plus"], "bin": { "oxlint": "bin/oxlint" } }, "sha512-QgQePuxIqKOzo1KSjG2EnITEeWvWnKAm77eq8nrMtf6AGoA+zyGc4PFYtDNJSD25g/ibOwfQ851hZ4/SPkMVoA=="], "package-json-from-dist": ["package-json-from-dist@1.0.1", "", {}, "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw=="], @@ -1885,7 +1890,7 @@ "possible-typed-array-names": ["possible-typed-array-names@1.1.0", "", {}, "sha512-/+5VFTchJDoVj3bhoqi6UeymcD00DAwb1nJwamzPvHEszJ4FpF6SNNbUbOS8yI56qHzdV8eK0qEfOSiodkTdxg=="], - "postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "postcss": ["postcss@8.5.26", "", { "dependencies": { "nanoid": "^3.3.17", "picocolors": "^1.1.1", "source-map-js": "^1.2.1" } }, "sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ=="], "postcss-selector-parser": ["postcss-selector-parser@6.0.10", "", { "dependencies": { "cssesc": "3.0.0", "util-deprecate": "1.0.2" } }, "sha512-IQ7TZdoaqbT+LCpShg46jnZVlhWD2w6iQYAcYXfHARZ7X1t/UGhhceQDs5X0cGqKvYlHNOuv7Oa1xmb0oQuA3w=="], @@ -1997,7 +2002,7 @@ "robust-predicates": ["robust-predicates@3.0.3", "", {}, "sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA=="], - "rolldown": ["rolldown@1.1.5", "", { "dependencies": { "@oxc-project/types": "0.139.0", "@rolldown/pluginutils": "1.0.1" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.1.5", "@rolldown/binding-darwin-arm64": "1.1.5", "@rolldown/binding-darwin-x64": "1.1.5", "@rolldown/binding-freebsd-x64": "1.1.5", "@rolldown/binding-linux-arm-gnueabihf": "1.1.5", "@rolldown/binding-linux-arm64-gnu": "1.1.5", "@rolldown/binding-linux-arm64-musl": "1.1.5", "@rolldown/binding-linux-ppc64-gnu": "1.1.5", "@rolldown/binding-linux-s390x-gnu": "1.1.5", "@rolldown/binding-linux-x64-gnu": "1.1.5", "@rolldown/binding-linux-x64-musl": "1.1.5", "@rolldown/binding-openharmony-arm64": "1.1.5", "@rolldown/binding-wasm32-wasi": "1.1.5", "@rolldown/binding-win32-arm64-msvc": "1.1.5", "@rolldown/binding-win32-x64-msvc": "1.1.5" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-t9z29cJjXf/vxQ8dyhCSpt6H6aSwHTk8cT5I3iy6SMXuFpk5mB6PL6XfC8PCwrPTx93udwKUm9HRteAlTGBLiA=="], + "rolldown": ["rolldown@1.2.4", "", { "dependencies": { "@oxc-project/types": "=0.144.0", "@rolldown/pluginutils": "^1.0.0" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.2.4", "@rolldown/binding-darwin-arm64": "1.2.4", "@rolldown/binding-darwin-x64": "1.2.4", "@rolldown/binding-freebsd-x64": "1.2.4", "@rolldown/binding-linux-arm-gnueabihf": "1.2.4", "@rolldown/binding-linux-arm64-gnu": "1.2.4", "@rolldown/binding-linux-arm64-musl": "1.2.4", "@rolldown/binding-linux-ppc64-gnu": "1.2.4", "@rolldown/binding-linux-s390x-gnu": "1.2.4", "@rolldown/binding-linux-x64-gnu": "1.2.4", "@rolldown/binding-linux-x64-musl": "1.2.4", "@rolldown/binding-openharmony-arm64": "1.2.4", "@rolldown/binding-win32-arm64-msvc": "1.2.4", "@rolldown/binding-win32-x64-msvc": "1.2.4" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-rSr7irW0K7QRWzjdJXqZowkcRdDtjRduh43rBltnVKd0VFq839l1lJoDvGJb6gl7+4rTTCrPWu+YfujUL8Ug7w=="], "router": ["router@2.2.0", "", { "dependencies": { "debug": "4.4.3", "depd": "2.0.0", "is-promise": "4.0.0", "parseurl": "1.3.3", "path-to-regexp": "8.4.2" } }, "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ=="], @@ -2041,7 +2046,7 @@ "shelljs": ["shelljs@0.10.0", "", { "dependencies": { "execa": "5.1.1", "fast-glob": "3.3.3" } }, "sha512-Jex+xw5Mg2qMZL3qnzXIfaxEtBaC4n7xifqaqtrZDdlheR70OGkydrPJWT0V1cA1k3nanC86x9FwAmQl6w3Klw=="], - "shiki": ["shiki@4.3.1", "", { "dependencies": { "@shikijs/core": "4.3.1", "@shikijs/engine-javascript": "4.3.1", "@shikijs/engine-oniguruma": "4.3.1", "@shikijs/langs": "4.3.1", "@shikijs/themes": "4.3.1", "@shikijs/types": "4.3.1", "@shikijs/vscode-textmate": "10.0.2", "@types/hast": "3.0.4" } }, "sha512-oR+qDVi2OjX1tmDpyv+3KviX01KzO6Af+0NNnKnsp9491UEGz2YpxTuJboS/6VhYpTdqzmuJBuiTlrAWWJAssw=="], + "shiki": ["shiki@4.4.3", "", { "dependencies": { "@shikijs/core": "4.4.3", "@shikijs/engine-javascript": "4.4.3", "@shikijs/engine-oniguruma": "4.4.3", "@shikijs/langs": "4.4.3", "@shikijs/themes": "4.4.3", "@shikijs/types": "4.4.3", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.5" } }, "sha512-Mb/GvXPHBAXdgGIcnfU5L3ldpn1XcxrGkPHwqgRx17/I2XRfqlFKk2vGkHWINn1kdXvzJZeuO3is6I9KLPFm0g=="], "side-channel": ["side-channel@1.1.1", "", { "dependencies": { "es-errors": "1.3.0", "object-inspect": "1.13.4", "side-channel-list": "1.0.1", "side-channel-map": "1.0.1", "side-channel-weakmap": "1.0.2" } }, "sha512-6x6dK6zJdpTzF4sQeNYxwtvBzf6Eg4GtlesS94HOvTudUeyK2WXAaIfmDgsyslYrRBeFIlsi54AYsFGUuhmvrQ=="], @@ -2185,7 +2190,7 @@ "typescript": ["typescript@6.0.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw=="], - "undici": ["undici@7.28.0", "", {}, "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA=="], + "undici": ["undici@8.10.0", "", {}, "sha512-HvltHd7avK13QIw/oLe4qoOLyoVSoafqJ2jYOrtMRBkbYT31eiBQ8O0ehRKZiEZCMEyLFQNIADpgCWC5fALvYQ=="], "undici-types": ["undici-types@8.3.0", "", {}, "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ=="], @@ -2237,7 +2242,7 @@ "vfile-message": ["vfile-message@4.0.3", "", { "dependencies": { "@types/unist": "3.0.3", "unist-util-stringify-position": "4.0.0" } }, "sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw=="], - "vite": ["vite@8.1.5", "", { "dependencies": { "lightningcss": "1.32.0", "picomatch": "4.0.5", "postcss": "8.5.23", "rolldown": "1.1.5", "tinyglobby": "0.2.17" }, "optionalDependencies": { "@types/node": "26.1.1", "esbuild": "0.28.1", "fsevents": "2.3.3", "jiti": "2.7.0", "terser": "5.49.0", "tsx": "4.23.1", "yaml": "2.9.0" }, "bin": { "vite": "bin/vite.js" } }, "sha512-7ULLwsCdYx/nRyrpiEwvqb5TFHrMVZyBt+rg/OAXT7rgj/z+DtTDyKFeLAdDkubDVDKD8jOsndmy7m55XcfUsw=="], + "vite": ["vite@8.2.1", "", { "dependencies": { "lightningcss": "^1.33.0", "picomatch": "^4.0.5", "postcss": "^8.5.25", "rolldown": "~1.2.1", "tinyglobby": "^0.2.17" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", "@vitejs/devtools": "^0.4.0", "esbuild": "^0.27.0 || ^0.28.0", "jiti": ">=1.21.0", "less": "^4.0.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", "sugarss": "^5.0.0", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "@vitejs/devtools", "esbuild", "jiti", "less", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-EU/eS7BH3XROHh2YnBefjM6DBKA6ZeMZEYQbj7NLWg5wHYlhB8B/Mayd5XsgWq+NFYccDOTemRpdETWR6Ka/lw=="], "vitest": ["vitest@4.1.10", "", { "dependencies": { "@vitest/expect": "4.1.10", "@vitest/mocker": "4.1.10", "@vitest/pretty-format": "4.1.10", "@vitest/runner": "4.1.10", "@vitest/snapshot": "4.1.10", "@vitest/spy": "4.1.10", "@vitest/utils": "4.1.10", "es-module-lexer": "2.3.0", "expect-type": "1.4.0", "magic-string": "0.30.21", "obug": "2.1.3", "pathe": "2.0.3", "picomatch": "4.0.5", "std-env": "4.2.0", "tinybench": "2.9.0", "tinyexec": "1.2.4", "tinyglobby": "0.2.17", "tinyrainbow": "3.1.0", "why-is-node-running": "2.3.0" }, "optionalDependencies": { "@opentelemetry/api": "1.9.1", "@types/node": "26.1.1", "@vitest/coverage-v8": "4.1.10", "jsdom": "29.1.1" }, "peerDependencies": { "vite": "8.1.5" }, "bin": { "vitest": "./vitest.mjs" } }, "sha512-R9jUTe5S4Qb0HCd4TNqpC7oGcrMssMRGXLW80ubjWsW9VH5GF8y1Y0SFLY9AbqSk6nt0PnOx4H4WNJYZ13GUPw=="], @@ -2259,7 +2264,7 @@ "whatwg-mimetype": ["whatwg-mimetype@5.0.0", "", {}, "sha512-sXcNcHOC51uPGF0P/D4NVtrkjSU2fNsm9iog4ZvZJsL3rjoDAzXZhkm2MWt1y+PUdggKAYVoMAIYcs78wJ51Cw=="], - "whatwg-url": ["whatwg-url@16.0.1", "", { "dependencies": { "@exodus/bytes": "1.15.1", "tr46": "6.0.0", "webidl-conversions": "8.0.1" } }, "sha512-1to4zXBxmXHV3IiSSEInrreIlu02vUOvrhxJJH5vcxYTBDAx51cqZiKdyTxlecdKNSjj8EcxGBxNf6Vg+945gw=="], + "whatwg-url": ["whatwg-url@17.1.0", "", { "dependencies": { "@exodus/bytes": "^1.15.1", "tr46": "^6.0.0", "webidl-conversions": "^8.0.1" } }, "sha512-3GeworPmc2ZfEEHP7lEbUfBX/L75wdEsi0rLNhXcXxnoN5jyq0SL5gCy06SGW2cyTIZdTvWIDQNQoza++vKeaw=="], "which": ["which@2.0.2", "", { "dependencies": { "isexe": "2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], @@ -2295,7 +2300,7 @@ "zwitch": ["zwitch@2.0.4", "", {}, "sha512-bXE4cR/kVZhKZX/RjPEflHaKVhUVl85noU3v6b8apfQEc1x4A+zBxjZ4lN8LqGd6WZ3dl98pY4o717VFmoPp+A=="], - "@actions/http-client/undici": ["undici@6.27.0", "", {}, "sha512-YmfV3YnEDzXRC5lZ2jWtWWHKGUm1zIt8AhesR1tens+HTNv+YZlN/dp6G727LOvMJ8xjP9Be7Y2Sdr96LDm+pg=="], + "@bahmutov/cypress-esbuild-preprocessor/esbuild": ["esbuild@0.28.1", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.1", "@esbuild/android-arm": "0.28.1", "@esbuild/android-arm64": "0.28.1", "@esbuild/android-x64": "0.28.1", "@esbuild/darwin-arm64": "0.28.1", "@esbuild/darwin-x64": "0.28.1", "@esbuild/freebsd-arm64": "0.28.1", "@esbuild/freebsd-x64": "0.28.1", "@esbuild/linux-arm": "0.28.1", "@esbuild/linux-arm64": "0.28.1", "@esbuild/linux-ia32": "0.28.1", "@esbuild/linux-loong64": "0.28.1", "@esbuild/linux-mips64el": "0.28.1", "@esbuild/linux-ppc64": "0.28.1", "@esbuild/linux-riscv64": "0.28.1", "@esbuild/linux-s390x": "0.28.1", "@esbuild/linux-x64": "0.28.1", "@esbuild/netbsd-arm64": "0.28.1", "@esbuild/netbsd-x64": "0.28.1", "@esbuild/openbsd-arm64": "0.28.1", "@esbuild/openbsd-x64": "0.28.1", "@esbuild/openharmony-arm64": "0.28.1", "@esbuild/sunos-x64": "0.28.1", "@esbuild/win32-arm64": "0.28.1", "@esbuild/win32-ia32": "0.28.1", "@esbuild/win32-x64": "0.28.1" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw=="], "@cypress/request/tough-cookie": ["tough-cookie@5.1.2", "", { "dependencies": { "tldts": "6.1.86" } }, "sha512-FVDYdxtnj0G6Qm/DhNPSb8Ju59ULcup3tuJxkFb5K8Bv2pUXILbf0xZWU8PX8Ov19OXljbUyveOFwRMwkXzO+A=="], @@ -2307,21 +2312,33 @@ "@emnapi/wasi-threads/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], - "@img/sharp-wasm32/@emnapi/runtime": ["@emnapi/runtime@1.11.2", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-kyOl3X0DuTiT1h2ft8r2fYO8JYtU9a9Xis/zBSiGArNaagCOWx90N1k2wxp18czFDH+OgcWGb5ZP/XMt3dcyPA=="], + "@mdx-js/mdx/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], "@mdx-js/mdx/acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], - "@posthog/nextjs-config/semver": ["semver@7.8.0", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-AcM7dV/5ul4EekoQ29Agm5vri8JNqRyj39o0qpX6vDF2GZrtutZl5RwgD1XnZjiTAfncsJhMI48QQH3sN87YNA=="], + "@napi-rs/wasm-runtime/@emnapi/runtime": ["@emnapi/runtime@1.11.1", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw=="], - "@posthog/webpack-plugin/@posthog/core": ["@posthog/core@1.40.1", "", { "dependencies": { "@posthog/types": "1.393.0" } }, "sha512-jXuMtZCwA7AMpYlo1wjm6GlC58YBlz/ZxpDIsF9hroUfqYoPPDmQbsQb4iiZAS43+o/kwloxdKJIlE0IiwQ5HQ=="], + "@posthog/webpack-plugin/@posthog/core": ["@posthog/core@1.48.1", "", { "dependencies": { "@posthog/types": "^1.404.1" } }, "sha512-mxw31XdYgt/SnlwqLPAcltK67q+QmsiYjVLGQ4GbBc8OJ7O4yRFSDwAXt8QsokHokeyEZtTRWM6jDHL7LYMx1A=="], "@react-three/fiber/buffer": ["buffer@6.0.3", "", { "dependencies": { "base64-js": "1.5.1", "ieee754": "1.2.1" } }, "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA=="], + "@rolldown/binding-wasm32-wasi/@emnapi/runtime": ["@emnapi/runtime@1.11.1", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw=="], + "@swc/helpers/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + "@tailwindcss/node/lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "2.1.2" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], + + "@tailwindcss/postcss/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "@tybys/wasm-util/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], - "@vercel/blob/undici": ["undici@6.27.0", "", {}, "sha512-YmfV3YnEDzXRC5lZ2jWtWWHKGUm1zIt8AhesR1tens+HTNv+YZlN/dp6G727LOvMJ8xjP9Be7Y2Sdr96LDm+pg=="], + "@types/adm-zip/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + + "@types/stream-chain/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + + "@types/stream-json/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + + "@vercel/analytics/next": ["next@16.2.12", "", { "dependencies": { "@next/env": "16.2.12", "@swc/helpers": "0.5.15", "baseline-browser-mapping": "2.10.42", "caniuse-lite": "1.0.30001803", "postcss": "8.5.23", "styled-jsx": "5.1.6" }, "optionalDependencies": { "@next/swc-darwin-arm64": "16.2.12", "@next/swc-darwin-x64": "16.2.12", "@next/swc-linux-arm64-gnu": "16.2.12", "@next/swc-linux-arm64-musl": "16.2.12", "@next/swc-linux-x64-gnu": "16.2.12", "@next/swc-linux-x64-musl": "16.2.12", "@next/swc-win32-arm64-msvc": "16.2.12", "@next/swc-win32-x64-msvc": "16.2.12", "@opentelemetry/api": "1.9.1", "sharp": "0.35.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" }, "bin": { "next": "dist/bin/next" } }, "sha512-iD59eYQWmbFcEbX7v/acG5DRym9iw1DdaPoD0WTA920naWsE25wShzJW4+UvAs8MK9EC2kBfIH6vtto1H1PHGw=="], "@vercel/cli-config/zod": ["zod@4.1.11", "", {}, "sha512-WPsqwxITS2tzx1bzhIKsEs19ABD5vmCVa4xBo2tq/SrV4RNZtfws1EnCWQXM6yh8bD08a1idvkB5MZSBiZsjwg=="], @@ -2329,6 +2346,10 @@ "@vercel/oidc/jose": ["jose@5.10.0", "", {}, "sha512-s+3Al/p9g32Iq+oqXxkW//7jk2Vig6FF1CFqzVXoTUXt2qz89YWbL+OwS17NFYEvxC35n0FKeGO2LGYSxeM2Gg=="], + "@vercel/speed-insights/next": ["next@16.2.12", "", { "dependencies": { "@next/env": "16.2.12", "@swc/helpers": "0.5.15", "baseline-browser-mapping": "2.10.42", "caniuse-lite": "1.0.30001803", "postcss": "8.5.23", "styled-jsx": "5.1.6" }, "optionalDependencies": { "@next/swc-darwin-arm64": "16.2.12", "@next/swc-darwin-x64": "16.2.12", "@next/swc-linux-arm64-gnu": "16.2.12", "@next/swc-linux-arm64-musl": "16.2.12", "@next/swc-linux-x64-gnu": "16.2.12", "@next/swc-linux-x64-musl": "16.2.12", "@next/swc-win32-arm64-msvc": "16.2.12", "@next/swc-win32-x64-msvc": "16.2.12", "@opentelemetry/api": "1.9.1", "sharp": "0.35.3" }, "peerDependencies": { "react": "19.2.8", "react-dom": "19.2.8" }, "bin": { "next": "dist/bin/next" } }, "sha512-iD59eYQWmbFcEbX7v/acG5DRym9iw1DdaPoD0WTA920naWsE25wShzJW4+UvAs8MK9EC2kBfIH6vtto1H1PHGw=="], + + "@vitest/mocker/vite": ["vite@8.1.5", "", { "dependencies": { "lightningcss": "1.32.0", "picomatch": "4.0.5", "postcss": "8.5.23", "rolldown": "1.1.5", "tinyglobby": "0.2.17" }, "optionalDependencies": { "@types/node": "26.1.1", "esbuild": "0.28.1", "fsevents": "2.3.3", "jiti": "2.7.0", "terser": "5.49.0", "tsx": "4.23.1", "yaml": "2.9.0" }, "bin": { "vite": "bin/vite.js" } }, "sha512-7ULLwsCdYx/nRyrpiEwvqb5TFHrMVZyBt+rg/OAXT7rgj/z+DtTDyKFeLAdDkubDVDKD8jOsndmy7m55XcfUsw=="], + "@vue/compiler-core/@babel/parser": ["@babel/parser@7.29.7", "", { "dependencies": { "@babel/types": "7.29.7" }, "bin": "./bin/babel-parser.js" }, "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg=="], "@vue/compiler-core/entities": ["entities@7.0.1", "", {}, "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA=="], @@ -2339,6 +2360,8 @@ "@vue/compiler-sfc/estree-walker": ["estree-walker@2.0.2", "", {}, "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w=="], + "@vue/compiler-sfc/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "accepts/mime-types": ["mime-types@3.0.2", "", { "dependencies": { "mime-db": "1.54.0" } }, "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A=="], "acorn-jsx/acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], @@ -2357,12 +2380,16 @@ "d3-dsv/commander": ["commander@7.2.0", "", {}, "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw=="], + "data-urls/whatwg-url": ["whatwg-url@16.0.1", "", { "dependencies": { "@exodus/bytes": "1.15.1", "tr46": "6.0.0", "webidl-conversions": "8.0.1" } }, "sha512-1to4zXBxmXHV3IiSSEInrreIlu02vUOvrhxJJH5vcxYTBDAx51cqZiKdyTxlecdKNSjj8EcxGBxNf6Vg+945gw=="], + "deep-equal/side-channel": ["side-channel@1.1.0", "", { "dependencies": { "es-errors": "1.3.0", "object-inspect": "1.13.4", "side-channel-list": "1.0.1", "side-channel-map": "1.0.1", "side-channel-weakmap": "1.0.2" } }, "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw=="], "dependency-tree/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="], "dependency-tree/typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], + "detective-postcss/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "esast-util-from-js/acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], "escodegen/estraverse": ["estraverse@5.3.0", "", {}, "sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA=="], @@ -2379,16 +2406,44 @@ "find-test-names/@babel/parser": ["@babel/parser@7.29.7", "", { "dependencies": { "@babel/types": "7.29.7" }, "bin": "./bin/babel-parser.js" }, "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg=="], + "hast-util-from-dom/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-from-html/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "hast-util-from-html/parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="], + "hast-util-from-html-isomorphic/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-from-parse5/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "hast-util-from-parse5/property-information": ["property-information@7.2.0", "", {}, "sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg=="], + "hast-util-is-element/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-parse-selector/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-to-estree/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-to-html/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "hast-util-to-html/property-information": ["property-information@7.2.0", "", {}, "sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg=="], + "hast-util-to-jsx-runtime/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-to-string/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-to-text/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hast-util-whitespace/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "hastscript/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "hastscript/property-information": ["property-information@7.2.0", "", {}, "sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg=="], "istanbul-lib-report/supports-color": ["supports-color@7.2.0", "", { "dependencies": { "has-flag": "4.0.0" } }, "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw=="], + "jest-worker/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + "katex/commander": ["commander@8.3.0", "", {}, "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww=="], "log-update/slice-ansi": ["slice-ansi@7.1.2", "", { "dependencies": { "ansi-styles": "6.2.3", "is-fullwidth-code-point": "5.1.0" } }, "sha512-iOBWFgUX7caIZiuutICxVgX1SdxwAVFFKwt1EvMYYec/NWO5meOJ6K5uQxhrYBdQJne4KxiqZc+KptFOWFSI9w=="], @@ -2399,22 +2454,38 @@ "magicast/@babel/types": ["@babel/types@7.29.7", "", { "dependencies": { "@babel/helper-string-parser": "7.29.7", "@babel/helper-validator-identifier": "7.29.7" } }, "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA=="], + "mdast-util-math/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "mdast-util-mdx-expression/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "mdast-util-mdx-jsx/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "mdast-util-mdxjs-esm/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "mdast-util-to-hast/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "micromark-extension-mdxjs/acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], "micromatch/picomatch": ["picomatch@2.3.2", "", {}, "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA=="], "module-lookup-amd/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="], + "next/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "next-mdx-remote/@babel/code-frame": ["@babel/code-frame@7.29.0", "", { "dependencies": { "@babel/helper-validator-identifier": "7.28.5", "js-tokens": "4.0.0", "picocolors": "1.1.1" } }, "sha512-9NhCeYjq9+3uxgdtp20LSiJXJvN0FeCtNGpJxuMFZ1Kv3cWUNb6DOhJwUvcVCzKGR66cw4njwM6hrJLqgOwbcw=="], "node-source-walk/@babel/parser": ["@babel/parser@7.29.7", "", { "dependencies": { "@babel/types": "7.29.7" }, "bin": "./bin/babel-parser.js" }, "sha512-hnORnjP/1P/zFEndoeX+n+t1RwWRJiJpM/jO7FW32Kn9r5+sJB2JWOdYo4L6k78j15eCwY3Gm/7364B1EMwtNg=="], "parse-entities/@types/unist": ["@types/unist@2.0.11", "", {}, "sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA=="], + "postcss-values-parser/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "posthog-js/fflate": ["fflate@0.4.8", "", {}, "sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA=="], "precinct/commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="], + "precinct/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + "precinct/typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], "raw-body/iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": "2.1.2" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="], @@ -2427,6 +2498,12 @@ "recma-jsx/acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], + "rehype-katex/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "rehype-recma/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + + "remark-rehype/@types/hast": ["@types/hast@3.0.4", "", { "dependencies": { "@types/unist": "3.0.3" } }, "sha512-WPs+bbQw5aCj+x6laNGWLH3wviHtoCv/P3+otBhbOhJgG8qtpdAMlTCxLtsTWA7LH1Oh/bFCHsBn0TPS5m30EQ=="], + "request-progress/throttleit": ["throttleit@1.0.1", "", {}, "sha512-vDZpf9Chs9mAdfY046mcPt8fg5QSZr37hEH4TXYBnDF+izxgrbRGUAAaBvIk/fJm9aOFCGFd1EsNg5AZCbnQCQ=="], "require-and-forget/debug": ["debug@4.3.4", "", { "dependencies": { "ms": "2.1.2" }, "optionalDependencies": { "supports-color": "8.1.1" } }, "sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ=="], @@ -2441,6 +2518,8 @@ "send/mime-types": ["mime-types@3.0.2", "", { "dependencies": { "mime-db": "1.54.0" } }, "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A=="], + "sharp/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + "shelljs/execa": ["execa@5.1.1", "", { "dependencies": { "cross-spawn": "7.0.6", "get-stream": "6.0.1", "human-signals": "2.1.0", "is-stream": "2.0.1", "merge-stream": "2.0.0", "npm-run-path": "4.0.1", "onetime": "5.1.2", "signal-exit": "3.0.7", "strip-final-newline": "2.0.0" } }, "sha512-8uSpZZocAZRBAPIEINJj3Lo9HyGitllczc27Eh5YYojjMFMn8yHMDMaUHE2Jqfq05D/wucwI4JGURyXt1vchyg=="], "slice-ansi/ansi-styles": ["ansi-styles@6.2.3", "", {}, "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg=="], @@ -2453,6 +2532,14 @@ "terser/commander": ["commander@2.20.3", "", {}, "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ=="], + "terser-webpack-plugin/esbuild": ["esbuild@0.28.1", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.1", "@esbuild/android-arm": "0.28.1", "@esbuild/android-arm64": "0.28.1", "@esbuild/android-x64": "0.28.1", "@esbuild/darwin-arm64": "0.28.1", "@esbuild/darwin-x64": "0.28.1", "@esbuild/freebsd-arm64": "0.28.1", "@esbuild/freebsd-x64": "0.28.1", "@esbuild/linux-arm": "0.28.1", "@esbuild/linux-arm64": "0.28.1", "@esbuild/linux-ia32": "0.28.1", "@esbuild/linux-loong64": "0.28.1", "@esbuild/linux-mips64el": "0.28.1", "@esbuild/linux-ppc64": "0.28.1", "@esbuild/linux-riscv64": "0.28.1", "@esbuild/linux-s390x": "0.28.1", "@esbuild/linux-x64": "0.28.1", "@esbuild/netbsd-arm64": "0.28.1", "@esbuild/netbsd-x64": "0.28.1", "@esbuild/openbsd-arm64": "0.28.1", "@esbuild/openbsd-x64": "0.28.1", "@esbuild/openharmony-arm64": "0.28.1", "@esbuild/sunos-x64": "0.28.1", "@esbuild/win32-arm64": "0.28.1", "@esbuild/win32-ia32": "0.28.1", "@esbuild/win32-x64": "0.28.1" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw=="], + + "terser-webpack-plugin/lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "2.1.2" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], + + "terser-webpack-plugin/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + + "tsx/esbuild": ["esbuild@0.28.1", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.1", "@esbuild/android-arm": "0.28.1", "@esbuild/android-arm64": "0.28.1", "@esbuild/android-x64": "0.28.1", "@esbuild/darwin-arm64": "0.28.1", "@esbuild/darwin-x64": "0.28.1", "@esbuild/freebsd-arm64": "0.28.1", "@esbuild/freebsd-x64": "0.28.1", "@esbuild/linux-arm": "0.28.1", "@esbuild/linux-arm64": "0.28.1", "@esbuild/linux-ia32": "0.28.1", "@esbuild/linux-loong64": "0.28.1", "@esbuild/linux-mips64el": "0.28.1", "@esbuild/linux-ppc64": "0.28.1", "@esbuild/linux-riscv64": "0.28.1", "@esbuild/linux-s390x": "0.28.1", "@esbuild/linux-x64": "0.28.1", "@esbuild/netbsd-arm64": "0.28.1", "@esbuild/netbsd-x64": "0.28.1", "@esbuild/openbsd-arm64": "0.28.1", "@esbuild/openbsd-x64": "0.28.1", "@esbuild/openharmony-arm64": "0.28.1", "@esbuild/sunos-x64": "0.28.1", "@esbuild/win32-arm64": "0.28.1", "@esbuild/win32-ia32": "0.28.1", "@esbuild/win32-x64": "0.28.1" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw=="], + "type-is/content-type": ["content-type@2.0.0", "", {}, "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ=="], "type-is/mime-types": ["mime-types@3.0.2", "", { "dependencies": { "mime-db": "1.54.0" } }, "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A=="], @@ -2461,6 +2548,8 @@ "use-sidecar/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + "vitest/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + "webpack/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="], "wrap-ansi/ansi-styles": ["ansi-styles@6.2.3", "", {}, "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg=="], @@ -2469,16 +2558,146 @@ "wrap-ansi/strip-ansi": ["strip-ansi@7.2.0", "", { "dependencies": { "ansi-regex": "6.2.2" } }, "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w=="], + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.1", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/android-arm": ["@esbuild/android-arm@0.28.1", "", { "os": "android", "cpu": "arm" }, "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.1", "", { "os": "android", "cpu": "arm64" }, "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/android-x64": ["@esbuild/android-x64@0.28.1", "", { "os": "android", "cpu": "x64" }, "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.1", "", { "os": "linux", "cpu": "arm" }, "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.1", "", { "os": "linux", "cpu": "ia32" }, "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.1", "", { "os": "linux", "cpu": "x64" }, "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.1", "", { "os": "none", "cpu": "x64" }, "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.1", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.1", "", { "os": "sunos", "cpu": "x64" }, "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg=="], + + "@bahmutov/cypress-esbuild-preprocessor/esbuild/@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.1", "", { "os": "win32", "cpu": "x64" }, "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A=="], + "@cypress/request/tough-cookie/tldts": ["tldts@6.1.86", "", { "dependencies": { "tldts-core": "6.1.86" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-WMi/OQ2axVTf/ykqCQgXiIct+mSQDFdH2fkwhPwgEwvJ1kSzZRiinb0zF2Xb8u4+OqPChmyI6MEu4EezNJz+FQ=="], - "@img/sharp-wasm32/@emnapi/runtime/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + "@napi-rs/wasm-runtime/@emnapi/runtime/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + + "@posthog/webpack-plugin/@posthog/core/@posthog/types": ["@posthog/types@1.404.1", "", {}, "sha512-i2Gei6ARfOSBeTN4s2yUP1p97s2UNI+1NWmtLjhnR/V6t3RFOfI1sBWKcJNWHjtoOCCWoAFU+PNPY6SgT2VtEQ=="], + + "@rolldown/binding-wasm32-wasi/@emnapi/runtime/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + + "@tailwindcss/node/lightningcss/lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], + + "@tailwindcss/node/lightningcss/lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.32.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ=="], + + "@tailwindcss/node/lightningcss/lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.32.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w=="], + + "@tailwindcss/node/lightningcss/lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.32.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig=="], + + "@tailwindcss/node/lightningcss/lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw=="], + + "@tailwindcss/node/lightningcss/lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ=="], + + "@tailwindcss/node/lightningcss/lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg=="], + + "@tailwindcss/node/lightningcss/lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA=="], + + "@tailwindcss/node/lightningcss/lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg=="], - "@posthog/webpack-plugin/@posthog/core/@posthog/types": ["@posthog/types@1.393.0", "", {}, "sha512-vzWeEJZ7ERQhFRoQYaP5jzN1JvIu46UJyHXsuv+dTGW2r3sMgREOhNxXLZjmFHwZ8/FOHQoyqqQmXTCXZSfMSg=="], + "@tailwindcss/node/lightningcss/lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.32.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw=="], + + "@tailwindcss/node/lightningcss/lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q=="], + + "@vercel/analytics/next/@next/env": ["@next/env@16.2.12", "", {}, "sha512-d0Z5Bc13Fa4nR8pFAKx2jay2yhJM16vlfHbTzYnUQAxlNb6B6lmn4hjt69lYNt4kRtyYP6gEM49lPRHNbIyneg=="], + + "@vercel/analytics/next/@next/swc-darwin-arm64": ["@next/swc-darwin-arm64@16.2.12", "", { "os": "darwin", "cpu": "arm64" }, "sha512-0W1R0teHWJrqKX0FH20IzzIWAOuGtBxPGuObrxy1lE8hQvCFj49KE8a3WUg0D7sq6rn6zkM4c7YGUnhudBS6oA=="], + + "@vercel/analytics/next/@next/swc-darwin-x64": ["@next/swc-darwin-x64@16.2.12", "", { "os": "darwin", "cpu": "x64" }, "sha512-Hy5Ls099+aFUmOLmIgPfLqNi6iCwhL3uQCssz5rWk+5Nkc6TUKCE83DY5BbNylfm3+mfwcSFnLRfrZDJhVxdtw=="], + + "@vercel/analytics/next/@next/swc-linux-arm64-gnu": ["@next/swc-linux-arm64-gnu@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-+YqU2h1cQkHsGfvjAsrSmst8UIFBibBGm5x3Xgel8NLMiDQtNOM4sM2GOEMvG5YiOBNeN/Ykk8cQC2S0Xrqljg=="], + + "@vercel/analytics/next/@next/swc-linux-arm64-musl": ["@next/swc-linux-arm64-musl@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-0qjhiYBaKAqF63LA1ZWAAnKTzFUguAaZiRa5etMLGGPj/B6uEVjtIZldIzFEp3wHlB0koK6aTzqPtSdplTCjoA=="], + + "@vercel/analytics/next/@next/swc-linux-x64-gnu": ["@next/swc-linux-x64-gnu@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-7A3q26W+h7gnA15uqBToNuDqBEFZZcqh0mW2mn4AJh/G5pdg2RVE3n4slzLEliASZFG3NmsbEzng/x2Sh09mBg=="], + + "@vercel/analytics/next/@next/swc-linux-x64-musl": ["@next/swc-linux-x64-musl@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-qSjL/uppm+cbh21s72Ss8gkiOhQ4dExWHNGOWy6eZV7STj5WsKehgxT61beSsOj+YYQuTplL376lOCdMQU5T8w=="], + + "@vercel/analytics/next/@next/swc-win32-arm64-msvc": ["@next/swc-win32-arm64-msvc@16.2.12", "", { "os": "win32", "cpu": "arm64" }, "sha512-X6hzsOUJac/e7AWSbn9gQ9nzHld1xWP5iyjHpYWvud8pufB679O1xg4JDyKr8Xd69Jvd+kM2Der6uftiZCmjYA=="], + + "@vercel/analytics/next/@next/swc-win32-x64-msvc": ["@next/swc-win32-x64-msvc@16.2.12", "", { "os": "win32", "cpu": "x64" }, "sha512-F6fakeHuFTLOPt0bslQJdf+xtT+WIP9DVn/m4y1w1mRnVPyh3D/cNvzlRkxM444xfm+IvvYNSOrKiA2CDJ0Uxw=="], + + "@vercel/analytics/next/@swc/helpers": ["@swc/helpers@0.5.15", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-JQ5TuMi45Owi4/BIMAJBoSQoOJu12oOk/gADqlcUL9JEdHB8vyjUSsxqeNXnmXHjYKMi2WcYtezGEEhqUI/E2g=="], + + "@vercel/analytics/next/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], "@vercel/cli-exec/execa/get-stream": ["get-stream@6.0.1", "", {}, "sha512-ts6Wi+2j3jQjqi70w5AlN8DFnkSwC+MqmxEzdEALB2qXZYV3X/b1CTfgPLGJNMeAWxdPfU8FO1ms3NUfaHCPYg=="], "@vercel/cli-exec/execa/human-signals": ["human-signals@2.1.0", "", {}, "sha512-B4FFZ6q/T2jhhksgkbEW3HBvWIfDW85snkQgawt07S7J5QXTk6BkNV+0yAeZrM5QpMAdYlocGoljn0sJ/WQkFw=="], + "@vercel/speed-insights/next/@next/env": ["@next/env@16.2.12", "", {}, "sha512-d0Z5Bc13Fa4nR8pFAKx2jay2yhJM16vlfHbTzYnUQAxlNb6B6lmn4hjt69lYNt4kRtyYP6gEM49lPRHNbIyneg=="], + + "@vercel/speed-insights/next/@next/swc-darwin-arm64": ["@next/swc-darwin-arm64@16.2.12", "", { "os": "darwin", "cpu": "arm64" }, "sha512-0W1R0teHWJrqKX0FH20IzzIWAOuGtBxPGuObrxy1lE8hQvCFj49KE8a3WUg0D7sq6rn6zkM4c7YGUnhudBS6oA=="], + + "@vercel/speed-insights/next/@next/swc-darwin-x64": ["@next/swc-darwin-x64@16.2.12", "", { "os": "darwin", "cpu": "x64" }, "sha512-Hy5Ls099+aFUmOLmIgPfLqNi6iCwhL3uQCssz5rWk+5Nkc6TUKCE83DY5BbNylfm3+mfwcSFnLRfrZDJhVxdtw=="], + + "@vercel/speed-insights/next/@next/swc-linux-arm64-gnu": ["@next/swc-linux-arm64-gnu@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-+YqU2h1cQkHsGfvjAsrSmst8UIFBibBGm5x3Xgel8NLMiDQtNOM4sM2GOEMvG5YiOBNeN/Ykk8cQC2S0Xrqljg=="], + + "@vercel/speed-insights/next/@next/swc-linux-arm64-musl": ["@next/swc-linux-arm64-musl@16.2.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-0qjhiYBaKAqF63LA1ZWAAnKTzFUguAaZiRa5etMLGGPj/B6uEVjtIZldIzFEp3wHlB0koK6aTzqPtSdplTCjoA=="], + + "@vercel/speed-insights/next/@next/swc-linux-x64-gnu": ["@next/swc-linux-x64-gnu@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-7A3q26W+h7gnA15uqBToNuDqBEFZZcqh0mW2mn4AJh/G5pdg2RVE3n4slzLEliASZFG3NmsbEzng/x2Sh09mBg=="], + + "@vercel/speed-insights/next/@next/swc-linux-x64-musl": ["@next/swc-linux-x64-musl@16.2.12", "", { "os": "linux", "cpu": "x64" }, "sha512-qSjL/uppm+cbh21s72Ss8gkiOhQ4dExWHNGOWy6eZV7STj5WsKehgxT61beSsOj+YYQuTplL376lOCdMQU5T8w=="], + + "@vercel/speed-insights/next/@next/swc-win32-arm64-msvc": ["@next/swc-win32-arm64-msvc@16.2.12", "", { "os": "win32", "cpu": "arm64" }, "sha512-X6hzsOUJac/e7AWSbn9gQ9nzHld1xWP5iyjHpYWvud8pufB679O1xg4JDyKr8Xd69Jvd+kM2Der6uftiZCmjYA=="], + + "@vercel/speed-insights/next/@next/swc-win32-x64-msvc": ["@next/swc-win32-x64-msvc@16.2.12", "", { "os": "win32", "cpu": "x64" }, "sha512-F6fakeHuFTLOPt0bslQJdf+xtT+WIP9DVn/m4y1w1mRnVPyh3D/cNvzlRkxM444xfm+IvvYNSOrKiA2CDJ0Uxw=="], + + "@vercel/speed-insights/next/@swc/helpers": ["@swc/helpers@0.5.15", "", { "dependencies": { "tslib": "2.8.1" } }, "sha512-JQ5TuMi45Owi4/BIMAJBoSQoOJu12oOk/gADqlcUL9JEdHB8vyjUSsxqeNXnmXHjYKMi2WcYtezGEEhqUI/E2g=="], + + "@vercel/speed-insights/next/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + + "@vitest/mocker/vite/@types/node": ["@types/node@26.1.1", "", { "dependencies": { "undici-types": "8.3.0" } }, "sha512-nxAkRSVkN1Y0JC1W8ky/fTfkGsMmcrRsbx+3XoZE+rMOX71kLYTV7fLXpqud1GpbpP5TuffXFqfX7fH2GgZREw=="], + + "@vitest/mocker/vite/esbuild": ["esbuild@0.28.1", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.28.1", "@esbuild/android-arm": "0.28.1", "@esbuild/android-arm64": "0.28.1", "@esbuild/android-x64": "0.28.1", "@esbuild/darwin-arm64": "0.28.1", "@esbuild/darwin-x64": "0.28.1", "@esbuild/freebsd-arm64": "0.28.1", "@esbuild/freebsd-x64": "0.28.1", "@esbuild/linux-arm": "0.28.1", "@esbuild/linux-arm64": "0.28.1", "@esbuild/linux-ia32": "0.28.1", "@esbuild/linux-loong64": "0.28.1", "@esbuild/linux-mips64el": "0.28.1", "@esbuild/linux-ppc64": "0.28.1", "@esbuild/linux-riscv64": "0.28.1", "@esbuild/linux-s390x": "0.28.1", "@esbuild/linux-x64": "0.28.1", "@esbuild/netbsd-arm64": "0.28.1", "@esbuild/netbsd-x64": "0.28.1", "@esbuild/openbsd-arm64": "0.28.1", "@esbuild/openbsd-x64": "0.28.1", "@esbuild/openharmony-arm64": "0.28.1", "@esbuild/sunos-x64": "0.28.1", "@esbuild/win32-arm64": "0.28.1", "@esbuild/win32-ia32": "0.28.1", "@esbuild/win32-x64": "0.28.1" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw=="], + + "@vitest/mocker/vite/lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "2.1.2" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], + + "@vitest/mocker/vite/postcss": ["postcss@8.5.23", "", { "dependencies": { "nanoid": "3.3.16", "picocolors": "1.1.1", "source-map-js": "1.2.1" } }, "sha512-g50586zr4bZmwFiTlflMu8E0bDTb5I5gertgwAKmsdUlTQIhZtunzUlD1WSzwcVWPoAVpsrA6vlfCD7oXvRwgg=="], + + "@vitest/mocker/vite/rolldown": ["rolldown@1.1.5", "", { "dependencies": { "@oxc-project/types": "0.139.0", "@rolldown/pluginutils": "1.0.1" }, "optionalDependencies": { "@rolldown/binding-android-arm64": "1.1.5", "@rolldown/binding-darwin-arm64": "1.1.5", "@rolldown/binding-darwin-x64": "1.1.5", "@rolldown/binding-freebsd-x64": "1.1.5", "@rolldown/binding-linux-arm-gnueabihf": "1.1.5", "@rolldown/binding-linux-arm64-gnu": "1.1.5", "@rolldown/binding-linux-arm64-musl": "1.1.5", "@rolldown/binding-linux-ppc64-gnu": "1.1.5", "@rolldown/binding-linux-s390x-gnu": "1.1.5", "@rolldown/binding-linux-x64-gnu": "1.1.5", "@rolldown/binding-linux-x64-musl": "1.1.5", "@rolldown/binding-openharmony-arm64": "1.1.5", "@rolldown/binding-wasm32-wasi": "1.1.5", "@rolldown/binding-win32-arm64-msvc": "1.1.5", "@rolldown/binding-win32-x64-msvc": "1.1.5" }, "bin": { "rolldown": "./bin/cli.mjs" } }, "sha512-t9z29cJjXf/vxQ8dyhCSpt6H6aSwHTk8cT5I3iy6SMXuFpk5mB6PL6XfC8PCwrPTx93udwKUm9HRteAlTGBLiA=="], + "@vue/compiler-core/@babel/parser/@babel/types": ["@babel/types@7.29.7", "", { "dependencies": { "@babel/helper-string-parser": "7.29.7", "@babel/helper-validator-identifier": "7.29.7" } }, "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA=="], "@vue/compiler-sfc/@babel/parser/@babel/types": ["@babel/types@7.29.7", "", { "dependencies": { "@babel/helper-string-parser": "7.29.7", "@babel/helper-validator-identifier": "7.29.7" } }, "sha512-4zBIxpPzowiZpusoFkyGVwakdRJUyuH5PxQ/PrqghfdFWWasvnCdPfQXHrenDai+gyLARulZjZowCOj6fjT4pA=="], @@ -2517,6 +2736,132 @@ "shelljs/execa/human-signals": ["human-signals@2.1.0", "", {}, "sha512-B4FFZ6q/T2jhhksgkbEW3HBvWIfDW85snkQgawt07S7J5QXTk6BkNV+0yAeZrM5QpMAdYlocGoljn0sJ/WQkFw=="], + "terser-webpack-plugin/esbuild/@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.1", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/android-arm": ["@esbuild/android-arm@0.28.1", "", { "os": "android", "cpu": "arm" }, "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.1", "", { "os": "android", "cpu": "arm64" }, "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg=="], + + "terser-webpack-plugin/esbuild/@esbuild/android-x64": ["@esbuild/android-x64@0.28.1", "", { "os": "android", "cpu": "x64" }, "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng=="], + + "terser-webpack-plugin/esbuild/@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q=="], + + "terser-webpack-plugin/esbuild/@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw=="], + + "terser-webpack-plugin/esbuild/@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.1", "", { "os": "linux", "cpu": "arm" }, "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.1", "", { "os": "linux", "cpu": "ia32" }, "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag=="], + + "terser-webpack-plugin/esbuild/@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.1", "", { "os": "linux", "cpu": "x64" }, "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA=="], + + "terser-webpack-plugin/esbuild/@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw=="], + + "terser-webpack-plugin/esbuild/@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.1", "", { "os": "none", "cpu": "x64" }, "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg=="], + + "terser-webpack-plugin/esbuild/@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.1", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q=="], + + "terser-webpack-plugin/esbuild/@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw=="], + + "terser-webpack-plugin/esbuild/@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg=="], + + "terser-webpack-plugin/esbuild/@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.1", "", { "os": "sunos", "cpu": "x64" }, "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ=="], + + "terser-webpack-plugin/esbuild/@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA=="], + + "terser-webpack-plugin/esbuild/@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg=="], + + "terser-webpack-plugin/esbuild/@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.1", "", { "os": "win32", "cpu": "x64" }, "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A=="], + + "terser-webpack-plugin/lightningcss/lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], + + "terser-webpack-plugin/lightningcss/lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.32.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ=="], + + "terser-webpack-plugin/lightningcss/lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.32.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w=="], + + "terser-webpack-plugin/lightningcss/lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.32.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig=="], + + "terser-webpack-plugin/lightningcss/lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw=="], + + "terser-webpack-plugin/lightningcss/lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ=="], + + "terser-webpack-plugin/lightningcss/lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg=="], + + "terser-webpack-plugin/lightningcss/lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA=="], + + "terser-webpack-plugin/lightningcss/lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg=="], + + "terser-webpack-plugin/lightningcss/lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.32.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw=="], + + "terser-webpack-plugin/lightningcss/lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q=="], + + "tsx/esbuild/@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.1", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ=="], + + "tsx/esbuild/@esbuild/android-arm": ["@esbuild/android-arm@0.28.1", "", { "os": "android", "cpu": "arm" }, "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ=="], + + "tsx/esbuild/@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.1", "", { "os": "android", "cpu": "arm64" }, "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg=="], + + "tsx/esbuild/@esbuild/android-x64": ["@esbuild/android-x64@0.28.1", "", { "os": "android", "cpu": "x64" }, "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng=="], + + "tsx/esbuild/@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q=="], + + "tsx/esbuild/@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ=="], + + "tsx/esbuild/@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw=="], + + "tsx/esbuild/@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ=="], + + "tsx/esbuild/@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.1", "", { "os": "linux", "cpu": "arm" }, "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ=="], + + "tsx/esbuild/@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g=="], + + "tsx/esbuild/@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.1", "", { "os": "linux", "cpu": "ia32" }, "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w=="], + + "tsx/esbuild/@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg=="], + + "tsx/esbuild/@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ=="], + + "tsx/esbuild/@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ=="], + + "tsx/esbuild/@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ=="], + + "tsx/esbuild/@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag=="], + + "tsx/esbuild/@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.1", "", { "os": "linux", "cpu": "x64" }, "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA=="], + + "tsx/esbuild/@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw=="], + + "tsx/esbuild/@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.1", "", { "os": "none", "cpu": "x64" }, "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg=="], + + "tsx/esbuild/@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.1", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q=="], + + "tsx/esbuild/@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw=="], + + "tsx/esbuild/@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg=="], + + "tsx/esbuild/@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.1", "", { "os": "sunos", "cpu": "x64" }, "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ=="], + + "tsx/esbuild/@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA=="], + + "tsx/esbuild/@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg=="], + + "tsx/esbuild/@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.1", "", { "os": "win32", "cpu": "x64" }, "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A=="], + "type-is/mime-types/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="], "wrap-ansi/string-width/emoji-regex": ["emoji-regex@10.6.0", "", {}, "sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A=="], @@ -2525,6 +2870,114 @@ "@cypress/request/tough-cookie/tldts/tldts-core": ["tldts-core@6.1.86", "", {}, "sha512-Je6p7pkk+KMzMv2XXKmAE3McmolOQFdxkKw0R8EYNr7sELW46JqnNeTX8ybPiQgvg1ymCoF8LXs5fzFaZvJPTA=="], + "@vercel/analytics/next/@swc/helpers/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + + "@vercel/speed-insights/next/@swc/helpers/tslib": ["tslib@2.8.1", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="], + + "@vitest/mocker/vite/esbuild/@esbuild/aix-ppc64": ["@esbuild/aix-ppc64@0.28.1", "", { "os": "aix", "cpu": "ppc64" }, "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/android-arm": ["@esbuild/android-arm@0.28.1", "", { "os": "android", "cpu": "arm" }, "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/android-arm64": ["@esbuild/android-arm64@0.28.1", "", { "os": "android", "cpu": "arm64" }, "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg=="], + + "@vitest/mocker/vite/esbuild/@esbuild/android-x64": ["@esbuild/android-x64@0.28.1", "", { "os": "android", "cpu": "x64" }, "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng=="], + + "@vitest/mocker/vite/esbuild/@esbuild/darwin-arm64": ["@esbuild/darwin-arm64@0.28.1", "", { "os": "darwin", "cpu": "arm64" }, "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q=="], + + "@vitest/mocker/vite/esbuild/@esbuild/darwin-x64": ["@esbuild/darwin-x64@0.28.1", "", { "os": "darwin", "cpu": "x64" }, "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/freebsd-arm64": ["@esbuild/freebsd-arm64@0.28.1", "", { "os": "freebsd", "cpu": "arm64" }, "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw=="], + + "@vitest/mocker/vite/esbuild/@esbuild/freebsd-x64": ["@esbuild/freebsd-x64@0.28.1", "", { "os": "freebsd", "cpu": "x64" }, "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-arm": ["@esbuild/linux-arm@0.28.1", "", { "os": "linux", "cpu": "arm" }, "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-arm64": ["@esbuild/linux-arm64@0.28.1", "", { "os": "linux", "cpu": "arm64" }, "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-ia32": ["@esbuild/linux-ia32@0.28.1", "", { "os": "linux", "cpu": "ia32" }, "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-loong64": ["@esbuild/linux-loong64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-mips64el": ["@esbuild/linux-mips64el@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-ppc64": ["@esbuild/linux-ppc64@0.28.1", "", { "os": "linux", "cpu": "ppc64" }, "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-riscv64": ["@esbuild/linux-riscv64@0.28.1", "", { "os": "linux", "cpu": "none" }, "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-s390x": ["@esbuild/linux-s390x@0.28.1", "", { "os": "linux", "cpu": "s390x" }, "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag=="], + + "@vitest/mocker/vite/esbuild/@esbuild/linux-x64": ["@esbuild/linux-x64@0.28.1", "", { "os": "linux", "cpu": "x64" }, "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA=="], + + "@vitest/mocker/vite/esbuild/@esbuild/netbsd-arm64": ["@esbuild/netbsd-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw=="], + + "@vitest/mocker/vite/esbuild/@esbuild/netbsd-x64": ["@esbuild/netbsd-x64@0.28.1", "", { "os": "none", "cpu": "x64" }, "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg=="], + + "@vitest/mocker/vite/esbuild/@esbuild/openbsd-arm64": ["@esbuild/openbsd-arm64@0.28.1", "", { "os": "openbsd", "cpu": "arm64" }, "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q=="], + + "@vitest/mocker/vite/esbuild/@esbuild/openbsd-x64": ["@esbuild/openbsd-x64@0.28.1", "", { "os": "openbsd", "cpu": "x64" }, "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw=="], + + "@vitest/mocker/vite/esbuild/@esbuild/openharmony-arm64": ["@esbuild/openharmony-arm64@0.28.1", "", { "os": "none", "cpu": "arm64" }, "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg=="], + + "@vitest/mocker/vite/esbuild/@esbuild/sunos-x64": ["@esbuild/sunos-x64@0.28.1", "", { "os": "sunos", "cpu": "x64" }, "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ=="], + + "@vitest/mocker/vite/esbuild/@esbuild/win32-arm64": ["@esbuild/win32-arm64@0.28.1", "", { "os": "win32", "cpu": "arm64" }, "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA=="], + + "@vitest/mocker/vite/esbuild/@esbuild/win32-ia32": ["@esbuild/win32-ia32@0.28.1", "", { "os": "win32", "cpu": "ia32" }, "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg=="], + + "@vitest/mocker/vite/esbuild/@esbuild/win32-x64": ["@esbuild/win32-x64@0.28.1", "", { "os": "win32", "cpu": "x64" }, "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-darwin-arm64": ["lightningcss-darwin-arm64@1.32.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-darwin-x64": ["lightningcss-darwin-x64@1.32.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-freebsd-x64": ["lightningcss-freebsd-x64@1.32.0", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-linux-arm-gnueabihf": ["lightningcss-linux-arm-gnueabihf@1.32.0", "", { "os": "linux", "cpu": "arm" }, "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-linux-arm64-gnu": ["lightningcss-linux-arm64-gnu@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-linux-arm64-musl": ["lightningcss-linux-arm64-musl@1.32.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-linux-x64-gnu": ["lightningcss-linux-x64-gnu@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-linux-x64-musl": ["lightningcss-linux-x64-musl@1.32.0", "", { "os": "linux", "cpu": "x64" }, "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-win32-arm64-msvc": ["lightningcss-win32-arm64-msvc@1.32.0", "", { "os": "win32", "cpu": "arm64" }, "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw=="], + + "@vitest/mocker/vite/lightningcss/lightningcss-win32-x64-msvc": ["lightningcss-win32-x64-msvc@1.32.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q=="], + + "@vitest/mocker/vite/rolldown/@oxc-project/types": ["@oxc-project/types@0.139.0", "", {}, "sha512-r9gHphtCs+1M7J0pw6Sn/hh/Wpa/iQrOOkrNAlVLF/gHq+/CJmHIWKKUUhdWjcD6CIa8idarspCsASiXCXvFUw=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-android-arm64": ["@rolldown/binding-android-arm64@1.1.5", "", { "os": "android", "cpu": "arm64" }, "sha512-lZg8fqIv2v7FF237bwMgzGZEJvGL79/s5knJ/i6FmsGF4XXlzccZ4jb+TrFIxtSSxFtIpdsgrPZeMk1I9AFcyQ=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-darwin-arm64": ["@rolldown/binding-darwin-arm64@1.1.5", "", { "os": "darwin", "cpu": "arm64" }, "sha512-51Bnx9pNiMRKSUNtBfySkNJ9vMU9Hh3I1ozDd6gyPPYzaXCfnptUcEZxXGYFn+ul2dtcMUiqGR1Yai2K10uoTw=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-darwin-x64": ["@rolldown/binding-darwin-x64@1.1.5", "", { "os": "darwin", "cpu": "x64" }, "sha512-Tm+gbfC0aHu1tBA/JvKQh32S0K6YgCHkiAF4/W6xX0K0RmNuc94VeK419dJoE65R5aRxmo+noZQSWrAMF6yb6g=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-freebsd-x64": ["@rolldown/binding-freebsd-x64@1.1.5", "", { "os": "freebsd", "cpu": "x64" }, "sha512-JMzDKCCXq93YccG5gz3hvOs1oXRKAf0XYpfOS88e+wZrC8Iugj6j68867vrYZkvpDDpKn/KoKORThmchMpF6TA=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-arm-gnueabihf": ["@rolldown/binding-linux-arm-gnueabihf@1.1.5", "", { "os": "linux", "cpu": "arm" }, "sha512-uML21j2K5TfPGutKxub+M+nLjZIrWjXQ5Grx4lCe/nimTj9B4L63zHpjXLl4y0L3mcm2htEQIb06oCG/szerNw=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-arm64-gnu": ["@rolldown/binding-linux-arm64-gnu@1.1.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-navSiuTMogvnQoZoM/v+l3ZWo50/NTwSHSzheABx/RCnmUPaKwq9qSo4Br2OYRs21+Fz8uFqITZM3H4opOB0/Q=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-arm64-musl": ["@rolldown/binding-linux-arm64-musl@1.1.5", "", { "os": "linux", "cpu": "arm64" }, "sha512-lAryqH7IteztmCXQXk0etKj4wBQ7Gx5S6LjKhsgp9zb8I5bsuvU/2llH1hDQcjsFeqIsovMVN339/8pUDDBXxA=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-ppc64-gnu": ["@rolldown/binding-linux-ppc64-gnu@1.1.5", "", { "os": "linux", "cpu": "ppc64" }, "sha512-fsK/sNBnxzBlL4O1JNrZakVQxPspqpED5dLtNsZS9oOKmtSpdNIzxH2kkol5HYTWJN47sE20ztMJPxfZ89qGOg=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-s390x-gnu": ["@rolldown/binding-linux-s390x-gnu@1.1.5", "", { "os": "linux", "cpu": "s390x" }, "sha512-gLYb4BIadlfTOYT5gO503n8zQjXflgzpD0FcyKh0Mzx3rqCZKnHoJWV9xe1KXUJ5lx2JfcSHr/mhzS0PC/McAA=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-x64-gnu": ["@rolldown/binding-linux-x64-gnu@1.1.5", "", { "os": "linux", "cpu": "x64" }, "sha512-FjcpEKUyJygHgs1o50VYNvkt5+7Le/VEdYt0AkRpkL33MnyQfwr8l5mXwMmfmTbyMPr5vJLC+8/Gd9gXnwU1QQ=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-linux-x64-musl": ["@rolldown/binding-linux-x64-musl@1.1.5", "", { "os": "linux", "cpu": "x64" }, "sha512-Me+PfPI2TMeOQk0gYWfLQZtTktrmzbr8cDboqX83XKc7UrgAi55gF+2dUkWdxd19n55Essp2yeca+O9N5rBxHg=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-openharmony-arm64": ["@rolldown/binding-openharmony-arm64@1.1.5", "", { "os": "none", "cpu": "arm64" }, "sha512-yc5WrLzXks6zCQfn9Oxr8pORKyl/pF+QjHmW/Qx3qu0oyrrNC+y2JLTU1E2rcWYAmzlnqngWXHQjy51VzW70Vw=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-win32-arm64-msvc": ["@rolldown/binding-win32-arm64-msvc@1.1.5", "", { "os": "win32", "cpu": "arm64" }, "sha512-gHv82k63z4qpV5+Q1y/12KrK0ltWBukVDI8nZcbT7Tt/ZlOIVwppazneq0F93oDxTo3IgAMEDIoQh3E2n6mVsw=="], + + "@vitest/mocker/vite/rolldown/@rolldown/binding-win32-x64-msvc": ["@rolldown/binding-win32-x64-msvc@1.1.5", "", { "os": "win32", "cpu": "x64" }, "sha512-tTZuDBPw85tEN5PQi1pnEBzDy0Z49HtScLAbD5t6hyeU92A95pRWaSMw1GZZi/RwgSgUIl0xrSlXIT/9QzvYSA=="], + "@vue/compiler-core/@babel/parser/@babel/types/@babel/helper-string-parser": ["@babel/helper-string-parser@7.29.7", "", {}, "sha512-Pb5ijPrZ89GDH8223L4UP8i6QApWxs04RbPQJTeWDV0/keR2E36MeKnyr6LYmUUvqRRI+Iv87SuF1W6ErINzYw=="], "@vue/compiler-core/@babel/parser/@babel/types/@babel/helper-validator-identifier": ["@babel/helper-validator-identifier@7.29.7", "", {}, "sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg=="], diff --git a/docs/architecture.md b/docs/architecture.md index bcf23b021..0b34558f1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -14,6 +14,7 @@ Some routes serve a purpose that raw DB rows cannot satisfy: - CollectiveX assembles stored sweep documents through its shared reader. - `tco-feed` performs server-side frontier interpolation for spreadsheet consumers that cannot run the TypeScript transforms. +- `server-log-search` searches immutable log bundles in bounded database slices and returns only contextual matches. Returning raw rows would require transferring complete files that can exceed 100 MB before the browser could search them. - `/api/v1/overview` is a page-owned backend-for-frontend (BFF). It returns the compact `OverviewPageData` shape used by the initial server render, allowing selector changes to update the matrix without downloading every model's raw benchmark history or triggering a React Server Component (RSC) round trip. It is not a reusable public data API. - `/api/v1/benchmarks?view=calculator&sequence=...` is a page-owned compact view. It returns only the selected calculator scenario and the metric keys needed for interpolation, while the default endpoint preserves the raw-row contract. This reduces transfer and browser parsing without narrowing the reusable inference response. @@ -24,13 +25,30 @@ Two rules keep that BFF honest, both enforced in `overview-navigation.tsx`: Because the selector commit uses `History.prototype.pushState` rather than Next's patched version, `useSearchParams()` and `usePathname()` stay at the load-time URL on `/overview`. Anything that needs the live URL there listens for `CLIENT_SEARCH_CHANGE_EVENT`; the provider emits its own `$pageview`. Do not add a `useSearchParams()` consumer to the overview tree. -## Hash-Based Tab Routing (Not Next.js Routes) +## Route-Based Dashboard Navigation -Tabs use `window.location.hash` instead of Next.js file-based routing because: +Every dashboard surface is a real Next.js route. The data-only registry in +`packages/app/src/lib/dashboard-routes.ts` owns canonical paths, navigation groups, +indexability, provider capabilities, locale mirroring, and share-parameter scopes. +`TabNav` renders links from that registry, so browser back/forward uses ordinary +navigation and a newly added route cannot silently disappear from the sitemap or +Chinese route map. -- The entire app is a single dashboard page. Separate routes would mean separate page loads, losing React state (zoom positions, filter selections, legend toggles). -- Hash changes don't trigger Next.js navigation, so context providers stay mounted. This is critical — rebuilding D3 charts from scratch on tab switch would cause visible jank. -- Browser back/forward still works (hashchange event listener updates tab state). +Chart state survives navigation through the URL-state contract rather than a +long-lived single-page tab tree. Each route mounts only the providers it consumes. + +### Per-Model Tab Routes + +`/calculator/` and `/historical/` (plus `/zh` siblings) give every +dashboard model an indexable URL. `packages/app/src/lib/model-routes.ts` derives +the slugs from the compare-page model registry (one slug vocabulary site-wide) +and stays a thin layer on top of `dashboard-routes.ts`, whose prefix matching +already resolves the child paths — it is not a second route registry. +`GlobalFilterProvider` seeds the model from the pathname; switching models on +the page rewrites the pathname with the pristine `History.prototype.replaceState` +(same trick as `replaceClientSearch`), so the address bar tracks the model +without an App Router navigation or remount. The bare tab paths remain the +canonical home of the default model; alias slugs 308-redirect like `/compare`. ## URL State Persistence @@ -38,23 +56,28 @@ Chart filter state (model, sequence, metric, precisions, date range, GPU selecti **Why debounced writes (150ms)?** Rapid filter changes (e.g., clicking multiple precision checkboxes) would spam `history.pushState`. Debouncing batches them into a single URL update. -**Why snapshot-and-clear on load?** Initial params are read into React state, then stripped from the URL via `history.replaceState`. This prevents stale params from accumulating across navigation — the URL always reflects current state, written back by the debounced sync. +**Why snapshot-and-clear on load?** Initial params are read into the URL-state snapshot, then stripped from the visible address bar with `history.replaceState` while preserving the pathname and hash. Route-owned entry points can also pass typed server seeds. Subsequent debounced writes update the in-memory share state. + +**Prefix convention**: `g_` (global), `i_` (inference), `e_` (evaluation), `r_` (reliability), and `c_` (calculator). The canonical dashboard route registry declares which scopes belong in each share URL. -**Prefix convention**: `g_` (global), `i_` (inference), `e_` (evaluation), `r_` (reliability). Prevents namespace collisions and allows `buildShareUrl()` to include only tab-relevant params. +## Provider Ownership -## Provider Nesting Order +`DashboardShell` reads provider capabilities from the route registry: ``` -QueryProvider → ThemeProvider → UnofficialRunProvider → GlobalStateProvider - → GlobalFilterProvider → InferenceProvider → EvaluationProvider → ReliabilityProvider +QueryProvider → ThemeProvider → DashboardShell + filtered routes: UnofficialRunProvider → GlobalFilterProvider + inference / historical: InferenceProvider + evaluation: EvaluationProvider + calculator: UnofficialRunProvider → route-owned seeded GlobalFilterProvider + reliability: route-owned ReliabilityProvider + static / internal routes: no data providers ``` -This isn't arbitrary. Each provider depends on the one above it: - -- `GlobalFilterProvider` needs React Query (`useAvailability()`, `useWorkflowInfo()`) -- `InferenceProvider` needs global model/date selection; gated by `activeTab` to skip heavy work on non-inference tabs -- Evaluation and Reliability need the hardware config from Inference context -- TCO Calculator and Historical Trends reuse InferenceContext state (sequence, precisions) without their own providers — local `useState` is sufficient since they don't share state with other tabs +This keeps availability, workflow, and unofficial-run requests off pages that do not +consume them. Compare routes sit outside `DashboardShell` and own their seeded +`GlobalFilterProvider` / `InferenceProvider` pair. Agentic point-detail routes are also +standalone because they fetch point-owned data rather than dashboard filters. ## Client-Side Caching (React Query — In-Memory Only) @@ -71,7 +94,9 @@ API route responses are cached at two layers before hitting the CDN. | Local (unstable_cache) | Next.js in-process | ~2 MB default | Small payloads (availability, workflow-info, etc.) | | Blob storage | Vercel Blob | No practical limit | Large payloads that exceed the 2 MB threshold | -`cachedQuery()` in `src/lib/api-cache.ts` wraps both tiers. Pass `{ blobOnly: true }` for payloads known to be large (e.g. `/api/v1/benchmarks`, which returns full benchmark rows for a model). The blob path is `{BLOB_CACHE_PREFIX}/{keyPrefix}:{args}.json`. +`cachedQuery()` in `src/lib/api-cache.ts` wraps both tiers. Pass `{ blobOnly: true }` for payloads known to be large (e.g. `/api/v1/benchmarks`, which returns full benchmark rows for a model). Blob keys encode canonical typed arguments as `{keyPrefix}:v2:{base64url}`; keys that would exceed the pathname limit use `{keyPrefix}:v2:sha256:{digest}` instead. `blob-cache.ts` stores that key beneath `BLOB_CACHE_PREFIX` with a `.json` suffix. + +`cachedDerivedData()` is the tagged, `unstable_cache`-only wrapper for compact server payloads. Overview caches its fully assembled selector response. Compare caches one pair-filtered, full-shape hydration row set per model/GPU pair, then recomputes the inexpensive sequence/precision table transform per request; nesting a selector cache around the pair cache would bypass Next.js incremental caching. Raw public benchmark responses keep their existing Blob-backed contract. `blobSet()` is no-op if the key already exists, making concurrent lambda invocations race-safe — only the first writer wins, subsequent calls skip silently. @@ -79,7 +104,7 @@ API route responses are cached at two layers before hitting the CDN. `unstable_cache` entries are tagged `'db'`. Calling `revalidateTag('db', { expire: 0 })` evicts all local cache entries in one call. Blob storage has no built-in tag system, so `blobPurge()` walks the paginated blob list and deletes every object under the prefix. -Both are called together by `purgeAll()`, which also writes a new `cache-version` timestamp to blob storage. +`purgeAll()` calls both mechanisms and also invalidates the separate CollectiveX tag. Blob keys are versioned and encoded deterministically so structured arguments cannot collide. ### CDN Cache Headers @@ -94,7 +119,7 @@ Vercel-Cache-Tag: db ### /api/v1/invalidate -`POST /api/v1/invalidate` requires a `Bearer {INVALIDATE_SECRET}` header (compared with `timingSafeEqual` to prevent timing attacks). On success it calls `purgeAll()` — which clears blob storage, bumps the cache-version timestamp, and revalidates the `'db'` tag — then returns `{ invalidated: true, blobsDeleted: N }`. This endpoint is called by the CI ingest pipelines after benchmark runs and by the run-override pipeline after it applies and verifies merged override changes. +`POST /api/v1/invalidate` requires a `Bearer {INVALIDATE_SECRET}` header, checked through the shared byte-safe constant-time bearer helper. On success it calls `purgeAll()`, which clears blob storage and revalidates the database and CollectiveX tags, then returns `{ invalidated: true, blobsDeleted: N }`. This endpoint is called by the CI ingest pipelines after benchmark runs and by the run-override pipeline after it applies and verifies merged override changes. ## React Query Configuration diff --git a/docs/chinese-copy.md b/docs/chinese-copy.md new file mode 100644 index 000000000..0e18c3331 --- /dev/null +++ b/docs/chinese-copy.md @@ -0,0 +1,192 @@ +# Chinese Copy Editorial Guide / 中文文案编辑规范 + +This guide is the editorial source of truth for user-visible Simplified Chinese in +InferenceX. It complements the route architecture in [i18n.md](./i18n.md): that document +explains where Chinese pages live; this one defines how their copy is written and reviewed. + +本文是 InferenceX 简体中文用户文案的编辑规范。它与 [i18n.md](./i18n.md) 分工: +后者说明中文页面的技术架构,本文规定中文文案的写法与审核方式。 + +## Audience and baseline tone / 读者与基本语气 + +Write for Chinese-speaking ML infrastructure engineers who already understand models, +hardware, serving frameworks, and performance metrics. The copy should be precise, +restrained, and easy to read without consulting the English source. + +目标读者是熟悉模型、硬件、推理框架和性能指标的中文 ML 基础设施工程师。文案应当 +准确、克制;即使不对照英文,也能独立读懂。 + +- Lead with the user action, result, or claim. Do not reproduce English modifier chains. +- Prefer ordinary technical Chinese over slogans, literal metaphors, and stacked nouns. +- Preserve facts, qualifications, modality, numbers, units, product names, and attribution. +- Do not add confidence, causality, or endorsement that is absent from the English source. +- Keep English when Chinese practitioners normally do: product and model names, hardware SKUs, + framework names, flags, identifiers, units, and established English technical terms or phrases. + Decide from real industry usage and the surface, not from a closed list. + +- 先说用户动作、结果或核心判断,不照搬英文的前置修饰链。 +- 使用工程师日常采用的技术中文,避免口号、直译隐喻和名词堆叠。 +- 保留事实、限定条件、语气强弱、数字、单位、产品名和署名。 +- 英文原文没有表达的确定性、因果关系或背书,不得自行添加。 +- 中文技术社区通常直接使用的英文应保留,包括产品与模型名称、硬件 SKU、框架名、flag、 + 标识符、单位,以及行业中已习惯使用英文的技术术语和短语。应结合真实行业用法和界面场景 + 判断,不能依赖封闭词表。 + +## Register by surface / 不同界面的语域 + +| Surface | Editorial register | 中文要求 | +| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | +| Buttons, navigation, and short labels | Brief and action-led. Omit subjects that the UI already makes obvious. | 简短、以动作或对象为中心;界面已经明确的主语不重复。 | +| Tooltips and dashboard explanations | Explain what the metric or control means and what changes when it is used. Prefer one to three direct sentences. | 说明指标或控件的含义,以及操作后会发生什么;通常用一至三句直述。 | +| Empty, loading, and error states | State the current condition, then the next useful action when one exists. Do not blame the user. | 先说明当前状态,再给出可执行的下一步;不要把问题归咎于用户。 | +| Metadata | Standalone, factual, and search-readable. Do not stuff keywords or repeat the title mechanically. | 脱离页面正文也能读懂;以事实为主,不堆关键词,也不机械重复标题。 | +| Technical prose | Preserve definitions and causal structure, but reorganize clauses for Chinese readability. Define an uncommon acronym on first use. | 保留定义和因果关系,但按中文逻辑重组分句;不常见缩写首次出现时补充说明。 | +| Landing and marketing copy | Evidence-led and restrained. Concrete capabilities and provenance are stronger than superlatives. | 以事实和证据为主,克制表达;具体能力和数据来源比夸张形容词更可信。 | +| Attributed quotations | Preserve the speaker's claim, degree of certainty, and voice. Do not polish a quotation into house marketing copy. | 保留说话者的观点、语气强弱和个人表达,不得改写成官方营销文案。 | + +Second-person wording is contextual, not a mechanical ban. Controls and tooltips normally omit +the pronoun. A deliberate `您` is acceptable in a direct, respectful call to action when the +surrounding surface uses that register. Do not replace every `您` with `你`, and do not reject an +otherwise natural sentence solely because it contains `您`. + +第二人称必须结合场景判断,不能全局禁用。控件和提示框通常省略第二人称;直接面向读者、 +且有意采用礼貌语气的 CTA 可以使用 `您`。不要把所有 `您` 机械替换为 `你`,也不要仅因 +一句话包含 `您` 就判定文案不合格。 + +## Rewrite method / 改写方法 + +For each changed UI element or paragraph: + +1. Read the complete English source and the surrounding rendered UI or paragraph. +2. Write one plain-language sentence stating the intended meaning. Record facts, scope, + modality, metrics, links, and attribution that cannot change. +3. Hide the old Chinese translation and rewrite from the intended meaning. +4. Compare the rewrite with the source for semantic fidelity. +5. Read the Chinese alone for naturalness, register, and information order. +6. Verify the desktop and mobile rendering when width, hierarchy, or interaction affects copy. + +逐个 UI 元素或段落执行: + +1. 阅读完整英文原文,并查看对应的页面或段落上下文。 +2. 用一句直白的中文写出原意,并记录不得改变的事实、范围、语气强弱、指标、链接和署名。 +3. 隐藏旧译文,根据原意重新写,不在旧句上逐词替换。 +4. 回到英文原文,检查语义是否完整、准确。 +5. 脱离英文单独阅读中文,检查自然度、语域和信息顺序。 +6. 如果宽度、层级或交互会影响文案,在桌面端和移动端实际页面中检查。 + +Semantic fidelity and natural Chinese are independent gates. A fluent mistranslation fails the +first gate; an accurate but translation-shaped sentence fails the second. Both must pass. + +语义准确和中文自然度是两道独立门槛。中文流畅但翻错原意,第一关不通过;语义准确但仍是 +英文句式,第二关不通过。两关都通过才算完成。 + +## Context-aware terminology / 上下文相关的术语表 + +The table records defaults, not blind substitutions. When an exception is needed, document the +surface and reason in the PR or add a reviewed example here. + +Established English technical terms and phrases that Chinese ML infrastructure engineers normally +use in English should not be translated merely to increase Chinese-language coverage. This is an +open, context-dependent category: `warmup`, `seed`, and `offload` are examples, not its boundaries. + +下表记录默认写法,不是全局替换表。需要例外时,应在 PR 中说明使用场景与理由,或将经过 +审核的案例补充到本文。 + +中文 ML 基础设施工程师在行业实践中通常直接使用的英文技术术语和短语,不应为了提高中文 +占比而强行翻译。这是一个需要结合上下文判断的开放类别;`warmup`、`seed`、`offload` 只是 +示例,并非完整清单。 + +| English | Preferred rendering | Context and exceptions | +| ---------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| benchmark | 基准测试 | Use for the process or result set. A short UI label may use “测试” only when the surrounding heading already establishes that it is a benchmark. / 表示测试过程或结果集;只有上下文已经明确是基准测试时,短标签才可简写为“测试”。 | +| dashboard | 仪表板 | Use for the interactive data surface. Do not use “完整仪表板” merely to mirror “full”; write “查看完整仪表板” when it is an action. / 表示交互式数据页面;按钮需要表达动作时写“查看完整仪表板”。 | +| chart | 图表 | Use “图” only when naming a specific conventional form such as a scatter plot. / 特指散点图等固定图形时可用“图”。 | +| configuration / config | 配置 | “测试配置” is preferred when the surrounding text could otherwise refer to deployment configuration. / 容易与部署配置混淆时写“测试配置”。 | +| inference | 推理 | Keep inside established product names and identifiers. / 产品名和标识符中的英文保持不变。 | +| throughput | 吞吐量 | Preserve the token type, denominator, and scope. / 必须保留 token 类型、分母和统计范围。 | +| latency | 延迟 | Preserve percentile and stage qualifiers such as TTFT or TPOT. / 必须保留分位数以及 TTFT、TPOT 等阶段限定。 | +| evaluation / eval | 评估 | Keep `eval` in code identifiers, flags, and filenames. / 代码标识符、flag 和文件名中的 `eval` 不翻译。 | +| artifact | 产物 | Use for CI or benchmark outputs. “文件” is acceptable only when the object is literally a user-facing file. / 表示 CI 或基准测试输出;确实指用户下载的文件时可写“文件”。 | +| warmup | `warmup` | Keep English in first-party ML infrastructure UI and technical prose. Chinese-first `预热(warmup)` is acceptable when introducing the concept to a broader audience. Never write the redundant `warmup 预热`. / 第一方 ML 基础设施 UI 与技术文案默认保留英文;面向更广泛读者首次解释概念时可写 `预热(warmup)`,不得使用重复表达 `warmup 预热`。 | +| seed | `seed` | Keep English in technical copy and preserve flags such as `--random-seed`. Chinese-first `随机种子(seed)` is acceptable in explanatory copy. / 技术文案默认保留英文,`--random-seed` 等 flag 保持不变;解释性文案可写 `随机种子(seed)`。 | +| offload | `offload` | Keep English in first-party ML infrastructure UI and technical prose. Chinese-first `卸载(offload)` is acceptable in explanatory copy; attributed quotations retain the speaker's wording. / 第一方 ML 基础设施 UI 与技术文案默认保留英文;解释性文案可写 `卸载(offload)`,署名引用保留说话者原有措辞。 | +| chip | 芯片 | Keep English inside units and identifiers such as `tok/s/chip`, `$/chip-hour`, and `ChipSKU`. / `tok/s/chip`、`$/chip-hour`、`ChipSKU` 等单位和标识符保持英文。 | +| framework | 框架 | Keep names such as vLLM, SGLang, and TensorRT-LLM unchanged. / vLLM、SGLang、TensorRT-LLM 等名称不翻译。 | +| agent / agentic | 智能体 / 智能体相关 | Keep `AgentX` as a product name. Translate according to the role, not by replacing every occurrence mechanically. / `AgentX` 作为产品名保持不变;其他场景根据实际角色翻译,不做全局替换。 | +| workload | 工作负载 | Avoid using “场景” when the source refers to the actual request distribution or task mix. / 原文指请求分布或任务组合时,不要弱化成“场景”。 | +| disaggregated serving | 分离式推理 | Use “分离式部署” when the sentence is about architecture or deployment rather than the serving process. / 强调架构或部署方式时可写“分离式部署”。 | +| speculative decoding | 投机解码 | Do not rewrite the ordinary verb “推测”. / 不影响作为普通动词使用的“推测”。 | +| recipe | 测试配置 / 配置方案 | Choose from context; do not leave `recipe` in user-facing prose merely because it appears in code. / 根据上下文选择,不因代码中使用 `recipe` 就在用户文案中保留英文。 | +| methodology | 测试方法 / 方法说明 | Use “方法论” only when the content actually discusses a methodology as a field or system of principles. / 只有内容确实讨论一套方法论或原则体系时才使用“方法论”。 | +| industry testimonials | 业界评价 | Use for navigation and section labels. Preserve the speaker's own wording inside each attributed quotation. / 用于导航和栏目名称;每条署名引用仍需保留说话者自己的表达。 | +| Land Acknowledgement | 原住民传统领地声明 | Use for the dedicated legal-cultural page and its link. Preserve community, nation, and place names exactly. / 用于对应的法律与文化说明页面及其链接;原住民族群、社群和地名必须准确保留。 | + +## Review workflow / 审核流程 + +1. The author supplies the English source, intended meaning, and rendered context for every + changed Chinese passage. +2. Claude loads `review-zh-copy`, evaluates semantic fidelity and natural Chinese separately, + and reports complete suggested replacements rather than isolated word swaps. +3. The skill's findings never block another contributor's merge. Claude must not label a + Chinese-copy finding `BLOCKING` or submit a request-changes review for it. Clean reviews, + routine coverage, and ordinary wording or naturalness suggestions do not mention the Chinese + maintainer. Only a high-confidence semantic or factual error, changed attribution or speaker + voice, or unresolved high-impact ambiguity is labeled `Needs Chinese maintainer confirmation` + and mentions `@edwingao28`. +4. The Chinese-speaking maintainer manually reads every changed sentence and makes the final + editorial decision. Attributed quotations receive an explicit attribution-and-voice check. + +中文步骤: + +1. 作者为每段改动的中文提供英文原文、核心意思和实际页面上下文。 +2. Claude 加载 `review-zh-copy`,分别检查语义准确度和中文自然度,并给出完整改写, + 而不是只替换孤立词语。 +3. skill 的 finding 不阻断其他贡献者合并。Claude 不得把中文文案 finding 标为 + `BLOCKING`,也不得因此提交 request-changes review。审核通过、例行 coverage 和普通措辞 + 或自然度建议都不提及中文维护者。只有高置信度的语义或事实错误、署名或说话者语气被 + 改变,或无法消除且影响较大的歧义,才标为 `Needs Chinese maintainer confirmation`,并 + `@edwingao28` 请中文维护者审核。 +4. 中文维护者逐句人工审阅并作出最终编辑决定;署名引用还要单独核对署名、观点和说话者 + 语气。 + +## Pull request checklist / PR 审核清单 + +- [ ] Every changed Chinese passage has an identified English source and intended meaning. + 中文:每段改动的中文都有对应英文来源和明确原意。 +- [ ] Facts, numbers, units, identifiers, links, modality, and attribution are unchanged. + 中文:事实、数字、单位、标识符、链接、语气强弱和署名均未改变。 +- [ ] The Chinese reads naturally without preserving English word order or noun stacking. + 中文:中文脱离英文后仍自然通顺,没有照搬英文语序或名词堆叠。 +- [ ] The register matches the surface. + 中文:文案语域与所在界面一致。 +- [ ] English UI strings remain byte-identical in a Chinese-only rewrite. + 中文:仅修改中文时,英文界面字符串保持逐字节不变。 +- [ ] The complete UI element or paragraph was reviewed, not only the changed word. + 中文:审核对象是完整 UI 元素或段落,而不是孤立的替换词。 +- [ ] Visible copy was checked in rendered desktop and mobile context. + 中文:用户可见文案已在桌面端和移动端实际页面中检查。 +- [ ] The Chinese-speaking maintainer manually reviewed every changed sentence. + 中文:中文维护者已逐句人工审核所有改动。 + +## Relationship to automation / 与自动化检查的关系 + +Deterministic CI may enforce only context-independent rules such as punctuation, protected +identifiers, dictionary completeness, and terminology with a genuinely universal rendering. +It must not decide fluency, sentence structure, register, or contextual pronoun choice. + +确定性 CI 只适合检查与上下文无关的规则,例如标点、受保护标识符、字典完整性,以及确实 +只有一种正确写法的术语。流畅度、句法、语域和第二人称选择不得交给机械规则判定。 + +The `review-zh-copy` skill assists the judgment layer and stays non-blocking. It mentions +`@edwingao28` only for high-confidence semantic or factual risk, attribution risk, or unresolved +high-impact ambiguity; ordinary findings do not CC the Chinese maintainer. CI does not globally ban +`预热`, `随机种子`, or `卸载`, because explanations and quotations can use them correctly. It may +reject context-independent duplication such as `warmup 预热`. Separately verified mechanical cases +from #819 may enter deterministic CI fixtures in #820, but editorial rewrites are never automatic +ground truth. + +`review-zh-copy` skill 用于辅助判断性审核,并保持非阻断模式。只有高置信度的语义或事实 +风险、署名风险,或无法消除且影响较大的歧义才 `@edwingao28`;普通 finding 不 CC 中文 +维护者。CI 不全局禁用 `预热`、`随机种子` 或 `卸载`,因为这些写法在解释与引用中可能 +正确;`warmup 预热` 这类与上下文无关的重复表达则可由 CI 拦截。#819 中经过单独验证的 +机械案例可以进入 #820 的确定性 CI fixtures,但编辑性改写不得自动视为标准答案。 diff --git a/docs/collectivex.md b/docs/collectivex.md index 973694f49..8ac376e50 100644 --- a/docs/collectivex.md +++ b/docs/collectivex.md @@ -44,6 +44,11 @@ Key invariants: `ON CONFLICT (run_id) DO NOTHING`; concurrent first-viewers can't double-ingest or expose a partial run. A GitHub re-run (newer `run_attempt`) is replaced through a `FOR UPDATE`-guarded refresh statement. +- **Reads after lazy persistence use the primary**: discovery state checks, writes, and + each route's post-discovery query use `DATABASE_COLLECTIVEX_WRITE_URL`. This guarantees + that a newly inserted or refreshed run is visible in the same request even when + `DATABASE_COLLECTIVEX_READONLY_URL` points at a lagging replica. Ordinary read-only + consumers can still use the read URL. - **Deletion tombstones** (`cx_runs.deleted_at`, documents freed): discovery must never resurrect a deleted run. Re-ingesting via the CLI (`bun run admin:db:ingest:collectivex `) clears the tombstone — that CLI is @@ -65,11 +70,12 @@ Key invariants: `s-maxage` (freshness bound for lazy discovery). Run deletion and `POST /api/v1/invalidate?scope=collectivex` purge only that tag; the main dashboard's blob cache is untouched by CollectiveX operations. -- **Env**: `DATABASE_COLLECTIVEX_READONLY_URL` (must be the same primary as the write URL - — the routes read their own writes), `DATABASE_COLLECTIVEX_WRITE_URL` (direct/unpooled; - also used by migrations via `bun run admin:db:migrate:collectivex`), - `COLLECTIVEX_ADMIN_SECRET` (delete route Bearer token — deliberately not - INVALIDATE_SECRET, since it is remembered in browser localStorage), and `GITHUB_TOKEN`. +- **Env**: `DATABASE_COLLECTIVEX_READONLY_URL` is the ordinary read-only connection and + may point at a replica. `DATABASE_COLLECTIVEX_WRITE_URL` is the primary used by lazy + persistence, its consistent reads, deletion, and migrations via + `bun run admin:db:migrate:collectivex`. `COLLECTIVEX_ADMIN_SECRET` is the delete route + Bearer token (deliberately not `INVALIDATE_SECRET`, since it is remembered in browser + localStorage), and `GITHUB_TOKEN` authorizes artifact reads. ## Multi-run explorer @@ -89,6 +95,8 @@ visible legend labels while retaining them in the legend item's accessible title The newest run with measured cases is checked by default; newer incomplete sweeps remain listed but cannot blank the initial explorer. Deletion is available per row or as one confirmed action for all currently shown runs; both paths keep the same tombstone semantics described above. +The Runs card's suite filter narrows the table to runs containing EP or KV cases; a run containing +both appears under both filters, and changing the filter does not alter the checked-run selection. ## The raw-rows exception diff --git a/docs/d3-charts.md b/docs/d3-charts.md index 4dc1a4bac..d2a63ae5a 100644 --- a/docs/d3-charts.md +++ b/docs/d3-charts.md @@ -1,17 +1,21 @@ # D3 Chart Design Rationale -## Why 4 Effects in ScatterGraph +## Four Invalidation Phases -A single useEffect for all D3 operations takes ~500ms (React reconciliation ~200ms + full SVG rebuild ~300ms). Splitting into 4 effects makes the most common interaction — switching Y-axis metric — take ~100ms instead: +One effect for every D3 operation takes roughly 500ms (React reconciliation ~200ms plus a full SVG update ~300ms). `D3Chart` separates structure, stable-identity data joins, metric coordinates, and display decoration. The most common interaction, switching the Y-axis metric, then takes roughly 100ms: | Effect | Trigger | Cost | What it does | | ----------------- | ---------------------- | ------ | ----------------------------------------------- | | 1. Structure | Mount, resize, theme | ~5ms | SVG skeleton, axes groups, defs, clip paths | | 2. Data render | Data shape changes | ~500ms | Full D3 join, bindpoints, rooflines, zoom setup | | 3. Metric update | Metric/scale selection | ~100ms | Reposition points in-place, rebuild rooflines | -| 4. Display toggle | Legend/label toggles | <20ms | Opacity transitions only | +| 4. Display toggle | Legend/label toggles | <20ms | Restyle marks or redraw changed decorations | -**The key insight**: Effect 2 depends on `dataIdentity` (a stable string hash of point keys), NOT on the `data` array reference. When users switch metrics, data points get new y-values (new array reference), but `dataIdentity` stays the same because the set of points hasn't changed. So Effect 2 skips entirely, and only Effect 3 runs. +**The key insight**: The data phase depends on `dataIdentity`, a stable string built from point join keys, rather than the `data` array reference. Metric changes produce new `x`/`y` values and a new array, but the set of points stays the same. The metric phase mutates bound coordinates by key and skips enter/update/exit joins. + +Custom layers that own scale-neutral decorations declare a `displayIdentity` and an `onDisplayUpdate` callback. The renderer snapshots each identity after the data and metric phases, then reruns only the custom layers whose identities changed. This removes stale labels and decorative overlays without rebuilding joins, scales, or unrelated paths. Callbacks receive the current zoomed scales so toggles do not jump decorations back to the unzoomed frame. + +`ScatterGraph` further partitions Effect 4 because its controls have disjoint mutation scopes. Mark visibility, palette/shape, trace metadata, point-label visibility/collision, line-label placement, and known-issue annotations each have a dedicated layout effect. It deliberately omits the generic scatter `displayIdentity`: point-label toggles update only `.point-label` nodes, while the unofficial overlay layer's own display callback updates only `.overlay-label` nodes. Recombining these passes causes label toggles to restamp thousands of unchanged shape, visibility, and trace attributes. ## In-Place Y-Value Mutation @@ -60,14 +64,14 @@ HTML strings bypass React entirely. The tooltip div is a plain DOM element appen Different metrics need different "optimal" directions: -| Direction | "Best" means | Example metric | -| ----------- | -------------- | --------------------------------------------------------- | -| upper_right | High x, high y | (not currently used) | -| upper_left | Low x, high y | Interactivity chart: low latency (x), high throughput (y) | -| lower_right | High x, low y | Cost chart: high interactivity (x), low cost (y) | -| lower_left | Low x, low y | (not currently used) | +| Direction | "Best" means | Example metric | +| ----------- | -------------- | --------------------------------------------------------------------- | +| upper_right | High x, high y | (not currently used) | +| upper_left | Low x, high y | Interactivity chart: low latency (x), high throughput (y) | +| lower_right | High x, low y | Cost/energy chart: high interactivity (x), low $/M tok or J/token (y) | +| lower_left | Low x, low y | (not currently used) | -The direction is declared per-metric in `inference-chart-config.json`, not computed. This makes the roofline direction a data concern, not a rendering concern. +The metric registry declares whether higher or lower values are preferable. Chart definitions derive the concrete corner from that polarity and the chart's x-axis direction, which keeps the choice in data rather than rendering code. ## Gradient Roofline Labels @@ -114,6 +118,11 @@ D3 bar charts measure actual Y-axis label widths using a temporary SVG `` Opacity animates via inline CSS `transition: opacity 150ms ease` (set on dots, rooflines, and labels in the render path); d3 `.transition()` is reserved for attributes CSS can't animate — the `data-update` entrance transitions on dot `transform` and roofline `d`. Never point both systems at the same property: a d3 transition re-writes the style every animation frame, and each write restarts the CSS transition, emitting `transitionrun`/`transitioncancel` per node per frame (a legend hover across a full chart used to produce tens of thousands of events per session, all of it also observed by PostHog's session-replay MutationObserver). Handlers like legend hover therefore write opacity **once** and let CSS do the animation. +Official coordinate transitions use one 300ms D3 tween for visible points and rooflines. Hidden +official marks snap directly to their final geometry and are excluded from the transition snapshot, +avoiding frame-by-frame writes for data the user cannot see. Unofficial overlays retain their +existing snap behavior. Replay passes a zero duration and remains frame-driven. + ## Batched Label Measurement Label loops that size a background rect to its text (`.ll-bg`/`.pl-bg`, point-label collision avoidance) must not interleave `getBBox()` with DOM writes — each read after a write forces a synchronous layout, turning N labels into N reflows. The pattern is two passes over the selection: write every label's text first, then measure every bbox (one forced layout for the whole batch), then write the rects. Same rule for `measureLegendRightInset`: it reads `getBoundingClientRect`, so it's skipped entirely when there are no known-issue annotations to place — it would otherwise run on every zoom frame. diff --git a/docs/data-pipeline.md b/docs/data-pipeline.md index 9d1ce61cc..176acc32c 100644 --- a/docs/data-pipeline.md +++ b/docs/data-pipeline.md @@ -2,24 +2,38 @@ ## DB Schema Decisions -### Why JSONB + Hot Columns (Hybrid) - -Benchmark metrics are stored in a JSONB `metrics` column AND extracted into dedicated "hot" columns (`tput_per_gpu`, `median_intvty`, `median_ttft`, `median_e2el`, `p99_ttft`, `median_tpot`). - -- **JSONB**: New metrics can be added by CI without schema migrations. Old data doesn't need backfilling — missing fields default to 0 at read time (`m.field ?? 0`). -- **Hot columns**: The most-queried metrics need B-tree indexes for `DISTINCT ON` queries. JSONB extraction (`metrics->>'field'`) can't use these indexes efficiently. -- **Trade-off**: ~6 duplicated values per row. Acceptable because benchmark_results is write-once/read-many, and the index speedup on daily queries is orders of magnitude. +### Why Metrics Stay in JSONB + +Benchmark measurements live only in the `benchmark_results.metrics` JSONB column; +there are no duplicated metric-specific columns on `benchmark_results` or +`latest_benchmarks`. + +- **Schema flexibility**: CI can add metrics without a database migration, and + historical rows simply omit keys that their artifact did not produce. +- **Read contract**: Raw benchmark endpoints pass the JSONB object through. + Specialized readers such as MCP select keys explicitly. A missing historical + metric remains absent until a presentation consumer applies its own fallback. +- **Indexes**: The benchmark indexes cover relational selection dimensions such + as config, scenario, concurrency, sequence lengths, date, offload mode, and + recipe fingerprint. Metric ranking and extraction use JSONB expressions rather + than nonexistent "hot" columns. ### Why Denormalized Dates `benchmark_results.date`, `workflow_runs.date`, and the `availability` table all store denormalized date values (derived from `workflow_runs.created_at`). This avoids JOINs in the hottest queries: -- `getLatestBenchmarks` uses `DISTINCT ON (config_id, conc, isl, osl) ORDER BY date DESC` — needs date on the same table as benchmark_results for the covering index. +- `getLatestBenchmarks` orders candidate line snapshots by `date`, `run_started_at`, + and workflow-run ID before applying the append-only curve rules. Keeping `date` + on each benchmark row avoids another join in that base-table scan. - `availability` is a separate denormalized table because the date-picker query (`SELECT DISTINCT model, date`) needs to be fast without scanning benchmark_results. ### Why a Materialized View (latest_benchmarks) -The `DISTINCT ON` query for "latest benchmark per config" is expensive on the full table (millions of rows). The materialized view pre-computes this, refreshed concurrently after each ingest. API routes use the view when no date filter is specified; date-filtered requests hit the base table. +Computing the newest logical curve from the full result table is expensive. The +materialized view pre-computes the newest successful snapshot for each config, +scenario, sequence, offload mode, recipe fingerprint, and concurrency, including +the same-image append-only history when applicable. API routes use the view when no +date filter is specified; date-filtered requests hit the base table. `REFRESH CONCURRENTLY` allows reads during refresh (no downtime). The trade-off is a brief window where the view is stale after ingest — acceptable since data changes at most daily. @@ -31,16 +45,40 @@ Every INSERT uses `ON CONFLICT DO UPDATE` or `DO NOTHING`. This means: - **Partial failures recover**: If ingest crashes mid-batch, re-running picks up where it left off. - **No cleanup needed**: No "delete old data first" step that could leave the DB empty on failure. -The unique constraints match natural keys (e.g., `(workflow_run_id, config_id, isl, osl, conc)` for benchmarks), not surrogate keys. +The unique constraints match natural keys (for benchmarks, workflow/config/scenario, +concurrency, offload mode, and the nullable recipe fingerprint), not surrogate keys. + +### Append-Only Curve Extensions + +Normal workflow runs are complete line snapshots: the latest run for a line replaces +the prior run as a unit, so partial re-sweeps cannot silently stitch points from +different recipes. A changelog containing only `append-only: true` entries marks the +one narrow exception. The latest-curve queries then walk backward through consecutive +append-only runs and include the nearest full snapshot, selecting the newest producer +for each recipe and concurrency. The producer stamps every new point with a deterministic +`recipe_fingerprint`, so variants that share the app's normalized topology and concurrency +remain separate. Historical rows have a null fingerprint and retain their legacy identity. + +The chain continues only while the image is identical. Each returned benchmark row +keeps its original workflow-run ID and run URL, so extending a curve does not erase +point provenance; `curve_date` and `curve_workflow_run_id` carry the separate logical +snapshot identity used by charts and history. InferenceX CI separately verifies that +the generated matrix is strictly additive: existing recipes and points are immutable, +while newly added concurrency points or complete recipe variants may run as deltas. +All appended points must retain the target curve's image, and benchmark-affecting code +changes must be isolated to the new points. ### Audited Point Purges `packages/db/src/etl/run-overrides.ts` is the durable audit record for exceptional data removal. Use `PURGED_BENCHMARK_POINTS` when a valid workflow run contains a specific invalid result, such as a server hang. Each record names `githubRunId`, -`runAttempt`, `configId`, `benchmarkType`, `isl`, `osl`, `conc`, and `offloadMode`, -with a dated reason comment. The full identity is required because one run can -contain multiple serving configurations at the same sequence lengths and concurrency. +`runAttempt`, `configId`, `benchmarkType`, `isl`, `osl`, `conc`, `offloadMode`, and +the optional nullable `recipeFingerprint`, with a dated reason comment. A fingerprint +targets exactly that recipe; omitting it or setting it to null targets only legacy rows +whose stored fingerprint is null. The full identity is required because one run can +contain multiple serving configurations and recipes at the same sequence lengths and +concurrency. `bun run db:apply-overrides` previews every matching row and requires confirmation before deleting it in a transaction. It also removes unreferenced server logs and @@ -54,14 +92,59 @@ conservative fallback prevents a failed GitHub lookup from restoring a suppresse point. It can only suppress an identical point from another attempt while that attempt remains unknown. -### Why Two Connection Types - -| Connection | Library | Use Case | Why | -| ------------------------------- | ----------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------- | -| `@neondatabase/serverless` HTTP | API routes | Stateless, read-only, scales to zero | Serverless functions can't hold persistent connections; HTTP driver works over Vercel's edge network | -| `postgres` TCP | ETL scripts | Bulk inserts, transactions, COPY | HTTP driver has per-query overhead that's unacceptable for 10K+ row batches | - -The read replica (`DATABASE_READONLY_URL`) is used by API routes to isolate read traffic from write load. ETL uses the primary writer (`DATABASE_WRITE_URL`). +### Audited Run Backfills + +Exceptional metadata corrections use the same reviewed, automatically enforced ledger +as purges. Add a `CHANGELOG_BACKFILLS` or `BENCHMARK_POINT_BACKFILLS` entry in +`packages/db/src/etl/run-overrides.ts`; never add a one-off production SQL statement. +Every entry has a stable kebab-case `id`, a human-readable `reason`, an exact run attempt, +and the target table's complete natural-key selector. The source-control history records +who approved the correction and why, while the stable ID appears in ingest and apply logs. + +Changelog backfills select `(githubRunId, runAttempt, baseRef, headRef)` and can patch +`configKeys`, `description`, `prLink`, or `appendOnly`. An `appendOnly` correction updates +both `changelog_entries.append_only` and `workflow_runs.append_only` atomically because +the materialized benchmark view reads the run-level value. + +Benchmark point backfills use the same complete point selector as audited point purges: +`githubRunId`, `runAttempt`, `configId`, `benchmarkType`, `isl`, `osl`, `conc`, +`offloadMode`, and nullable `recipeFingerprint`. Their `set` block supports a shallow +`metricsMerge`, top-level `metricsRemove`, and `offloadMode`. Setting `offloadMode` +updates both the first-class column and `metrics.offload_mode`; for example, a missed CPU +KV offload annotation can set `offloadMode: 'on'` and merge `kv_offloading` plus backend +metadata in one correction. Unrelated metrics remain unchanged. + +Run `bun run admin:db:apply-overrides` to preview the exact rows, reasons, and patches; +the command requires confirmation unless passed `--yes`. It fails before writing when a +target is missing, when a point's desired identity already exists, or when registry +validation finds an overlap or duplicate. Writes are verified against the declared state +and `latest_benchmarks` is refreshed afterward. Merges to `master` run the same command in +CI, followed by database verification, cache invalidation, and cache warmup. + +Point corrections are additionally applied before each CI or GCS benchmark insert. This +prevents a later idempotent re-ingest from restoring the artifact's original value. The +ingest tracks source and desired identities across the run and fails on a collision instead +of silently collapsing two distinct artifact points. When GCS cannot recover an attempt +number, it follows the purge path's conservative fallback and matches the exact run and +point across attempts; the log records the applied backfill ID. Changelog corrections are +likewise applied to artifact metadata before `appendOnly`/`evalsOnly` run semantics are +resolved and before the changelog is upserted. + +### Connection drivers and policy + +| Connection | Library | Use case | +| ------------------------------- | ------------------------ | ------------------------------------ | +| `@neondatabase/serverless` HTTP | Application API reads | Stateless serverless queries | +| `postgres` TCP | ETL, migrations, and MCP | Bulk work, transactions, and MCP SQL | + +`resolveDatabaseConnection` in `packages/db/src/connection.ts` is the single URL and TLS +policy. Each caller selects its environment variable and driver deliberately. An explicit +URL can override the environment value, `DATABASE_SSL` can override host detection, and +loopback hosts disable TLS by default. Application reads may select Neon HTTP, while admin +and MCP clients remain on postgres.js. + +Normal API reads use `DATABASE_READONLY_URL` so a replica can isolate read traffic from +writes. ETL uses the primary writer in `DATABASE_WRITE_URL`. ### Why CHECK Constraints for Lowercase @@ -73,7 +156,7 @@ All text keys (model, hardware, framework, precision) have `CHECK (field = lower Phase 1 (parallel 20): ZIP reading + JSON parsing + row mapping. IO-bound (network + disk), so high parallelism. -Phase 2 (parallel 5): DB writes. Connection-limited (max 20 connections), and each write does config lookup + bulk insert. Lower parallelism prevents connection exhaustion. +Phase 2 (parallel 10): DB writes. Connection-limited (max 20 connections), and each write does config lookup + bulk insert. Half-pool concurrency leaves capacity for metadata and summary queries while improving ingest throughput. ### Config Cache @@ -89,12 +172,105 @@ AIPerf defines the `server_metrics_export.json` envelope, but labels such as wor Adapters are selected from the benchmark's canonical framework, and per-worker series are only emitted for disaggregated configs with a recognized adapter. Unknown orchestrators and non-disaggregated configs retain their aggregate-only series; roles are never guessed from ports or metric names. The frontend only consumes the canonical source identity and never interprets orchestrator-native labels. +### Logical Engines vs Raw Series + +A raw series in the blob is one `(scrape endpoint × phase block × label set)` tuple, which is **not** the same as one engine. The KV-cache chart needs one entry per _logical engine_ — one KV pool — so `compute-chart-series.ts` groups series by their Prometheus label set (`seriesIdentityKey`) rather than emitting one entry per raw series. Three kinds of duplication collapse there: + +- **Phase blocks.** The warmup and profiling blocks each carry their own series for the same engine, so keying by scrape instant unions them into one continuous line. The two blocks' first/last bounds can look overlapping — the profiling series often emits a single boundary sample and then gaps until warmup ends — but they never share a scrape instant, so the union neither drops nor double-counts a sample. +- **Mirrored API-server frontends.** vLLM run with several API servers exposes the _same_ engine set on every `/metrics` endpoint, a few hundred ms apart. Identity ignores `endpoint_url` (Prometheus treats the label set as the series), and the endpoint covering the most wall-clock wins — merging the mirrors instead would interleave near-duplicate samples and halve the effective span of the frontend's fixed-width rolling average. Endpoints are only fused when their values actually agree; same-label endpoints whose readings diverge are independent replicas behind a router, and are kept as separate engines so none is silently discarded. +- **Intra-engine shard ranks.** `tp_rank` / `pp_rank` / `ep_rank` / `moe_ep_rank` shard one pool and all report the same utilization, so they are excluded from the identity and averaged. `engine` / `engine_idx` / `dp_rank` are _not_ excluded — those do name distinct pools. + +The cluster average is then a mean across those logical engines on the union of their scrape instants, with each engine holding its last sample until its next one and contributing only inside its own observed window (and only while that sample is fresher than 5× the engine's own median scrape gap, so a reporting hole drops the engine out of the mean rather than pinning it to a stale reading). Grouping on an exact `start_ns` instead would average whichever engines happened to share that nanosecond — on a disaggregated run that alternates between "prefill only" and "decode only" and reads as a full-scale sawtooth. + +Engines are ordered by role, then numeric rank, then worker — never by the composed display string, which would sort `"decode 10"` before `"decode 2"` and scramble DP ranks. The role is only shown when engines actually differ in role, so an aggregated deployment reads `DP 0…DP 3` rather than `decode 0…decode 3`. + +### Summed Series and the Canonical Grid + +The same "components aren't scraped in lockstep" problem hits the **summed** series — prefill/decode/prefix-hit rates, queue depth, host KV usage, and the prompt-token source breakdown — but it cannot be solved the same way. A mean is scale-free, so `averageAcrossEngines` can evaluate on the union of scrape instants; a sum is not. `cumulativeUniqueInputTokens` turns these rates into token totals with `sum += value` and `rollingAverage` is a sample-count mean, so both only stay correct at **one point per scrape bucket**. + +Summing on an exact `start_ns` emitted one point per component per tick, each holding that component's share alone. Downstream that reads as a comb: the rolling average mixed the real samples with the other components' structural gaps and drew roughly 1/N of the cluster total. Two shapes produced it in the corpus — a disaggregated run puts every worker on its own `/metrics` endpoint and its own sub-second offset (~7× low on the 7-worker rows), and even a single-endpoint SGLang run splits its token counters into `is_streaming="true"`/`"false"` series ~16 ms apart, where the `"false"` half is an all-zero run for a fully streaming benchmark (2.2× low). + +Every summed series is therefore evaluated by `sumOntoGrid` on one canonical grid at the blob's native scrape cadence (`canonicalTickNs`, the median gap of the best-sampled series), with each component holding its last sample between its own scrapes and contributing only inside its observed window. The lattice is anchored at `t=0` and shared by every metric, so the pairs that get divided or added downstream (hits/queries, used/total, running+waiting) land on identical `t` values — anchoring each metric at its own first sample instead silently emptied the prefix-cache-hit-rate chart, because `sglang:cached_tokens` starts ~0.18 s off the grid `sglang:prompt_tokens` starts on. + +Mirrored endpoints are collapsed here too, on a **relative** tolerance rather than the gauge path's absolute one — a throughput mean is O(10⁵), so an absolute threshold would never fire and every mirror would be counted twice. This is a correction as well as a de-duplication: the two-API-server vLLM rows were double-counting their queue depth and token totals exactly 2× before v14. + +Nothing groups on an exact `start_ns` any more; `aggregateByStart` was removed in v14. + +The detail page does not average the resulting per-tick cache-hit ratios. Cache-hit and +prompt-token counters can publish logically related deltas in adjacent scrape buckets, so a +quiet denominator bucket can make a pointwise ratio exceed 100% even though the run-wide totals +are valid. The displayed line is instead a centered 50-sample ratio of sums — +`Σ(cache-hit rate) / Σ(cache-query rate)` — computed in O(n) with prefix sums. The query-rate +denominator is recovered from the stored pointwise ratio and hit rate, preserving vLLM's +`prefix_cache_queries` semantics; prompt-token rate is only a proxy weight for zero-hit intervals +where `hits / ratio` cannot be inverted. Hit-counter ticks with no stored ratio (zero queries) are +also unioned back into the window rather than dropped. Only after that volume-weighted +aggregation is the semantic `[0, 1]` bound applied as a guard against residual counter timing +skew. Older rows without the component rate series retain the stored-ratio fallback. + +### Server-Log Artifact Bundles + +Server-log artifacts are stored as filename-keyed bundles rather than as a hard-coded list of +router, worker, or benchmark roles. Ingest recursively retains every regular `.log` and `.out` +file and preserves its artifact-relative path. This applies to both `server_logs_*` ZIPs and +`multinode_server_logs_*` artifacts whose files are nested inside +`multinode_server_logs.tar.gz`. + +The primary file remains in `server_logs.server_log` for compatibility and KV-cache metadata +extraction; `server_log_files` holds the remaining files. `server_logs.file_name` records the +primary file's original path, and `files_complete` distinguishes fully scanned bundles from +legacy single-file rows. Run `bun run admin:db:backfill-server-log-files` to use the public GCS +artifact backup first and fall back to retained GitHub artifacts for recent runs. Pass `--all` +for complete DB history, `--source gcs` to exercise only the backup path, and `--dry-run` to +inventory pair counts and compressed download size without writing. `--run `, `--limit `, +and `--yes` remain available for targeted, idempotent recovery. Bundles already marked +`files_complete` are skipped after their small benchmark artifact is mapped, avoiding repeated +large log downloads. + +Full-bundle search stays inside PostgreSQL: each file is scanned through overlapping 16 MiB +character slices, and the API returns at most 50 match contexts rather than transferring the +complete file. The overlap preserves literal matches that cross a slice boundary. Files are +processed sequentially so a multinode bundle cannot materialize every large log in one query. + ### Agentic Dataset Provenance AIPerf exports public-dataset provenance in `metadata.dataset`, including the Hugging Face dataset ID. InferenceX preserves that object as `dataset` on each agentic aggregate benchmark row. During benchmark ingest, `ingest-ci-run.ts` derives the dashboard slug from `hf_dataset_name` (for example, `semianalysisai/cc-traces-weka-062126` becomes `cc-traces-weka-062126`) and upserts `run_datasets` for the workflow run. Legacy artifacts without provenance leave any existing mapping untouched. A workflow run can map to only one dataset; conflicting dataset IDs fail ingest rather than silently linking the run to an arbitrary dataset. +### Agentic Replay-Lane Identity + +AIPerf can sample one dataset conversation into multiple trajectory lanes, and +those replays may execute concurrently. The source `conversation_id` therefore +identifies dataset provenance, not a unique execution lane. During trace-replay +ingest, `computeRequestTimeline()` groups requests by `root_correlation_id`. +For older exports without that field, it walks `parent_correlation_id` ancestry +from each per-session `x_correlation_id`, so nested subagents resolve to the +main session rather than becoming separate replay lanes. The extractor replaces +the repeated root UUID with a compact, deterministic numeric `ri` ranked by each +lane's first dispatch. + +The request-timeline frontend groups by `(conversation_id, ri)`, keeping all +main-agent, subagent, warmup, and profiling requests from one replay together. +Legacy timelines without correlation metadata omit `ri` and retain the older +conversation-only grouping. `REQUEST_TIMELINE_VERSION` invalidates derived +JSONB when this extraction changes; `db:backfill-request-timeline` recomputes it +from the raw gzipped `profile_export.jsonl` already stored in Neon, so no GitHub +artifact refetch or schema migration is required. + +Timeline extraction reads the gzip stream twice instead of materializing the +entire JSONL. The first pass retains only timeline bounds and one first-dispatch +timestamp per replay; the second emits compact request records. This bounds +peak memory for high-throughput traces whose decompressed profiles exceed +400 MB, at the cost of a second sequential decompression pass during ingest or +backfill. The backfill also downloads the stored gzip and uploads the derived +JSONB in 4 MiB chunks, matching trace ingest and avoiding single-value transport +limits for oversized runs. The request-timeline reader checks only compact +metadata inline, then reconstructs every current timeline from bounded text +chunks. It deliberately does not choose by compressed JSONB storage size, so a +highly compressible timeline cannot cross Neon's 64 MiB response limit or force +the database to materialize the serialized size just to select a read path. + ### Agentic Full-Response Interactivity Agentic charts use full-response inter-token latency as their canonical ITL and @@ -109,6 +285,24 @@ minus TTFT, then divides by `output_sequence_length - 1`. The same helper powers the one-time `db:backfill-full-response-interactivity` data migration, keeping historical rows and newly ingested rows on one definition. +### Agentic Point-Detail Payloads + +The point-detail page deliberately splits its immutable trace data by use case. +The default charts fetch `request-chart-data`, a dictionary-encoded tuple +projection containing only conversation, phase, request timing, latency, and +sequence-length fields. The source-rich `request-timeline` document is fetched +only after the user opens the Timeline view. Likewise, `trace-server-metrics` +returns the aggregate series plus source descriptors; a selected endpoint's +full arrays come from `trace-server-metric-source` on demand. + +This split is a scale invariant, not only a transport optimization. A high- +throughput point can contain more than 150,000 requests and several copies of +each server series. Reusing either full document for the default view multiplies +JSON parsing, React Query memory, and cache-transfer cost by data the visible +charts do not consume. The Timeline view also virtualizes rows, so the number of +SVG bar/link nodes is bounded by the scroll viewport rather than total request +count. + ## Frontend Transform Pipeline ### Why transformBenchmarkRows Exists diff --git a/docs/data-transforms.md b/docs/data-transforms.md index b04ae973e..b09c43d06 100644 --- a/docs/data-transforms.md +++ b/docs/data-transforms.md @@ -21,7 +21,7 @@ RenderableGraph[] (consumed by ScatterGraph) **`AggDataEntry`** (`components/inference/types.ts`) — flattened, fully-typed working representation. All metric keys are promoted to top-level fields with `?? 0` defaults. The display model name is resolved here. `hwKey` starts as an empty string and is filled in by `transformBenchmarkRows` after calling `getHardwareKey`. The `actualDate` field holds the real DB date when the `date` field has been overridden to a user-selected comparison date. -**`InferenceData`** (`components/inference/types.ts`) — chart-ready scatter point. Extends `Partial` and adds: chart coordinates (`x`, `y`), all derived metrics as `{ y: number; roof: boolean }` objects, and narrowed boolean types for the `dp_attention`/`disagg` family. The `roof` flag is always `false` at creation and set to `true` by `markRooflinePoints` during the render phase. +**`InferenceData`** (`components/inference/types.ts`) is a chart-ready scatter point. It extends `Partial` with chart coordinates (`x`, `y`), derived metric objects shaped as `{ y: number; roof: boolean }`, and narrowed boolean types for the `dp_attention`/`disagg` family. Derived metric objects start with `roof: false`; current frontier membership is computed separately from the selected metric and chart direction. **`RenderableGraph`** — final output of `useChartData`. Bundles `model`, `sequence`, `chartDefinition`, and `InferenceData[]` for a single scatter chart panel. @@ -44,27 +44,29 @@ RenderableGraph[] (consumed by ScatterGraph) 1. Converts every row to `AggDataEntry` once (via `rowToAggDataEntry`). 2. Calls `getHardwareKey(entry)` and writes the result back into `entry.hwKey`. 3. Calls `getHardwareConfig(hwKey)` with a per-call cache to build the `HardwareConfig` map (hardware display metadata — label, color, GPU title). -4. For each `ChartDefinition` in `inference-chart-config.json`, calls `createChartDataPoint` with that definition's `x`/`y` field keys to produce one `InferenceData` array per chart. The same `AggDataEntry` objects are reused across chart definitions — they are not re-created. +4. Builds canonical derived fields once per prepared entry, then calls `createChartDataPoint` for each chart definition with that definition's `x`/`y` keys and the shared fields. The same `AggDataEntry` and derived objects are reused across both charts. Returns `{ chartData: InferenceData[][], hardwareConfig: HardwareConfig }`. ### Step 2: AggDataEntry to InferenceData (`lib/chart-utils.ts`) -**`createChartDataPoint(date, entry, xKey, yKey, hwKey)`** — spreads `entry` first, then overrides with derived fields: +**`createChartDataPoint(date, entry, xKey, yKey, hwKey, derivedFields?)`** spreads `entry` first, then overrides chart coordinates and metadata. The full transform passes precomputed derived fields; direct callers may omit them. - `x` / `y`: read directly from `entry[xKey]` and `entry[yKey]` (set per chart definition). - `tp`: for disaggregated configs, set to `num_prefill_gpu + num_decode_gpu` instead of `decode_tp`. - Boolean narrowing: `dp_attention`, `prefill_dp_attention`, `decode_dp_attention`, and `is_multinode` are coerced from `boolean | string` to `boolean | undefined`. - Disagg fields: `num_prefill_gpu` / `num_decode_gpu` are only set when `entry.disagg` is true; otherwise they are dropped. -**Derived metric fields** — each uses `{ y: number; roof: boolean }` so the chart layer can switch between them without re-fetching data: +**`buildDerivedChartFields(entry, hwKey, requestedMetrics?)`** is the sole owner of inference and historical trend formulas. The full transform requests every benchmark-backed field. History requests only the selected metric and its interpolation dependencies, so it stays lightweight. Each emitted value uses `{ y: number; roof: boolean }`. - `tpPerGpu`, `outputTputPerGpu`, `inputTputPerGpu` — raw throughput from the entry (tok/s/gpu). +- `tokenRevenuePerGpuHour` — gross revenue in $/GPU/hr. `buildDerivedChartFields` emits the normalized default (`tputPerGpu × 3,600 ÷ 1,000,000`, with input and output both priced at $1/M). `applyTokenRevenuePricing` can replace it with the selected model's current public OpenRouter prompt/completion prices. Aggregate rows use their measured input/output split; disaggregated rows derive a like-for-like split from fixed ISL:OSL or measured agentic prompt:generation totals because their raw input/output rates use different prefill/decode GPU denominators. - `tpPerMw` — `(tputPerGpu * 1000) / hardwarePower` (GPU power in kW, result in tok/s/MW). -- Cost fields — GPU hourly cost divided by tokens-per-hour (in millions): `costh` / `costn` / `costr` for hyperscaler / neocloud / 3-year-rental pricing respectively. Three token variants exist: combined (`costh`/`costn`/`costr`), output-only (`costhOutput`/`costnOutput`/`costrOutput`), input-only (`costhi`/`costni`/`costri`). +- Cost-per-million fields — GPU hourly cost divided by tokens-per-hour (in millions): `costh` / `costn` / `costr` for hyperscaler / neocloud / 3-year-rental pricing respectively. Three token variants exist: combined (`costh`/`costn`/`costr`), output-only (`costhOutput`/`costnOutput`/`costrOutput`), and input-only (`costhi`/`costni`/`costri`). +- Tokens-per-dollar fields — tokens-per-hour divided by the same hourly costs. They are separate Y-axis metrics rather than a display toggle: combined (`tokensPerDollarH`/`N`/`R`), output-only (`outputTokensPerDollarH`/`N`/`R`), and input-only (`inputTokensPerDollarH`/`N`/`R`). This keeps existing `i_metric=y_cost*` links and cost semantics unchanged. - Energy fields — `jTotal` / `jOutput` / `jInput`: `(hardwarePower * 1000) / tputPerGpu` (Joules per token, where power in kW is converted to W). -**GPU specs lookup** — `createChartDataPoint` calls `getGpuSpecs(hwKey)` (`lib/constants.ts`) which splits on `[-_]` to extract the base GPU token (e.g. `"b200_trt_mtp"` → `"b200"`) and looks it up in `HW_REGISTRY`. `HW_REGISTRY` stores power (kW per GPU) and three cost tiers ($ per GPU-hour). Missing keys return zeroed specs, which produces `0` cost/energy values rather than crashing. +**GPU specs lookup** happens inside `buildDerivedChartFields`. `getGpuSpecs(hwKey)` (`lib/constants.ts`) splits on `[-_]` to extract the base GPU token (for example, `"b200_trt_mtp"` becomes `"b200"`) and looks it up in `HW_REGISTRY`. Missing keys return zeroed specs, producing `0` cost/energy values rather than crashing. ### Step 3: Filtering, memoization, and rendering (`hooks/useChartData.ts`) @@ -78,7 +80,7 @@ The hook runs a 5-step memoized pipeline: 4. **Sort `hardwareConfig`** — the `HardwareConfig` object is sorted by `getModelSortIndex` and stabilized with a ref: if the sorted key string matches the previous render, the same object reference is returned. This prevents D3 Effect 2 (data bind) from firing when a sequence change returns the same GPU set. -5. **Build renderable graphs** — `stableChartDefinitions` is computed in a separate `useMemo` that depends only on metric/axis selections (not on data). This decouples Y-axis changes from data changes so D3 Effect 3 (metric repositioning) does not fire alongside Effect 2 (data bind). Within this memo, the x-axis field is resolved per chart type, roofline directions are flipped when the x-axis polarity reverses (e.g. interactivity → TTFT), and Y-axis labels are looked up. The final `graphs` memo applies GPU filtering, cost-limit clamping, optional user-cost/user-power overrides, and remaps each point's `x`/`y` from the selected metric's `{ y, roof }` object. +5. **Build renderable graphs** — `stableChartDefinitions` is computed in a separate `useMemo` that depends only on metric/axis selections (not on data). This decouples Y-axis changes from data changes so D3 Effect 3 (metric repositioning) does not fire alongside Effect 2 (data bind). Within this memo, the x-axis field is resolved per chart type, roofline directions are flipped when the x-axis polarity reverses (e.g. interactivity → TTFT), and Y-axis labels are looked up. The final `graphs` memo applies GPU filtering, cost-limit clipping, optional user-cost/user-power overrides, and remaps each point's `x`/`y` from the selected metric's `{ y, roof }` object. --- @@ -106,7 +108,7 @@ The resulting key's base GPU must exist in `HW_REGISTRY`. Display fields (label, ## Chart Configuration -`inference-chart-config.json` defines exactly two `ChartDefinition` objects: +`metric-registry.ts` owns metric fields, English and Chinese labels, titles, polarity, and control ordering. It derives exactly two `ChartDefinition` objects: | `chartType` | Default x-axis | x meaning | | --------------- | --------------- | -------------------------------------------------------------- | @@ -115,13 +117,13 @@ The resulting key's base GPU must exist in `HW_REGISTRY`. Display fields (label, Both charts share the same Y-axis options. The `y` field is the default `AggDataEntry` key used for raw Y values; each `y_{metric}` field overrides this with a dotted path into the `InferenceData` derived fields (e.g. `"tpPerGpu.y"`). -**Per-metric Y-axis schema** — for each metric key (e.g. `y_costh`), the config carries: +**Per-metric Y-axis schema** is owned once in `METRIC_REGISTRY`: -- `y_{metric}`: dotted path for the value (e.g. `"costh.y"`). -- `y_{metric}_label`: Y-axis label string. -- `y_{metric}_title`: dropdown/UI title string. -- `y_{metric}_roofline`: Pareto direction (`upper_left`, `upper_right`, `lower_left`, `lower_right`). Roofline direction differs between chart types for the same metric because the x-axis polarity differs (interactivity: higher-is-better; E2EL: lower-is-better). +- `field`: dotted path for the value (for example, `"costh.y"`). +- `label` / `labelZh`: bilingual Y-axis labels. +- `title` / `titleZh`: bilingual dropdown and chart titles. +- `polarity`: whether higher or lower values are preferable. The derived chart definitions combine this with x-axis polarity to produce the concrete Pareto corner. -**Input-metric x-axis override** — when the selected Y metric's title contains `"input"`, the interactivity chart switches its x-axis to `p99_ttft` (or the user-overridden x metric). This is detected in `stableChartDefinitions` via `metricTitle.toLowerCase().includes('input')`. The config encodes the default override fields: `y_inputTputPerGpu_x: "p99_ttft"` and `y_inputTputPerGpu_x_label`. +**Input-metric x-axis override** switches the interactivity chart to TTFT when the selected Y metric is input-related. `inputTputPerGpu` declares `p90_ttft` and its label in the registry; a user-selected TTFT metric may override it. -**Limits** — both charts include `y_cost_limit: 5` (clamp cost-metric y-axis to $5/M tokens) and `y_latency_limit: 60` (filter x-axis outliers beyond 60s when TTFT is on x). +**Limits** — both charts include `y_cost_limit: 5` (clip cost-per-million metrics above $5/M tokens) and `y_latency_limit: 60` (clip x-axis outliers beyond 60s when TTFT is on x). Tokens-per-dollar metrics are not cost-clipped; clipped cost points remain available to the dashed continuation layer. diff --git a/docs/i18n.md b/docs/i18n.md index 2a6c5dde5..4063f5502 100644 --- a/docs/i18n.md +++ b/docs/i18n.md @@ -4,7 +4,7 @@ Why the Simplified Chinese site is a hand-authored `/zh` page tree instead of an ## Why hand-authored pages, not next-intl / `[locale]` routing -- **SEO is the goal, not full UI translation.** The objective is Chinese pages that crawlers can index: Chinese titles, meta descriptions, server-rendered Chinese content, and bidirectional hreflang. The interactive dashboard (charts, filters, tooltips) stays English — model/GPU/framework names are English-first terms in Chinese ML writing anyway, and translating deep chart UI would touch hundreds of client components for little search value. +- **SEO and a complete Chinese user experience are both goals.** Chinese titles, meta descriptions, server-rendered content, and bidirectional hreflang make the pages indexable; localized controls, tooltips, states, and explanations make them usable without falling back to English. Product names, model and hardware SKUs, framework names, units, flags, and identifiers remain English where Chinese ML infrastructure writing normally keeps them. - **A `[locale]` root segment would move every route** and force middleware rewrites to keep existing URLs stable — high blast radius for a two-locale site. A parallel `/zh` tree adds pages without touching English URLs. - **No message-catalog indirection.** With exactly two locales and mostly page-level content, colocated `STRINGS = { en, zh }` dictionaries (landing page, quotes content) and dedicated zh files (`tab-meta-zh.ts`, `faq-data-zh.ts`) are easier to review than a parallel key-file hierarchy, and dead strings die with their page. @@ -26,5 +26,6 @@ Why the Simplified Chinese site is a hand-authored `/zh` page tree instead of an - **zh OG images reuse the English post meta** — the `next/og` default Satori font has no CJK glyphs, so a Chinese title would render as tofu. Loading a subset CJK font is a known follow-up. - **`/zh/inference` canonicalizes to `/zh`**, mirroring the English quirk where `/inference` canonicalizes to `/`. - **All UI strings are localized in place** via `useLocale()` + component-local `STRINGS = { en, zh }` dicts (footer, TabNav, dashboard display headings/labels, nudges, preset cards, legend toggles, select placeholders), and registry-defined display names (Y-axis metrics, toggle configs) carry `labelZh` fields resolved through locale-aware helpers. The `en` values keep the exact original strings so English pages are byte-identical. Only brand/product/hardware/framework/precision names, units, code identifiers, and DB-stored content stay English. +- **Editorial decisions are contextual.** [Chinese Copy Editorial Guide](./chinese-copy.md) defines the audience, surface-specific register, terminology exceptions, rewrite workflow, and manual-review process. It is the source of truth for how Chinese copy should read; this document remains the source of truth for route and rendering architecture. - **Compare slug pages are mirrored** at `/zh/compare/[slug]` and `/zh/compare-per-dollar/[slug]`. The Chinese narrative templates live in `compare-ssr-zh.ts` as a 1:1 port of `compare-ssr.ts` (data logic is imported, only sentence templates differ) — the two files must change together. - **Sitemap pairs**: `localizedPair()` in `sitemap.ts` emits the EN and zh URL together, both carrying the same `alternates.languages` map. Blog posts without a translation fall back to an English-only entry, so a missing translation degrades gracefully instead of 404-ing crawlers. diff --git a/docs/index.md b/docs/index.md index c4a72633b..7a22a0d1b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,7 +4,7 @@ Design rationale and non-obvious conventions. See [CLAUDE.md](../CLAUDE.md) for ## Docs -- [Architecture](./architecture.md) — Why client-first, hash routing, URL state, provider nesting, server-side caching (unstable_cache + blob), in-memory client cache, color system, analytics enforcement +- [Architecture](./architecture.md) — Why client-first, route navigation, URL state, provider nesting, server-side caching (unstable_cache + blob), in-memory client cache, color system, analytics enforcement - [D3 Charts](./d3-charts.md) — Why 4 effects, in-place mutation, refs for zoom, rAF throttling, HTML tooltips, Pareto directions, gradient labels - [Data Pipeline](./data-pipeline.md) — DB schema reasoning, ETL design, transform pipeline, spline method choice, normalizer resolution order (model/GPU/framework) - [Pitfalls](./pitfalls.md) — Failure modes: token type consistency, schema evolution, empty objects, zoom loss, stale closures, disaggregated metrics, negative splines, date stamping, ref stability, cost inheritance @@ -12,8 +12,10 @@ Design rationale and non-obvious conventions. See [CLAUDE.md](../CLAUDE.md) for - [TCO Calculator](./tco-calculator.md) — Why interpolation, composite keys, cost matrix, token type bugs, badge logic, state design - [Adding Entities](./adding-entities.md) — Step-by-step checklists for adding new models, GPUs, precisions, sequences, frameworks (ingest + constants + frontend), plus featuring a day-0 model (launch banner, modal, Quick Comparisons preset) - [Testing](./testing.md) — Requirements, quality standards, pre-commit checklist +- [Typography](./typography.md) — Type tokens (text-2xs/3xs, named tracking), Heading/Eyebrow components, TS-based chart font sizes, and the check:typography ratchet gate + allowlist burndown - [Data Transforms](./data-transforms.md) — Full pipeline from BenchmarkRow to RenderableGraph: type hierarchy, hardware key construction, derived metrics, memoization strategy - [State Ownership](./state-ownership.md) — Which context owns which state, availability filtering cascade, comparison date mechanics, URL param sync - [Blog](./blog.md) — MDX content system, SEO features (OG images, RSS, llms.txt, JSON-LD), TOC sidebar, reading progress, heading links, analytics events - [CollectiveX](./collectivex.md) — lazy ingest-on-read pipeline (separate Neon DB as a durable GitHub-artifact cache), tombstoned deletes, raw-docs storage + shared reader - [Chinese Pages (/zh)](./i18n.md) — Why hand-authored /zh pages instead of an i18n framework, hreflang pairing, blog translation pairing, html lang workaround, CJK reading time/slugs +- [Chinese Copy Editorial Guide](./chinese-copy.md) — Audience, surface-specific register, context-aware terminology, rewrite workflow, manual-review process, and PR checklist diff --git a/docs/pitfalls.md b/docs/pitfalls.md index 756de2dde..ffe3af17c 100644 --- a/docs/pitfalls.md +++ b/docs/pitfalls.md @@ -4,7 +4,7 @@ Non-obvious failure modes discovered during development. Each entry explains wha ## Token Type Consistency -**Bug pattern**: Chart title says "Output Token Cost" but bars show total token cost. +**Bug pattern**: Chart title says "Output Tokens per $1" or "Cost per Million Output Tokens" but values use total-token throughput. When a UI has a Token Type selector (Total/Input/Output), **every** downstream value must use the selected type: chart title, bar values, table values, tooltips, sorting, comparison text, axis labels. The common mistake is updating the title and sorting but forgetting the tooltip or comparison banner. @@ -40,9 +40,9 @@ D3 event handlers capture variables from their creation scope. If a handler refe ## Disaggregated Config Metrics -**Bug pattern**: Cost per token is 2x too high for disaggregated setups. +**Bug pattern**: Tokens per $1 is 2x too low—or cost per token is 2x too high—for disaggregated setups. -Disaggregated configs (MoRI SGLang, Dynamo TRT) have separate prefill and decode GPU counts. Cost and throughput metrics must be calculated per decode/prefill GPU, not per total GPU count. The disclaimer in ChartDisplay.tsx explains this to users. +Disaggregated configs (MoRI SGLang, Dynamo TRT) have separate prefill and decode GPU counts. Cost, purchasing-power, and throughput metrics must be calculated per decode/prefill GPU, not per total GPU count. The disclaimer in ChartDisplay.tsx explains this to users. When adding metrics that involve per-GPU calculations, check if the config is disaggregated (`d.disagg === true`) and use the appropriate GPU count. diff --git a/docs/state-ownership.md b/docs/state-ownership.md index 07e5a2bcf..84f546bbb 100644 --- a/docs/state-ownership.md +++ b/docs/state-ownership.md @@ -2,25 +2,32 @@ Reference for agents modifying filter behavior. Explains which context to touch for a given change, how availability cascades, and how URL params are kept in sync. -## Provider Nesting +## Provider Ownership + +`DashboardShell` resolves provider capabilities from +`packages/app/src/lib/dashboard-routes.ts`: ``` QueryProvider ThemeProvider - UnofficialRunProvider - GlobalFilterProvider ← availability, model/sequence/precision/date/runId - InferenceProvider ← GPU selection, comparison dates, chart UI, y-axis metric - inference tab - historical tab (no own provider — reads InferenceContext) - EvaluationProvider ← benchmark, eval-specific date, hardware toggle - evaluation tab - ReliabilityProvider ← date range, model toggle - reliability tab - calculator tab ← no provider; local useState in ThroughputCalculatorDisplay - gpu-specs tab ← no provider; static data + DashboardShell + inference / historical / evaluation + UnofficialRunProvider + GlobalFilterProvider + route-owned InferenceProvider or EvaluationProvider + calculator + UnofficialRunProvider + route-owned, URL-seeded GlobalFilterProvider + reliability + route-owned ReliabilityProvider + static / internal routes + no data providers ``` -Source: `packages/app/src/components/page-content.tsx` lines 203–239. +Compare routes live outside `DashboardShell` and own seeded +`GlobalFilterProvider` / `InferenceProvider` pairs. Agentic point-detail routes are +standalone. Provider changes must update the canonical route registry and its parity +tests rather than adding pathname conditionals in the shell. --- @@ -30,27 +37,40 @@ Source: `packages/app/src/components/page-content.tsx` lines 203–239. File: `packages/app/src/components/GlobalFilterContext.tsx` -**Selection state** (user-settable, URL-initialised): +**Requested selection state** (user or URL intent): + +- `selectedModel` (`g_model`) +- `selectedSequence` (`i_seq`) +- `selectedPrecisions` (`i_prec`) +- requested run date and run ID (`g_rundate`, `g_runid`) -- `selectedModel` — active model (`g_model`) -- `selectedSequence` — active ISL/OSL sequence (`i_seq`; owned here because it gates availability for all tabs) -- `selectedPrecisions` — active precision list (`i_prec`; same reason) -- `selectedRunDate` / `selectedRunId` — active benchmark run date and run ID (`g_rundate`, `g_runid`) +**Effective values** (purely derived): -**Effective (auto-corrected) values** (derived, not settable directly): +- `effectiveSequence` resolves the requested scenario against availability +- `effectivePrecisions` resolves requested precisions against available curves +- `effectiveRunDate` resolves the requested date or latest valid date +- `selectedRunId` resolves requested run intent against `availableRuns` -- `effectiveSequence` — `selectedSequence` if valid for current model, else first available -- `effectivePrecisions` — subset of `selectedPrecisions` that are available; falls back to `[availablePrecisions[0]]` -- `effectiveRunDate` — latest available date unless user explicitly picked one +`GlobalFilterProvider` owns the state, but it does not publish one response-shaped +context value. Consumers subscribe through five independently memoized domains: -**Derived availability** (memos over `availabilityRows`): +- `useGlobalFilterSelection`: requested model, sequence, and precision state plus + their effective values and `sequenceResolved` +- `useGlobalFilterActions`: stable setters for selection, run date, and run ID +- `useGlobalFilterRun`: effective date and run ID selectors plus the manual date revision +- `useGlobalFilterAvailability`: availability options, raw rows, settled state, and errors +- `useGlobalFilterWorkflow`: the run map plus workflow loading and error state -- `availableModels`, `availableSequences`, `availablePrecisions`, `availableDates` -- `availabilityRows` — raw `AvailabilityRow[]` from `useAvailability()`, passed down to InferenceProvider for GPU filtering +Workflow query updates therefore do not notify selection or action consumers. Selection +updates do not notify workflow-only consumers. Components should subscribe only to the +domains they read. -**Workflow / run info** (derived from `useWorkflowInfo(effectiveRunDate)`): +Availability owns explicit settled and error states. Consumers must distinguish an +unresolved request from a settled empty result. Inference fetches are gated on +`sequenceResolved` and render `availabilityError` rather than an indefinite skeleton. -- `availableRuns`, `workflowInfo`, `workflowLoading`, `workflowError` +There is no response-shaped workflow adapter. `RunInfo` and `availableRuns` are the +canonical run contract. **Why here, not InferenceProvider**: Model, sequence, and precision are cross-tab. EvaluationContext consumes `selectedModel` and `availableModels` directly. If these lived in InferenceProvider, EvaluationProvider would need an indirect coupling or duplicate state. @@ -60,55 +80,55 @@ File: `packages/app/src/components/GlobalFilterContext.tsx` File: `packages/app/src/components/inference/InferenceContext.tsx` -Depends on: `GlobalFilterProvider` (reads all filter state and availability, including `availabilityRows`). - -**GPU comparison state** (inference-only, URL-initialised): - -- `selectedGPUs` — hardware keys selected for GPU filter/comparison (`i_gpus`) -- `selectedDates` — discrete comparison dates (`i_dates`) -- `selectedDateRange` — `{startDate, endDate}` for range comparisons (`i_dstart`, `i_dend`) -- `activeDates` — `Set` toggle controlling visible comparison overlays (keyed by `${date}_${gpuKey}`) +Depends on `GlobalFilterProvider` through its narrow selection, run, availability, +workflow, and action hooks. -**Chart axis / display state** (URL-initialised): +The provider owns inference-specific effects and URL synchronization, then exposes +four independently memoized domains. Consumers must subscribe only to the domains +they read. -- `selectedYAxisMetric` (`i_metric`), `selectedXAxisMetric` (`i_xmetric`), `selectedE2eXAxisMetric` (`i_e2e_xmetric`) -- `scaleType` — `auto | linear | log` (`i_scale`) -- `hideNonOptimal` (`i_optimal`), `showPointLabels` (`i_label`), `logScale` (`i_log`) -- `highContrast` (`i_hc`), `isLegendExpanded` (`i_legend`) -- `useAdvancedLabels` (`i_advlabel`), `showGradientLabels` (`i_gradlabel`) -- `colorShuffleSeed` — no URL param; ephemeral +**`useInferenceData`** -**Derived availability** (GPU-level, computed from `availabilityRows` inherited from GlobalFilterContext): +Fetched and derived benchmark results: -- `availableGPUs` — hardware configs that have data for the current model + sequence + precisions AND have a known base GPU in `HW_REGISTRY` -- `dateRangeAvailableDates` — dates available for the current filter combination, further narrowed by `selectedGPUs` -- `hwTypesWithData` — `Set` of GPU keys currently present in fetched chart data +- `graphs`, `hardwareConfig`, `loading`, and `error` +- `hwTypesWithData` +- GPU, date, precision, sequence, model, run, and quick-filter availability -**Hardware toggle set**: +**`useInferenceFilters`** -- `activeHwTypes` — subset of `hwTypesWithData` that are visible (managed by `useChartDataFilter`) +Workflow, filter, and date selection state: -**Tracked configs / presets**: +- effective model, sequence, and precision selections read from global filter state +- `selectedGPUs`, `selectedDates`, `selectedDateRange`, and `activeDates` +- `activeHwTypes`, `bestPerSku`, and `quickFilters` +- effective run selection, custom cost and power values, preset state, and compare scope -- `trackedConfigs` — up to 6 pinned data points for cross-chart comparison -- `activePresetId`, `pendingHwFilter` — active favourite preset and its deferred GPU filter +This domain owns inference-only selection state. GPU selections and comparison dates +do not belong in global filter state because evaluation and reliability do not use +them. -**User overrides**: +**`useInferenceDisplay`** -- `userCosts`, `userPowers` — per-GPU cost/power overrides for custom cost metric; reset when `selectedYAxisMetric` changes away from `y_costUser`/`y_powerUser` +Axis and presentation state: -**Run filtering** (inference-local, not written back to GlobalFilterContext): +- selected x-axis and y-axis metrics, percentile, and effective x-axis mode +- token-revenue price source (`i_revenue`): normalized $1/M or the selected model's live OpenRouter catalog prices +- scale, optimal-point, label, contrast, legend, and overlay controls -- `filteredAvailableRuns` — `availableRuns` filtered to runs matching `selectedModel` + `effectivePrecisions` -- `effectiveSelectedRunId` — validated run ID within `filteredAvailableRuns`; intentionally NOT synced back to GlobalFilterContext to avoid full-tree re-renders on precision change +Display changes stay in this domain. A contrast or label toggle therefore does not +notify filter-only or data-only consumers. -**Charts data** (from `useChartData`): +**`useInferenceActions`** -- `graphs` — `RenderableGraph[]` used by all D3 charts -- `hardwareConfig` — config map derived from benchmark rows -- `loading`, `error` +All inference commands and setters. The context value and every exposed action keep +stable identities. The provider routes each stable action to the latest implementation, +so new availability or chart data cannot invalidate action-only consumers. -**Why not in GlobalFilterContext**: GPU selection and comparison dates are meaningless outside the inference/historical tabs. Putting them in the global context would pollute the interface for evaluation and reliability. +Run filtering remains inference-local. `filteredAvailableRuns` narrows global run +availability to the effective model and precisions. `effectiveSelectedRunId` is +validated within that set but is not written back through the global run action, so a +precision-specific fallback cannot replace the user's global run intent. --- @@ -116,12 +136,16 @@ Depends on: `GlobalFilterProvider` (reads all filter state and availability, inc File: `packages/app/src/components/evaluation/EvaluationContext.tsx` -Depends on: `GlobalFilterProvider` (reads `selectedModel`, `setSelectedModel`, `selectedRunDate`, `selectedRunDateRev`, `setSelectedRunDate`, `availableModels`, `availableDates`). +Depends on `GlobalFilterProvider` for model and global date intent. **Selection state**: -- `selectedRunDate` — evaluation-specific date; initialised from `e_rundate` URL param or `globalRunDate`. Bidirectionally synced: when the user picks a date, it calls `setGlobalRunDate` (if the date exists in inference availability). When `globalRunDate` changes (from another tab), the effect at line 124 applies it here. -- `selectedBenchmark` — active eval task (`e_bench`) +- requested evaluation date plus a reducer-derived nearest valid `selectedRunDate` +- `selectedBenchmark` (`e_bench`) + +Explicit evaluation date actions synchronize the global requested date only when that +date exists in inference availability. Global changes are reducer inputs rather than a +pair of bidirectional mirroring effects. **UI state**: @@ -165,15 +189,19 @@ Does NOT consume `GlobalFilterProvider`. Fully standalone — reliability data h --- -### Tabs without providers +### Route-Owned Providers -**TCO Calculator** (`calculator` tab): All state is local `useState` inside `ThroughputCalculatorDisplay`. It reads `effectiveSequence` and `effectivePrecisions` from `useGlobalFilters()` for the initial GPU list but does not share state back. +**TCO Calculator**: the route parses a typed URL seed and mounts the sole +`GlobalFilterProvider` for the calculator. Calculator-specific visibility, target, and +bar selection transitions live in one reducer. `visibleHwKeys` remains the single +source of truth for official and unofficial bars. -It reads `unofficialBenchmarkRows` / `unofficialRunInfos` / `runIndexByUrl` from `useUnofficialRun()` to render unofficial-run overlay bars, but it neither reads nor writes the shared `activeOverlayHwTypes` that the inference and evaluation tabs use: the calculator's local `visibleHwKeys` governs its official and overlay bars alike. Two visibility sets behind one legend can only drift — see [TCO Calculator › Unofficial-Run Overlays](./tco-calculator.md#unofficial-run-overlays-unofficialrun). +**Historical Trends**: mounts `InferenceProvider` and shares the global filter provider +declared by its route capability. -**Historical Trends** (`historical` tab): Rendered inside `InferenceProvider` (shares the `inference` + `historical` `isActive` gate). It reads `useInference()` directly — no additional provider. Uses InferenceContext's model/sequence/precision/date state. +**Reliability**: mounts only `ReliabilityProvider`. -**GPU Specs**: Static data, no provider. +**GPU Specs and other static/internal pages**: mount no data providers. --- @@ -190,14 +218,16 @@ GlobalFilterProvider → selectedModel (user pick) → modelRows = availabilityRows filtered to selectedModel (internal memo) → availableSequences = unique sequences in modelRows - → effectiveSequence = selectedSequence if in availableSequences, else availableSequences[0] + → effectiveSequence = Agentic (if unchosen and in availableSequences), + else selectedSequence if in availableSequences, + else 8k/1k if available, else availableSequences[0] → availablePrecisions = unique precisions in modelRows where sequence = effectiveSequence → effectivePrecisions = selectedPrecisions ∩ availablePrecisions; falls back to [availablePrecisions[0]] → availableDates = unique dates in modelRows where sequence = effectiveSequence AND precision ∈ effectivePrecisions → effectiveRunDate = latest of availableDates (unless user explicitly picked a date) -InferenceProvider (receives availabilityRows from GlobalFilterContext) +InferenceProvider (receives availability rows from `useGlobalFilterAvailability`) → availableGPUs = availabilityRows filtered to (model, effectiveSequence, effectivePrecisions) → hwKey extracted via buildAvailabilityHwKey() → filtered by isKnownGpu() (base GPU in HW_REGISTRY) @@ -235,62 +265,82 @@ Source files: `packages/app/src/lib/url-state.ts`, `packages/app/src/hooks/useUr | Prefix | Scope | | ------ | ----------------------------------------------------------------------------------- | -| `g_` | GlobalFilterContext — model, run date, run ID | -| `i_` | InferenceProvider — sequence, precision, GPUs, dates, metrics, display toggles | +| `g_` | Global filter run and selection contexts: model, run date, run ID | +| `i_` | Global selection plus InferenceProvider: sequence, precision, GPUs, metrics | | `e_` | EvaluationProvider — eval date (only when it differs from globalRunDate), benchmark | | `r_` | ReliabilityProvider — date range, display toggles | -Note: `i_seq` and `i_prec` are written by `GlobalFilterProvider` (not InferenceProvider) because they live in GlobalFilterContext. +Note: `i_seq` and `i_prec` are written by `GlobalFilterProvider` because selection state is owned by `useGlobalFilterSelection`, not InferenceProvider. + +### Snapshot, external subscriptions, and writes + +`url-state.ts` snapshots known share parameters at module load and removes them from the +visible address bar after hydration while preserving the pathname and hash. Route-owned +entry points pass typed seeds where they render URL-controlled state on the server. -### Snapshot-and-clear on load +Live address-bar consumers use the shared `useClientSearch` external store. It subscribes +once to `popstate` and `CLIENT_SEARCH_CHANGE_EVENT` through `useSyncExternalStore`; +components do not mirror search strings through independent effects. -`url-state.ts` runs at module-load time (before any React render). It reads all known keys from `window.location.search` into `_initialParams` and copies them into `currentState`. A `setTimeout(0)` callback then strips those keys from the URL via `history.replaceState`. This ensures the URL is clean after load — subsequent writes reflect only current state. +`writeUrlParams` batches state serialization for 150 ms. `useUrlStateSync` and explicit +state actions write requested/user state, not derived fallbacks. -`useUrlState` caches `_initialParams` in a ref via `readUrlParams()` on first call. All `getUrlParam()` calls return from this snapshot, so providers initialize from the original share URL even if the cleanup timer has already run. +The mutable remount snapshot and explicit URL intent are separate. Debounced +filter writes update the remount snapshot so providers preserve current state, +but they do not become explicit share-link choices that block automatic +unofficial-run model selection. A real navigation rebuilds the explicit-intent +set from the destination URL. -### Debounced write-back (150 ms) +### Carrying state across a full-document navigation -`writeUrlParams(params)` merges incoming params into `pendingParams` and sets a 150 ms debounce timer. On flush, params matching their default in `PARAM_DEFAULTS` are deleted from `currentState`; non-defaults are stored. This keeps share URLs short. +The two effects above combine into a trap: filter changes reach only the +in-memory `currentState`, and the address bar is stripped clean, so **the URL of +the `/inference` history entry describes nothing**. Any full-document navigation +away from the chart therefore destroys the state entirely — the module is torn +down, and Back returns to a bare `/inference` that rebuilds from defaults. -`useUrlStateSync` (used by InferenceProvider, EvaluationProvider, ReliabilityProvider) skips the first render via `isMountedRef` to avoid overwriting the just-snapshotted URL params. GlobalFilterProvider uses a manually-written equivalent (lines 288–308 in `GlobalFilterContext.tsx`). +The agentic point-detail links (`/inference/agentic/`, rendered by +`tooltipUtils.viewChartsButtonHTML` and `legend-points-table.pointDetailHref`) +are exactly that: plain ``, deliberately, so browsers can offer +open-in-new-tab. Three helpers in `url-state.ts` bridge the gap: + +| Helper | Used by | Why | +| --------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------- | +| `rememberChartStateInUrl()` | the point-click handlers (Scatter / GPU graph, legend table) | `history.replaceState`s the chart state onto the entry Back will return to | +| `withChartState(href)` | `agenticDetailHref()` | appends it to the outbound link so the detail page can link back to that view | +| `currentChartSearch()` | both of the above | resolves the tab through the `/zh` prefix, flushes pending writes, filters | + +The detail page reads the state back through `withChartState` (its own URL was +snapshotted into `currentState` at load, then stripped as usual), which is what +makes its "Inference chart" link land on the chart the reader left rather than +on defaults. + +### Re-hydration on client-side navigation + +`useUrlState` calls `refreshUrlParamsOnNavigation(pathname)` **during render**, +guarded by a module-level last-pathname so it runs at most once per navigation. +The refresh first flushes pending debounced writes, then applies explicit +destination parameters over that current state. This prevents a rapid filter +change followed by a retained-provider tab switch from replaying the previous +selection. + +That ordering also matters because providers read `getUrlParam` in `useState` +initialisers and mount effects, both of which run before any parent's layout +effect. With the hook-level refresh, every provider sees the params of the page +it actually landed on. The once-per-pathname guard keeps a late-mounting +component from replaying a stale URL over subsequent filter changes. ### Share URL construction -`buildShareUrl()` flushes pending writes, reads `currentState`, and filters to only the param prefixes relevant to the current tab (defined in `TAB_PARAM_PREFIXES`). Inference share URLs include `g_` + `i_` params; evaluation includes `g_` + `e_`; reliability includes only `r_`. - -Historical Trends and TCO Calculator share the inference tab's URL path (`/inference` and `/calculator` respectively) but have no dedicated param prefix — they inherit whichever `i_` params are relevant. - -### Full parameter list - -| Param | Owner | Default | -| --------------- | ------------------- | --------------------------------- | -| `g_model` | GlobalFilterContext | `DeepSeek-R1-0528` | -| `g_rundate` | GlobalFilterContext | `''` | -| `g_runid` | GlobalFilterContext | `''` | -| `i_seq` | GlobalFilterContext | `8k/1k` | -| `i_prec` | GlobalFilterContext | `fp4` | -| `i_metric` | InferenceProvider | `y_tpPerGpu` | -| `i_xmetric` | InferenceProvider | `p99_ttft` | -| `i_e2e_xmetric` | InferenceProvider | `''` | -| `i_scale` | InferenceProvider | `auto` | -| `i_gpus` | InferenceProvider | `''` | -| `i_dates` | InferenceProvider | `''` | -| `i_dstart` | InferenceProvider | `''` | -| `i_dend` | InferenceProvider | `''` | -| `i_optimal` | InferenceProvider | `''` (truthy = hide non-optimal) | -| `i_label` | InferenceProvider | `''` (truthy = show point labels) | -| `i_nolabel` | InferenceProvider | `''` (legacy, read-only) | -| `i_hc` | InferenceProvider | `''` | -| `i_log` | InferenceProvider | `''` | -| `i_legend` | InferenceProvider | `''` | -| `i_advlabel` | InferenceProvider | `''` | -| `i_gradlabel` | InferenceProvider | `''` | -| `e_rundate` | EvaluationProvider | `''` | -| `e_bench` | EvaluationProvider | `''` | -| `e_hc` | EvaluationProvider | `''` | -| `e_labels` | EvaluationProvider | `''` | -| `e_legend` | EvaluationProvider | `''` | -| `r_range` | ReliabilityProvider | `last-3-months` | -| `r_pct` | ReliabilityProvider | `''` | -| `r_hc` | ReliabilityProvider | `''` | -| `r_legend` | ReliabilityProvider | `''` | +`buildShareUrl()` preserves the exact current pathname, locale prefix, dynamic slug, and +hash. It filters parameters using `shareParamScopes` from the canonical dashboard route +registry and carries the canonical `unofficialruns` value from the live URL. + +### Parameter registry + +`UrlStateKey`, `URL_STATE_KEYS`, and `PARAM_DEFAULTS` in +`packages/app/src/lib/url-state.ts` are the exhaustive parameter source of truth. +Dashboard scope membership is declared by `shareParamScopes` in +`packages/app/src/lib/dashboard-routes.ts`. Tests enforce completeness and route-specific +share behavior, so this document deliberately does not duplicate a manually maintained +parameter table. diff --git a/docs/tco-calculator.md b/docs/tco-calculator.md index 2002753d3..d25a1c1ce 100644 --- a/docs/tco-calculator.md +++ b/docs/tco-calculator.md @@ -53,16 +53,42 @@ Badges change based on metric because showing power badges when the metric is "C - **Cost metric**: TCO $/GPU/hr badges (assumed hourly rates per GPU, sourced from SemiAnalysis AI Cloud TCO Model) - **tok/s/MW metric**: Power/GPU badges (assumed power draw per GPU, sourced from SemiAnalysis Datacenter Industry Model) -## Why No Separate Context Provider +## Provider and State Ownership -The calculator reuses `GlobalStateContext` (model, run date) and `InferenceChartContext` (sequence, precisions). Calculator-specific state (cost provider, token type, bar metric, target interactivity, selected bars) is local `useState`. +The calculator route parses its URL seed on the server and mounts the sole +`GlobalFilterProvider` for calculator model, sequence, precision, date, and run state. +`DashboardShell` deliberately does not mount a second global provider on this route. -Adding another context provider to the nesting hierarchy would increase re-render surface for unrelated tabs. Since calculator state doesn't need to be shared, local state is simpler and more performant. +Calculator-specific visibility, target/input, metric, and selected-bar transitions are +owned by one local reducer. Derived ranges and valid selections are computed from current +data rather than copied back through reconciliation effects. ## Bar Selection & Comparison Click-to-compare uses `resultKey` (not hwKey) because multi-precision mode produces multiple bars per GPU. Comparison ratios use the lower value as denominator (ratio >= 1.0). Both metric and token type are reflected in the comparison text to avoid ambiguity. +## Folding sections away (`CollapsibleSection`) + +The page is long — chart, cost target, lifecycle — and most readers want one of them +at a time. The chart section and Interactivity Within a Cost Target each carry a +chevron toggle (`components/ui/collapsible-section.tsx`). + +Two decisions in that component: + +- **Folded means unmounted, not hidden.** These bodies hold D3 charts and tables, and + leaving them mounted under `display: none` keeps them measuring and re-rendering + off-screen for a reader who said they are not interested. The trade is that chart + zoom and a frozen tooltip do not survive a fold. +- **`titleWhenOpen`.** The two fleet-planner sections own their `

`, so the header + shows the title always. The chart section's title lives in the chart's own caption + (and is part of the PNG export), so its header shows the title **only while + folded** — two copies would be worse than none. That is why `getChartTitle` is + hoisted out of the caption in `ThroughputCalculatorDisplay`: the fold header needs + the same words, and they carry the current metric and target. + +State is local `useState`, not a URL param: it is a reading preference, not a view of +the data worth sharing in a link. + ## Unofficial-Run Overlays (`?unofficialrun=`) A loaded unofficial run contributes an extra bar per (hardware × run) to the bar chart, in the run's palette color (`overlayRunColor`) and labeled `B300 (✕ my-branch)`. The label keeps the branch inside the same paren group as the precision so the `twoRowYAxisLabels({ split: 'parens' })` y-axis customizer still splits it into two rows. @@ -102,7 +128,7 @@ The reset's early-out guards on the **merged** list, not the official one. An em - **Clamped values.** `interpolateForGPU` clamps the target into each series' measured range and always returns a value, so a bar can be showing its nearest edge point rather than an interpolation. This is pre-existing across GPUs with different ranges, but widening the slider to cover overlay operating points makes it reachable for every official bar at once — which would turn a side-by-side overlay delta into a real-vs-clamped comparison. Results carry a `clamped` flag and the tooltip says so. (Narrowing the slider back is not the fix: it only moves the clamping onto the overlay bars, and an overlay-only model loses its bounds entirely.) - **Escaping.** The tooltip is a hand-built HTML string injected with `.html()`, and branch names and run URLs come from the GitHub API for whatever run id the user pasted. Everything untrusted goes through `escapeHtml` (`lib/utils`). The y-axis tick labels render the same branch but go through d3 `.text()`, and the legend entry is React — both already safe. -The calculator supports both fixed-sequence and Agentic Traces scenarios through +The calculator supports both fixed-sequence and Agentic scenarios through the shared `rowToSequence` classifier. Agentic rows carry null `isl`/`osl`, so filtering them by numeric sequence lengths would silently drop every point. @@ -141,6 +167,28 @@ at every interpolated point. compare interpolation models on a fixed snapshot, but they must not be presented as permanent impact figures for the changing live dataset. +The `/inference` page also exposes tokens-per-dollar and normalized token revenue +as separate Y-axis metrics; neither replaces the cost-per-million metrics. +Historical trends for these proportional metrics select Pareto knots from the +matching throughput, spline that throughput, and apply the constant multiplier. +Tokens-per-dollar uses `3600 / $/GPU-hr`; normalized token revenue uses +`3600 / 1,000,000` at the axis's explicit `$1/M tok` sale price. Reusing the +matching throughput frontier is essential: total-throughput knots are not +necessarily the output- or input-throughput Pareto knots. + +The revenue axis defaults to pricing every input and output token at the same +`$1/M tok` rate, making it a model-independent SLA comparison. Its OpenRouter +option fetches the selected model's current public prompt/completion prices from +`https://openrouter.ai/api/v1/models` and prices the streams separately. Aggregate +rows use their measured input/output split. Disaggregated rows cannot use the raw +per-prefill/per-decode rates together, so they apply the fixed ISL:OSL shape or +the measured agentic prompt:generation mix to total tok/s/GPU. The OpenRouter +option uses standard prompt/completion prices and does not apply cache discounts. + +Neither mode is the fleet lifecycle section's realized-revenue model: it does not +model availability, rollout, or a user-supplied cached-input discount. Those +business assumptions remain in Fleet Lifecycle below. + ### The consistency guard `recoverReciprocalNumerator` returns the constant only if **every** usable point @@ -154,3 +202,799 @@ The rate is recovered across **all three token types at once** (`recoverCostRate Checking one family alone and falling back to another would recover a rate from output tokens and then apply it to total throughput; the existing `maxInteractivityAtCost` tests caught exactly that mistake. + +## Fleet Lifecycle (`FleetLifecycle.tsx`) + +> **Now on its own page.** The section no longer renders on `/calculator`; it is +> the body of the `/fleet` route (and `/zh/fleet`), hosted by +> `FleetLifecycleDisplay.tsx`, which owns the same model/scenario/precision/ +> cost-tier/target controls the calculator page does and renders the chip +> legend in its controls card, where it stays reachable from the table view and +> when nothing is plottable. The calculator keeps a pointer card linking to `/fleet`, and +> its cost-target panel now renders the MW budget input (`c_mw` is shared +> between the two pages, so a budget set on one seeds the other). Everything +> below about the section's internals still applies unchanged. + +Everything above answers "which chip is cheapest at this interactivity, right +now?". This section answers "what has a fleet of it earned and cost since the +model shipped?" — and the answer is measured, not assumed. + +### Why it absorbed Fleet Projection, and why the cost target moved instead + +The page used to carry three fleet sections: **Fleet Projection** (a MW budget → +chips, fleet tok/s, concurrent users, $/hr, $/mo), **Interactivity Within a Cost +Target**, and this one. Two of those three are now one. + +Fleet Projection was folded in here because it was half of this section already. +Its MW input was this section's _primary_ input — the empty state read "enter a +facility power budget in the Fleet Projection section above", which is a control +in one section gating a table in another. Its `Fleet $/hr` and `$/mo` are this +section's `Cost $/day` in other units, and its `Fleet tok/s` is `tok/s/MW now` +times the budget. What it had that this did not was the physical sizing: **chip +count** and **concurrent users**. Those are now columns, so the budget, the sizing +it produces and the economics that rest on it read across one row. + +Two consequences worth knowing: + +- **The MW input renders outside `body()`.** Every other control here only means + something once a fleet exists, but this is the control that brings one into + being, so it must be present in the empty state — otherwise the section + deadlocks on its own precondition. +- **"Nothing to plot" needed splitting.** Fleet Projection had a dedicated message + for a budget too small to power one chip. Falling back to this section's + `noneMeasured` would have blamed the interactivity slider for a budget problem + and sent the reader to the wrong control, so an unsizeable budget (`unplottable` + non-empty with no rows) says so in its own words. + +The cost target was **not** folded in, because it is not a slice of this section at +any point on the axis: it is the _inverse_ of the target-interactivity slider. The +slider fixes a speed and the cost falls out; the cost target fixes a cost ceiling +and the speed falls out. Nothing on a lifecycle answers "how fast can I go under +$0.50/M tok". So it moved rather than merged — out of the fleet-economics stack and +up to sit directly beneath the slider it inverts (`CostTargetPanel.tsx`), which is +also why it is deliberately independent of the slider's current value: it reads the +unfiltered frontier, so a chip that interpolates to nothing at the present target +still gets an answer. + +### Why it reads the full run history + +The rest of the calculator reads one run date, so a chip is credited with +whatever its latest sweep found. Measured against production history for dsr1 +8k/1k (7,715 rows, 134 dates), roughly 37% of configs have their best read at a +typical target on an **earlier** date, worth up to ~3.6× — usually because a +later sweep explored a different part of the space rather than because the chip +got worse. For a lifecycle projection the honest number is the best config the +chip has ever demonstrated, so this section is the one place that reads +`/api/v1/benchmarks/history` instead of `/api/v1/benchmarks`. + +### One line per chip, not per config + +`hwKey` encodes the software, so a single piece of silicon appears in the history +many times over: `b200_trtllm`, `b200_sglang`, `b200_sglang_mtp` and so on are all +one B200. Drawing a line each says a fleet operator ran seven fleets, when they ran +one and kept re-deploying it onto whichever config was ahead. `mergeProgressionsByChip` +therefore collapses a chip's hwKeys into their **upper envelope**, and the winning +config becomes part of what each step reports (`Config Now` in the table, named per +rung in the tooltip). + +Three consequences worth knowing: + +- **Disagg competes for the same line.** Disaggregated configs report throughput + per decode or prefill chip, so a step won by one is not sized on quite the same + basis as a step won by an aggregated config; an earlier iteration split them into + two lines for that reason. They are pooled by product decision — the operator's + question is what the silicon can be made to do — so the caveat travels with the + step instead: `ChipProgression.disagg` flags a chip whose progression involves + any, the amber note explains the basis, and the config named on each step says + which kind won it. Per-line dashing was dropped when this changed, because a + single line can now contain both and a dashed stroke would assert something + untrue of half of it. +- **The legend still filters hwKeys**, one level below the lines — hiding a config + removes it from candidacy. Because the legend isolates on click, isolating a + single config leaves at most one chip, and none at all when that config was never + measured at the target. That is a real state, and `calculator-lifecycle.cy.ts` + asserts it rather than assuming a row always survives. +- **Colour resolves from the winning hwKey**, not the base GPU. The palette is + built over the _active_ hwKeys, so a bare base key falls through to the fallback + grey — which silently drained the colour out of every series when first wired. + +Cost stays flat across the merge because power and $/chip/hr come from the base +GPU: it is the same silicon whichever framework is in front. + +`bestSoFarProgression` supplies the per-hwKey input: each hwKey's **running +maximum** over run dates — the sequence of dates whose +interpolated read at the target beat every date before it. That progression is +the section's subject. A sweep that failed to beat the incumbent is not a step, +because the fleet kept serving the config it already had, and the last rung of +the staircase is by construction the same all-time best the table reports +(`selectBestFromGroups` and `bestSoFarProgression` share one selection basis, so +the headline figure can never disagree with the plotted line). + +That is only defensible with two rules, both in `historical-best.ts`: + +- **Clamped reads are discarded, not clamped.** `interpolateForGPU` always + returns a value, clamping the target into each frontier's measured range. + Searching every date multiplies the chance of picking up such an edge read — + at target 20 tok/s/user, 63% of naive winners are clamped, which would credit + a sweep's peak throughput at an interactivity it never served. This section + therefore applies the no-extrapolation rule (the same one + `useInterpolatedTrendData` uses) and lists the hwKeys it consequently has no + read for, with their measured ranges. Chips are never silently dropped: at a + low or extreme target that empty list _is_ the finding. +- **Provenance travels with every number.** Each row renders its winning date + and links its run, because the run date stamped above the chart no longer + describes these numbers. A row whose winner beat the latest date is marked. + There is no per-row trust annotation in the data — `PURGED_RUNS` is + destructive so retracted runs cannot win, but a merely-superseded run can now + win permanently — so the run link is the audit trail. + +Precisions are pooled into one frontier per hwKey: the question is what the +chip's best config does, and precision is part of the config. + +### Two-stage memoization + +`groupHistoryByHwKeyAndDate` (rows → one sweep per hwKey per date) is separate +from `selectBestFromGroups` (read every frontier at the target) so moving the +interactivity slider re-reads without rebuilding ~790 frontiers, each of which +costs ten splines. For the same reason the hook computes **every** hwKey and the +component filters by `visibleHwKeys` for display — filtering in the data layer +would rebuild every frontier on each legend toggle. + +### `view=calculator` on the history route + +The history response is ~8.6 MB per model. `useHistoricalBest` gates the fetch +on a facility power budget being set, and requests `view=calculator` to trim it +to the calculator's metric allowlist (~24% smaller). + +**That trim must stay opt-in.** `CALCULATOR_METRIC_KEYS` excludes the +measured-power metrics Historical Trends plots, so applying it to every history +response would silently blank those charts. `measured-power-overlay.cy.ts` is +the regression guard. + +### Lifecycle math (`lifecycle.ts`) + +Pure and React-free, like `fleet.ts`. It takes a `ThroughputStep[]` — one entry +per measured improvement — and turns it into a revenue staircase over calendar +time against a flat cost line: + +``` +revenue │ ,──── each config rolls out to its own numbers + │ ,───╯ + │ ,───╯ +────────┼──╱────────────────────────────── cost: flat, the racks + │ ╱ the first config rolls out from zero + └──────────────────────────────────▶ months since model release +``` + +An earlier iteration modelled a TTFI delay, a ramp to a flat plateau and a +decommissioning taper, with no measured steps at all. That collapsed each chip's +optimisation history into a single number held constant for years, so every chip +drew the same assumed shape. The steps are measured; only the ramp survived, and +it survived as an explicit user assumption. + +**A config is rolled out, not switched on.** So every config gets its own ramp: +when it lands, the fleet climbs from whatever it was already serving to that +config's numbers over `rampMonths`. The first config climbs from zero — nothing is +being served before it lands. + +A rollout starts from the level the previous one **actually reached**, not from the +previous config's nominal rate. With a ramp longer than the gap between sweeps that +matters: the second config picks up mid-climb instead of jumping, so the line stays +continuous no matter how the ramp length and the sweep cadence interact. + +Four conventions the numbers depend on: + +- **Cost is flat, including through every rollout.** It is `chips × $/chip/hr`, and + neither term moves when a config rolls out — racks bill from the moment they are + energised, not from the moment they are loaded. So the opening rollout is paid for + in full while it earns its way up from zero, which is what puts the first months + below the rule and gives payback its meaning. +- **The ramp is an assumption; the steps are not.** It defaults to half a month + (`c_ramp`) and moves in quarter-month increments; 0 means every config takes + effect instantly. +- **Markers sit at the release instant** — the foot of each rollout, not its top — + so every dot on the chart is a sweep the user can open. +- **Interrupts are an availability haircut, not drawn events.** A 24-day MTBI + over a multi-year window is thousands of interruptions; at any sane chart width + each is far under a pixel, so drawing them yields aliasing noise rather than + information. They scale revenue via `mtbi / (mtbi + recovery)`. + +A blank or zero MTBI means "no interruptions modelled", not "always down" — the +input is an optional refinement and a hostile default would be worse than none. + +### Anchoring and the x axis + +Time is measured from the model's release date, from `MODEL_RELEASE_DATES` in +`@semianalysisai/inferencex-constants` (DeepSeek-V4-Pro: 2026-04-24, confirmed +against `/api/v1/availability`, whose first `dsv4` row is that date). A model with +no release date on file falls back to its earliest measured run, and the caption +states which anchor is in use. A chip's line starts at its **own** first measured +run, not at the anchor: before that there is no data, so there is no line. + +The horizon defaults to the measured window rather than a fixed number of months, +and stops re-seeding once the user edits it. A fixed 60-month default left ~80% of +the chart as flat tail extrapolated from the last sweep. + +The time scale passes `nice: false`. `buildScale` niceing is on by default +(`scale-builders.ts:35`), which rounds a multi-year span out to whole years and +leaves dead space before the release date and after the horizon; both ends of this +domain are meaningful. + +The zero rule is a `type:'custom'` layer, and it removes `.lifecycle-zero-rule` +before appending. The chart re-renders into the same zoom group, so appending +unconditionally leaves a stale rule and a duplicate "break-even" label behind at +the previous y-scale on every data change. + +### Y-axis selector (margin / revenue / cumulative revenue) + +`c_ly` switches the chart between margin/day (default), revenue/day, and cumulative +revenue. The table is unchanged — it already carries both rates and cumulative +margin — so this only affects the plot. + +Three things move with it, and they all matter for honesty: + +- **The break-even rule is drawn only for margin.** Zero is break-even on a margin + axis; on a revenue axis it is just the bottom of the scale, and each chip breaks + even at its **own** cost line rather than at zero revenue. Labelling the zero + gridline "break-even" there would be false, so `renderZeroRule` returns early. + Per-chip cost rules would be the honest equivalent but add one horizontal line + per series, which is why they are not drawn. Cumulative revenue is the same case: + a running total that starts at zero and only grows. +- **The readout's value column follows the plotted quantity**, with cumulative + margin beside it, so it never depends on the axis to be read. +- **Cumulative revenue is accumulated, not derived.** `LifecyclePoint` carries a + second running total, `cumulativeRevenue`, integrated by the same trapezoid over + the same intervals as `cumulative` (cumulative margin). It is tempting to + reconstruct it as `cumulative + costPerDay × elapsed`, and that identity holds + only when every interval carries the same flat cost — which is true today but is + a property of the assumptions, not of the shape. Accumulating both keeps the two + curves exactly one flat cost apart by construction, which is what makes them + comparable, and `isCumulative(metric)` is what the axis formatting and the + break-even suppression key off rather than a per-metric special case. + +Revenue mode is for comparing rollout _shapes_ across chips whose costs differ a +lot; a chip sitting higher there does not mean it is more profitable. Cumulative +revenue compounds that caveat — a chip with the largest area under its revenue +curve may still never have covered its cost. The control's tooltip says so. + +### One line layer, interpolated linearly + +Because every riser is now a sampled rollout curve and every plateau is flat, the +shape is already in the points: `curveLinear` draws exactly what was computed. A +step curve would flatten the rollouts back into stairs, and a spline would overshoot +on the near-vertical ones (a zero-length ramp emits two samples at the same month to +force a true vertical). + +An intermediate version drew the ramp and the steps as two `line` layers with +different curves. **That does not work, and the reason is worth recording:** +`useD3ChartRenderer` renders every layer into one shared `renderGroup`, and +`renderLines` does a keyed join on `.line-path`. Two line layers with the same +series keys join against each other's paths, and the second silently overwrites the +first — one of the two segments just vanishes. Before adding a second layer of an +existing type to a `D3Chart`, remember that the shared render group makes layer keys +non-isolating; a `type:'custom'` layer with its own class is the way out. + +### End-of-line chip labels + +Each line is named by its chip at its right end, in the line's own colour, so a +series can be identified without crossing back to the sidebar legend. Two details +matter: + +- The labels sit **past the plot width** (`x = width + 6`, with `CHART_MARGIN.right` + widened to hold them), and `clipContent` defaults to true — layers render into a + clipped zoom group, which would erase them entirely. They are drawn into + `ctx.layout.g`, the unclipped parent, instead. +- End values cluster, so labels are placed by a greedy pass that pushes each label + down to keep `LABEL_MIN_GAP`, then slides the whole block up if it has run off the + bottom. A line ending at the very top or (after a zoom) outside the visible range + is pinned to the nearest edge rather than allowed to escape the SVG. + +Under an x zoom or pan, a line's last data point can sit well outside the plot, so +the label is placed at the value where the line **crosses the visible right edge** +(interpolated on the straddling segment), not at its final value. A line that ends +inside the plot is labelled at its own end instead, and one panned entirely out of +view is not labelled at all. Getting this wrong is not cosmetic: the two chips that +are still climbing at the horizon read several million dollars a day lower at a +zoomed-in edge than at their plateau. + +Both the labels and the break-even rule are `type:'custom'` layers, and **both +declare `onZoom` as well as `render`**. A custom layer without `onZoom` stays pinned +to the base scales while the lines move, so a zoomed chart shows break-even — or a +chip's name — at the wrong height. This only matters because the chart passes +`zoom={{enabled: true, axes: 'x'}}`; it shipped for a while with the wrapper's +default caption promising shift+scroll zoom and no `zoom` prop at all, which made +both `onZoom` handlers dead code and the caption a lie. A Cypress test now asserts +the axis actually moves. Zoom is x-only on purpose: rescaling y would slide +break-even under the reader. + +The custom layers read the current metric and line data through a **ref**, not +through the closure they were created in. The zoom behaviour installed by a render +captures that render's callbacks, and d3's double-click reset is a transition, so its +trailing events can fire after a later render has already redrawn — a stale callback +then re-appends what the new render removed. Switch to the revenue axis during a +reset animation and the break-even rule used to come back and stay, because nothing +renders again afterwards. + +**Known limitation, in the shared renderer rather than here:** those same trailing +events also repaint the axes and grid from the scales captured when the transition +started, so changing the metric inside the ~750ms reset window leaves a stale y axis +until the next interaction. Fixing it means giving `useD3ChartRenderer` a +latest-context ref, which every zoomable chart shares; it is deliberately not done as +part of this section. The Cypress spec waits out the transition for this reason. + +### The hover readout: one rule, every chip + +Hovering anywhere in the plot draws a vertical rule and reads **every** plotted chip +at that instant into one popup; clicking freezes it, and clicking again releases it. +This replaced per-dot tooltips, which answered "what is this dot?" — the wrong +question for a chart whose lines are only interesting against each other. Comparing +two chips at one date meant hovering twice and holding the first number in your head. + +It is built on `TooltipConfig`'s `proximityHover` + `getDataX`, so the shared +renderer's overlay rect owns hover for the plot area. Three consequences worth +knowing: + +- **The hover positions are a uniform grid** (`HOVER_SLICE_COUNT` slices across the + window), not the sample points. Sample density varies by two orders of magnitude + along a line — 24 samples across each rollout ramp, one across a multi-month + plateau — so snapping to samples makes the cursor crawl through a riser and then + jump a year. Values come from linear interpolation between the bracketing samples, + which is exactly what `curveLinear` draws, so the number always matches the pixel. + A chip has no row before its own first measurement rather than a fabricated one. +- **Steps match by proximity, not equality** (`STEP_MATCH_SLICES`). A slice is a + fraction of a pixel, so a step's instant essentially never coincides with one; + matching exactly would make the run links unreachable by pointing at the dot, + which is the only place a reader looks for them. +- **A step's config detail is frozen-only.** Hovering is for scanning, and the popup + keeps one shape wherever the cursor is; a step block appearing as the cursor + crosses a dot reflows the rows under the reader and buries the comparison they + came for. A click means "tell me about this instant", which is where the config, + its gain over the first run, and its run links belong. +- **The dots no longer own hover.** The overlay rect sits above them, so a Cypress + test cannot click a `.dot-group` — it clicks the overlay at the dot's x instead. + +Freeze is a toggle, which the shared overlay handler cannot express on its own: it +pins on every click, so a second click would just re-pin. A capture-phase listener on +the SVG records whether the readout was already frozen _before_ the pin lands, and +`onPointClick` dismisses when it was. Reading the pinned state afterwards would +always say "frozen". + +Two testing notes, both about the portal tooltip rather than this chart. It is keyed +`data-chart-tooltip=""` so a page with several charts can be asserted +per-chart. And `:visible` does not work on an **unfrozen** readout: Cypress treats a +fixed-position element as hidden when something else answers `elementFromPoint` at +its centre, and an unfrozen readout sets `pointer-events: none`, so the plot answers +instead. Assert `display` instead. Relatedly, any toast nudge re-renders the chart +tree and rewrites the portal's inline styles, wiping an open hover readout — app-wide +behaviour, which the spec sidesteps by suppressing the timed nudge. + +Every rung links its run in the frozen readout, not just the opening and closing +sweeps the table links. Intermediate rungs are exactly where a superseded-but-never- +purged anomalous run would sit, so leaving them unlinked would have removed the audit +trail at the one place the trust argument depends on it. + +The rule's group is `.lower()`ed on every draw. `renderLines` keeps its paths across +re-renders through a data join while this layer removes and re-appends, so without +that the dashed rule climbs above the lines after the first repaint. + +### Known visual limitation: the palette is not colourblind-safe here + +An adversarial review ran the series colours through a CVD validator and the result +is worth recording rather than hiding. With the default fleet the chart draws four +NVIDIA chips plus one AMD chip, which under the site-wide vendor convention +(`dynamic-colors.ts` gives NVIDIA the green hue zone and AMD the red one) means four +greens and a red. Seven of ten adjacent pairs fail: worst are MI355X↔B200 at ΔE 1.2 +for protanopia — indistinguishable, and those two lines cross mid-plot — and +GB200↔GB300 at ΔE 7.7 for normal vision. + +This is a property of the vendor convention every chart and legend shares, not of +this section, so it is deliberately **not** worked around locally: giving this one +chart its own palette would desync it from the sidebar legend and from every other +calculator chart. Two things are unavailable as local mitigations too — marker shape +already encodes _precision_ app-wide and there are only four shapes for five-plus +chips, and `POINT_SIZE` is a shared constant with no per-layer override, so the dots +cannot be enlarged to the 8px spec here alone. What the chart does carry is a +non-colour decoder: every line is named at its right end. Fixing the palette +properly means re-stepping the vendor zones in `dynamic-colors.ts` for all charts. + +Related, and also left alone: the rollout ramps and the post-last-sweep hold are +assumptions drawn in the same stroke as the measured window, with only the step dots +marking measurements. The ramp input and the notes say so in words; distinguishing +them visually was considered and declined to keep one line per chip readable. + +### Token price defaults to break-even + +Break-even is per-chip, so a single global price input needs one anchor: the +**cheapest visible chip's** break-even, i.e. the competitive floor at which that +chip earns exactly nothing and everything pricier is underwater. It re-seeds when +the tier, token type or target changes, and stops the moment the user edits the +field (or arrives with `c_price` in the URL); a reset link restores it. The +figure comes from the TCO model — everything above that line is the user's +assumption, and the section says so. + +It is derived as `costPerHour / fleetTokPerSec` for that fleet, so the plotted +margin is exactly zero at the default — for that chip, at its **latest** config. +Under the default price every other series therefore sits below the rule, and each +one's own earlier steps sit lower still. That is the competitive floor being read +correctly, not a scaling bug; raising the price lifts the whole fleet. + +That is deliberately **not** the `$/M tok` the calculator's cost bars show for +the same config, and the reason is worth recording because it is a property of +the existing calculator rather than of this section. + +`interpolateForGPU` splines every metric independently. Per point, +`cost = $/GPU-hr / (tput x 3600)` — cost is a **convex** function of throughput — +so splining the cost values directly lands above the curve implied by splining +throughput (Jensen), by a gap that widens with knot spacing. Measured over the +captured fixture across every frontier and every date: + +| target sits… | n | median ratio | p95 | max | share biased high | +| -------------------- | --- | ------------ | ------ | ------- | ----------------- | +| exactly on a knot | 470 | 1.0000 | 1.0000 | 1.0000 | 0% | +| midway between knots | 307 | 1.0568 | 3.9661 | 25.3237 | 73.6% | + +(ratio = splined `result.cost` / cost derived from the splined throughput.) + +Agreement at every knot and one-sided divergence between them identifies this +as interpolation bias, not a modelling disagreement. + +Scored against the oracle, the splined read is simply **less accurate**. +`/inference` plots cost only at measured points, as `specs.costh / tokensPerHour` +(`lib/chart-utils.ts:380`, `roof: false`). Holding out each interior frontier +knot in turn, rebuilding the frontier from the rest and predicting the held-out +point's real cost: + +| method | mean err | p50 | p90 | p99 | closer to oracle | +| ------------------------------- | -------- | ----- | ------ | ------- | ---------------- | +| splined `result.cost` | 162.8% | 86.3% | 528.1% | 1402.1% | 36 / 144 | +| derived `k / interpolated tput` | 42.2% | 23.0% | 72.0% | 245.3% | **108 / 144** | + +The structural argument is stronger still. That oracle construction makes +`cost x throughput = $/GPU-hr` an identity at every real point. Splining the two +independently breaks it: the calculator's pair violates it by a median 71.6% and +up to 2026%, i.e. it reports an operating point no real config could occupy. +Deriving cost from the interpolated throughput satisfies the identity by +construction. + +This section therefore derives break-even from throughput, which also keeps it +consistent with the fleet planner directly above (fleet cost is +`gpus x $/GPU-hr`, a constant with no interpolation in it, so no other price can +zero the plotted margin). + +**Follow-up worth taking.** The same defect affects the calculator's own cost +bars and table, not just this section: `interpolateForGPU` should build `cost` +from its interpolated throughput rather than splining the cost values. That is a +small change, but it lives in a Python-synced function (`iso_interactivity.py`) +and would move published cost numbers, so it needs its own change. Until then the +price tooltip states the direction of the divergence. + +### Input and output are priced separately + +Revenue is `input x inputPrice + output x outputPrice`, not one blended price +against one throughput. The two streams are wildly unequal and the split is +measured, not assumed — `input_tput_per_gpu` and `output_tput_per_gpu` are +already on every row, the same figures `/inference` plots: + +| scenario | input : output throughput | +| -------------- | ---------------------------------------------------------------------------- | +| fixed 8k/1k | 8.0 : 1 (p10 7.9, p90 8.0 — it is the sequence shape, not per-config spread) | +| agentic traces | ~130 : 1 (p10 114, p90 145) | + +At a 4x output premium on 8k/1k, output is a ninth of the tokens and a third of +the revenue, so a blended price is not a rounding difference. Three consequences: + +- **Revenue no longer depends on the Input/Output/Total cost-type selector.** The + fleet sells everything it produces; the selector still drives the cost matrix, + the bar chart, and which config wins the staircase. This also retires a live + trap — on `costType=output` the old model counted revenue from a ninth of the + tokens while cost covered the whole rack. +- **Break-even is a line, not a point.** Any (input, output) pair on it zeroes the + margin. The section resolves that by fixing the ratio currently in the two + fields and solving the input price against `effectiveTokPerSec` — + `billableInput + multiple x output` — so a reset scales both and lands back on + the line at the ratio the user chose. Unseeded, the pair starts at 4x, which is + roughly where the major vendors price (DeepSeek publishes $0.27 / $1.10). +- **Gain stays a token multiple.** `improvementFactor` counts both streams and no + price, so it remains a statement about the hardware. The fixture that pins this + grows the streams _disproportionately_ — input flat, output tripled — because a + proportional one cannot tell "counted both" from "counted input only". + +Cached input tokens are discounted into the input stream before pricing (see +below), so the input price is the fresh-token price. + +`c_price` is the input price — unchanged, so a link written before the split +still seeds the field it always seeded — and `c_oprice` the output price. When a +legacy link has `c_price` alone, the missing output price is derived at the +default 4x ratio without overwriting the explicit input price; the symmetric +rule completes a manually authored output-only link. + +### The disaggregated token-split trap + +`input_tput_per_gpu` is per **prefill** chip, `output_tput_per_gpu` per **decode** chip, +and `tput_per_gpu` per chip **overall**. Three denominators. On an aggregated run they +coincide and the two rates sum to the total on all 937 production rows. On a +disaggregated run they do not, and the gap is large: across 308 disagg rows the two +rates sum to between 1.00× and **16.11×** the total, tracking +`(prefill + decode) × (isl/prefill + osl/decode) / (isl + osl)` to four decimals. + +Pricing the two streams off those rates therefore billed a disaggregated config for +tokens its chips never served. The visible symptom was a **dip**: MI355X on +DeepSeek-V4-Pro 8k/1k stepped from `mori-sglang` (disaggregated, 2026-07-22) to +`atom_mtp` (aggregated, 2026-07-28) with a _higher_ total tok/s/MW — a legitimate +best-so-far rung — and the margin line fell, because the outgoing config had been +credited with ~2× its tokens. Eleven such dips existed across 215 step transitions on +eight interactivity targets; every one was a handover where the prefill:decode split +changed or disappeared. + +`tput_per_gpu` itself is **not** affected: it is per chip overall for both kinds, which +reproduces exactly from the absolute rates and the chip counts — on the first disagg row +in production, `(54.521×8 + 6.857×8) / 16 = 30.68886 = tput_per_gpu` to five decimals. +So the ranking, the sizing and the cost line were always comparable across an +aggregated→disaggregated handover; it was only the two per-stream rates that were not. +The section's disagg note used to claim otherwise and has been corrected. + +One place had the same defect and is fixed alongside: **concurrent users**, which divided +the per-_decode_-chip output rate into a whole-fleet chip count and so overstated the +stream count by `(prefill + decode) / decode` — median 2×, up to 7× in production history. +It now uses `outputTokPerChip`, which re-bases onto chips overall and is a bit-for-bit +no-op wherever no share is known. + +What remains an assumption rather than a measurement: the winning config's prefill:decode +ratio is taken to apply across the whole fleet from the moment it rolls out. A switch from +aggregated to disaggregated serving is a redeployment, and the ramp window is what stands +in for it. Chip counts also will not generally divide into the config's ratio, a remainder +bounded by one config-unit that the model ignores. + +The fix is to use a **share of the per-chip total** wherever a token-type figure is +needed, never the two rates directly, with the share recovered as: + +1. `input / (input + output)` when those agree with `tput_per_gpu` to within 1% — every + aggregated row, where the measurement is self-consistent and is the truth; +2. otherwise `isl / (isl + osl)` — for a fixed sequence the mix is the sequence shape, + and no config can change it; +3. otherwise the run's own `total_prompt_tokens : total_generation_tokens` — agentic + traces, which have no ISL/OSL; +4. otherwise nothing is charged as input, which understates revenue rather than + inventing a mix. + +After the fix: 0 dips across the same 215 transitions, and the streams sum to the rate +the fleet was sized and costed on by construction. + +**Selection needed the same correction, and it is a second bug.** Fixing revenue alone +left the _ranking_ — which config wins each step — reading `inputTpPerMw` / +`outputTpPerMw` raw, so a disaggregated config could win a rung purely for having been +divided by fewer chips. Measured across seven interactivity targets: on total pricing the +basis makes no difference (46 of 46 chip-winners identical, confirming `tput_per_gpu` is +genuinely per total chip), but on **output** pricing 10 of 46 winners change and 5 are +handed to a disagg config a comparable basis rejects; on **input**, 17 of 46 change. +`mi355x_mooncake-atom@2026-06-12` won the output ranking at 25 and 30 tok/s/user on the +raw basis; on a comparable one `mi355x_atom_mtp@2026-07-28` wins. + +So `getComparableTpPerMwForType` splits `tpPerMw` by the same share, and the +lifecycle selection (`useHistoricalBest`) uses it. +For an aggregated config it is an exact identity — the rates already sum to the total, so +`tpPerMw × share` _is_ `inputTpPerMw` — so only disaggregated rows move. + +The bar chart and cost matrix above still show the raw per-prefill / per-decode figures +with their existing disclaimer, deliberately: a chart the reader is told how to interpret +can carry a footnote, and changing the headline chart's numbers is a separate decision. +What is no longer acceptable is a _ranking_ that silently picks a winner on that basis. A corollary worth stating: on a fixed +sequence the input/output split changes the **level** and the break-even price, but it +cannot reorder chips, because the mix is the same for all of them. On agentic traces the +mix is measured per config, so there it can. + +### Margin and revenue per megawatt + +Two y-axis metrics normalize the daily rates by provisioned power: +`marginPerMw` (`c_ly=marginPerMw`) plots `revenue − TCO` in `$/MW/day`, while +`revenuePerMw` (`c_ly=revenuePerMw`) plots revenue before TCO in the same unit. +Two things about them are worth being explicit, because a reader could reasonably +expect more than they deliver: + +- **They re-rank almost nothing.** Every chip in this section is sized to the same + power budget, so each per-MW rate is its unnormalized rate divided by very nearly + the same number for every series. The only spread comes from how completely each + chip's power density fills the budget. What the metrics buy is a figure that does + not move when the budget does — the unit a power-constrained plan is written in — + not a new ordering. +- **The denominator is provisioned power, not the budget.** `chips × kW/chip ÷ 1000`, + because chip counts are whole and the remainder of the budget is stranded. Using + the typed budget would credit each chip with power it never provisioned. An + unusable figure (no registered power, a budget too small for one chip) leaves the + metrics at zero rather than dividing by it. + +Because the rescale is by a positive constant, zero on `marginPerMw` is still +break-even. That is why `isBreakEvenAnchored` includes it and the chart retains +the dashed rule. `revenuePerMw`, like unnormalized revenue, does not draw the +rule: zero revenue is not break-even for a fleet with non-zero cost. + +### Agentic traces, and cached input tokens + +The section refused Agentic Traces until it was shown that the refusal's stated +reason — "run history is keyed by input/output sequence length" — was no longer +true. `/api/v1/benchmarks/history` takes `benchmarkType=agentic_traces` and drops +the ISL/OSL filter server-side; `buildGpuGroups` already maps agentic rows and +already takes a percentile. Only the fetch in `useHistoricalBest` had to change. + +What did need real work is the revenue term. Measured against production history +(`inferencex.semianalysis.com`, DeepSeek-V4-Pro and GLM-5.2, August 2026): + +| | agentic | fixed 8k/1k | +| -------------------------------- | ------------------------------ | ------------------------------ | +| median input:output token ratio | **133:1** | ~8:1 | +| median GPU prefix-cache hit rate | **0.92** (min 0.005, p90 0.96) | metric **absent on every row** | +| rows carrying a hit rate | 217/223 DSV4, 45/45 GLM-5.2 | 0/1245, 0/541 | + +Charging the input price against the raw input rate therefore bills ~13.5k tok/s +of which ~92% are cache _reads_, which providers charge a fraction of the fresh +input rate for — overstating agentic margin by close to an order of magnitude. At +8:1 with no prefix reuse the distinction is immaterial; at 133:1 with 92% hits it +is the dominant term. + +So the fleet is sized on the physical rate and the **input stream is discounted +before it is priced**: + +``` +billableInputTokPerSec = inputTput × (1 − clamp01(cacheHitRate) × (1 − cacheReadRatio)) +``` + +Selling `hit` of the stream at `ratio` of the price is exactly the same revenue as +selling `hit × ratio` of it at full price, which is why one factor on the rate +does the whole job and the input price stays the fresh-token price. + +Three properties are worth stating, because they are what make this safe: + +- **It is a subtraction, not a recomposition.** Writing it as + `output + billableInput` would re-derive the total from two independently + splined series and drift against it. As a subtraction the result is _exactly_ + `base` whenever there is nothing to discount. +- **Fixed sequences are unaffected by construction.** They carry no cache metric + on any row, so `cacheHitRate` is `undefined` and the term is zero — not "close + to zero", identical. That is asserted with `toBe`, not `toBeCloseTo`. +- **A partly-measured frontier opts out.** `interpolateForGPU` splines the rate + only when _every_ frontier point carries one; substituting 0 for the missing + points would invent a dip in the cached fraction and overstate the billable + rate. Opting out bills every input token at full price instead — wrong in the + direction that understates margin. + +`cacheReadRatio` is the one user input here, labelled `Cached input (% of price)` +and defaulting to 10% (the ratio DeepSeek and Anthropic both publish). The cached +_fraction_ it multiplies is measured per config, not assumed. The control is +hidden for fixed sequences rather than shown as a no-op. There is no circularity +with the break-even seed: billable throughput depends only on the ratio, and +break-even then solves the price given that throughput. + +**What counts as cached.** Three tiers are reported and they do not all stack, so +the rule is conditional rather than a sum. Measured across 326 production agentic +rows, each checked against its own `theoretical_cache_hit_rate` ceiling: + +| row shape | rows | `gpu+ext` > ceiling | `gpu+ext+cpu` > ceiling | +| ---------------------------- | ---- | ------------------- | ----------------------- | +| external absent, CPU absent | 92 | 8 | 8 | +| external absent, **CPU > 0** | 26 | 0 | **0** | +| external > 0, CPU > 0 | 106 | 1 | **56** | +| external 0, CPU 0 | 96 | 0 | 0 | + +The middle two rows carry the decision. Where an external rate is reported, +adding CPU on top breaches the ceiling 56 times out of 106 — the router-side +external figure already contains the offload tier, so summing double-counts it. +Where no external rate is reported the CPU tier is real and disjoint: 0 breaches +across all 26, every one of them an offload-on row. Hence `cacheHitRateOf` adds +external when present and CPU only in its absence. + +Dropping the CPU tier outright, which this did until 2026-08-19, understated the +cached share on those 26 rows by a median 5.54pp (p90 39.7pp, max 76.9pp — the +median CPU rate is 0.055 but the tail is long). An understated cached share bills +more input at the fresh price, so it **overstates** revenue: on those rows the +input revenue leg was too high by a median 27%, p90 67%. That is the opposite +direction from the all-or-nothing rule below, which deliberately errs +conservative, and the reason this was worth a branch. Fleet-wide the median hit +rate moves only 0.9255 → 0.9279, because 294 of 320 rows are unaffected. + +Two honest caveats on the rule. A reported external `0` is treated as a +measurement that suppresses CPU rather than as an absence, on the reasoning that a +router reporting zero external hits has still accounted for the offload tier — +but **no production row currently has external `0` alongside a non-zero CPU rate**, +so that branch is a deliberate choice about an unobserved shape, covered by a test +rather than by data. Separately, `gpu` alone already breaches the ceiling on 8 +rows that report neither of the other tiers; that is a data-quality question about +those rows, partly masked by the `[0,1]` clamp, and this rule neither causes nor +fixes it. + +The clamp to `[0,1]` is not decorative — the GPU figure alone reaches 1.185 on +rows without an external one. + +**Interpolating the cached fraction is safe, and this is the evidence.** The rate +is splined along the frontier like any other metric, which raises the fair +objection that two adjacent frontier points can describe different serving +regimes. Measured by leave-one-out over 141 interior points of 26 production +frontiers — drop the point, read its interactivity from its neighbours, compare to +what was measured there: + +| metric | median | p90 | max | +| -------------------------------------- | ------ | ------ | ------- | +| cache hit rate, absolute error | 0.0076 | 0.0631 | 0.4890 | +| throughput, relative error _(control)_ | 8.60% | 34.85% | 245.79% | + +At the median the hit rate is off by 0.8% relative, against 8.6% for the +throughput the whole calculator already rests on — so the cached fraction is about +an order of magnitude better behaved than the number nobody questions. Adjacent +frontier points differ by under one percentage point at the median (p90 0.079, +max 0.510). The wide 0.005–0.96 spread quoted above is across _all_ of a chip's +configs, not along a single Pareto frontier, which is what the spline walks. + +**A limitation the UI discloses.** Agentic run history is roughly a week deep: +DSV4 has 7 run dates with 4 of 19 configs measured on more than one; GLM-5.2 has +4 dates and **0 of 9** configs on more than one. Fixed 8k/1k has 61 dates over +3.7 months with 30 of 45 configs on several. Since this section plots a +best-so-far staircase, agentic lines are often a single rung. When no visible +chip has two, the section says so — a flat line here is a gap in the history, not +a finding about the hardware. This resolves itself as runs accumulate. + +### Overlay exemption + +This section is official-only, and unlike the fleet planner that is not a policy +choice: unofficial runs are not ingested, so `/api/v1/benchmarks/history` cannot +serve them. Taking AGENTS.md's documented exemption; the section states the +exclusion in its own note rather than leaving a silent gap. + +### What is still not apples-to-apples, and why it was left alone + +The bar chart and the cost matrix still show +`input_tput_per_gpu` / `output_tput_per_gpu` on their reported per-prefill / per-decode +basis, and their `$/M tok` with them. On the Input and Output token types a disaggregated +config therefore reads **faster per chip and cheaper per token than it is** — a median 2×, +up to 18× on input and 7× on output across run history. Total-token figures are unaffected. + +That was a deliberate decision not to move published numbers, taken after establishing +that the two cannot be fixed independently: **per-token-type cost is not stored data.** +`interpolateForGPU` recovers the constant `$/chip-hr × 1e6/3600` from the points and +re-derives cost as `rate ÷ throughput` — verified on all 37 disaggregated sweeps in the +fixture, where the rate is recovered every time and `cost × throughput` holds to 1e-6 on +the interpolated result for all three token types. So: + +- re-basing the point's `costhi` has no effect, because `costInput` is recomputed from + `inputTputValue` downstream; +- re-basing the divisor instead would break the identity `recoverReciprocalNumerator` + licenses, which `maxInteractivityAtCost` depends on — the cost-cap card would answer + against a different cost curve than the bars, the one thing its comment says must never + happen; +- and dividing the displayed `$/chip-hr` by the displayed throughput would stop giving the + displayed cost, for disaggregated rows only. + +Fixing cost alone is therefore not available. The choice is all of it or none of it, and +the notes now carry the correction instead: which figures are affected (Input and Output, +never Total), in which direction (flattering), by how much, and that the Fleet Lifecycle +section next to them is deliberately on a different basis. + +### Chart and table are tabs, with the figure's own header + +The section renders as a `
` in the same shape as the bar chart above it and +the `/inference` charts: a `ChartButtons` row carrying a Chart/Table `SegmentedToggle` +and the PNG/CSV export menu, and a caption with the title plus the provenance a reader +needs — model, scenario, target interactivity, power budget, source. The caption is +passed to the chart as its `caption` and rendered as a `
` in the table view, +so switching tabs never loses the heading. + +PNG export is disabled on the table tab: a picture of a paginated HTML table is not an +artefact anyone wants. The CSV is the export for that view, and it carries the +assumptions in its notes preamble — a CSV read six months later cannot be reconstructed +from the rows alone. + +The table's search box is off (`searchable={false}` on `DataTable`). It is one row per +chip, and every chip is already named in the legend and on the chart; a search field +over five rows is furniture. The prop is new and defaults to true, so no other table +changed. + +### URL params + +`c_price`, `c_life` (horizon), `c_ramp`, `c_cache` (cached-input price, % — agentic +only), `c_mtbi`, `c_rec`, `c_ly` (y-axis metric — +`margin` | `marginPerMw` | `revenue` | `revenuePerMw` | `cumulativeRevenue`, parsed +against that allowlist so a stale or hand-edited link falls back to `margin` rather +than seeding an unknown metric). +The first two default to +`''` in `PARAM_DEFAULTS` because their real defaults are derived, not constant — see +the comment there. The MW budget is `c_mw`, defaults to 10 MW, and is shared by +the calculator and Fleet Lifecycle page so a budget set on either seeds the +other. diff --git a/docs/testing.md b/docs/testing.md index dfdcd1224..3303b8607 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -3,16 +3,18 @@ ## Structure **Unit tests**: Vitest, colocated with source (`*.test.ts` next to `*.ts`). Config: `packages/app/vitest.config.ts`. -**E2E tests**: Cypress, in `packages/app/cypress/e2e/`. Config: `packages/app/cypress.config.ts`. +**Component tests**: Cypress, in `packages/app/cypress/component/`. Use these for isolated UI behavior with controlled providers, routes, and API intercepts. +**Integration tests**: Cypress, in `packages/app/cypress/e2e/`. Use these for page routing, browser history, API fixture mode, and cross-component workflows. Config: `packages/app/cypress.config.ts`. ## Requirements (Mandatory) Enforced by `@pr-claude` — missing/low-quality tests are flagged 🔴 BLOCKING. 1. New utility functions → colocated unit test -2. New UI features → E2E test in `cypress/e2e/.cy.ts` -3. Bug fixes → regression test reproducing the bug -4. Run `bun run test:unit` and the local smoke suite, `bun run test:e2e`, before considering a task complete. The full E2E suite runs in CI and is available locally as `bun run test:e2e:full`. +2. Component-local UI behavior → Cypress component test in `cypress/component/` +3. Route, navigation, or cross-component behavior → Cypress integration test in `cypress/e2e/` +4. Bug fixes → regression test at the narrowest layer that reproduces the bug +5. Run `bun run test:unit` and the local smoke suite, `bun run test:e2e`, before considering a task complete. The full E2E suite runs in CI and is available locally as `bun run test:e2e:full`. ## Pre-commit Checklist @@ -29,6 +31,26 @@ The local `bun run test:e2e` command is a curated smoke suite across the core pa The complete suite is `bun run test:e2e:full`. It runs all Cypress component and integration specs. GitHub Actions runs that same coverage as one component job plus four integration shards per browser, Chrome and Firefox. The CI workflow is the merge gate for the full E2E suite. +`packages/app/timings.json` is the committed `cypress-split` baseline. Its unit guard requires one positive timing for every integration spec and rejects removed entries. Regenerate the baseline from an observed full integration run when specs are added, removed, or materially rebalanced. + +With an `E2E_FIXTURES=1` app server running on port 3000, run this from the repository root: + +```bash +E2E_FIXTURES=1 \ + SPLIT=1 \ + SPLIT_INDEX1=1 \ + SPLIT_FILE=timings.json \ + SPLIT_OUTPUT_FILE=timings.json \ + SPLIT_SUMMARY=false \ + bun run --cwd packages/app test:e2e:integration +``` + +`SPLIT=1` intentionally runs every spec in one chunk so one process records the complete baseline. CI uses `SPLIT=4` only for parallel execution and writes each shard's timing output to a throwaway file. + +`E2E_FIXTURES=1` serves the committed API snapshots under `packages/app/cypress/fixtures/api/`. Refresh them with `bun run --cwd packages/app capture:fixtures`. The capture script updates `_manifest.json`, which records the fixture shape, byte length, checksum, source, and capture timestamp. The manifest guard rejects partial or hand-edited snapshots; benchmark history must also contain at least two dates so replay tests cannot silently skip their substantive path. + +Server-cache performance regressions must assert source-reader call counts for repeated identical inputs. Do not use elapsed-time thresholds: CI scheduling and cold-start variance make wall-clock assertions nondeterministic. + ## Quality Standards 1. **No tautological tests** — every test must verify a real transformation diff --git a/docs/typography.md b/docs/typography.md new file mode 100644 index 000000000..bae6881d4 --- /dev/null +++ b/docs/typography.md @@ -0,0 +1,49 @@ +# Typography + +How text is sized and spaced in the app, and the ratchet gate that keeps it that way. + +## Tokens + +Defined in the `@theme` block of `packages/app/src/app/globals.css`: + +| Token | Class | Value | Use for | +| ------------------------- | ----------------------- | ---------------- | ------------------------------------------------- | +| `--text-2xs` | `text-2xs` | 0.6875rem (11px) | Micro labels one step below `text-xs` | +| `--text-3xs` | `text-3xs` | 0.625rem (10px) | Densest labels: table meta, tooltips, log gutters | +| `--tracking-eyebrow` | `tracking-eyebrow` | 0.16em | Uppercase eyebrow/kicker labels | +| `--tracking-eyebrow-wide` | `tracking-eyebrow-wide` | 0.2em | Extra-wide uppercase labels | +| `--tracking-heading` | `tracking-heading` | -0.04em | Tightened display/hero headings | + +Rules of thumb: + +- Body and heading sizes use the standard Tailwind scale (`text-xs` … `text-6xl`). Do **not** add wrapper components for body text. +- Never write `text-[11px]`, `text-[0.65rem]`, `tracking-[0.16em]`, etc. — the gate (below) blocks new arbitrary font sizes and letter-spacing values. `tracking-[0.1em]` is exactly `tracking-widest`; use the built-in. +- The micro size tokens deliberately define **no paired line-height**: the arbitrary values they replaced set only `font-size` and inherited line-height, and dense tables depend on that. Don't add `--text-2xs--line-height` without re-checking those surfaces. +- The custom tracking tokens are registered with tailwind-merge in `src/lib/utils.ts` (`extendTailwindMerge`); if you add a token, add it there too or `cn()` will keep conflicting classes. + +## Components + +- `` (`src/components/ui/heading.tsx`) — cva-based visual heading levels (`display`, `page`, `section`, `card`, `label`) with the rendered tag chosen via `as`, so outline and visual size stay independent. New pages should use it instead of retyping size/weight/tracking recipes; existing headings migrate file-by-file as pages get touched — no sweep. +- `` (`src/components/ui/eyebrow.tsx`) — the uppercase mono kicker label (`tone="brand" | "muted"`, `wide` for 0.2em tracking). + +Blog/article typography is owned by `@tailwindcss/typography` (`prose`) and is out of scope — don't migrate MDX content styles to these tokens. + +## Chart text + +Chart font sizes live in TypeScript, not CSS variables: `CHART_TYPE` in `src/lib/d3-chart/typography.ts`, with the `px()` helper for `.attr('font-size', …)`. The PNG export path (`useChartExport`) serializes the chart with html-to-image, which cannot resolve `var(--*)`; its `resolveCssVarsForExport()` only bakes color-type attributes, so a CSS-variable font-size silently collapses in exports. The shared export font stacks (`CHART_FONT_SANS`, `CHART_FONT_MINECRAFT`) live in the same module. + +OG-image renderers (Satori: `opengraph-image.tsx`, `og-image-render.tsx`, `compare-og.tsx`) draw on a fixed canvas and intentionally hardcode display sizes — they are exempt from all of this. + +## The ratchet gate + +`bun run check:typography` (wired into CI's lint workflow and lefthook pre-commit) scans `packages/app/src` for: + +1. `arbitrary-text-size` — `text-[…]` in any `.ts`/`.tsx` (hex colors like `text-[#ed1c24]` are out of scope); +2. `arbitrary-tracking` — any `tracking-[…]`; +3. `chart-font-literal` — quoted `font-size` literals inside `src/lib/d3-chart/`. + +Per-file counts are compared against `packages/app/scripts/typography-allowlist.json`: + +- **A count above the allowlist (or a hit in an unlisted file) fails immediately** — fix it with a token, `CHART_TYPE`, or the standard scale. +- **A count below the allowlist also fails**, telling you to run `bun run check:typography --update` — so the allowlist only ever shrinks. It doubles as the migration burndown; there is no deadline, offenders migrate when their file is next touched. +- Matching logic lives in `src/lib/typography-gate.ts` (unit-tested); exemptions (OG renderers, tests, the gate itself) are defined there. diff --git a/lefthook.yml b/lefthook.yml index 09753abf7..af93e4507 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -11,3 +11,6 @@ pre-commit: - name: typecheck run: bun run typecheck + + - name: typography + run: bun run check:typography diff --git a/package.json b/package.json index 65db68a61..20d593638 100644 --- a/package.json +++ b/package.json @@ -4,7 +4,7 @@ "private": true, "repository": { "type": "git", - "url": "https://github.com/SemiAnalysisAI/inferencemax-app" + "url": "https://github.com/SemiAnalysisAI/InferenceX-app" }, "workspaces": [ "packages/*" @@ -24,6 +24,7 @@ "lint:fix": "oxlint --fix --no-error-on-unmatched-pattern", "fmt": "oxfmt --check --no-error-on-unmatched-pattern", "fmt:fix": "oxfmt --write --no-error-on-unmatched-pattern", + "check:typography": "bun packages/app/scripts/check-typography.ts", "test": "bun run --cwd packages/app test", "test:e2e": "bun run --cwd packages/app test:e2e", "test:e2e:full": "bun run --cwd packages/app test:e2e:full", @@ -43,19 +44,31 @@ "admin:db:migrate:collectivex": "bun run --cwd packages/db db:migrate:collectivex", "admin:db:apply-overrides": "bun run --cwd packages/db db:apply-overrides", "admin:db:backfill-full-response-interactivity": "bun run --cwd packages/db db:backfill-full-response-interactivity", + "admin:db:backfill-server-log-files": "bun run --cwd packages/db db:backfill-server-log-files", "admin:db:reset": "bun run --cwd packages/db db:reset", "admin:db:verify": "bun run --cwd packages/db db:verify" }, "devDependencies": { "@babel/core": "^8.0.1", - "@types/node": "^26.1.1", + "@types/node": "^26.2.0", "is-ci": "^4.1.0", "lefthook": "^2.1.10", - "oxfmt": "^0.60.0", - "oxlint": "^1.75.0", + "oxfmt": "^0.63.0", + "oxlint": "^1.78.0", "rimraf": "^6.1.3", "typescript": "^6.0.3", - "vite": "^8.1.5" + "vite": "^8.2.1" + }, + "overrides": { + "brace-expansion": "5.0.9", + "dompurify": "3.4.13", + "fast-uri": "4.1.2", + "hono": "4.13.2", + "ip-address": "10.5.0", + "js-yaml": "3.15.1", + "jsdom": "30.0.1", + "nanoid": "3.3.18", + "undici": "8.10.0" }, "engines": { "bun": ">=1.3.14", diff --git a/packages/app/content/blog/agentic-benchmark-agent-benchmark-guide.mdx b/packages/app/content/blog/agentic-benchmark-agent-benchmark-guide.mdx new file mode 100644 index 000000000..24a7dd667 --- /dev/null +++ b/packages/app/content/blog/agentic-benchmark-agent-benchmark-guide.mdx @@ -0,0 +1,289 @@ +--- +title: 'Agentic Benchmark for LLM Inference: Metrics and Methodology' +subtitle: 'How an agent benchmark replays long-context, multi-turn workloads to measure latency, throughput, cache behavior, and serving cost' +seoDescription: 'Agentic benchmark guide for LLM inference: learn how an agent benchmark measures latency, throughput, prefix caching, concurrency, and cost.' +date: '2026-08-19' +publishDate: '2026-08-19' +tags: + - benchmark + - agents + - agentic + - inference + - latency + - throughput +--- + +An **agentic benchmark for LLM inference** measures how a model-serving system behaves under the traffic generated by AI agents. It does not grade whether an agent fixed a bug or completed a workflow. It measures the infrastructure serving each model call: latency, throughput, cache reuse, concurrency, and cost. + +That distinction changes the workload. A conventional inference benchmark can send independent prompts with fixed input and output lengths. An agent benchmark must represent sessions in which context grows across turns, requests share long prefixes, tools insert pauses, and subagents create parallel branches. The next request often cannot start until an earlier request finishes. + +AgentX is InferenceX's open benchmark for this workload. It converts privacy-preserving shapes from opt-in coding-agent traces into deterministic AIPerf replays, then compares complete hardware and software serving configurations across concurrent agent clients. + + + Compare live AgentX inference results → + + +## What is an agentic inference benchmark? + +An agentic inference benchmark is a systems benchmark for multi-turn agent traffic. The system under test includes the accelerator, server topology, inference engine, model checkpoint, precision, parallelism strategy, cache configuration, and request scheduler. + +It answers questions such as: + +- How many concurrent agent sessions can the deployment serve? +- How long does each agent wait for its first token? +- How quickly does each active agent receive output tokens? +- How much output throughput does each chip deliver at a target latency? +- Do repeated prefixes hit HBM, host memory, or recompute paths? +- What does a million served tokens cost at a usable operating point? + +The word “benchmark” is also used for agent task evaluations, so the scope must be stated. The two test types answer different questions: + +| Question | Test type | Typical output | +| -------------------------------------------- | --------------------------- | --------------------------------------------------- | +| Can the agent complete the task correctly? | Agent task evaluation | Success rate, policy compliance, or grader score | +| Can the inference stack serve agent traffic? | Agentic inference benchmark | Latency, throughput, cache behavior, capacity, cost | + +AgentX covers the second row. Semantic task quality requires a separate evaluation. + +## Why fixed-sequence inference benchmarks miss agent traffic + +Fixed-sequence benchmarks remain useful for controlled comparisons. An 8K input and 1K output workload isolates a common request shape and makes configurations easy to reproduce. It does not reproduce a long-running agent session. + +| Workload property | Fixed-sequence inference | Agentic inference | +| ----------------- | ------------------------------------------- | ------------------------------------------------------ | +| Request shape | One fixed input and output length | Length changes on every turn | +| Arrival pattern | Independent requests at a set rate or batch | Dependent requests triggered by earlier completions | +| Prefix reuse | Optional or synthetic | Large shared prefixes grow through the session | +| Tool time | Usually absent | Uneven pauses between model calls | +| Parallelism | Request-level concurrency | Main-agent and subagent branches can overlap | +| Cache lifetime | Short and regular | Long-lived state competes for HBM and offload capacity | +| Useful result | Performance for one request distribution | Capacity and responsiveness for complete sessions | + +These differences can change which system wins. More HBM can keep long-lived KV cache resident. Better prefix caching can avoid reprocessing hundreds of thousands of prompt tokens. Scheduler and router costs that are small for independent prompts can grow with the number and length of live sessions. + +## An agent benchmark is a request graph + +An agent session is not a flat prompt list. Main-agent turns form a dependency chain. A turn can spawn one or more subagents, and the main chain may wait for those branches before it continues. Auxiliary calls can run without joining the main path. + +
+ +AgentX replays this graph in a [closed loop](/glossary/closed-loop-benchmark). Each client sends its next eligible request only after the required earlier work completes. Faster servers advance through a session sooner and create later work sooner. The instantaneous HTTP request count can exceed client concurrency when subagents fan out, then fall when branches finish. + +This is why AgentX concurrency means **live agent clients**, not a fixed request batch. A run at concurrency 50 can place more than 50 requests in flight if several clients have active subagents. + +## How AgentX builds a reproducible workload + +AgentX turns recorded session structure into a content-free replay in four stages. + +### 1. Capture request shape and timing + +An opt-in HTTP proxy records request arrival and completion times, input and output token counts, conversation IDs, and subagent IDs. It does not publish prompts, source code, tool arguments, or tool results. + +### 2. Preserve prefixes without preserving content + +Each input becomes a chain of session-scoped hashes in 64-token blocks. Matching block IDs retain the prefix relationships between turns. AIPerf replaces those blocks with deterministic synthetic coding and tool-use tokens before replay. + +### 3. Reconstruct dependencies + +Main-agent requests become a linear chain. Subagent requests become separate chains with spawn and join dependencies. Recorded inter-turn gaps reproduce the time spent using tools or waiting for other work. The replay keeps client-visible ordering and overlap without claiming access to provider-internal events. + +### 4. Replay the same scenario across systems + +A fixed seed controls session sampling, replay start points, and synthetic payloads. Each hardware and software configuration receives the same scenario definition, so differences in the resulting curve come from the serving system rather than a hand-written prompt batch. + +AgentX v1.0 contains 393 published sessions selected from opt-in Claude Code traces. Each selected session has at least 20 requests and no more than 10 concurrent subagents. + +| AgentX v1.0 property | Value | +| -------------------------------- | ------: | +| Published sessions | 393 | +| Median input tokens per request | 142,016 | +| Median output tokens per request | 444 | +| Sessions containing subagents | 44% | +| Full dataset maximum context | 1M | +| Context-limited dataset variant | 256K | + +
+ +The full variant retains contexts up to 1M tokens. The 256K variant removes requests above its context cap while preserving the timing and subagent overlap of the requests that remain. The [AgentX methodology](/agentx/methodology) publishes the dataset rules, figures, and source links. + +## Benchmark controls that make comparisons valid + +Long-running agent workloads are sensitive to starting state, cache contents, speculative decoding, and host memory. AgentX locks those variables with explicit controls. + +### Cache warmup and measurement window + +A fixed seed selects a starting point between 25% and 75% of each recorded session. A `max_tokens=1` primer materializes active main-agent and subagent prefixes. Each replay lane then completes 10 additional warmup requests before measurement starts. + +Only the following one-hour profiling window contributes to reported metrics. Each recycled replay receives a unique cache-bust marker so unrelated sessions cannot accumulate a shared prefix. + +### Synthetic payloads and speculative decoding + +Synthetic tokens preserve length and prefix structure, but they do not naturally reproduce draft-token acceptance. AgentX therefore uses an acceptance length measured from the coding category of [SPEED-Bench](https://arxiv.org/abs/2604.09557) for each model, speculative method, draft length, and thinking mode. The value is recorded in versioned golden files and applied through engine controls. + +This keeps the comparison focused on serving performance instead of accidental differences in how synthetic text interacts with a speculator. + +### DRAM and KV-cache offload + +Host-memory capacity can change whether a long prefix is reloaded or recomputed. Servers without a standardized DRAM configuration are capped at 3 TB. Standard systems such as GB200 NVL72, GB300 NVL72, and TPUv7 can use their installed capacity. A benchmark configuration can access only the host-memory share proportional to its GPU allocation. + +The replay format and dependency rules are documented in the [AIPerf WEKA trace guide](https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/weka-trace.md). + +## Metrics for an agentic benchmark + +One number cannot describe agent serving. Report throughput, latency, and interactivity together across a concurrency sweep. + +| Metric | What it reveals | +| --------------------------------- | --------------------------------------------------------------------- | +| Output throughput per chip | Generated tokens completed by each accelerator per second | +| P90 interactivity | Per-client generation speed derived from p90 inter-token latency | +| Time to first token (TTFT) | Delay before an agent starts receiving a response | +| Inter-token latency (ITL) | Time between streamed output tokens | +| Input throughput | Prompt tokens processed per second | +| Cache-source distribution | Prompt tokens served from HBM, host memory, or recomputation | +| Queue and in-flight request depth | Scheduler pressure as branches spawn and join | +| Cost per million tokens | Serving economics under a stated latency or interactivity requirement | + +Percentiles matter. A low mean TTFT can coexist with a poor p90 or p99 experience if a subset of agent branches waits behind long prefills. Aggregate output throughput can rise while each individual client receives tokens more slowly. + +## Read the full throughput-interactivity curve + +AgentX sweeps concurrent agent clients and plots the resulting operating points. Low concurrency usually favors per-client responsiveness. Higher concurrency can improve batching and aggregate throughput until queueing, memory pressure, or another bottleneck reduces interactivity. + +
+ +The useful result is the [Pareto frontier](/glossary/pareto-frontier), not the single point with maximum throughput. Hardware A may produce the highest tok/s/chip at low interactivity while hardware B leads at the response speed required by an interactive coding product. + +Closed-loop replay also means a faster configuration can progress farther through its sampled sessions during the same profiling hour. The exact request mix can vary slightly, especially at low concurrency. Results should include the full curve and enough concurrency to reduce sampling noise. + +## What agent traffic stresses in an inference stack + +Agentic inference exposes several bottlenecks that fixed prompt batches can understate. + +- **KV-cache capacity:** Long contexts and many live sessions keep large caches resident for longer. +- **Prefix-cache correctness:** A false hit produces wrong state; a missed hit repeats expensive prefill work. +- **Offload bandwidth:** Moving a long prefix from host memory can be faster than recomputing it, but transfer granularity and descriptor overhead matter. +- **Scheduler fairness:** Parallel subagents can raise throughput while an individual branch waits too long. +- **Router and frontend CPU:** Prefix matching, expiry tracking, serialization, and streamed response handling scale with live session state. +- **Long-context kernels:** Attention, sparse attention, normalization, and graph-capture choices can behave differently across varying sequence lengths. +- **Speculative decoding:** Acceptance controls and scheduler lifetime affect both throughput and interactivity. + +The [AgentX optimization tracker](/agentx/optimizations) documents concrete changes in vLLM, SGLang, TensorRT-LLM, ATOM, Dynamo, and KV-cache systems that were exposed by this workload. + +## How to compare AgentX results + +Use the same scenario contract on both sides of a comparison: + +1. Match the model, checkpoint, precision, and thinking mode. +2. Match the AgentX dataset variant and context limit. +3. Record the inference engine, container image, parallelism, and speculative-decoding method. +4. Apply the same golden acceptance-length rule. +5. Match DRAM capacity, KV-cache offload policy, and the proportional memory rule. +6. Use the same seed, warmup procedure, profiling duration, and client-concurrency sweep. +7. Compare per-chip and whole-system results when system sizes differ. +8. Compare at the same p90 interactivity or latency target. +9. Include failed and unstable configurations instead of reporting only surviving points. + +An iso-interactivity comparison is usually more useful than peak throughput. It answers a deployment question directly: at the response speed users need, which system serves more agent work per chip or per dollar? + +## What this agent benchmark can and cannot answer + +AgentX can measure serving capacity, responsiveness, cache reuse, memory pressure, throughput, and cost under a trace-derived agent workload. It can show whether a software change improves long-context multi-turn serving on the same hardware, or whether a different hardware topology changes the useful frontier. + +AgentX cannot tell whether the model wrote correct code, selected the right tool, followed a business policy, or completed the source task. The synthetic payload deliberately removes those semantics. Use a task evaluation for agent quality and AgentX for inference performance. Keep the scores separate because they describe different systems questions. + + + Explore the AgentX throughput-interactivity frontier → + + +## Frequently asked questions + +### What is an agentic benchmark for LLM inference? + +It is a systems benchmark that measures how an inference stack serves multi-turn agent traffic. It replays growing contexts, shared prefixes, tool delays, and subagent branches, then reports latency, throughput, interactivity, cache behavior, capacity, and cost. + +### How is an agentic benchmark different from a fixed-sequence inference benchmark? + +A fixed-sequence benchmark sends independent requests with a specified input and output length. An agentic benchmark replays dependent sessions whose request lengths, timing, prefix reuse, and parallel branches change over time. The latter measures the serving conditions created by working agents. + +### What does the AgentX benchmark measure? + +AgentX measures complete inference configurations under trace-derived coding-agent traffic. It sweeps concurrent agent clients and reports metrics including output throughput per chip, p90 interactivity, TTFT, ITL, cache behavior, and serving cost. + +### Why does AgentX use closed-loop concurrency? + +An agent cannot issue every turn in advance because later actions depend on earlier results. In closed-loop replay, each client sends its next eligible request after its dependencies complete. This preserves the relationship between server speed and how quickly a session creates more work. + +### Which metrics matter most for agent inference? + +Use output throughput, p90 interactivity, TTFT, ITL, cache-source distribution, queue depth, and cost together. Compare full concurrency curves at the same latency or interactivity target. A maximum-throughput point alone does not describe an interactive deployment. + +### Does an agent benchmark measure model or agent quality? + +Some benchmarks do, but AgentX does not. AgentX measures inference-system performance with synthetic payloads that preserve workload shape. Task correctness, tool selection, and policy compliance require a separate agent evaluation. + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "What is an agentic benchmark for LLM inference?", + "acceptedAnswer": { + "@type": "Answer", + "text": "It is a systems benchmark that measures how an inference stack serves multi-turn agent traffic. It replays growing contexts, shared prefixes, tool delays, and subagent branches, then reports latency, throughput, interactivity, cache behavior, capacity, and cost." + } + }, + { + "@type": "Question", + "name": "How is an agentic benchmark different from a fixed-sequence inference benchmark?", + "acceptedAnswer": { + "@type": "Answer", + "text": "A fixed-sequence benchmark sends independent requests with a specified input and output length. An agentic benchmark replays dependent sessions whose request lengths, timing, prefix reuse, and parallel branches change over time." + } + }, + { + "@type": "Question", + "name": "What does the AgentX benchmark measure?", + "acceptedAnswer": { + "@type": "Answer", + "text": "AgentX measures complete inference configurations under trace-derived coding-agent traffic. It sweeps concurrent agent clients and reports output throughput per chip, p90 interactivity, time to first token, inter-token latency, cache behavior, and serving cost." + } + }, + { + "@type": "Question", + "name": "Why does AgentX use closed-loop concurrency?", + "acceptedAnswer": { + "@type": "Answer", + "text": "An agent cannot issue every turn in advance because later actions depend on earlier results. In closed-loop replay, each client sends its next eligible request after its dependencies complete. This preserves the relationship between server speed and how quickly a session creates more work." + } + }, + { + "@type": "Question", + "name": "Which metrics matter most for agent inference?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Use output throughput, p90 interactivity, time to first token, inter-token latency, cache-source distribution, queue depth, and cost together. Compare full concurrency curves at the same latency or interactivity target." + } + }, + { + "@type": "Question", + "name": "Does an agent benchmark measure model or agent quality?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Some benchmarks do, but AgentX does not. AgentX measures inference-system performance with synthetic payloads that preserve workload shape. Task correctness, tool selection, and policy compliance require a separate agent evaluation." + } + } + ] +}`} diff --git a/packages/app/content/blog/agentx-inferencexv3-does-cuda-moat.mdx b/packages/app/content/blog/agentx-inferencexv3-does-cuda-moat.mdx new file mode 100644 index 000000000..9877fef69 --- /dev/null +++ b/packages/app/content/blog/agentx-inferencexv3-does-cuda-moat.mdx @@ -0,0 +1,1031 @@ +--- +title: 'AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?' +subtitle: '$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200' +seoDescription: 'AgentX 1.0 is the first fully open source multi-turn agentic coding inference benchmark at 1M context. Agentic results for DeepSeek V4 Pro, Kimi K3, MiniMax M3, Qwen3.5, and GLM 5.3 across GB300 NVL72, B300, B200, MI355X, and H200, plus the 70+ upstream PRs the benchmark drove.' +date: '2026-08-24' +publishDate: '2026-08-24' +tags: + - agentx + - agentic + - benchmark + - inference + - gpu + - nvidia + - amd + - announcement +--- + +Since the [Claude Code inflection point](https://newsletter.semianalysis.com/p/claude-code-is-the-inflection-point) in November 2025, long-context, multi-turn agentic workloads have grown rapidly. They now dominate traffic for production inferencing. In April 2026, OpenAI’s Enterprise agentic spending overtook ChatGPT spending. + +[Claude Code is the Inflection Point](https://newsletter.semianalysis.com/p/claude-code-is-the-inflection-point) — 4% of GitHub public commits are being authored by Claude Code right now. At the current trajectory, we believe that Claude Code will be 20%+ of all daily commits by the end of 2026. While you blinked, AI consumed all of software development. + +Agentic workflows have decisively taken the baton. **Today, we announce AgentX 1.0 - the world’s first fully open source, multi-turn agentic coding inference benchmark at 1 million context, released under Apache 2.0. [Our full dashboard is available here.](https://inferencex.semianalysis.com/)** + +
+ +In the past most measured performance based on fixed sequence length prefill and decode workloads, but this is an inaccurate way to measure workloads. Reality is multi-turn, long context, high prefill reuse, with sub agent bursts, KVCache offload, and numerous tool calls. As such we aimed to build the correct way for the industry to measure AI hardware and software performance. + +[We have spent more than $3M building this dataset. Today, we open source everything.](https://inferencex.semianalysis.com/) InferenceXv3 implements AgentX, a new realistic scenario in addition to the existing “fixed sequence length” scenarios (8k1k, 1k1k, 1k8k). It improves the benchmark scenarios by using agentic coding traffic instead of the previous single-turn traffic of 8k input and 1k output tokens. + +The full matrix runs on ~2MW of continuously operated compute across over 1000 chips spanning a wide range of SKUs, featuring the MI355X, GB300 NVL72, GB200 NVL72, B300, B200, MI325, MI300X, H200, and RTX Pro Servers. Rubin arrives later this month, and TPUs and Mi455X UALoE72 arrive later this year. [Please drop a star if you found our free open source work valuable.](https://github.com/SemiAnalysisAI/InferenceX) + +It is great to see amazing performance from both NVIDIA and AMD on agentic workloads. NVIDIA does very good on a lot of frontier models while AMD also does well on some frontier models for specific comparsions. + +
+ +The most valuable thing AgentX produced in its first months was not the initial results. It was the massive industry impact the benchmark is already having. **Over 70+ upstream PRs** for optimizing real world production agentic workloads across vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, LMCache, and Mooncake, uses AgentX as the north star benchmark proxy. Most of these optimization improvements are transferable to production traffic. We deep dive into each of these optimizations later in the article. + +
+ +Open source is a core principle for InferenceX and thus, we open more of the stack than most people who use that word. That includes an open frontend, a public database served through an easily consumable REST API **that multiple tier 1 AI lab’s capacity planning teams already consume**, public GitHub Actions CI provenance, [logs](https://inferencex.semianalysis.com/inference/agentic/440193?i_seq=agentic-traces&i_xmode=interactivity&view=logs), and accuracy validation on every single point. Crucially, our benchmark configs mainly track [recipes.vllm.ai](http://recipes.vllm.ai) and [SGLang cookbook](https://docs.sglang.io/cookbook/intro) on upstream images such that we are measuring the performance actual customers are experiencing instead of measuring benchmax’ed images. + +In three to four weeks, we will release an AgentX update article. It will cover further optimizations to agentic workloads, plus updated performance results from AMD and Nvidia. It is important to understand that the profile of agentic workloads is updating fast. InferenceX will continue to move swiftly to benchmark the relevant workloads. + +InferenceX is 100% committed to being open-source - this would not be possible without the contributions and support from our OSS partners. We would like to thank the following people that have made massive contributions to the AgentX 1.0 release: + +- **Inferact/vLLM**: Roger Wang, Yifan Qiao, Simon Mo, Jeff Ma, and many others +- **RedHat/llm-d**: Michael Goin, Robert Shaw, Tyler Michael Smith +- **RadixArk/SGLang**: Baizhou Zhang, Yuwei An, Mingyi Lu, and many others +- **LMCache/TensorMesh**: Samuel Shen +- **Weka**: Callan Fox, Val Bercovici +- **MoonCake Maintainers:** Teng Ma, Xu Wenjie, Ke Yang +- **AMD**: Thomas Wang, HaiShaw, Andy Luo, Seungrok Jung, Chun Fang, Parth Panchal, Bill He, Theresa Shan, Hongxia, Fangzhou, Gilbert Lei, Yanfei Wang, Duyi Wang, Peng Sun, Lingpeng Jin, Simon Danielsson, Xiaohu Guo, Haichen Zhang, Chang Liu, Doug Lehr, Poovaiah Palangappa, and many others in the AMD Shanghai Development Centre +- **Nvidia**: Xin Li, Anthony Casagrande, Kedar Potdar, Ankur Singh, Ishan Dhanani, Nick Comly, Nvidia Shanghai TensorRT-LLM team, and many others +- **Anthropic staff, for promptly fixing multiple bugs that made implementing AgentX possible** +- **GitHub:** Austen Stone for helping with reliability of GitHub Actions that AgentX uses +- And many others + +[In addition, we are thankful to all who support our open source InferenceX initiative, including Meta, Microsoft, Oracle, OpenAI, MiniMax, Moonshot Kimi, Alibaba Qwen, and Zhipu GLM.](https://inferencex.semianalysis.com/quotes) + +
+ +## A Brief Overview of Agentic Workloads + +At a high level, an agentic workload is characterized by four elements: + +1. **Multi-turn:** a session includes many user / assistant interactions (tens or hundreds) compared to a handful in a chatbot scenario. Multi-turn, long context, high prefill reuse, with sub agent bursts and numerous tool calls. +2. **Long context:** system prompts, tool definitions, and the large number of turns make context accumulate quickly. +3. **High prefix reuse:** since the conversation progresses linearly, where output from turn n-1 is concatenated to turn n (typically), most context can be served from KV cache rather than recomputed (this depends on amount of storage available to store KV tensors). As n grows, the ratio of cached input relative to uncached typically tends towards 1. +4. **Sub-agent bursts:** a session launches multiple short-lived sub-agents with fresh context, which create bursty KVCache patterns. + +
+ +Considering the characteristics above, benchmarking these workloads is fundamentally different from the existing fixed sequence length benchmarks. Namely, agentic inference is inherently a _systems problem_. Because of extremely high prefix reuse, KV tensors must be efficiently transferred across nodes/ranks (NIXL, MORI-IO, Mooncake). Additionally, different conversations should be routed to different nodes/ranks depending on where the appropriate prefix resides in order to maximize cache hit rate (LLM-d, Dynamo, vLLM/SGLang router). Long context conversations stress the HBM capacity for KV cache and necessitate offloading KV tensors to different tiers of memory (DRAM, SSD), a process that needs to be carried out efficiently (Mooncake Store, LMCache, vLLM Simple Offloading, SGLang HiCache). + +This is in contrast to fixed sequence length, single turn workloads where prefix reuse is not relevant and inference performance is largely reflective of baseline chip/kernel performance. This is not to say that the plethora of fixed sequence length data on InferenceX is not important. In fact, stripping away the complexities of agentic serving shows clearly how low-level inference performance optimizations are progressing. It also provides an important baseline for AgentX results. + +In an attempt to make the AgentX workloads as realistic as possible, we collected an initial corpus of [393 internal SemiAnalysis anonymous Claude Code traces](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126) to replay. To anonymize the content while keeping the original prefix reuse pattern, we use a method similar to the [Qwen-Bailian dataset](https://github.com/alibaba-edu/qwen-bailian-usagetraces-anon), one of the earliest corpora of production traces. We then use [AIPerf](https://github.com/ai-dynamo/aiperf) to reconstruct the traces according to the original schedule of requests at varying levels of concurrent clients. We worked with Anthropic to ship two Claude Code features to make the AgentX dataset possible. We thank the Anthropic staff for their help. + +1. [https://github.com/anthropics/claude-code/issues/49207](https://github.com/anthropics/claude-code/issues/49207) +2. [https://github.com/anthropics/claude-code/issues/66761](https://github.com/anthropics/claude-code/issues/66761) + +This brief introduction to agentic workloads should give the reader enough context to understand the results in the next section. In a later section, we will provide a deeper technical dive into the methodology, replay harness, and dataset. + +## Agentic Coding Inference Performance + +When looking at inference coding performance, OpenAI, Anthropic, xAI, and other frontier labs focus on three things. They look at performance per dollar versus interactivity (TPOT), TTFT (time to first token), and overall end-to-end task completion. Performance per megawatt is also important, considering that terrestrial datacenter power is a critical constraint (money is a social construct and it appears the labs have an unlimited supply, but power is physically hard to come by in this day and age). [Our datacenter model has estimates of quarter by quarter build up of power demand and supply.](https://semianalysis.com/datacenter-industry-model/) + +In this section, we highlight some of the overall agentic performance themes across frontier models. We strongly encourage the reader to use this as a guide to [investigate the results for themselves](https://inferencex.semianalysis.com/). **All of the data is open source and the community has the opportunity to draw their own conclusions on the current state of real world inference performance.** + +## DeepSeek V4 Pro 0813 + +DeepSeek V4 Pro 0813 is an ultra popular frontier open weight model from China. It has ~1.6 Trillion parameters with 49 Billion active parameters. + +As of August 21, the following graph shows the best performance per SKU for all submissions, normalized by total cost of ownership (TCO). + +The ISL/OSL distribution of _all_ requests among all DeepSeek v4 runs was as follows: ISL p50=88k, p90=272k, p95=404k, p99=675k and OSL p50=413, p90=2.2k, p95=3.7k, p99=8.6k. + +
+ +
+ +In general, **it is important to consider both tokens per second per user** (TPS - also known as interactivity) and TTFT, **since these often come at the expense of one another**. For instance, in the graph above, some SKUs achieve very high throughput at decent interactivity, however TTFT is severely degraded. What is an “acceptable” p90 TTFT varies heavily depending on the application. For most production systems serving agentic workloads, you can expect p90 TTFT to be anywhere from 200-5,000ms. Anything over 5-10s is pushing the boundary of what can be considered “online inference.” There are still practical applications for the ultra-high throughput sector of the curve, where latency does not matter and peak system utilization is desirable (batch processing, _very_ long running agents, etc). + +In terms of single node performance, MI355X open-source performance (vLLM) trails behind vendor specific ATOM (AMDs equivalent of TensorRT LLM). We think it’s great that AMD is pushing the frontier quickly with ATOM, however we encourage them to make a higher priority of upstreaming these improvements into vLLM. + +
+ +AMD’s distributed inference (DI) team has made great progress on 8k1k scenarios over the past six months. The team still has some way to go before DI is a viable solution for realistic workloads. In terms of throughput per GPU vs. interactivity, we observe the 1xDEP8+1xDEP8 disagg config is only able to realize slight performance gains in the high throughput scenarios, while actually performing worse in low latency scenarios. + +To make matters worse, any increase in throughput at the middle-to-high interactivity configs is overshadowed by the significant spike in p90 TTFT. One of the reasons for this is the use of SGLang’s --enable-prefill-delayer argument above concurrency 64, [which postpones prefill admission so DP ranks can form fuller batches](https://github.com/sgl-project/sglang/blob/v0.5.17/python/sglang/srt/server_args.py#L3180-L3194) (for up to 30 forward passes). Additionally, these points also increase chunked prefill size from 8,192 to 65,536. + +
+ +
+ +On e2e latency, ATOM MI355X beats B200 vLLM (it does not beat B300 or B200 SGLang though). The issue with this is that most AI labs in China or the west do not want to use ATOM in production besides 1 small advertising business unit at Alibaba Corp due to tons of missing features. The main Qwen LLM org at baba does not use ATOM in production. + +
+ +Before August 21, 2026, AMD’s MI355X strong SGLang development team was matching B200 vLLM on performance per dollar on end to end (e2e) performance. + +
+ +However, B300 vLLM and B200 SGLang still beat AMD’s MI355X. + +
+ +After August 21, 2026, due to optimizations in vLLM from Inferact and Nvidia, the performance per dollar of Nvidia’s B200 has surpassed that of the MI355X. This is a close race and we are excited to see the performance optimizations over the next couple weeks. We will be publishing an AgentX update article very soon. + +[AMD has listed their DeepSeekv4 vLLM optimization and includes lots of exciting things that they can do to improve their performance](https://github.com/vllm-project/vllm/issues/52911). + +
+ +Now turning to Nvidia. Their most competitive solutions are GB300 Dynamo TRTLLM and GB200 Dynamo vLLM. Both configs rely on PD disagg to achieve high throughput at reasonable interactivity. Additionally, GB300 configs employ wide-EP (DEP32) decode instances in order to achieve higher throughput at the middle of the frontier. + +Note that the 2xDEP8+1xDEP12 GB200 point is significantly closer to the 3xDEP8+1xDEP16 GB300 point in terms of TPS compared to TTFT. Again, TTFT is, in general, more sensitive to the “spikiness” of the workload. Since the GB300 point achieves much higher overall concurrency, it incurs more subagent traffic and hence more cold prefills. We can see this in the TTFT chart for the point: + +
+ +
+ +
+ +When normalized by TCO, B300 vLLM versus B200 vLLM aggregated performance is quite similar. The main difference being that B300 can “squeeze” out extra throughput, given its 50% increase in HBM capacity over B200. + +
+ +We can further visualize this difference using our server metric visualizations, which are new to AgentX. + +Under the load of 384 concurrent agentic traces, B300 vLLM DEP8 w/ 3TB DRAM via vLLM simple offloading achieved a 91% HBM cache hit rate with an additional 1.36% DRAM cache hit rate. This is because the **HBM KV cache working set size** is approximately 43M tokens with this configuration, and the load barely exceeds this number of tokens in flight at any given time. + +
+ +
+ +With B200 concurrency 196 (all other parameters stay the same), we see only 73% HBM cache hit rate and rely more heavily on DRAM with an offload cache hit rate of nearly 20%. We observe that the HBM KV cache working set size is 22M tokens, roughly half that of B300. + +DRAM KV offloading is typically implemented as a write-through cache, meaning every prefix written to the HBM cache is also written to the DRAM cache. Therefore, it is most effective when the amount of DRAM available for offloading is significantly bigger (a multiple of 1.5-3) than HBM KV cache capacity. + +
+ +
+ +H200 SGLang FP8 is able to serve DeepSeek v4 at low concurrency, and is even competitive with B200/MI355X SGLang from a perf/$ standpoint. However, it cannot compete with the newer SKUs in high throughput scenarios due to lack of HBM. + +
+ +Furthermore, the reliance on DRAM KV offloading at higher concurrencies leads to unreasonable latency as the number of users scales. + +
+ +Overall, MI355X performs decently well compared to its main competitors B200 and B300. Performance is most comparable at the lower throughput / lower latency parts of the curve, where only tensor parallelism and more rudimentary kernels are deployed. AMD needs to work on optimizing DEP kernels on MI355X to be more competitive in the high throughput scenarios, especially given the 1.5x HBM over B200. + +
+ +
+ +## Kimi K3 2.8 Trillion Parameters + +Kimi K3 is another frontier open weight model from China that has 2.8 Trillion total parameters. This is in the same range in terms of number of parameters vs Claude’s Mythos/Fable5 model architecture. We use this as an open weights proxy model architecture. The Kimi K3 model is so big that it does not even fit on a single B200 server and requires using wide EP/wide TP or pipeline parallelism in order to fit all of the weights. On vLLM, speculation decoding/DSpark did not compose at all with pipeline parallelism until very recently, so B200 performance on Kimi K3 was horrible and was getting mogged by MI355X since B200 was unable to use speculative decoding with pipeline parallelism. + +MI355X vLLM worked out of the box on day 0 for short context single turn workloads, but for long context multi turn workloads, MI355X AITER and Triton kernels suffered a massive panic attack on the first week and upstream vLLM was completely unusable for MI355X on realistic workloads. + +
+ +Hopper struggles to serve the AgentX workload for Kimi K3 since Kimi is a massive model and because vLLM maintainers/NVIDIA have not been focusing on optimizing Hopper for Kimi K3. Hopper (SM90) requires custom tuned kernels for K3 along with TP32/EP32 tuned shapes for serving at high interactivity. + +
+ +We think it is great that AMD is quickly pushing K3 performance forward with ATOM. However, we encourage AMD to further prioritize upstreaming these improvements into vLLM. ATOM is currently AMD's best-performing engine, but vLLM remains the more relevant comparison for customers using an upstream open-source serving stack. + +
+ +
+ +On part of the curve between 40 to 60 second e2e latency, MI355X ATOM beats even GB300 NVL72 vLLM on performance per dollar. + +
+ +## MiniMax M3 + +Nvidia absolutely destroys all competitors on MiniMax M3 432B. AMD software performance is horrible on MiniMax especially at high context length due to AMD engineering leadership incentivizing tuning only for short context single turn workloads and ignoring long context multi turn workloads. + +
+ +
+ +B300 TRT-LLM TP2 owns the M3 crown. There is a lack of DP-attention points as it is non-optimal on M3 since KV cache locality becomes a routing constraint. This is further explained later on. For GB200 at concurrency 40, TP4/EP4/DPA gets 0.60x the throughput of plain TP4 at a >3x p90 TTFT. At concurrency 32 it hits 28.8% of cache vs 96.0% theoretical. Each DP rank owns a private quarter of the pool; a 300k-token session re-landing on the wrong rank recomputes everything. No decode config with EP appears on the M3 frontier, likely as the concurrency is not high enough to balance the loads on all experts. + +B200/B300 also completely beat their rack-scale counterparts for MiniMax M3 on TCO-normalized throughput. On AgentX, the rack-scale advantage isn’t as pronounced as the Dynamo router can become the bottleneck because its work scales with the number and length of live prefixes. Optimizations on this and the several fixes which moved throughput by double-digit percentages are discussed later on in the article. Also, there are no well tuned kernels for wideEP, wide DCP, nor wide TP. And since GB200/300 have higher TCO so without wide ep/wide DCP, it shows up as worse perf per TCO. + +With that being said, we also expect further optimizations from Nvidia on their rack scale solutions for this SKU. We will make sure to highlight these in our follow up article. + +No submission currently runs context parallelism, despite P90 ISL of 317k. With 4 KV heads, DCP caps at 2 even at TP8, and the MSA indexer needs its own context-parallel handling (a vLLM PR is opened), see the Context Parallelism section for more discussion on this topic. + +
+ +
+ +All of Nvidia’s Pareto optimal points include KV offload above concurrency 20, but for AMD none of the Pareto optimal points use KV offload to DRAM. AMD also uses KV offload less than Nvidia on the other models. The reason for this is that GPU-to-CPU transfers for CPU KVCache offloading are highly inefficient on AMD vLLM. The hipMemcpyBatchAsync API was missing until ROCm 7.14. Without hipMemcpyBatchAsync, vLLM’s native Simple CPUOffloading requires doing serialized Memcpy from CPU to GPU instead of batching them into larger message sizes. + +It is also worth mentioning that vLLM performance is very comparable to TRT-LLM in terms of throughput versus p90 interactivity. Additionally, vLLM performs better in terms of throughput versus p90 TTFT. + +
+ +
+ +## Qwen3.5 397B + +Qwen3.5 397B uses GatedDeltaNet instead of vanilla attention for every couple of layers. GatedDeltaNet was invented at MIT/Nvidia Research and has a theoretically constant state storage requirement instead of vanilla attention’s linear storage requirements. This means that it has lower storage requirements compared to an equivalent dense attention model. Unlike end to end model training research like the Nemotron disaster, Nvidia Research is great at fundamental research like GDN and LatentMoE which is used on frontier models. + +Note that this model’s native max context length is 262k tokens, so we use the [truncated dataset](https://inferencex.semianalysis.com/agentx/cc-traces-weka-062126-256k). This simulates a workload on a smaller model where the max context length would be frequently reached with many compactions, how users would actually use this model. + +Qwen3.5 397B is a strong hold for NVIDIA on SGLang versus SGLang, with over 20x better performance at 90 tok/s/user. There is currently zero competition from AMD for Qwen3.5 SGLang. + +
+ +
+ +Again, we observe Nvidia over optimizing for interactivity at the cost of TTFT, especially in the case of TRT-LLM. In the graph above, all of the Nvidia SGLang submissions have much lower p90 TTFT when compared to TRT-LLM. + +Compared to H100, on Qwen3.5, B300 FP4 has 12x better performance per dollar. + +
+ +## GLM 5.3 + +GLM 5.3 builds on top of GLM5.2 744B with additional post training. This is a frontier level model. + +In terms of OSS SGLang performance, this is another model where Nvidia again beats AMD on realistic agentic inference performance. At 150 tok/s/user p90 interactivity, Nvidia has up to 5x better cost efficiency, With the current state of AMD software, at 150 tok/s/user, Nvidia’s performance advantage is so great that even if the competitor chip hardware was sold for free (but with providers still of course paying for datacenter hosting and power and other operating costs), cost per token would still be cheaper when using Nvidia. + +We look forward to AMD’s performance optimizations in the upcoming AgentX update article in a couple of weeks, which will also include some other very exciting results. + +
+ +
+ +When looking at ATOM, AMD has better performance per dollar than GB300 NVL72 SGLang and even TRTLLM for some parts of the range of p90 E2E Normalized Interactivity. Great work to the AMD team on these results. Again, we look forward to AMD porting over these optimizations to SGLang. We also are looking forward to NVIDIA quickly optimizing GB300 NVL72 in the coming weeks. + +
+ +We take a moment to introduce an experimental metric which we call **E2E Normalized Interactivity**. At a high level, this metric is supposed to evaluate how fast a user experiences responsiveness when considering both TTFT as well as TPS. It is defined by OSL/E2EL. Substituting the fact that E2EL equals TTFT plus OSL times TPOT (in reality only OSL - 1 tokens are decoded), we get the following equation. + +This is effectively interactivity (the 1/TPOT portion) plus an additional penalty proportional to TTFT. + +
+ +Please note that this metric is experimental and is not perfect. For instance, it heavily penalizes high TTFT and doesn’t capture all the nuances of certain optimizations such as PD disaggregation. All submissions for AgentX v1.0 optimize for both regular interactivity and TTFT separately. We will continue working on new north star metrics that reflect all nuances of modern agentic inference. + +## AgentX Industry Impact - Optimizations for Agentic Workloads + +The most impactful result from AgentX in its first months was not producing open source datasets, instead, it has been the **industry impact of 50+ upstream PRs** created by AgentX partners to optimize real world agentic workloads using AgentX as the north star. AgentX’s real agentic traffic benchmarks not only the raw prefill and decode kernels, but also tests the entire end to end token generation process from KV cache lifecycle, hybrid-attention cache correctness, CPU KV offload, transfer progress, routing affinity, to incremental tokenization, request serialization, and scheduler bookkeeping. All of these steps matter for every production agentic deployment. + +This is just a continuation of our ongoing mission to help the ecosystem accelerate improvement and deliver light speed improvement in software. A great example is SemiAnalysis’s multi-year collaboration with AMD’s software development team in which we have been providing continual feedback and input to help to modernize their software development principles. This has not only led to many changes that have accelerated AMD’s progress, but also has been instrumental towards getting AMD open source closer to first class on agentic workloads. + +## A Brief Introduction to the Distributed Inference Ecosystem + +As mentioned, agentic inference is inherently a system-wide problem as opposed to just a chip/kernel level problem. Additionally, when there are large _distributed systems_ handling hundreds of thousands of agentic requests, the scheduling of requests and management of KV cache becomes non-trivial and has legitimate performance implications. For example, sub-agents give bursty KVCache patterns where not properly optimizing, it will improperly evict the main agents cache. + +The following diagram illustrates the stack at a high level. At the top, routers (sometimes referred to as “frontends”) route requests to different workers. For instance, in the case a server is running data parallel attention, there are separate KV caches for each DP rank. In order not to thrash any one of the KV caches, requests are routed according to different policies such as [consistent hash](https://github.com/vllm-project/router/blob/main/src/policies/consistent_hash.rs), where requests in the same session/subagent are routed by their unique ID. + +
+ +For most routing policies, there is nothing significantly different about each router implementation. Some are separate components such as [vLLM router](https://github.com/vllm-project/router) and [llm-d router](https://github.com/llm-d/llm-d-router) while others are integrated in the engine such as [SGLang model gateway](https://github.com/sgl-project/sglang/tree/main/sgl-model-gateway) and [ATOM Mesh](https://github.com/ROCm/ATOM/tree/main/atom/mesh). + +After a request is routed, it is handled by the scheduler of an inference engine such as vLLM, SGLang, etc. The engine is responsible for actually performing the inference and returning the result over an API. Additionally, each engine has an interface for connecting the engine’s internal KV cache to external KV cache managers. This allows for a “pluggable” ecosystem where different KV cache managers can integrate with a variety of inference engines. + +A simple deployment, used in the current AgentX results, [runs Mooncake](https://pypi.org/project/mooncake-transfer-engine/) alongside vLLM on the same node. Each vLLM worker embeds a Mooncake Store client and contributes a portion of host DRAM to the external KV-cache pool. vLLM connects to this pool through the MooncakeStoreConnector interface which loads reusable KV blocks into GPU memory and saves newly computed blocks back to host memory. Mooncake Store manages the external cache, including placement and eviction, while Mooncake Transfer Engine performs the actual movement of data between GPU and CPU memory. + +Different KV cache managers may use different transfer engines to physically move bytes between memory tiers or machines, such as between prefill and decode workers. Mooncake Store, for example, uses [Mooncake Transfer Engine](https://github.com/kvcache-ai/Mooncake/tree/main/mooncake-transfer-engine) to move KV blocks between GPU memory, host DRAM, and remote nodes. + +A deployment can use Mooncake Store to offload reusable KV blocks to host DRAM while simultaneously using NIXL to transfer request-specific KV directly from prefill GPUs to decode GPUs. Mooncake TE handles movement for the Mooncake Store path, while NIXL handles the separate prefill-decode path using UCX and GPUDirect RDMA where supported. Multiple KV-management and transfer paths can therefore coexist within the same inference engine. + +The ecosystem consists of many independent components, including inference engines, routers, KV-cache managers, data-transfer libraries, and cluster controllers. Platforms such as Nvidia Dynamo, llm-d, and AMD Infera “package” selected combinations of these components into complete software distributions. They publish compatible container images, connectors, deployment manifests, and orchestration logic that allow the components to be deployed and operated as one system. The resulting product is usually a collection of coordinated containers rather than a single monolithic service (for instance: Dynamo, llm-d, and Infera are typically deployed on k8s and co-ordinate large, distributed systems). + +
+ +## Context Parallelism + +Long context benefits parallelism techniques that a fixed 8k prompt cannot strongly exercise, because at 8k there is little to divide and TTFT is already short. Moreover, parallelism strategies like TP and DP attention are not optimal at longer context lengths, TP can result in the full KV being replicated on each rank. Although KV is shared for DP attention, it can get hung up on longer contexts, as long context workloads also result in a higher variance of possible context lengths. + +
+ +Context parallelism is a parallelism technique that splits query tokens across GPUs. It comes in two forms: PCP for prefill context parallelism and DCP for decode context parallelism. In PCP, each rank prefills its query chunk (KV ring-passed), since prefill tends to be compute-bound, this parallelizes FLOPs resulting in faster prefill with no giant-prompt prefill spike on one rank. For DCP, each rank scans its KV shard, and the partial attention is then merged flash-decode style. Since decode is memory-BW-bound, parallel KV reads can result in faster tok/s. + +
+ +
+ +This parallelism technique was invented partially by Nvidia Research. Nvidia Research is great at fundamental research like this while for end to end training research, they are embarrassing America with their horrible Nemotron3 Ultra model which is currently getting massively beaten by even tiny Qwen3.8 27B model. DCP/PCP forms part of CUDA moat as the AMD implementation of DCP/PCP isn’t optimized yet. In the [vLLM support matrix](https://docs.vllm.ai/en/latest/design/attention_backends/#standard-attention-mha-mqa-gqa-backends), every single AMD backend is unsupported. + +A few of the changes mentioned in the next few sections focus on DCP/PCP. + +## vLLM Agentic Optimizations + +Working alongside vLLM maintainers from Inferact, Red Hat, NVIDIA, and AMD, we used AgentX’s realistic replayer as a north star, with the resulting fixes landing upstream where most of the optimizations are highly transferable to production. A few examples follow: + +vLLM improved hybrid-attention prefix caching so short-lived sliding-window allocations do not evict useful long-context checkpoints. [Selective retention](https://github.com/vllm-project/vllm/pull/43447) preserves sparse replay boundaries and reported a prefix-cache hit rate above 95% with fourteen concurrent requests and contexts up to one million tokens. The same reachability policy [was applied to Mooncake](https://github.com/vllm-project/vllm/pull/44774), and [unreachable sliding-window lookups were removed](https://github.com/vllm-project/vllm/pull/45444). Earlier follow-up work also [stopped offloading sliding-window blocks that could never be reused](https://github.com/vllm-project/vllm/pull/42258) and [kept the speculative lookahead block in the retained prefix](https://github.com/vllm-project/vllm/pull/44082). + +
+ +Agentic Workloads for high concurrent agents require offloading. Thanks to the aforementioned influence of AgentX, there are already workstreams on vLLM that aim to allow CPU KV offload for hybrid models rather than only for uniform full-attention models. This distinction matters because a uniform model has one KV layout per token, so a connector can describe what to save with a single block geometry. A hybrid model carries several cache groups at once, each with a different shape and a different lifetime, and a connector that assumes one uniform layout cannot express which group a given block belongs to. Offload was therefore unavailable for the models whose long sessions needed it most. The general [SimpleCPU connector](https://github.com/vllm-project/vllm/pull/37160) came first, was [enabled on ROCm](https://github.com/vllm-project/vllm/pull/40549), and was then [extended to DeepSeek-V4 hybrid attention](https://github.com/vllm-project/vllm/pull/42296), reporting 81.7% higher output throughput and 46.6% lower mean e2e latency against recomputing the prefix once it no longer fits in HBM. + +Mooncake has also gained [equivalent hybrid-memory allocation support](https://github.com/vllm-project/vllm/pull/42828). The same layout problem resurfaces in disaggregated serving, where a [pending change](https://github.com/vllm-project/vllm/pull/51052) transfers Kimi-K3’s conv+ssm recurrent state alongside the attention KV over MoRI-IO for 1P1D prefill/decode splits; without it the decode side starts from an uninitialized recurrent state. The recurrent-state slot rides the existing remote-block-ids channel, so the disaggregation router needs no model-specific changes, and the path was exercised end-to-end on MI355X, TP8 per leg over cross-node RDMA, with DSpark speculative decoding on both legs. + +When profiling realistic workloads, vLLM maintainers noticed that during offload, the cost moved to the store path, which was writing too much and too often. Three new fixes have now addressed this issue: + +A store is now [skipped while an identical transfer is already in flight](https://github.com/vllm-project/vllm/pull/41289), so concurrent sessions sharing a prefix pay for it once rather than once each. A store [covers only newly generated KV ranges](https://github.com/vllm-project/vllm/pull/46412), so a session that extends its history writes the delta instead of rewriting the whole prefix on every turn. Finally, a store no longer [depends on whether the same blocks still sit in HBM](https://github.com/vllm-project/vllm/pull/46906), so work already scheduled is not discarded when an eviction lands underneath it. + +The load path was tuned separately, because lookups happen on every scheduling decision rather than only when data actually moves. Making [lookups asynchronous in the scheduler path](https://github.com/vllm-project/vllm/pull/45659) keeps the connector off the step’s critical path, so a step no longer waits on CPU-side cache queries before it can admit work. [Compact zero-copy lookup keys](https://github.com/vllm-project/vllm/pull/45969), [parallel receive-side loading](https://github.com/vllm-project/vllm/pull/45971), and [prebuilt Mooncake key strings](https://github.com/vllm-project/vllm/pull/46188) then removed the CPU and transport overhead that remained. + +
+ +Long-lived hybrid state also forced correctness and accounting fixes that fixed-shape requests rarely reach. vLLM [emits cache events per hybrid cache group](https://github.com/vllm-project/vllm/pull/44103), [strides distributed-context stores correctly](https://github.com/vllm-project/vllm/pull/45371), and [computes lookup prefixes correctly under distributed context and prefill](https://github.com/vllm-project/vllm/pull/46855). A related [context-parallel accounting change](https://github.com/vllm-project/vllm/pull/45340) aligns cache ownership with sharded token ranges. Speculative state is now [propagated across merged Mooncake groups](https://github.com/vllm-project/vllm/pull/49069) and [through the SimpleCPU coordinator](https://github.com/vllm-project/vllm/pull/49071), preventing the repeated-turn cache from silently losing EAGLE state. + +Looking at the ROCm side of optimizing Agentic workloads in vLLM, the work continues below the cache layer, where the remaining cost is per-layer rather than per-request. Once the prefix survives and arrives on time, what is left is the decode step itself, and a decode step that runs thousands of times per session pays for every avoidable copy and every mismatched kernel. + +Three open changes attack that layer: + +- A Kimi-K3 change [writes KDA decode results directly into the layer output buffer](https://github.com/vllm-project/vllm/pull/51183), removing one device copy per KDA layer; the saving is small in isolation and repeated for every layer of every decoded token. + +- A second change [selects an AITER sparse-MLA decode kernel](https://github.com/vllm-project/vllm/pull/51714) in place of the generic path, and reported 5.22% higher AgentX output throughput with substantially lower inter-token latency. +- The third is a useful illustration of how much the measurement shape matters. A companion change [routes full-graph attention projections through tuned AITER GEMMs](https://github.com/vllm-project/vllm/pull/51713) and reached a 2.3% gain on fixed sequences at low concurrency. This shows a kernel-level change can show a clean gain on uniform shapes and then be swamped, on an agentic trace, by the cache and scheduling variance that the trace introduces. A [pending change](https://github.com/vllm-project/vllm/pull/52882) turns that shape sensitivity into the dispatch criterion itself: it replaces the DeepSeek V4 C4A selector’s ROCm top-k bottleneck with a hybrid AITER/native path on gfx950, routing short and medium contexts through AITER and long contexts through a graph-safe tuned native fallback. It reports end-to-end selector speedups of 1.21x to 1.76x, with decode-kernel geomeans of 1.2x to 2.9x across an 84-shape matrix. + +## SGLang Agentic Optimizations + +The AgentX team has been working closely alongside SGLang maintainers from RadixArk, Meta, Nvidia, and AMD to drive optimizations to Agentic workloads run using SGLang, resulting in massive improvements in production inference performance. Let’s discuss these optimizations in more depth. + +We will start by explaining how, from the allocator side, SGLang’s sliding-window work addresses the same conflict vLLM’s retention policy does. Window pages and prefix pages are drawn from one pool, and the window is the greedier consumer: it turns over constantly while the prefix sits still and so, under pressure, the transient allocation displaces the durable one. + +Three design improvements attack this problem that from different angles. One [proactively frees pages as they leave the window](https://github.com/sgl-project/sglang/pull/26907) rather than waiting for eviction pressure to find them, so dead window state stops competing for pages it can no longer use. Another [caps compute locks to a single window](https://github.com/sgl-project/sglang/pull/27210), bounding how much of the pool an in-flight request can hold pinned at once. A third [removes stale full-KV entries](https://github.com/sgl-project/sglang/pull/29369) that outlive their usefulness. + +Proactive freeing has a fork-shaped blind spot, though: a request branching from a shared prefix can still hold reusable full-KV while the window state at the branch point has already been released, and the whole prefix gets recomputed for want of the cheap half. [Open work](https://github.com/sgl-project/sglang/pull/34565) preserves the SWA state at those branch points so forks inherit the window instead of rebuilding it. + +Alongside those, [the ROCm ring-cache fix](https://github.com/sgl-project/sglang/pull/30339) is a correctness rather than a capacity change: a ring buffer reuses slots by construction, and reusing one whose old contents are still referenced yields wrong output rather than slow output. None of this is visible on a single 8k prompt, where the window never laps the prefix and the pool is never contended. On a multi-turn hybrid session, these changes are what decide whether the expensive full-attention history is still there on the next turn. + +HiCache is SGLang’s first class in-tree offloading mechanism. It faced the same hybrid problem vLLM’s connectors did, and solved it with an asymmetry: [offload the full-attention cache and reconstruct the short sliding-window tail on the way back](https://github.com/sgl-project/sglang/pull/29417). Only the expensive half is worth moving across the bus, and the cheap half can be rebuilt more quickly than it can be fetched. On AMD, [staged write-back](https://github.com/sgl-project/sglang/pull/28534) keeps that movement from blocking the engine while it happens. Recurrent state was the remaining gap, because it cannot be rebuilt from neighbouring tokens the way a window tail can; [FlashInfer GDN checkpoints](https://github.com/sgl-project/sglang/pull/29735) let it participate in prefix reuse at all, and raised throughput from 47,771 to 53,004 tok/s/GPU at a 92.4% cache-hit rate. + +
+ +Two further changes address how variable-length traffic affects kernel pipeline. AgentX realistic sessions tend to arrive at continuously varying context lengths, a similar pattern observed for production traffic. A naive runtime that specializes on length will compile a fresh kernel for nearly every request it sees. SGLang maintainers solved this by passing [context length as a runtime scalar](https://github.com/sgl-project/sglang/pull/30255) instead collapsing it into one compilation, improving AgentX concurrency 384 output throughput by 26.75% and mean TTFT by 36.25%, by removing compilation, not from computing anything faster. + +
+ +In the same spirit, [removing a per-step device-to-host sequence-length synchronization](https://github.com/sgl-project/sglang/pull/30365) eliminates a decode bubble that exists only because the host wanted to know a length the device already had. + +
+ +Variable length also bites inside the attention kernel itself. On GB300, a mixed-context decode batch pays a tail tax: a matched profile attributed 8.4 ms of a 9.8 ms decode-step delta to attention, with the longest requests dragging out the persistent kernel’s shared wave. [Open work ](https://github.com/sgl-project/sglang/pull/34888)splits TRTLLM MHA decode batches into KV-length-sorted groups so the short requests stop waiting on the longest ones. + +Decode can starve one level up as well, at the scheduler rather than in a kernel. Under DP attention, every rank joins the same MoE collective while scheduling attention work locally, so a rank fed a stream of chunked-prefill continuations keeps winning the prefill-first decision while peer ranks’ running batches sit waiting, observed on AgentX runs. [A configurable decode interval](https://github.com/sgl-project/sglang/pull/35017) after prefill forces decode rounds between prefills; on AgentX DSv4 Pro, output throughput rose 141% and p99 inter-token latency fell 97.3%, at the cost of median TTFT rising from 36.5 to 59 seconds, trading off first-token wait for stream smoothness. + +
+ +When a request carries no reusable history such as the start of a subagent, any worker will do just fine, and load balancing is the only question worth asking. When the request carries a MB of cached prefix, sending it to an idle worker that does not hold that prefix is the expensive choice, and the router needs to know where the state already lives. SGLang added [DP cache affinity](https://github.com/sgl-project/sglang/pull/26091), so a session is sticky to the rank holding its cache. In this PR, [DP-aware prefill and decode routing](https://github.com/sgl-project/sglang/pull/26245) is both implemented so that both halves of a disaggregated deployment make that decision consistently, and [cache balance as a routing signal](https://github.com/sgl-project/sglang/pull/26293) so affinity does not degenerate into one hot worker. A router can only act on what it is told, so hybrid cache events also become [radix-cache aware](https://github.com/sgl-project/sglang/pull/26387) and [sliding-window aware](https://github.com/sgl-project/sglang/pull/26579). + +Speculative decoding receives special attention, because MTP adds a second, smaller piece of per-request state that has to survive everything the main cache survives. SGLang [fixed draft-window transfer in disaggregated serving](https://github.com/sgl-project/sglang/pull/30461) so that state crosses the prefill-to-decode boundary intact, [added overlap scheduling for high-concurrency online decoding](https://github.com/sgl-project/sglang/pull/30497), [removed a no-op EAGLE renormalization](https://github.com/sgl-project/sglang/pull/31294), and [avoided host synchronizations during EAGLE prefill](https://github.com/sgl-project/sglang/pull/33662). The open [resource-lease scheduling work](https://github.com/sgl-project/sglang/pull/32042) and [data-parallel graph-metadata fix](https://github.com/sgl-project/sglang/pull/32196) continue the same effort, which is to make overlap safe when requests can be retracted and resumed rather than simply run to completion. + +When looking at Agentic workloads with Heterogeneous prefill and decode topologies, prefix-aware staging is needed, and this is where prefix caching and disaggregation interact badly. When the two sides are not sharded identically, KV cannot be copied across as one contiguous stream; it has to be split on a transfer grid and reassembled at the offsets the decode side expects. A prefix hit makes that harder, not easier, because the prefill worker now sends only the uncached remainder while the decode side still expects a complete, correctly positioned cache. [Radix-cache support in the staging buffer](https://github.com/sgl-project/sglang/pull/30545) splits cached sends on that grid and scatters them at the correct decode offsets. + +
+ +However, this results in correctness problems. A 127,500-token shared-prefix test went from 2 correct needles out of 128 to 128 out of 128, meaning the cache had been silently landing in the wrong places, which a throughput benchmark would have scored as a fast, confident, but wrong answer. The AgentX comparison additionally raised median per-user output throughput by 9.6% at nearly unchanged total throughput per GPU. The transfer itself also carried dead weight: decode-side PREBUILT batches never enter a model forward, yet every transferred prompt was still flattened and copied into a CUDA input tensor that the first decode step reconstructs from relay metadata anyway. [Dropping that unused prompt transfer](https://github.com/sgl-project/sglang/pull/35070) is the largest single win, driving +18.0% per-user output throughput and +12.7% decode throughput per GPU on AgentX GB300. A [follow-up](https://github.com/sgl-project/sglang/pull/35071) moved the prefill DP-rank bootstrap query off the decode scheduler’s critical path, overlapping an HTTP round trip that had been paid synchronously at result consumption, driving a further +1.36% per-user output throughput on the same deployment. Open work continues along the same seam: [multi-pool DeepSeek-V4 support in UMBP](https://github.com/sgl-project/sglang/pull/30762), [unified-KV HiSparse state carried over MoRI](https://github.com/sgl-project/sglang/pull/32368), and [preserving the prefill-owned token when decode terminates without visible content](https://github.com/sgl-project/sglang/pull/34216). The HiSparse work should be read as a capacity and correctness enabler for long contexts rather than as a throughput win at high concurrency, which it is not yet. + +
+ +## TensorRT-LLM Agentic Optimizations + +Next, we shall move onto explaining a couple of the recent optimizations from TensorRT-LLM. First, we look at TRTLLM’s distinct frontend optimization for repeated chat turns, a cost that only exists because the workload is multiturn. Every turn of a conversation re-sends the entire history plus a little more, and the naive implementation re-tokenizes all of it. Tokenization is cheap per kilobyte, but is ruinous when the same 100,000 tokens are tokenized again on every turn. + +The obvious fix, to tokenize only the new suffix, is wrong in a way that is easy to miss, because byte-pair encoding is not position-independent. Tokens can merge across the join, so splitting the text at the boundary and concatenating the two token sequences can produce a different sequence than tokenizing the whole string, which quietly diverges from the sequence the prefix cache was built against. + +TRTLLM implements [Boundary-aware incremental tokenization](https://github.com/NVIDIA/TensorRT-LLM/pull/17462) which handles this by finding the rendered-text common prefix, rolling back one complete token so any merge that spans the join is recomputed, and tokenizing only the changed suffix from there. On the Qwen3.5 AgentX trace, it matched full tokenization on all 1,087 transitions — the correctness claim, tested rather than assumed — and reduced mean processing time from 185.1 ms to 11.3 ms. + +
+ +A fixed 8k1k request has no prior rendered turn to reuse, so none of this appears there. Relatedly, [chat-template rendering was moved into the input-processing pool](https://github.com/NVIDIA/TensorRT-LLM/pull/16231), so a long template no longer serializes the main request loop behind it. + +
+ +The MiniMax-M3 work focuses on disaggregated KV movement, where the failure is one of granularity. When prefill and decode do not agree on head layout, the KV for one logical request stops being a few large contiguous regions and becomes thousands of small strided pieces, each of which turns into its own transfer descriptor. The bytes moved are unchanged; the per-descriptor overhead is what explodes, and it explodes in the worst way on exactly the long prompts that matter! [Corrected multi-pool mapping and a chunked NIXL bounce path](https://github.com/NVIDIA/TensorRT-LLM/pull/17518) coalesce those pieces through a bounded reusable arena, trading an extra staging copy for orders of magnitude fewer descriptors. Its AgentX diagnostic reduced request-critical KV p99 from 26.74 seconds to 125 ms at concurrency five, and from 10.15 seconds to 288 ms at concurrency forty. + +
+ +[Nonblocking context-transfer polling](https://github.com/NVIDIA/TensorRT-LLM/pull/17428) protects the same path by reaping completed transfers even when scheduling stalls. This breaks a feedback loop in which finished KV blocks stay pinned and prevent new admissions. The stalls themselves also had a [removable cause](https://github.com/NVIDIA/TensorRT-LLM/pull/16734): avoiding implicit device-scalar syncs in DeepSeek-V4’s context sparse-attention metadata eliminated 18 four-byte device reads per step that each forced a cudaStreamSynchronize, on a GB300 disaggregated context worker. The fix threads host-side counts through as plain Python ints, so the executor thread no longer holds the GIL for most of a step while the KV-transfer and response threads wait behind it. + +TensorRT-LLM also moved irregular long-context work onto more efficient execution paths. [Context graph producers for MiniMax-M3](https://github.com/NVIDIA/TensorRT-LLM/pull/17473) capture stable sparse producers while leaving request-dependent attention eager, and per-user output throughput improved by 12.58 percent in its AgentX test. An open [native KV-event production change](https://github.com/NVIDIA/TensorRT-LLM/pull/16876) reduces allocation and conversion work on the KV-aware routing path. + +AgentX also exposed kernel-selection and scheduler-lifetime failures that only appear at scale and duration. Two are about which kernel gets picked. MiniMax-M3 [added CuTeDSL choices to MXFP8 autotuning](https://github.com/NVIDIA/TensorRT-LLM/pull/17316), widening the candidate set and improving output throughput per GPU by roughly 7 to 10% at low-concurrency aggregate points. In the opposite direction, TensorRT-LLM [disabled corrupt split-K MoE tactics](https://github.com/NVIDIA/TensorRT-LLM/pull/17105) after it crashed five of seven AgentX runs, with no crashes in seven matched runs afterwards. A tactic that is fast and wrong is worse than one that is merely slow, and an autotuner will select it enthusiastically unless it is removed from the pool. Selection is not the only way a kernel goes wrong: MiniMax-M3’s legacy sparse-attention path for short queries could hand the SM100 kernel a non-contiguous, head-major block-index view that it read as contiguous, selecting the wrong KV pages and producing incorrect or non-finite output. [Honoring the block-index strides](https://github.com/NVIDIA/TensorRT-LLM/pull/17285) for q_len ≤ 32 fixed the indexing without materializing the tensor or adding a kernel, and five matched full AgentX pairs on GB300 afterwards completed with zero serving errors and no non-finite markers. + +
+ +The other two are lifetime bugs, which are the characteristic failure of long runs rather than large ones. [Sequence-slot headroom and consistent slot-indexed buffer sizing](https://github.com/NVIDIA/TensorRT-LLM/pull/16279) handle the transient overlap where a completing request and a newly admitted one both need a slot, a window that a steady stream of arrivals and departures hits constantly and a fixed batch never hits at all. A later [attention-data-parallel dummy-request fix](https://github.com/NVIDIA/TensorRT-LLM/pull/17278) kept nine Qwen3.5 disaggregated cells alive where most earlier cells had failed within minutes, the difference between a configuration that benchmarks and one that survives a session. + +Two open transfer changes target very long disaggregated prompts, and together they show how a fix can create the next bottleneck. In the default arrangement, a decode worker cannot start until the entire prompt has been prefilled and then transferred, so two expensive phases run back to back even though the first produces its output incrementally. [Pipelined KV transfer](https://github.com/NVIDIA/TensorRT-LLM/pull/15727) begins sending each completed prefill chunk as it lands, so transfer overlaps prefill compute and only the final chunk is on the critical path. + +That change makes chunk handling frequent, which exposes work that used to happen once. Its follow-up [retrieves only the block IDs belonging to the current chunk](https://github.com/NVIDIA/TensorRT-LLM/pull/17526) rather than the whole prompt’s block list each time. For a 128,000-token prompt split into 1,024-token chunks, that is the difference between building a 4,096-entry list once and rebuilding it 128 times for every layer group. A per-chunk cost that scales with total prompt length is a cost scaling shape that eats the gain the pipelining just bought. + +
+ +## AMD ATOM Agentic Optimizations + +AMD’s ATOM engine was originally designed only for single turn workloads instead of real world agentic multi-turn production workloads, so there were a lot of changes needed to the core fundamental ATOM engine and kernels to enable good support for long context multi turn workloads. ATOM still has a long way to go to support agentic workloads relative to where vLLM/SGLang are at present. AgentX is used as the realistic north star target for ATOM’s refactor to support agentic workloads. The first optimization we will chat about that ATOM implemented is smartly using sparse checkpoint retention for DeepSeek-V4 paged sliding-window attention. [The merged implementation](https://github.com/ROCm/ATOM/pull/1640) keeps selected window tails alive so branch and replay requests can resume at useful boundaries. Its measurements separate the two effects cleanly: on the same AgentX trace at concurrency 48, the actual prefix hit rate rose from 5.6% to 96.45%, and losses at the sliding-window gate fell from 91.35% to 0.16%. The second number is the mechanism behind the first. Nine out of ten prefix matches were being found and then discarded for want of a window tail, so the cache was not missing but being overruled. + +
+ +Two earlier cache-manager fixes had to land before any of this could be measured, and both fixes are worth noting as examples of a cache that reports itself healthy while doing nothing. One [stopped free-pool hits from destroying shared cache entries](https://github.com/ROCm/ATOM/pull/902); the other, [a deferred-output fix](https://github.com/ROCm/ATOM/pull/939), restored prefix hashing in the default scheduler mode and moved repeated long prompts from zero cached tokens to reuse of every complete prefix block. A separate change lets prefix-hit prefill [stay on the optimized sink attention kernel](https://github.com/ROCm/ATOM/pull/1345) rather than falling back to the generic path, so a cache hit does not quietly cost part of what it saves. + +Hybrid models also carry a recurrent or compressor state, which differs from ordinary KV in one decisive way: it cannot be reconstructed from the tokens around it. A window tail can be recomputed from neighboring context, but recurrent state is the accumulated result of everything that came before, so if it is dropped, the only way back is to replay the sequence. ATOM [gave this per-request state a content-addressed checkpoint lifecycle](https://github.com/ROCm/ATOM/pull/1771), letting generated turns leave reusable resume points without reserving a separate protected cache for them. In one test, a request reused 512 generated tokens and computed only a two-token suffix. + +The tuning detail matters as much as the feature. Publishing a checkpoint unconditionally costs 17.5% throughput on zero-hit traffic, the price paid by every session that never comes back, in order to help the ones that do. Spacing checkpoints by token interval avoided that penalty, and fixed 1k1k throughput stayed within measurement noise, which is the relevant safety property: a feature aimed at agentic reuse should not tax workloads that will never use it. + +ATOM’s AgentX-relevant CPU path starts from the arithmetic that justifies offloading at all. [Standalone LMCache offload](https://github.com/ROCm/ATOM/pull/1318) reloads a 32,000-token prefix from CPU in about 0.32 seconds against roughly 2.5 seconds to recompute it, an eight-fold margin. This makes crossing the bus worth doing at these context lengths and would not hold for a short prompt. + +The rest of the path is about ownership and index placement rather than bandwidth. ATOM copied vLLM’s [multi-connector](https://github.com/ROCm/ATOM/pull/1406) design which lets a prefill worker send KV to a remote decode worker and save the same prefix to CPU at once, without freeing the blocks until both consumers are finished; two independent readers of the same blocks is a situation where single-turn won’t get. + +[Promoting restored blocks back into the GPU prefix index](https://github.com/ROCm/ATOM/pull/1725) fixes a subtler waste: without it, a prefix loaded from CPU is used and then not registered as resident, so the next turn fetches the same hot prefix across the bus again, paying the transfer repeatedly for a cache that was already in HBM. Follow-up work [fixed asynchronous save ordering, packed-KV geometry, unaligned handoffs, and remote request accounting](https://github.com/ROCm/ATOM/pull/1807) together, eliminating reload corruption across a two-round, 2,638-request validation. This bug surfaces only when the same blocks are saved, evicted, and restored many times over. + +
+ +The distributed path repeats, in a different codebase, and the pattern is already visible in SGLang and Dynamo: routing has to know where state lives. ATOM’s router called ATOMesh is a fork of SGLang’s router with most of the features removed. Unfortunately, ATOMesh required SGLang’s cache-aware routing feature, so that feature had to be added back. ATOM gained KV lifecycle events for cache-aware routers, so the router can know where state lives at all. It also gained multi-node prefill and decode routing, and session-sticky data-parallel routing. The sticky policy is a two-sided compromise worth stating explicitly: a conversation returns to the healthy worker that owns its state, but idle assignments expire so that stickiness does not permanently unbalance the cluster on behalf of sessions that have gone away. + +Disaggregation then has to move whatever the model actually keeps, which is not always one uniform cache. DeepSeek-V4 [transfers both buffers of its mixed FP8 and BF16 cache layout](https://github.com/ROCm/ATOM/pull/1737), and EAGLE disaggregation [moves the draft model’s independent KV cache](https://github.com/ROCm/ATOM/pull/1331) alongside the target cache, the same second-cache problem TensorRT-LLM and SGLang each had to solve. [Remote-KV admission and backpressure](https://github.com/ROCm/ATOM/pull/1647) closes the loop by stopping the decode side from accepting more parked transfers than it can safely resume, which is the disaggregated form of accepting work you cannot finish. + +On ATOM, [PCP reported 35 to 43% lower mean TTFT](https://github.com/ROCm/ATOM/pull/1220), with total throughput gains of up to about 49% at a 64,000-token input - a gain that grows with input length rather than with batch size. Making that usable in practice required it to compose with everything else a session relies on, so DCP was [made compatible with prefix caching, chunked prefill, and FP8 KV](https://github.com/ROCm/ATOM/pull/1701) and then [extended to MTP](https://github.com/ROCm/ATOM/pull/1746). Parallelism that cannot coexist with the prefix cache would trade one long-context win for another. The same scarcity of parallelism exists inside a single GPU: a batch-1 MLA decode has no head or query dimension to spread, only the KV walk, and a hardcoded split budget of 16 left that walk running on 16 of a gfx950’s 256 CUs. A [still-open change](https://github.com/ROCm/ATOM/pull/1911) stops overriding the kernel’s own split derivation, so Aiter cuts the walk into as many parts as the machine has clusters. + +[Chunked pipeline-parallel prefill](https://github.com/ROCm/ATOM/pull/1552) attacks the same problem from the memory side, replacing repeated tensor-parallel collectives with streamed layer-stage handoffs. Its GLM-5.2 result at high load is the most complete in this section: output throughput doubled, median time to first token fell from 28.6 seconds to 8.7 seconds, and each prefill GPU held 3.68 times as many KV blocks. That last figure is the one to read first, because capacity per prefill GPU is what decides how many long sessions can be in flight before the deployment hits the HBM cliff at all. + +
+ +## ROCm AITER Agentic Optimizations + +ATOM/AMD vLLM/AMD SGLang’s long-context execution depends on matching lower-level AITER kernels, because a parallelism strategy at the engine layer is only real if the kernels can express it. [Prefill context-parallel process groups](https://github.com/ROCm/aiter/pull/3728) provide the extra query-sharding dimension that prefill context parallelism needs, and also widen fused-kernel row indexing for prompts above 131,000 tokens. [Decode context parallelism](https://github.com/ROCm/aiter/pull/3267) (DCP) shards KV across the tensor-parallel GPUs already present, so a longer sequence or a larger batch fits without replicating the whole cache on every rank. + +Large caches also exposed a class of failure that short fixed requests essentially never reach: address width. A 32-bit offset is entirely adequate until a single cache pool crosses the boundary, at which point the arithmetic wraps and the kernel addresses the wrong row without any error being raised. AITER added [runtime 64-bit dispatch for batch prefill above 4 GB](https://github.com/ROCm/aiter/pull/2893), [64-bit MLA offsets above 2 GB](https://github.com/ROCm/aiter/pull/4474), and [64-bit addressing throughout DeepSeek-V4’s unified cache paths](https://github.com/ROCm/aiter/pull/4680), the last preventing silent reads and writes to the wrong row in pools of roughly 150 million rows. + +
+ +DeepSeek-V4 decode also gained [a persistent MLA kernel for 64-head and 128-head MTP packings](https://github.com/ROCm/aiter/pull/3459). Those two head counts are what ordinary decoding and speculative verification actually produce, so this gives the engine a dedicated long-context path for its common shapes instead of treating them as incidental variants of a kernel written for short contexts. It is the same argument as the vLLM AITER sparse-MLA selection above: in long context, the generic path is not a modest compromise, it is the wrong kernel. + +## Dynamo Agentic Workload Optimizations + +A good chunk of Nvidia submissions use the Dynamo Inference Orchestration and Router Systems. Dynamo’s AgentX series shows that the distributed serving layer can become the bottleneck once engine kernels improve. The router’s work is proportional to the number and length of live prefixes rather than to the number of tokens generated, so a workload of many long, overlapping, long-lived sessions loads it in a way that fixed-shape traffic never does. The first series of PRs reduced the cost of each routing decision: [less work on the lookup hot path](https://github.com/ai-dynamo/dynamo/pull/10540), [no redundant suffix invalidation](https://github.com/ai-dynamo/dynamo/pull/10836), and finally [batched KV matching, registration, ownership, and terminal dereferences](https://github.com/ai-dynamo/dynamo/pull/11095), which reported a 22.2% median output-throughput gain at concurrency 512. Batching helps here for the same reason it helps in an engine: the per-item overhead was dominating the item. + +
+ +The second series of PRs changed how ownership is represented, which is the harder problem underneath. Every cached block needs to be attributed to the requests relying on it, so it is not freed while still in use and not pinned after everyone has finished. With thousands of concurrent sessions sharing overlapping prefixes, the bookkeeping itself becomes significant. Dynamo moved from [shared block chains](https://github.com/ai-dynamo/dynamo/pull/11503) to [arena-level ownership counts](https://github.com/ai-dynamo/dynamo/pull/11508) and finally to [backend-specific request leases](https://github.com/ai-dynamo/dynamo/pull/12329), each step coarsening the unit being tracked. The lease design reduced AgentX replay time by 23.7% for the vLLM backend and 22.0% for SGLang, and lowered peak memory at the same time, a sign that the previous representation was the problem rather than the traffic. + +
+ +Further router profiles removed costs with the same shape, where a periodic sweep or a full recomputation had been acceptable only because live state used to be small. [Bucketed expiry pruning](https://github.com/ai-dynamo/dynamo/pull/10521) replaced a scan proportional to everything tracked and improved high-churn AgentX throughput by 13.7%. [Delta-only suffix cleanup](https://github.com/ai-dynamo/dynamo/pull/10676) processes only what changed and absorbed about 28 times as many store and remove events in the same window. [Compressed prompt paths](https://github.com/ai-dynamo/dynamo/pull/11644) cut front-end CPU by 35.3% and materially improved tail time to first token, which matters because prompts in this workload are long and largely repeated. Overload state is now [tracked incrementally](https://github.com/ai-dynamo/dynamo/pull/10645) rather than recomputed. + +
+ +One routing change is a deliberate trade rather than a pure win. Dynamo can now [charge active decode requests in its routing score](https://github.com/ai-dynamo/dynamo/pull/12158), so a worker already committed to long-running decodes looks more expensive than its queue depth alone suggests. That improved median AgentX latency at a small throughput cost in the reported tuning point, which is the kind of choice that only becomes visible when requests occupy a worker for a long time. An [open follow-up](https://github.com/ai-dynamo/dynamo/pull/13447) packages that trade into an new agentic router preset that pushes further in the same direction, crediting prefix overlap at 2, scaling prefill load by 4, and weighting active decode requests at 64. At that tuning point, the trade stops costing throughput: on an 8xH200 AgentX run, the preset improved fixed-window completed-output throughput by 8.26% over the default cost function, cut run-level p95 time to first token by 43.1% and p95 inter-token latency by 22.6%, and completed one more full trajectory. + +The request plane was optimized next, because an agentic trace does not send one request and one response. It sends many related requests carrying largely identical prompts, and streams every token back as its own frame, so serialization and copying are paid per turn and per token rather than once. Switching to [MessagePack request payloads](https://github.com/ai-dynamo/dynamo/pull/10437) improved throughput by 8.1% and reduced average time to first token by 9.7% in its AgentX test, and [direct Python transcoding](https://github.com/ai-dynamo/dynamo/pull/11104) removed an intermediate value tree from that path entirely. + +What followed is a sequence of changes that all remove a copy rather than speed one up: not copying [MessagePack event payloads](https://github.com/ai-dynamo/dynamo/pull/11539), not copying [received ZeroMQ frames](https://github.com/ai-dynamo/dynamo/pull/11574), and not paying full [inter-token-latency metrics overhead](https://github.com/ai-dynamo/dynamo/pull/11569) on every token. [The chat streaming hot path](https://github.com/ai-dynamo/dynamo/pull/10433) was shortened for the same reason. Individually, these are unremarkable; multiplied by every streamed token of every concurrent session, they are what determines how many requests per second a frontend can sustain. + +High-concurrency profiling then found costs that had nothing to do with moving data. [Static logging filters](https://github.com/ai-dynamo/dynamo/pull/11820) removed a shared span-matcher lock, a contention point rather than a volume problem, and raised reported frontend throughput from 932 to 1,133 requests per second. [Simpler positional radix buckets](https://github.com/ai-dynamo/dynamo/pull/12161) reduced peak memory in the mocker by 5.51 GiB in a 32-worker run. An open change [flushes detokenization metrics once per response](https://github.com/ai-dynamo/dynamo/pull/12999) rather than updating cumulative counters on every streamed chunk, approximately halving frontend CPU time in its matched diagnostic profile. That last one is the clearest example of the category: the instrumentation was cheap per call and ruinous at one call per token. + +## LMCache Agentic Optimizations + +LMCache is an open-source KV cache layer that sits under inference engines like vLLM, storing reusable KV chunks keyed by prefix hash across CPU DRAM, local NVMe, and remote backends (Mooncake, Redis, S3). LMCache can be used as an alternative to vLLM’s native offloading connectors. + +
+ +LMCache’s multiprocess path was changed for the volume and shape of agentic cache movement, beginning with a failure that is not a slowdown but a stop. When each of many requests with contexts above 100,000 tokens reserves the blocks for its whole load before starting, the pool is exhausted by requests that are all waiting and none progressing. [Chunked external-cache loading](https://github.com/LMCache/LMCache/pull/3382) reserves per chunk instead, so loads interleave and drain. At concurrency 32, the validation completed 120 requests where the old path deadlocked after 28, and concurrency 48 kept running with the KV pool 98.5 percent full. + +The other changes reduce how much is moved and how often the runtime gets in the way. Storing only the useful portions of DeepSeek-V4’s hybrid groups cut [storage per token by almost twenty times](https://github.com/LMCache/LMCache/pull/3635), and sliding-window prefetch now [loads only the live window](https://github.com/LMCache/LMCache/pull/3869) rather than window state that will never be read, the same reachability argument vLLM applied to offload, approached from the storage side. [One native transfer call per object group](https://github.com/LMCache/LMCache/pull/3908) then removes repeated Python lock handoffs across staging copies and kernel launches, which is overhead proportional to the number of pieces rather than to the bytes in them. + +Two current LMCache changes are especially specific to AgentX but remain open. [The hybrid lock-accounting fix](https://github.com/LMCache/LMCache/pull/4524) stops one request from releasing another request’s read locks on shared sliding-window or recurrent-state chunks. Several requests must share the same chunks, the accounting must be per-chunk rather than per-holder, and eviction must actually start. Sustained Kimi-K3 runs with DRAM offload supplied all three and produced tens of thousands of warnings, corrupt generations, and eventually GPU crashes once eviction began. Anything short of a long, shared, memory-pressured run leaves it dormant. + +A parallel line of LMCache work made all of the above reachable on AMD Instinct hardware. CacheBlend’s non-prefix reuse depended on flashinfer, which is CUDA-only, so [a Triton block-sparse attention backend](https://github.com/LMCache/LMCache/pull/3092) reimplements the three kernels it needs: block-sparse attention with CSR indices and log-sum-exp output, causal prefill, and log-sum-exp output blending. It then routes to them automatically when ROCm is detected or flashinfer is missing. [ROCm Dockerfiles](https://github.com/LMCache/LMCache/pull/3101) mirror the CUDA build and lightweight images. [An AMD hipFile backend](https://github.com/LMCache/LMCache/pull/3843) extends the GDS L1 slab-file tier, which reached storage only through NVIDIA cuFile, by binding ROCm’s hipFile through ctypes and dispatching on torch.version.hip; the cuFile path is unchanged. + +Distribution was the remaining gap. CUDA users installed a prebuilt wheel; AMD users built it from source. We worked together with AMD to publish a [prebuilt gfx942 and gfx950 wheel](https://github.com/LMCache/LMCache/pull/4273) which closes that. It installs into the upstream image and passes all 56 KV-transfer kernel tests on MI350X, and it publishes to a GitHub release rather than PyPI so a plain pip install lmcache stays the CUDA build. [A one-line follow-up](https://github.com/LMCache/LMCache/pull/4363) marks the bind-mounted repository as a git safe directory, which only fails in CI because the container runs as root over a runner-owned checkout and the version introspection in setup.py refuses to read it. + +
+ +[DCP-aware CPU offload](https://github.com/LMCache/LMCache/pull/3561) resolves a straightforward incompatibility between two features that long contexts make mandatory together. With decode context parallelism enabled, each rank holds only a stride of the KV, so what any one rank could save is not a usable prefix; the fix gathers the strided shards before saving and redistributes them after loading. Without it, enabling context parallelism silently disables CPU cache hits for exactly the long prefixes that motivated both features. Its validation recorded more than 30,000 CPU hit events, with single-request loads reaching hundreds of thousands of tokens. + +## Mooncake Agentic Optimizations + +Mooncake serves Moonshot’s Kimi production traffic along with production traffic at many labs, and is a transfer engine underneath disaggregated vLLM and SGLang configurations. Until recently, Mooncake’s AMD support stopped short of both RDMA registration and offering installable packages. + +
+ +Registering GPU memory for RDMA on Nvidia either uses the nvidia-peermem kernel module or exports a dmabuf file descriptor. AMD has no nvidia-peermem equivalent, so GPU-direct RDMA had no path at all and deployments fall back to staging KV through host DRAM. [A HIP dmabuf registration branch](https://github.com/kvcache-ai/Mooncake/pull/2225) adds the mirror of the existing CUDA dmabuf path, exporting through ROCm instead of the CUDA handle call, and resolving the true allocation base first because caching allocators pack tensors at an offset inside a larger allocation. Host memory still registers directly. + +Support that cannot be installed is not support. Mooncake published CUDA and MUSA wheels but no ROCm package, so AMD users built the engine from source inside every image. [A ROCm wheel, CI, and release path](https://github.com/kvcache-ai/Mooncake/pull/3184) publishes mooncake-transfer-engine-rocm to PyPI alongside them. This workstream from Andy Luo, AMD engineer, was due to noticing a pattern when dogfooding agentic workloads with AgentX that building MoonCake from source in ROCm is not an first class citizen pattern. + +
+ +The transfer engine has no device kernels and does not depend on torch, so one architecture-agnostic wheel covers gfx942 and gfx950, and the ROCm runtime is bound at load time rather than vendored, which means the same wheel works unmodified in both the upstream vLLM ROCm image and the SGLang ROCm image. That was verified as a full cross product: MI300X and MI355X, each under vllm/vllm-openai-rocm and lmsysorg/sglang, running the master binary and a HIP buffer transfer test with data verification. The pull request adds a tag-triggered publish across Python 3.10 through 3.13. An open follow-up [adds a self-hosted two-node MI350X external prefill and decode tier](https://github.com/kvcache-ai/Mooncake/pull/3338) so the ROCm disaggregated path is exercised on real hardware rather than only compiled. + +Together, these PRs mean an AMD AgentX run can now install the transfer engine and the KV cache layer from published artifacts into stock upstream images, and move KV directly between GPU memory and the fabric. + +## Other Optimizations + +The changes above address long-context costs: a prefix that has to survive, a hybrid cache that has to stay correct, a transfer that has to keep up. But there are a whole host of day-zero enablement and correctness bugs that break requests just as badly as a million-token session. + +MiniMax-M3 tested whether that ROCm work compounds into day-zero readiness, and the [Advancing AI writeup](https://newsletter.semianalysis.com/p/can-amd-break-the-cuda-moat-amd-advancing) draws the comparison directly: AMD’s first public disaggregated recipe, MI355X FP4, reached InferenceX in January months behind Nvidia, while M3 FP4 disaggregation landed on day zero. This is an improvement from the DeepSeek-R1 period, when parity took months. Three vLLM fixes sat on that day-zero path, and each was a correctness failure rather than a performance one. + +Disaggregation was blocked first. NixlConnector’s handshake asserted that the SPLIT-region block_len scales with the prefill-to-decode TP ratio, but block_len follows per-rank KV heads. M3 has 4 KV heads, so a TP4 prefill paired with a TP8 decode is GQA-capped to one head per rank on both sides and the two lengths are equal where the assertion demanded a factor of two. The handshake was rejected, no KV moved, decode regenerated everything from scratch, and gsm8k scored 0. [Validating against the actual head ratio](https://github.com/vllm-project/vllm/pull/45879) fixed this. + +The other two were platform splits. M3’s sparse-attention backend read the byte-backed FP8 cache as float8_e4m3fn for every E4M3 configuration. But gfx942’s platform dtype is e4m3fnuz, and the two encodings differ. K and V were therefore altered before the kernels consumed them. The prefill and decode wrappers had also omitted the FNUZ types from their FP8 checks. [Using the platform dtype for the cache view](https://github.com/vllm-project/vllm/pull/45720) fixed both halves. Separately, M3 ships as separate NVIDIA and AMD model files, and only the NVIDIA one implemented the EAGLE3 interface, so speculative decoding aborted at engine init on ROCm with a model-does-not-support error. [Bringing the AMD model to parity](https://github.com/vllm-project/vllm/pull/45546) restored it, with MI355X gsm8k matching both the non-EAGLE3 MI355X run and B200. + +The TensorRT-LLM section above covers the M3 work that is long-context specific: descriptor explosion in disaggregated KV transfer, context graph capture, sparse block strides, autotuner candidates, and the corrupt split-K MoE tactics that had to be removed from the pool. + +The local AgentX matrix combines session-aware or KV-aware routing, long and variable conversation histories, MTP, hybrid attention, aggregate and disaggregated serving, and concurrency sweeps that cross the HBM capacity cliff. It includes GPU-resident comparisons and CPU DRAM offload through vLLM SimpleCPU, Mooncake, LMCache, and SGLang HiCache. That combination is what activates the upstream work above. The old fixed-sequence matrix usually creates one prompt, performs one prefill, decodes one fixed continuation, and discards the request. It therefore does not measure cache survival across turns, repeated tokenization, session affinity, cache-event traffic, offload churn, transfer progress during scheduler stalls, or long-lived ownership bookkeeping. + +The allowed optimization policy treats CPU KV offload as optional. A vendor may use vLLM connectors, LMCache, SGLang HiCache, Mooncake, Dynamo KVBM, or another CPU DRAM connector, or disable offload when the resulting latency and throughput point is better. NVMe offload is deferred. CPU DRAM must scale with the fraction of GPUs used, including the 3 TB cap for non-standardized-DRAM systems. Standardized-DRAM systems have no hard cap but retain the same proportionality rule. The local generator currently applies the 3 TB cap to every runner, so it does not yet implement the standardized-DRAM exception. + +The net new optimization surface is not simply longer attention. It is the preservation, movement, routing, reconstruction, and repeated processing of a growing session state. AgentX made those costs large enough to drive generic upstream changes across vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, and LMCache. Direct searches of NIXL and Mooncake did not identify additional AgentX-tagged runtime PRs, so their relevant effects remain represented through the engine connector changes above. + +## AgentX Methodology Deep Dive + +AgentX is a massive shift in open source real world long context multi turn agentic trace replaying and we collected traces over $3M worth of tokens within our own dataset consisting of real world traffic from Claude Code, OpenAI Codex, etc. In addition to the dataset, we developed a comprehensive methodology for replaying the traffic patterns fairly. The goal is to stay as authentic to the organic traffic as possible while being equitable about GPU resource requirements. + +We will deep dive into the agentic trace datasets, replay methodology, and agentic behavior in general. It is recommended reading for readers that would like to better understand the overall shape of agentic workloads as well as how harnesses orchestrate requests under the hood. + +## $3M USD Agentic Traffic Trace Collector + +When initially designing AgentX, our north star goal was to make the benchmark as realistic as possible in terms of KV workload shape and KV reuse patterns. We began experimenting with replaying some existing datasets, such as SWE-bench, Qwen-Bailian, and other random Claude Code traces from HuggingFace. At the time, these datasets did not include significant use of subagents, 1M context, compactions, dynamic workflows, or many other of the recent defining characteristics of an agentic trace. At SemiAnalysis, most of the team are AI power users and use agents for a broad variety of tasks including coding, analyst research, excel modeling, social media operations, and many more. Therefore, we decided that the most achievable and realistic traces could be captured in house. + +To collect a large number of traces, we created a proxy that intercepts HTTP requests to Claude / Codex. Then, users that wished to upload traces simply changed the base URL in their Claude / Codex setup to point to the proxy. At the time of writing, we have collected over 8,000 sessions, 3.4 million requests, and 610 billion tokens. Together these represent more than $3M USD in spend. [We open sourced a representative subset of these sessions for the AgentX v1.0 benchmark](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126). + +Although agentic harnesses can appear complex, they ultimately orchestrate a sequence of HTTP requests. Each request contains some combination of system instructions, tool definitions, and accumulated conversation history. As a session progresses, this history grows and is repeatedly sent back to the model, creating the long contexts and high prefix reuse that AgentX is designed to reproduce. + +Our proxy records these requests and responses as they occur. It also extracts metadata/HTTP headers such as timestamps, conversation IDs, and subagent IDs, which lets us recover the _structure_ of the conversation (request ordering, concurrent branches, and the approximate parent/child structure of each session). This metadata is what allows us to replay the traces approximately how they would have been seen by the original Anthropic API server. + +To protect employee privacy, the replay dataset contains no original prompts, source code, tool arguments, or tool results. Instead, we tokenize each requests’ content and then group it into 64-token blocks, finally replacing each block with a session-scoped chained hash. Matching prompt prefixes therefore produce matching hash prefixes without revealing their contents ([this paper](https://arxiv.org/abs/2506.02634) talks more about this strategy). During replay, these hash blocks can then be replaced with tokens from, say, a coding dataset. So we preserve the approximate context growth and conversation KV-reuse patterns of the original workload. + +It’s worth noting that this process is necessarily imperfect, mostly due to the fact that when using a frontier model provider’s API, much of the content that the end LLM server actually sees is hidden. For instance, thinking/reasoning content for SOTA models are now encrypted in HTTP requests and replaced with a deterministic hash in an attempt to hinder distillation attacks. However, it seems like [this didn’t work out exactly as well as the big labs had planned…](https://arxiv.org/html/2608.09867v1) + +
+ +Furthermore, while we have access to all the raw _content_ of the user/assistant messages, system prompts, tool usage, etc., API providers apply additional chat templating server side that aren’t transparent. We also cannot observe Anthropic’s proprietary tokenizer or context introduced by server-side tools. Images and documents also do not have a straightforward correspondence between their wire representation and the number of tokens processed by the model. We use deterministic placeholders and empirically calibrated, model-specific padding to bring reconstructed prompt lengths to best estimate the content that is actually seen by the server. + +We can’t _perfectly_ capture and replay Claude Code / Codex traces as they are _actually seen_ by the Anthropic / OpenAI servers due to incomplete information, but we can get pretty close. The chart below shows the ratio of hash tokens (after our approximations/processing) and the true API provider token count across all request lengths and models. + +
+ +To summarize, collecting real Claude Code traces in the exact way that they would have been replayed against the original server is not easy due to incomplete information. However, we have enough context to collect and replay traces with extremely high fidelity to match original traffic patterns, timing, prefix caching, and DAG patterns. + +## AgentX $3Mil Dataset + +The dataset used for AgentX v1.0 can be found on [HuggingFace](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126). It is a 393 session subset of 8.3k session proxy corpus mentioned in the previous section. Additionally, we applied some post-processing to clean up anomalies, such as: + +- Removing Claude Code security monitor (auto mode) requests and title generation requests as these are specific to Claude Code and not necessarily representative of general agentic traffic +- Removing requests with a reconstructed input length greater than 990k tokens (where our approximation overcounted) +- Remove duplicate requests (sometimes the proxy received identical requests if the connection was dropped) + +Additionally, each conversation is formatted into the WEKA trace format, proposed by Callan Fox as part of his [kv-cache-tester](https://github.com/callanjfox/kv-cache-tester) project. We chose this format for storing trace information primarily because we worked closely with Callan to develop the benchmark and found it intuitive for storing per-session traces. All in all, the trace format is quite arbitrary and our proxy dataset could be mapped to other formats such as [Mooncake](https://docs.nvidia.com/aiperf/benchmark-modes/trace-replay-with-mooncake-traces). + +
+ +After these are applied, we get the following dataset. Note that not all X-axes are the same. + +
+ +The distributions for ISL/OSL and inter-turn latency (in agentic work, this is mainly time taken for tool use) are relatively log-normal. The median ISL is 142k tokens and the median OSL is 444 tokens. The median inter-turn latency (or “tool use time”) was 3.84 seconds. Only ~10% of inter-turn latency was greater than 1 minute. These are likely made up of gaps where the harness is waiting on an actual response from a human. + +One thing worth mentioning is that these request distributions will look different depending on which harness is being used, since different amounts/types of context are injected (for instance Pi is known to be minimalist in terms of harness-injected context while Claude Code is known for the opposite. Additionally, the ISL/OSL distributions will depend on the model, as different models have tokenizers that can produce either more/fewer tokens. However, given a significant portion of the world’s agentic coding traffic goes through Claude Code, we believe this is rather representative. + +The dataset also has 175 sessions with at least one subagent (~44% of all sessions). There are 1,697 total subagent rollouts in the dataset, with a median of 4 per session. The median wall-clock time for a subagent (beginning of first request to end of last request) is 2.27 minutes. This distribution again follows a relatively log-normal distribution. + +
+ +This dataset includes context up to 1M context, meant to test the more recent frontier open-weight models. Additionally, we have [a truncated 256k context length dataset](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126-256k) which we replay against models with a max context length of 256k or less. + +## Agentic Traffic Trace Replayer + +Rather than build a replay solution from the ground up, we decided to partner with [AIPerf](https://github.com/ai-dynamo/aiperf), a vendor agnostic HTTP replayer tool from Nvidia that is adopted by many in the industry including tenstorrent, AWS, AMD, etc. While the intention is to integrate AgentX features into the upstream repo, we maintain a separate [fork](https://github.com/SemiAnalysisAI/aiperf) to be even more vendor neutral such that we have control over allowing even more 3rd more contributions. Again, thank you to the AIPerf team, especially Anthony Casagrande, for the help and dedication to building a realistic and representative agentic benchmark. + +An agentic session is naturally described as a directed acyclic graph (DAG). Each request is a node, and an edge means the request at its head cannot be issued until the one at its tail has completed. Every edge additionally carries a delay, specifying how long to wait once that precondition is met. + +The simplest session is completely linear, with no subagents and no parallel requests, each request depends on exactly one predecessor. The graph degenerates to a line and the only thing an edge encodes is the inter-turn latency (aka, tool use time or “think” time), which is the client’s local work rather than the model’s. + +
+ +In agentic traces, subagents can also be spawned. A subagent is a separate stream of requests that has its own context, typically to do a focused task. Multiple subagents can run in parallel to do more aggregate work, and subagents can run in parallel to the main agent in some cases. The main agent then waits on groups of subagents to finish, and then incorporates their outputs back into the main agent’s context (while this is not _always_ the case, this is the most common pattern). + +This behavior is responsible for turning the linear chain of requests above into a DAG, where certain requests are dependent on others. When a group of requests belonging to one subagent is identified, AIPerf finds the most recent main agent predecessor and designates it as the “spawning” request. Similarly, the “join” request is identified by the subsequent main agent request after the duration of the subagent group completes. + +In the example below, the subagent group consists of a single subagent (001), which runs two requests. Its first request goes out as soon as the main agent’s opening request completes. When that request completes, a 2.2-second inter-turn delay stands in for tool-use wall-clock time, and then the subagent’s second request is sent. The main agent’s second request is the join point for subagent 001. It goes out once both conditions are met: at least 17 seconds have elapsed since the main agent’s first response _and_ subagent 001’s second request has completed. + +One small limitation is that HTTP timestamps reveal timing, but not always causality. In the example below, if subagent 001 finishes with seven seconds remaining before main-agent request 2’s recorded start, we cannot tell whether those seven seconds represent work performed after the subagent returned or independent work already underway. AIPerf therefore preserves both constraints: request 2 waits for its recorded main-path delay and for subagent 001 to finish. This reproduces the observed timing and workload topology, but not any dependencies hidden inside the harness. These are things we hope to improve on in subsequent versions of AgentX. + +
+ +Multiple subagents can also be spawned from a single request. In the example below, subagents 001 and 002 both identify their spawning parent as main agent request 1, and then join at main agent request 2. + +AIPerf can also identify “auxiliary” requests, which are one off requests that do not share context with any other requests in the stream. These branch off of the main agent and never join back. In practical terms, these are requests like Claude Code’s “summarize this session” requests that are unrelated to the conversations context. Another good example is Claude Code’s “/btw” feature. + +
+ +The example below brings it all together. This is the type of trace snippet you’d see in the actual dataset. We have five parallel streams leaving a single main agent request, separated into groups defined by what main agent request they join upon. + +Subagents 001 and 002 finish at 20 and 23 seconds, so the first main agent request starting after that, at 25 seconds, is their join. Subagents 003 and 004 are spawned in the same gap but run much longer, finishing at 46 and 50 seconds, so they join at 52 seconds instead. AIPerf keys each subagent by the pair (spawning request, join request), which means these four streams collapse into two branches even though all four leave the same node. The branch takes the name of its first member, which is why the join edges are labelled with subagent 001 and subagent 003. + +This is also the first case where the main agent overlaps its own subagents. The request at 25 seconds goes out while 003 and 004 are still running: the main agent is blocked only on the group that joins it, not on every subagent in flight. + +The auxiliary chain attaches to whichever main agent request most recently preceded it, which here is the request at 52 seconds rather than the one that opened the session. That node therefore does two things at once — it receives the second subagent group’s join, and it spawns the one-off. Of course the auxiliary request never joins back. + +
+ +## Further AgentX Deep Dive + +### Pareto Frontiers Sweeping + +In the AgentX workload, in order to generate a Pareto frontier, we sweep over the number of concurrent Claude Code sessions against a single deployment. Since each conversation has realistic inter-turn delays and subagent usage, we get a spikier, more realistic traffic pattern. The example below is an example of replaying 40 concurrent clients against a B200 TP4 vLLM server running MiniMax M3. + +
+ +
+ +A final point worth discussing is what metrics are important to consider when evaluating agentic workloads. We believe interactivity (TPS - tokens per second) and time-to-first-token (TTFT) are still important, and these are the industry standard for evaluating SLOs. When viewing AgentX results, it is _extremely important_ to consider both TPS and TTFT together, since there are inference optimizations that can improve one at the cost of the other. + +We are currently working on defining a new metric that combines TPS and TTFT in a meaningful way. This should also take into account that in agentic workloads, people often care more about the end-to-end speed at which a _task_ finishes rather than how fast they receive tokens or TTFT. + +Finally, it’s worth mentioning that the end-to-end latency, in its current form, is now less meaningful because end-to-end latency is directly proportional to OSL. Therefore, P90 E2E latency is heavily affected by the 10% tail of longest output sequence lengths. While it can still be good to holistically compare the overall performance of certain configurations, we recommend instead looking at a combination of TPS and TTFT. + +### Warmup, Timing, Determinism, and Conversation Reuse + +The goal of AgentX is to benchmark systems that are already in a steady state. With agentic workloads, this means that profiling should start from a point where some context trajectories are already cached. To mimic a steady state, it is also desirable that not all conversations start at turn 0, which may cause a “[thundering herd](https://en.wikipedia.org/wiki/Thundering_herd_problem)” effect. + +Warmup proceeds in two stages. First, AIPerf uses a fixed random seed to select a wall-clock point between 25% and 75% of each conversation. At that point, it identifies every active request stream, including the main agent and any active subagents, and sends the most recent request before the selected point for each stream. These primer requests reconstruct the conversation state at that point and are dispatched together. AIPerf waits for them to drain before continuing. + +
+ +In the second phase, each replay lane is advanced by 10 additional requests to give additional opportunity for the KV cache to materialize. All warmup requests omit inter-turn delay and use a maximum output length of one token, substantially reducing warmup time. + +
+ +
+ +When warmup is complete, profiling begins and lasts one hour. All metrics are collected strictly over this duration. For reproducibility, AIPerf accepts a seed that ensures each run samples conversations deterministically, conversations start at the same point, and that each conversation is reconstructed with the same synthetic content run-to-run. During profiling, we impose a 5 minute idle time cap on each stream, so that long inter turn gaps don’t “acquire” a worker lane for the duration of the benchmark. We enforce this so that we can effectively run the benchmark in 1 hour. In later versions of AgentX where we include NVMe offloading, we may choose to increase this so that we can measure a longer TTL. For now, 5 minutes is reasonable as this is [Anthropic’s default KV cache TTL](). + +This level of determinism ensures that runs using the same inference engine, hardware, concurrency, and server settings are reproducible. However, because AgentX is a [closed-loop benchmark](https://notpeerreviewed.com/blog/tail-latency/), different configurations will complete different numbers of requests at different rates, introducing some natural variation in the workloads they encounter. This is most noticeable at lower concurrency, where fewer requests are completed (naturally) and the workload has less opportunity to converge toward the dataset’s overall distribution. + +When a conversation completes during profiling, its replay lane selects another conversation from the dataset sampler. Each replay receives a unique, deterministic cache-bust marker that is prepended to every independent prefix chain, including the main-agent chain and any fresh-context subagent or one-off chains. Forked subagents inherit the marker from their parents. The marker stays the same within a replay, preserving its KV-reuse patterns, but changes between replays to prevent artificially high cache-hit rates. This also enables scenarios to run where concurrency is greater than the number of conversations in the dataset (393). + +### AgentX’s Fair Speculative Decoding Methodology + +As mentioned, the dataset is anonymized upon collection. This means that the 64-token hash blocks must be synthetically filled in before replay. AIPerf accomplishes this by deterministically sampling from a synthetic coding/tool-use token pool. + +Importantly, the KV reuse patterns as well as request timing are maintained, however the synthetic request data does lead to some additional considerations. Namely, running speculative decoding methods on synthetic data may lead to the speculator rejecting/accepting an abnormal number of tokens when compared to non-synthetic data (since the speculator is not trained on synthetic data). + +We talked about this shortcoming in our [InferenceX v2 article](https://newsletter.semianalysis.com/i/188090866/multi-token-prediction-mtp), and have since then improved on our methodology. We have worked closely with the community to ensure a mechanism exists in most OSS inference engines that allows users to force how many draft tokens to accept from the speculator (aka, “acceptance length” or “acceptance rate”). Then, for each (model, speculator, draft length, and thinking mode) combination, [we collect the average AL](https://github.com/SemiAnalysisAI/InferenceX/tree/main/golden_al_distribution) on the [SPEED-Bench](https://huggingface.co/datasets/nvidia/SPEED-Bench) agentic coding dataset, a “unified benchmark designed to evaluate speculative decoding (SD) across diverse semantic domains and realistic serving regimes.” + +Then, at runtime we apply these realistic Speculative decoding acceptance lengths to AgentX to ensure vendor neutral fairness. + +
+ +## Exploring Agentic Workloads Detailed Telemetry Tutorial + +AgentX required more than a new benchmark harness and dataset. [We also spent some time rebuilding parts of the InferenceX visualization to make agentic results easier to explore and digest](https://inferencex.semianalysis.com/inference/agentic/439903). A single AgentX datapoint represents thousands of requests across growing conversations, subagents, warmup periods, cache states, and dynamically changing in-flight load. Due to this, having a single point on a Pareto curve can hide a lot of useful information. As we have said many times - there is never just a one size fits all solution for inference serving. + +
+ +One of our major changes is how we construct the curves themselves. In previous versions of InferenceX, configurations with speculative decoding enabled and disabled were often displayed as separate curves. However, we are now moving away from this approach. The frontend now combines allowed inference optimizations and displays the best available curve for each model, SKU, and inference engine combination. Due to this, individual points along a single curve may use different optimization techniques and configurations, including speculative decoding, disaggregation, or KV cache offload. + +Our goal is to show the best production performance available from each hardware and software stack, rather than creating a separate curve for every possible combination of optimizations. However, we still expose the underlying configuration and provenance for every point. Clicking a point shows a tooltip with a detailed view showing exactly which configuration produced it, along with the run metadata, links to the publicly viewable CI provenance, and AgentX specific statistics. From there, the “View charts” link opens the full point-detail page with AgentX specific statistics. + +
+ +The detailed point view provides a much deeper look into the selected AgentX run. It includes input and output sequence length distributions, interactivity and TTFT over time, KV cache utilization, request queue depth, prefix cache hit rate, input and decode throughput, prompt-token source breakdown, and unique input tokens over time. These metrics make it easier to understand why two points with similar aggregate throughput may behave differently throughout the replay. + +The page also separates warmup and profiling data. Readers can switch between the two phases to inspect how the system behaves while its cache state is being established and during the profiling period used for the benchmark run. + +
+ +Points using KV cache offload are surrounded by an additional dotted circle on the main chart, which is used to distinguish points with KV offload enabled. When one of these points is selected, the detail page shows the offload type, KV offload engine, chip cache-hit rate, and CPU cache-hit rate. This makes it possible to see where KV offload contributes to the best curve without creating a separate curve for every offload configuration. + +Another new feature is the request timeline. This view shows the individual requests replayed during a selected AgentX run and can be organized either by conversation or by worker. The conversation view groups subagents underneath their corresponding root conversation, making it easy to see when conversations and subagents overlap. Warmup and profiling requests can also still be viewed separately. + +
+ +Each request in the timeline is clickable and links directly to the corresponding conversation and turns on the InferenceX datasets page. This allows readers to move from an aggregate point on the Pareto curve to the exact anonymized request that was replayed. + +[The AgentX page](https://inferencex.semianalysis.com/datasets) also includes a flamegraph for visualizing the structure of an individual conversation. Each bar represents one turn and is scaled relative to the largest turn in that conversation. The bar is divided into cached prefix tokens, uncached input tokens, and generated output tokens. This gives a visual representation of how the context grows throughout a conversation and how much of each request can be reused from KV cache. + +
+ +## InferenceX/AgentX Future Next Steps + +We are excited to continue our goal of making AgentX the most realistic and representative benchmark. For the foreseeable future, we will continue making [small bug fixes to the current v1.0.x harness](https://github.com/SemiAnalysisAI/aiperf/releases), but have plans to make bigger changes to the v1.1 harness. Submissions for each distinct model will always run the same minor version to ensure all results are comparable. + +As a fast follow, we will add SSD/NVMe KV offloading. This will allow an even larger KV cache working set size than DRAM allows, which will allow the high throughput left side of the pareto curve. + +We will also capture a larger, more diverse, and more recent dataset of agentic traces across a wider variety of models and harnesses. Rather than representing each request as one contiguous list of hash IDs, the next dataset will preserve the boundaries between system instructions, user and assistant messages, tool calls, and tool results. This will allow AgentX to evaluate workload-aware serving techniques that use information available to the agent harness but normally hidden from the inference engine. For example, a router could direct low-reuse tool traffic to dedicated prefill workers, retain an agent’s prefix during a long tool call, or prefetch and share prefixes when subagents fork. The current format preserves request sizes and KV-reuse patterns, but it lacks the structure needed to evaluate these optimizations. + +The following [SGLang RFC](https://github.com/sgl-project/sglang/issues/27574) by Ishan provides a concrete example of why this richer trace structure matters. It proposes a router-initiated hint interface that uses information such as session lifecycles, shared-prefix boundaries, tool-call duration, and subagent state to tell the engine when KV should be shared, prefetched, demoted, pinned, or retained. Capturing this structure would allow future AgentX versions to evaluate these workload-aware cache policies instead of only replaying flat token prefixes. + +
+ +Lastly, we are capturing fine-grained and coarse grain power telemetry data to have an even more accurate view of the efficiency of the Joules per intelligence of different software and hardware stacks. + +We have so much data and so many possible visualizations. Please let us know any visualizations you would like to see, as well as any more general feature requests! + +[Suggest Frontend Feature!](https://github.com/SemiAnalysisAI/InferenceX-app/issues/new/choose) + +## Performance Throughout the Model Lifecycle + +In the following sections, we turn to our historical single turn data (8k1k), which covers each model from launch day through the point we retired it from active testing. There are strong results here from both Nvidia and AMD, including several fixed sequence length configurations where MI355X comes out ahead. + +AgentX better represents today’s agentic inference workloads. However, the historical fixed-sequence InferenceX results still remain useful for tracking the performance over time. Workloads such as 8k1k and 1k1k strip away most session-level behavior, including prefix reuse, persistent KV cache state, and routing affinity. This makes them less representative of current production traffic, but still useful for tracking how inference performance improves as software support matures. + + + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. + + + +--- + +_This article continues on our Substack. [Subscribe to SemiAnalysis](https://newsletter.semianalysis.com/subscribe) to read the complete article._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "What is AgentX 1.0?", + "acceptedAnswer": { + "@type": "Answer", + "text": "AgentX 1.0 is the world's first fully open source, multi-turn agentic coding inference benchmark at 1 million context, released under Apache 2.0. It replays real anonymized Claude Code and Codex traces instead of fixed sequence length prompts, and runs on roughly 2MW of continuously operated compute across more than 1000 chips including MI355X, GB300 NVL72, GB200 NVL72, B300, B200, MI325X, MI300X, H200, and RTX Pro servers." + } + }, + { + "@type": "Question", + "name": "What makes an agentic workload different from a fixed sequence length benchmark?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Agentic workloads are multi-turn, long context, have very high prefix reuse, and launch bursty short-lived subagents. That makes agentic inference a systems problem rather than a kernel problem: KV tensors must be transferred across nodes, requests must be routed to the rank holding the right prefix, and long contexts force KV offload to DRAM or SSD. Fixed sequence length workloads such as 8k1k strip all of that away and mostly measure baseline chip and kernel performance." + } + }, + { + "@type": "Question", + "name": "Does the CUDA moat hold up on agentic workloads?", + "acceptedAnswer": { + "@type": "Answer", + "text": "It depends on the model. On DeepSeek V4 Pro the race is close: MI355X SGLang matched B200 vLLM on performance per dollar until August 21, 2026, when vLLM optimizations from Inferact and NVIDIA pushed B200 ahead, although B300 vLLM and B200 SGLang were already beating MI355X. On MiniMax M3, Qwen3.5, and GLM 5.3, NVIDIA leads by a wide margin, with over 20x better performance at 90 tok/s/user on Qwen3.5 SGLang and up to 5x better cost efficiency on GLM 5.3 at 150 tok/s/user. Context parallelism is a concrete part of the moat: every AMD backend is listed as unsupported in the vLLM attention backend support matrix." + } + }, + { + "@type": "Question", + "name": "What is the AgentX dataset made of?", + "acceptedAnswer": { + "@type": "Answer", + "text": "SemiAnalysis collected traces through a proxy that intercepts HTTP requests to Claude and Codex, gathering over 8,000 sessions, 3.4 million requests, and 610 billion tokens, representing more than 3 million USD of spend. The open sourced AgentX v1.0 subset contains 393 sessions with a median input sequence length of 142k tokens, a median output sequence length of 444 tokens, and a median inter-turn latency of 3.84 seconds. 175 sessions, roughly 44 percent, contain at least one subagent, with 1,697 subagent rollouts in total. Content is anonymized into 64-token blocks replaced by session-scoped chained hashes, so prefix reuse patterns survive but prompts do not." + } + }, + { + "@type": "Question", + "name": "What is E2E Normalized Interactivity?", + "acceptedAnswer": { + "@type": "Answer", + "text": "E2E Normalized Interactivity is an experimental metric defined as output sequence length divided by end-to-end latency. Because end-to-end latency equals TTFT plus output length times TPOT, the metric works out to interactivity plus a penalty proportional to TTFT, capturing both responsiveness dimensions in one number. It is not perfect: it heavily penalizes high TTFT and does not capture every nuance of optimizations such as prefill-decode disaggregation, so AgentX v1.0 submissions still optimize for interactivity and TTFT separately." + } + }, + { + "@type": "Question", + "name": "What industry impact has AgentX had so far?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Over 70 upstream pull requests across vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, LMCache, and Mooncake used AgentX as the north star benchmark. Examples include hybrid-attention prefix caching in vLLM that reported over 95 percent prefix cache hit rate at 1M context, CPU KV offload extended to DeepSeek V4 for 81.7 percent higher output throughput and 46.6 percent lower mean end-to-end latency, boundary-aware incremental tokenization in TensorRT-LLM that cut mean processing time from 185.1 ms to 11.3 ms on Qwen3.5, and sparse checkpoint retention in AMD ATOM that lifted the DeepSeek V4 prefix hit rate from 5.6 percent to 96.45 percent." + } + } + ] +}`} diff --git a/packages/app/content/blog/brief-overview-of-agentic-workloads.mdx b/packages/app/content/blog/brief-overview-of-agentic-workloads.mdx new file mode 100644 index 000000000..5555973b7 --- /dev/null +++ b/packages/app/content/blog/brief-overview-of-agentic-workloads.mdx @@ -0,0 +1,136 @@ +--- +title: 'A Brief Overview of Agentic Workloads' +subtitle: 'Multi-turn sessions, long contexts, and near-total prefix reuse make agentic inference a systems problem, and change what a benchmark has to measure' +seoDescription: 'What defines an agentic workload: multi-turn sessions, long context, high prefix reuse, and bursty subagent fan-out, and why benchmarking them is a systems problem.' +date: '2026-08-19' +publishDate: '2026-08-19' +tags: + - agentic + - agents + - agentx + - benchmark + - inference +--- + +At a high level, an [agentic workload](/glossary/agentic-coding-workload) is characterized by four things: + +- **Multi-turn.** A session includes tens or hundreds of user and assistant interactions, compared to a handful in a chatbot scenario. +- **Long context.** Between system prompts, tool definitions, and the sheer number of turns, context accumulates quickly. +- **High [prefix reuse](/glossary/prefix-caching).** The conversation progresses linearly, with the output of turn _n-1_ concatenated into the input of turn _n_, so most context can be served from [KV cache](/glossary/kv-cache) rather than recomputed. How much depends on the storage available to hold KV tensors. As _n_ grows, the ratio of cached to uncached input tends toward 1. +- **Bursty [subagent](/glossary/subagent) fan-out.** A turn can launch several short-lived subagents at once, which creates spiky KV cache allocation and eviction patterns. + +
+ +## Agentic inference is a systems problem + +Considering the characteristics above, benchmarking these workloads is fundamentally different from the existing fixed-sequence-length benchmarks. + +Because prefix reuse is so high, KV tensors have to move efficiently across nodes and ranks — the job of transfer layers like NIXL, MORI-IO, and [Mooncake](/agentx/optimizations/mooncake). Conversations then have to be routed to the node or rank where their prefix already resides in order to maximize cache hit rate, which is what llm-d, [Dynamo](/agentx/optimizations/dynamo), and the vLLM and SGLang routers do. Long-context conversations stress HBM capacity for KV cache and force offload to slower memory tiers such as DRAM and SSD, which has to be done efficiently — Mooncake Store, [LMCache](/agentx/optimizations/lmcache), vLLM's CPU offloading, and SGLang HiCache all attack this layer. + +This is in contrast to fixed-sequence-length, single-turn workloads, where prefix reuse is not relevant and inference performance is largely reflective of baseline chip and kernel performance. That does not make the plethora of fixed-sequence-length data on InferenceX less important. Stripping away the complexities of agentic serving gives a clear picture of how low-level inference optimizations are progressing, and provides the baseline that AgentX results are read against. + +## Building a realistic replay corpus + +To make the AgentX workloads as realistic as possible, we collected an initial corpus of 393 internal SemiAnalysis Claude Code traces to replay. To anonymize the content while keeping the original prefix-reuse pattern intact, we use a method similar to [Qwen-Bailian](https://github.com/alibaba-edu/qwen-bailian-usagetraces-anon), one of the earliest corpora of production traces. We then use [AIPerf](https://github.com/ai-dynamo/aiperf) to reconstruct the traces according to the original schedule of requests, at varying levels of concurrent clients. + +Reconstructing subagent structure required changes in Claude Code itself. We worked with Anthropic to ship two features that make the AgentX dataset possible, both adding agent-instance identity to outgoing requests so that parallel branches can be told apart at the proxy: + +- [anthropics/claude-code #49207](https://github.com/anthropics/claude-code/issues/49207) — add an agent instance ID (`x-claude-code-agent-id`) and its parent ID (`x-claude-code-parent-agent-id`) to API request headers, so interleaved subagent requests are attributable instead of collapsing into one session-level stream. +- [anthropics/claude-code #66761](https://github.com/anthropics/claude-code/issues/66761) — extend those same headers to subagents spawned by the `Workflow` tool's fan-out, which previously carried only the parent conversation's session ID. + +Without those headers, a session that fans out N concurrent subagents is indistinguishable from N unrelated conversations, and the spawn and join dependencies that shape the [replay graph](/glossary/trace-replay) cannot be recovered. + +The full dataset rules, distributions, and source links are published in the [AgentX methodology](/agentx/methodology), and the concrete engine changes this workload has already exposed are tracked in the [AgentX optimization tracker](/agentx/optimizations). + +## Acknowledgments + +Thanks to the Anthropic staff who shipped the agent-identification headers in Claude Code, and to the Alibaba and NVIDIA teams behind Qwen-Bailian and AIPerf, whose work the AgentX replay pipeline builds on. + + + Compare live AgentX inference results → + + +## Frequently asked questions + +### What is an agentic workload? + +An agentic workload is inference traffic generated by AI agents rather than by a person typing into a chat box. It is characterized by multi-turn sessions of tens to hundreds of interactions, long contexts built from system prompts and tool definitions, very high prefix reuse across turns, and bursts of short-lived subagents. + +### Why does prefix reuse matter so much for agentic inference? + +Each turn's input contains the previous turn's input plus its output, so nearly all of the context has already been processed. If the KV tensors for that prefix are still resident, the prefill work is skipped. As a session grows, the ratio of cached to uncached input tends toward 1, so cache capacity and cache hit rate dominate the cost of serving. + +### How is benchmarking an agentic workload different from a fixed-sequence-length benchmark? + +A fixed-sequence-length benchmark sends independent requests with a set input and output length, so prefix reuse is not a factor and the result mostly reflects chip and kernel performance. An agentic benchmark replays dependent sessions where context grows, prefixes are shared, and subagents fan out, which makes KV transfer, prefix-aware routing, and offload tiering part of what is being measured. + +### What does bursty subagent fan-out do to the KV cache? + +A single turn can spawn several subagents at once. Each one allocates its own KV cache, runs briefly, and finishes, so cache pressure arrives in spikes rather than a steady state. Schedulers and eviction policies tuned for uniform request streams behave differently under that pattern. + +### How was the AgentX replay corpus built? + +We collected 393 internal SemiAnalysis Claude Code traces and anonymized them with a block-hash method similar to Qwen-Bailian, preserving prefix relationships without preserving content. AIPerf then reconstructs the traces on their original request schedule across varying levels of concurrent clients. + +### Do fixed-sequence-length benchmarks still matter? + +Yes. Fixed-sequence-length results isolate low-level chip and kernel performance without the complexity of agentic serving, which makes them the baseline that AgentX results are interpreted against. + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "What is an agentic workload?", + "acceptedAnswer": { + "@type": "Answer", + "text": "An agentic workload is inference traffic generated by AI agents rather than by a person typing into a chat box. It is characterized by multi-turn sessions of tens to hundreds of interactions, long contexts built from system prompts and tool definitions, very high prefix reuse across turns, and bursts of short-lived subagents." + } + }, + { + "@type": "Question", + "name": "Why does prefix reuse matter so much for agentic inference?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Each turn's input contains the previous turn's input plus its output, so nearly all of the context has already been processed. If the KV tensors for that prefix are still resident, the prefill work is skipped. As a session grows, the ratio of cached to uncached input tends toward 1, so cache capacity and cache hit rate dominate the cost of serving." + } + }, + { + "@type": "Question", + "name": "How is benchmarking an agentic workload different from a fixed-sequence-length benchmark?", + "acceptedAnswer": { + "@type": "Answer", + "text": "A fixed-sequence-length benchmark sends independent requests with a set input and output length, so prefix reuse is not a factor and the result mostly reflects chip and kernel performance. An agentic benchmark replays dependent sessions where context grows, prefixes are shared, and subagents fan out, which makes KV transfer, prefix-aware routing, and offload tiering part of what is being measured." + } + }, + { + "@type": "Question", + "name": "What does bursty subagent fan-out do to the KV cache?", + "acceptedAnswer": { + "@type": "Answer", + "text": "A single turn can spawn several subagents at once. Each one allocates its own KV cache, runs briefly, and finishes, so cache pressure arrives in spikes rather than a steady state. Schedulers and eviction policies tuned for uniform request streams behave differently under that pattern." + } + }, + { + "@type": "Question", + "name": "How was the AgentX replay corpus built?", + "acceptedAnswer": { + "@type": "Answer", + "text": "We collected 393 internal SemiAnalysis Claude Code traces and anonymized them with a block-hash method similar to Qwen-Bailian, preserving prefix relationships without preserving content. AIPerf then reconstructs the traces on their original request schedule across varying levels of concurrent clients." + } + }, + { + "@type": "Question", + "name": "Do fixed-sequence-length benchmarks still matter?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Yes. Fixed-sequence-length results isolate low-level chip and kernel performance without the complexity of agentic serving, which makes them the baseline that AgentX results are interpreted against." + } + } + ] +}`} diff --git a/packages/app/content/blog/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx b/packages/app/content/blog/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx new file mode 100644 index 000000000..4f1834ae7 --- /dev/null +++ b/packages/app/content/blog/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx @@ -0,0 +1,86 @@ +--- +title: 'DeepSeek V4 Pro on AgentX: B200 vs B300 and the KV Cache Working Set' +subtitle: '50% more HBM squeezes out extra throughput, and the per-point telemetry shows exactly where it comes from' +seoDescription: 'B200 against B300 on DeepSeek V4 Pro AgentX runs. TCO-normalized throughput is close, but the telemetry shows a 91% HBM cache hit rate on B300 against 73% on B200, and a KV working set of 43M tokens against 22M.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - b200 + - b300 + - h200 + - nvidia +--- + +When normalized by [total cost of ownership](/glossary/total-cost-of-ownership), B300 vLLM and B200 vLLM aggregated performance on DeepSeek V4 Pro is quite similar. The main difference is that B300 can squeeze out extra throughput, given its 50% increase in HBM capacity over B200. + +
+ +The interesting part is not the summary number, it is what the AgentX server telemetry shows underneath it. + +## 43 million tokens against 22 million + +Under the load of 384 concurrent agentic traces, B300 vLLM DEP8 with 3TB of DRAM using vLLM simple offloading achieved a 91% HBM cache hit rate, with an additional 1.36% DRAM cache hit rate. The reason is that the HBM [KV cache](/glossary/kv-cache) working set size is approximately 43 million tokens with this configuration, and the load barely exceeds that number of tokens in flight at any given time. + +
+ +With B200 at concurrency 196, all other parameters unchanged, the picture inverts. The HBM cache hit rate falls to 73% and the deployment leans much harder on DRAM, with an [offload](/glossary/kv-cache-offload) cache hit rate of nearly 20%. The HBM KV cache working set size here is 22 million tokens, roughly half that of B300. + +
+ +## Why the offload tier needs headroom + +DRAM KV offloading is typically implemented as a write-through cache, meaning every prefix written to the HBM cache is also written to the DRAM cache. It is therefore most effective when the amount of DRAM available for offloading is significantly bigger than HBM KV cache capacity, by a multiple of roughly 1.5 to 3. + +That property is what makes the extra HBM on B300 compound rather than merely add. A larger resident working set keeps more of the session in the fast tier, which leaves the slower tier serving a genuine remainder rather than absorbing a constant stream of evictions. + +## The H200 counterexample + +H200 SGLang FP8 is able to serve DeepSeek V4 at low concurrency, and is even competitive with B200 and MI355X SGLang from a performance per dollar standpoint. It cannot compete with the newer SKUs in high throughput scenarios, because it lacks the HBM to do so. + +The failure mode is instructive: the reliance on DRAM KV offloading at higher concurrencies leads to unreasonable latency as the number of users scales. Offload buys capacity, but it is not free, and a deployment that depends on it to cover a structural memory shortfall pays for that at the tail. + +
+ +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "How much better is B300 than B200 on DeepSeek V4 Pro?", + "acceptedAnswer": { + "@type": "Answer", + "text": "When normalized by total cost of ownership, aggregated B300 vLLM and B200 vLLM performance is quite similar. B300 squeezes out extra throughput thanks to a 50% increase in HBM capacity, which shows up in the telemetry as a 43 million token HBM KV cache working set at concurrency 384 with a 91% cache hit rate, against 22 million tokens and 73% on B200 at concurrency 196." + } + }, + { + "@type": "Question", + "name": "Why does H200 fall behind at high concurrency?", + "acceptedAnswer": { + "@type": "Answer", + "text": "H200 SGLang FP8 serves DeepSeek V4 well at low concurrency and is competitive with B200 and MI355X SGLang on performance per dollar there. It lacks the HBM to compete in high throughput scenarios, so it leans on DRAM KV offloading as concurrency rises, and that reliance leads to unreasonable latency as the number of users scales." + } + } + ] +}`} diff --git a/packages/app/content/blog/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx b/packages/app/content/blog/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx new file mode 100644 index 000000000..bdff8ae75 --- /dev/null +++ b/packages/app/content/blog/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx @@ -0,0 +1,72 @@ +--- +title: 'DeepSeek V4 Pro on AgentX: GB200 vs GB300 Rack-Scale Disaggregation' +subtitle: 'Both lean on PD disagg, GB300 adds DEP32 wide-EP decode, and the gap shows up in first-token latency rather than token rate' +seoDescription: 'NVIDIA rack-scale results for DeepSeek V4 Pro on AgentX. GB300 Dynamo TRTLLM and GB200 Dynamo vLLM both use prefill-decode disaggregation, and the GB300 point trails on TTFT because higher concurrency brings more subagent traffic and more cold prefills.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - gb200 + - gb300 + - nvl72 + - disagg + - wide-ep + - nvidia +--- + +On DeepSeek V4 Pro, NVIDIA's most competitive solutions are GB300 Dynamo TRTLLM and GB200 Dynamo vLLM. Both configurations rely on prefill-decode [disaggregation](/glossary/disaggregated-inference) to achieve high throughput at reasonable [interactivity](/glossary/interactivity). GB300 configurations additionally employ [wide expert parallelism](/glossary/wide-expert-parallelism) decode instances at DEP32 in order to achieve higher throughput in the middle of the frontier. + +
+ +## The two points are closer on token rate than on first token + +The 2xDEP8+1xDEP12 GB200 point is significantly closer to the 3xDEP8+1xDEP16 GB300 point in terms of tokens per second per user than it is in terms of time to first token. Reading only the interactivity axis would therefore understate how differently these two configurations behave. + +TTFT is, in general, more sensitive to the spikiness of the workload. The GB300 point achieves much higher overall concurrency, so it incurs more subagent traffic and hence more cold prefills. A [subagent](/glossary/subagent) starts with fresh context and no reusable prefix, which means its first request is a full prefill rather than a cache hit, and enough of them arriving together is what moves the [tail](/glossary/tail-latency). + +
+ +## Why this is an agentic effect rather than a hardware one + +None of this appears on a fixed-sequence workload. At 8k1k every request is independent and identically shaped, so concurrency scales the batch without changing the mix of cached and uncached work. On AgentX, raising concurrency changes the composition of the traffic itself: more sessions means more parallel subagent branches, and those branches arrive in bursts rather than as a smooth stream. + +The practical reading is that a rack-scale configuration tuned to sit further right on the throughput curve is also accepting a burstier prefill load, and the cost of that decision lands on first-token latency rather than on the token rate the same chart advertises. + +
+ +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "Which NVIDIA rack-scale configuration is fastest on DeepSeek V4 Pro?", + "acceptedAnswer": { + "@type": "Answer", + "text": "GB300 Dynamo TRTLLM and GB200 Dynamo vLLM are the most competitive NVIDIA configurations on this model, and both rely on prefill-decode disaggregation to reach high throughput at reasonable interactivity. GB300 additionally runs DEP32 wide expert parallelism decode instances to lift throughput in the middle of the frontier." + } + }, + { + "@type": "Question", + "name": "Why does the GB300 point have worse TTFT than GB200?", + "acceptedAnswer": { + "@type": "Answer", + "text": "The GB300 point achieves much higher overall concurrency, which brings more subagent traffic and therefore more cold prefills. Because a subagent starts with fresh context and no reusable prefix, its first request is a full prefill, and bursts of those requests move the tail. TTFT is generally more sensitive to workload spikiness than the per-user token rate is." + } + } + ] +}`} diff --git a/packages/app/content/blog/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx b/packages/app/content/blog/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx new file mode 100644 index 000000000..32c0cfe0c --- /dev/null +++ b/packages/app/content/blog/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx @@ -0,0 +1,84 @@ +--- +title: 'DeepSeek V4 Pro on AgentX: MI355X vs B200, and the August 21 Flip' +subtitle: 'AMD matched B200 vLLM on performance per dollar for end-to-end latency, then upstream vLLM work moved the line' +seoDescription: 'AgentX 1.0 results for DeepSeek V4 Pro 0813 on MI355X against B200. MI355X SGLang matched B200 vLLM on performance per dollar until August 21, 2026, when vLLM optimizations from Inferact and NVIDIA put B200 ahead.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - mi355x + - b200 + - amd + - nvidia +--- + +DeepSeek V4 Pro 0813 is an ultra popular frontier open weight model from China, with roughly 1.6 trillion parameters and 49 billion active. It is also the model where the AMD and NVIDIA race on [AgentX](/glossary/agentx) was closest, and where the ranking changed inside a single week. + +The workload matters here. AgentX replays real agentic coding sessions rather than a fixed prompt, and across all DeepSeek V4 runs the input length distribution came out at p50 88k, p90 272k, p95 404k, and p99 675k tokens, against an output distribution of p50 413, p90 2.2k, p95 3.7k, and p99 8.6k. That is a different machine problem from 8k1k. + +
+ +Before August 21, 2026, AMD's strong MI355X SGLang development team was matching B200 vLLM on performance per dollar for end to end performance. That is a real result and worth stating plainly. + +It was also not the whole board. B300 vLLM and B200 SGLang were already beating MI355X at the same time. + +
+ +After August 21, optimizations in vLLM from Inferact and NVIDIA pushed B200 performance per dollar past MI355X. This is a close race rather than a settled one, and [AMD has published its own list of DeepSeek V4 vLLM optimizations](https://github.com/vllm-project/vllm/issues/52911) with plenty left on it. + +## The ATOM asterisk + +On end to end latency, ATOM on MI355X beats B200 vLLM. It does not beat B300 or B200 SGLang, and the more important caveat is deployment: most AI labs in China or the west do not want to run ATOM in production, so a win that exists only under the vendor engine does not describe what a customer on an upstream stack receives. Single node open-source performance on MI355X, meaning vLLM, still trails ATOM. + +
+ +## Where MI355X is actually competitive + +Overall MI355X performs decently against B200 and B300. Performance is most comparable at the lower throughput and lower latency part of the curve, where only tensor parallelism and more rudimentary kernels are deployed. The gap opens at high throughput, where AMD needs to optimize its DEP kernels, and it opens despite MI355X carrying 1.5x the HBM of B200. + +
+ +Read both axes together. A configuration can look strong on throughput against [interactivity](/glossary/interactivity) and give that back on [p90](/glossary/tail-latency) time to first token, which is why the comparison above is plotted against TTFT rather than interactivity alone. + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "Did MI355X beat B200 on DeepSeek V4 Pro?", + "acceptedAnswer": { + "@type": "Answer", + "text": "For a period before August 21, 2026, MI355X SGLang matched B200 vLLM on performance per dollar for end to end performance, and ATOM on MI355X beat B200 vLLM on end to end latency. B300 vLLM and B200 SGLang were beating MI355X over the same period, and after August 21 vLLM optimizations from Inferact and NVIDIA put B200 performance per dollar ahead of MI355X." + } + }, + { + "@type": "Question", + "name": "Why does MI355X lose ground at high throughput despite more HBM?", + "acceptedAnswer": { + "@type": "Answer", + "text": "MI355X carries 1.5x the HBM of B200, but on AgentX its performance is most comparable to B200 and B300 at the lower throughput and lower latency part of the curve, where only tensor parallelism and more rudimentary kernels are used. The high throughput end depends on DEP kernels that AMD still needs to optimize, so the extra memory capacity is not converted into frontier throughput." + } + } + ] +}`} diff --git a/packages/app/content/blog/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx b/packages/app/content/blog/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx new file mode 100644 index 000000000..ffea55d49 --- /dev/null +++ b/packages/app/content/blog/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx @@ -0,0 +1,75 @@ +--- +title: 'GLM 5.3 on AgentX: MI355X ATOM Beats GB300 NVL72 on Part of the Curve' +subtitle: 'Where AMD’s vendor engine wins on performance per dollar, and what E2E Normalized Interactivity actually measures' +seoDescription: 'On GLM 5.3, AMD ATOM has better performance per dollar than GB300 NVL72 SGLang and even TRTLLM for parts of the p90 E2E Normalized Interactivity range. An explainer on what that experimental metric measures and where it misleads.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - glm5 + - mi355x + - gb300 + - nvl72 + - trtllm + - sglang + - amd + - nvidia +--- + +When looking at ATOM, AMD has better [performance per dollar](/glossary/performance-per-dollar) than GB300 NVL72 SGLang and even TRTLLM for some parts of the range of p90 E2E Normalized Interactivity. Great work to the AMD team on these results. + +The usual two requests apply to both vendors. We look forward to AMD porting these optimizations over to SGLang, and we are also looking forward to NVIDIA quickly optimizing GB300 NVL72 in the coming weeks. + +
+ +## What E2E Normalized Interactivity measures + +This result is plotted against an experimental metric worth understanding before drawing conclusions from it. [E2E Normalized Interactivity](/glossary/e2e-normalized-interactivity) is meant to evaluate how fast a user experiences responsiveness when considering both time to first token and tokens per second. It is defined as output sequence length divided by end to end latency. + +Substituting the fact that end to end latency equals TTFT plus output length times time per output token, and noting that in reality only output length minus one tokens are decoded, gives the following. + +
+ +This is effectively [interactivity](/glossary/interactivity), the reciprocal-TPOT portion, plus an additional penalty proportional to TTFT. + +## Where the metric misleads + +Please note that this metric is experimental and is not perfect. For instance, it heavily penalizes high TTFT and does not capture all the nuances of certain optimizations such as prefill-decode [disaggregation](/glossary/disaggregated-inference). All submissions for AgentX v1.0 optimize for both regular interactivity and TTFT separately, and we will continue working on new north star metrics that reflect all nuances of modern agentic inference. + +That caveat cuts in both directions here. A configuration that trades first-token latency for streaming speed is penalized on this axis whether or not that trade suits the product, so a win measured only against E2E Normalized Interactivity should be checked against the separate interactivity and TTFT curves before it becomes a procurement argument. + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "What is E2E Normalized Interactivity?", + "acceptedAnswer": { + "@type": "Answer", + "text": "It is an experimental metric defined as output sequence length divided by end to end latency, intended to evaluate how fast a user experiences responsiveness when considering both TTFT and tokens per second. Because end to end latency equals TTFT plus output length times time per output token, it works out to interactivity plus a penalty proportional to TTFT." + } + }, + { + "@type": "Question", + "name": "Where does AMD ATOM beat GB300 NVL72 on GLM 5.3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "For some parts of the p90 E2E Normalized Interactivity range, ATOM has better performance per dollar than GB300 NVL72 SGLang and even TRTLLM. The metric is experimental and heavily penalizes high TTFT, so the result should be checked against the separate interactivity and TTFT curves." + } + } + ] +}`} diff --git a/packages/app/content/blog/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx b/packages/app/content/blog/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx new file mode 100644 index 000000000..d3fd0ad64 --- /dev/null +++ b/packages/app/content/blog/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx @@ -0,0 +1,69 @@ +--- +title: 'GLM 5.3 on AgentX: NVIDIA Is Up to 5x Cheaper per Token at 150 tok/s/user' +subtitle: 'At this operating point, free AMD silicon would still not close the gap' +seoDescription: 'GLM 5.3 AgentX results on open source SGLang. At 150 tok/s/user p90 interactivity, NVIDIA has up to 5x better cost efficiency, a gap so large that free competitor hardware would still leave NVIDIA cheaper per token.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - glm5 + - b200 + - b300 + - mi355x + - sglang + - nvidia + - amd +--- + +GLM 5.3 builds on top of GLM 5.2 744B with additional post training. This is a frontier level model. + +In terms of open source SGLang performance, this is another model where NVIDIA again beats AMD on realistic agentic inference performance. At 150 tok/s/user p90 [interactivity](/glossary/interactivity), NVIDIA has up to 5x better cost efficiency. + +
+ +## The free-hardware test + +There is a useful way to size a gap like this. With the current state of AMD software, at 150 tok/s/user, NVIDIA's performance advantage is so great that even if the competitor chip hardware was sold for free, with providers still of course paying for datacenter hosting, power, and other operating costs, cost per token would still be cheaper when using NVIDIA. + +That framing matters because hardware price is the lever most often reached for in a procurement conversation, and at this operating point it is not a lever that reaches far enough. The constraint is software, not the bill of materials. + +
+ +## This is a snapshot, not a verdict + +The gap is measured against the current state of AMD software, and that state has been moving quickly elsewhere in these results. We look forward to AMD's performance optimizations in the upcoming AgentX update article in a couple of weeks, which will also include some other very exciting results. + +It is also worth being precise about scope: this is the open source SGLang comparison. AMD's vendor engine tells a different story on parts of this same model, which is [covered separately](/blog/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72). + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "How much cheaper is NVIDIA than AMD on GLM 5.3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "On open source SGLang at 150 tok/s/user p90 interactivity, NVIDIA has up to 5x better cost efficiency. With the current state of AMD software at that operating point, the advantage is large enough that even free competitor hardware, with providers still paying for hosting, power, and other operating costs, would leave cost per token cheaper on NVIDIA." + } + }, + { + "@type": "Question", + "name": "Does AMD win anywhere on GLM 5.3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Yes, but not on this comparison. The 5x figure is the open source SGLang result. With ATOM, AMD has better performance per dollar than GB300 NVL72 SGLang and even TRTLLM for parts of the p90 E2E Normalized Interactivity range." + } + } + ] +}`} diff --git a/packages/app/content/blog/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx b/packages/app/content/blog/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx new file mode 100644 index 000000000..0c18cb56e --- /dev/null +++ b/packages/app/content/blog/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx @@ -0,0 +1,70 @@ +--- +title: 'Kimi K3 on AgentX: MI355X ATOM Beats GB300 NVL72 on Part of the Curve' +subtitle: 'AMD’s vendor engine wins a real slice of the performance per dollar frontier, while Hopper struggles to serve K3 at all' +seoDescription: 'On Kimi K3 AgentX runs, MI355X ATOM beats GB300 NVL72 vLLM on performance per dollar between 40 and 60 seconds of end-to-end latency, while Hopper struggles because K3 needs custom SM90 kernels and tuned TP32 and EP32 shapes.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - kimi + - mi355x + - gb300 + - nvl72 + - h200 + - amd + - nvidia +--- + +On the part of the curve between 40 and 60 seconds of end to end latency, MI355X ATOM beats even GB300 NVL72 vLLM on performance per dollar. On a 2.8 trillion parameter model, against a rack-scale NVIDIA system, that is a result AMD earned. + +
+ +## The upstream caveat + +It is great that AMD is quickly pushing K3 performance forward with [ATOM](/glossary/atom). The encouragement that goes with that is to prioritize upstreaming these improvements into vLLM. ATOM is currently AMD's best-performing engine, but vLLM remains the more relevant comparison for customers using an upstream open-source serving stack, and a curve that only exists under a vendor runtime does not describe what those customers can deploy. + +## Hopper cannot really play + +Hopper struggles to serve the AgentX workload for Kimi K3. Part of that is simply that Kimi is a massive model, and part of it is that vLLM maintainers and NVIDIA have not been focusing on optimizing Hopper for K3. + +The specifics are concrete rather than vague: Hopper, at SM90, requires custom tuned kernels for K3, along with TP32 and EP32 tuned shapes for serving at high [interactivity](/glossary/interactivity). Neither exists yet, so the hardware is not the binding constraint so much as the absence of anyone working on it. + +
+ +This is a useful reminder that a generation gap in a benchmark chart is partly a generation gap in engineering attention. Hopper served DeepSeek R1 well for years because that model received sustained kernel work. K3 has not received the same, and the curve reflects the difference. + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "Does AMD beat NVIDIA anywhere on Kimi K3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Yes, on part of the curve. Between 40 and 60 seconds of end to end latency, MI355X ATOM beats even GB300 NVL72 vLLM on performance per dollar. The caveat is that ATOM is AMD's vendor engine rather than an upstream open-source stack, so vLLM remains the more relevant comparison for most customers." + } + }, + { + "@type": "Question", + "name": "Why does Hopper struggle on Kimi K3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Kimi K3 is a massive model, and vLLM maintainers and NVIDIA have not focused on optimizing Hopper for it. Serving K3 at high interactivity on SM90 requires custom tuned kernels plus TP32 and EP32 tuned shapes, and that work has not been done." + } + } + ] +}`} diff --git a/packages/app/content/blog/kimi-k3-the-manos-the-mythos-the.mdx b/packages/app/content/blog/kimi-k3-the-manos-the-mythos-the.mdx index 43e244de7..f4d11a898 100644 --- a/packages/app/content/blog/kimi-k3-the-manos-the-mythos-the.mdx +++ b/packages/app/content/blog/kimi-k3-the-manos-the-mythos-the.mdx @@ -686,7 +686,7 @@ This benchmark is a big step up from our previous 8k1k/1k1k benchmark, as it tru
For Kimi K3, Day 0 bringup was easier than DSv4 due to better documentation and preparation ahead of weights release. Appropriate images and a speculative decoder model were released at the same time as the weights. diff --git a/packages/app/content/blog/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx b/packages/app/content/blog/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx new file mode 100644 index 000000000..7007b1598 --- /dev/null +++ b/packages/app/content/blog/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx @@ -0,0 +1,75 @@ +--- +title: 'MiniMax M3 on AgentX: Why B200 and B300 Beat Their Rack-Scale GB200 NVL72 & GB300 NVL72 Counterparts' +subtitle: 'The Dynamo router becomes the bottleneck, no submission runs context parallelism, and AMD leaves KV offload on the table' +seoDescription: 'On MiniMax M3, B200 and B300 beat GB200 and GB300 on TCO-normalized throughput. The Dynamo router can bottleneck because its work scales with live prefixes, no submission runs context parallelism despite a 317k p90 input, and AMD Pareto points use no KV offload.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - minimax + - b200 + - b300 + - gb200 + - gb300 + - nvl72 + - dynamo + - amd + - rocm +--- + +B200 and B300 completely beat their rack-scale counterparts for MiniMax M3 on TCO-normalized throughput. For a model where NVIDIA sweeps the field, that is a surprising internal result, and the explanation is not about the fabric. + +On AgentX the rack-scale advantage is not as pronounced, because the Dynamo router can become the bottleneck: its work scales with the number and length of live prefixes rather than with tokens generated. There are also no well tuned kernels for wide expert parallelism, wide DCP, or wide tensor parallelism on this model. Since GB200 and GB300 carry higher TCO, without wide EP or wide DCP that shows up directly as worse performance per TCO. + +
+ +We also expect further optimizations from NVIDIA on their rack-scale solutions for this SKU, and will highlight them in the follow up article. + +## Nobody is running context parallelism + +No submission currently runs [context parallelism](/glossary/context-parallelism), despite a p90 input sequence length of 317k tokens. That is exactly the regime the technique exists for, so the absence needs an explanation. + +M3 has 4 KV heads, so DCP caps at 2 even at TP8, and the MSA indexer needs its own context-parallel handling, for which a vLLM pull request is open. The ceiling is therefore architectural rather than a tuning oversight, and it bounds how much a long-context parallelism strategy can help on this particular model. + +## The AMD offload gap + +All of NVIDIA's Pareto optimal points include [KV offload](/glossary/kv-cache-offload) above concurrency 20. For AMD, none of the Pareto optimal points use KV offload to DRAM, and AMD uses offload less than NVIDIA on the other models too. + +The reason is mechanical. GPU-to-CPU transfers for CPU KV cache offloading are highly inefficient on AMD vLLM, because the hipMemcpyBatchAsync API was missing until ROCm 7.14. Without it, vLLM's native simple CPU offloading has to perform serialized memcpy operations from CPU to GPU instead of batching them into larger message sizes. + +
+ +A missing batched-copy API is not the kind of thing that shows up in a specification sheet, and on a fixed-sequence benchmark it would never be exercised at all. On long agentic sessions it decides whether an entire tier of memory is usable. + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "Why do B200 and B300 beat GB200 and GB300 on MiniMax M3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "On TCO-normalized throughput, the rack-scale advantage is muted on AgentX because the Dynamo router can become the bottleneck, since its work scales with the number and length of live prefixes rather than with tokens generated. There are also no well tuned kernels for wide expert parallelism, wide DCP, or wide tensor parallelism on this model, and the higher TCO of GB200 and GB300 is not earned back without them." + } + }, + { + "@type": "Question", + "name": "Why does AMD use no KV offload on MiniMax M3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "GPU-to-CPU transfers for CPU KV cache offloading are highly inefficient on AMD vLLM because the hipMemcpyBatchAsync API was missing until ROCm 7.14. Without it, native simple CPU offloading performs serialized memcpy operations instead of batching them into larger messages, so no AMD Pareto optimal point uses DRAM offload while every NVIDIA one above concurrency 20 does." + } + } + ] +}`} diff --git a/packages/app/content/blog/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx b/packages/app/content/blog/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx new file mode 100644 index 000000000..69d8f62d0 --- /dev/null +++ b/packages/app/content/blog/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx @@ -0,0 +1,78 @@ +--- +title: 'MiniMax M3 on AgentX: B300 TRT-LLM TP2 Owns the Crown' +subtitle: 'NVIDIA sweeps the 432B model, and the missing DP-attention points explain why cache locality became a routing constraint' +seoDescription: 'MiniMax M3 432B AgentX results. B300 TRT-LLM TP2 leads the frontier, DP attention is absent because KV cache locality becomes a routing constraint, and a GB200 TP4/EP4/DPA point reaches 0.60x the throughput of plain TP4 at over 3x the p90 TTFT.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - minimax + - b300 + - b200 + - gb200 + - trtllm + - vllm + - nvidia +--- + +NVIDIA absolutely destroys all competitors on MiniMax M3 432B. AMD software performance is horrible on MiniMax, especially at high context length, which we attribute to AMD engineering leadership incentivizing tuning only for short context single turn workloads and ignoring long context multi turn workloads. + +Within the NVIDIA field, B300 TRT-LLM at TP2 owns the M3 crown. + +
+ +## The missing DP-attention points + +There is a lack of [DP-attention](/glossary/dp-attention) points on the M3 frontier, and the absence is informative. Data-parallel attention is non-optimal on M3 because KV cache locality becomes a routing constraint. + +The numbers make the mechanism concrete. For GB200 at concurrency 40, TP4/EP4/DPA gets 0.60x the throughput of plain TP4 at more than 3x the p90 TTFT. At concurrency 32 it hits 28.8% of cache against 96.0% theoretical. Each DP rank owns a private quarter of the pool, so a 300k-token session re-landing on the wrong rank recomputes everything. + +That gap between 28.8% measured and 96.0% theoretical is the whole story of [prefix reuse](/glossary/prefix-cache-hit-rate) under data parallelism: the cache was large enough, and the session simply was not sent to the rank that held it. + +No decode config with expert parallelism appears on the M3 frontier either, likely because the concurrency is not high enough to balance the loads across all experts. + +## vLLM against TRT-LLM + +It is worth mentioning that vLLM performance is very comparable to TRT-LLM in terms of throughput versus p90 interactivity. On throughput versus p90 TTFT, vLLM performs better. + +
+ +Which of the two you prefer therefore depends on which axis your product is written against, which is the recurring argument for reading interactivity and first-token latency together rather than picking whichever chart flatters a conclusion. + +
+ +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "Which configuration is fastest on MiniMax M3?", + "acceptedAnswer": { + "@type": "Answer", + "text": "B300 TRT-LLM at TP2 owns the M3 crown. NVIDIA leads the model overall, while AMD software performance is poor on MiniMax, especially at high context length." + } + }, + { + "@type": "Question", + "name": "Why is data-parallel attention missing from the M3 frontier?", + "acceptedAnswer": { + "@type": "Answer", + "text": "DP attention is non-optimal on M3 because KV cache locality becomes a routing constraint. On GB200 at concurrency 40, TP4/EP4/DPA reaches 0.60x the throughput of plain TP4 at more than 3x the p90 TTFT, and at concurrency 32 the measured cache hit rate is 28.8% against a 96.0% theoretical ceiling, because each DP rank owns a private quarter of the pool and a session landing on the wrong rank recomputes everything." + } + } + ] +}`} diff --git a/packages/app/content/blog/openai-jalapeno-better-than-nvidia.mdx b/packages/app/content/blog/openai-jalapeno-better-than-nvidia.mdx new file mode 100644 index 000000000..20fa26656 --- /dev/null +++ b/packages/app/content/blog/openai-jalapeno-better-than-nvidia.mdx @@ -0,0 +1,383 @@ +--- +title: 'OpenAI Jalapeño: Better Than Nvidia Blackwell' +subtitle: 'OpenAI’s self-designed ASIC compared with Rubin, Jalapeño’s TCO, throughput per MW, and spicy deets' +seoDescription: 'OpenAI’s first-generation inference ASIC, built with Broadcom and benchmarked on InferenceX. Perf per MW against Blackwell and Vera Rubin, the HBM4 and MXFP4 specs, the Gluon software stack, why OpenAI skipped prefill-decode disaggregation, and the Katsu, Vindaloo, and Chana rack architecture.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - inference + - benchmark + - gpu + - nvidia + - openai + - asic + - rubin +--- + +OpenAI has spent the past couple years quietly building “Jalapeño,” an inference chip just announced at Hot Chips. Rumors of a successful tapeout had been swirling for a while. But now we have details. OpenAI invited us to look at their chip, go to their labs to check out how real it is, and [benchmark](https://openai.com/index/jalapeno-first-results/) it with our [InferenceX](https://inferencex.semianalysis.com/) suite. + +In June, [OpenAI unveiled the chip program](https://openai.com/index/openai-broadcom-jalapeno-inference-chip/) in partnership with Broadcom, built from a blank slate exclusively for LLM inference. [Design work began in the middle of 2024](https://newsletter.semianalysis.com/p/openai-chip-team-is-now-serious), going from initial team hiring to manufacturing tape-out in ~16 months, an extremely fast ASIC development cycle. + +In general first generation chips are not competitive, but OpenAI bucks the trend by being industry leading and beating every Nvidia, AMD, and Google chip we have been able to test on multiple top open source models. OpenAI does this with extreme hardware software codesign. Surprisingly, OpenAI is not over specialization on any specific part of model inference, but instead by focusing on being a general chip that delivers high performance in all scenarios. + +In this article, we will go into architectural details, software details and performance results for Jalapeño on InferenceX. + +
+ +## A generalized inference chip + +Everyone says that OpenAI’s chip is specialized for OpenAI models, but that’s wrong, OpenAI made a generalized chip for AI inference. + +The timelines are insane. It shows that claims that use of AI is being used to accelerate chip design are real. Regardless of the quick timelines,Open AI spent a bunch of money, made pragmatic design decisions and their team is cracked, so this comes as no surprise. + +Just looking at the specs, it is an immediate contender: + +
+ +And the use of HBM4 makes it stand out as comparable to flagship GPUs from NVIDIA and AMD: + +
+ +A lot of the media coverage of this chip has followed a few throwaway comments from OpenAI that claim the chip will be optimized for their models in a way that other chips are not. This is wrong. Jalapeño is a generalized inference chip capable of running all sorts of models, and all sorts of workloads, including our benchmark InferenceX, where we ran the benchmark with OpenAI engineers in the lab. As a joke, OpenAI even showed us it running Doom, which was ported to their chip with just Codex prompts. + +The following is our headline perf/W result, looking at token throughput per All-in utility MW. **Jalapeño smokes every other chip**. All this is done without Multi Token Prediction (MTP), while the other chips on the chart are the best performing configs of each respective SKU, all with MTP. + +
+ +Jalapeño beats Blackwell on perf/W across almost all scenarios without being tuned for any specific point in the curve. It excels not only in low-latency scenarios but also in high-throughput scenarios. A more apples to apples comparison is against Single Token Prediction results, it knocks every competitor out of the water. At low concurrency scenarios, Jalapeño demonstrates remarkable interactivity, hitting over 700 tokens per sec per user at concurrency 1 on the DeepSeek R1 model. + +Incredibly, this is all achieved with single-token prediction (STP), no speculative decoding and no prefill-decode disaggregation. In addition to DeepSeek R1, we also got to see some other models, including Kimi-K2.5 and GPT-OSS which ran at approximately 1,400 tok/sec/user. For all models, we confirmed that Jalapeño’s GSM8k evals attained results on par with Nvidia chips. + +Some caveats on this. First, all numbers are provided to us by OpenAI. We verified the InferenceX runs in person in the lab, but we did not run the full suite of [InferenceX](https://inferencex.semianalysis.com/) benchmarks nor have we seen [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) results. AgentX is our preferred suite for comparing chip performance due to the datasets’ long context and multi-turn characteristics that reflect the cache behavior of realistic production workflows. Frameworks that perform well on 8k1k may perform worse on AgentX as real production loads stress components like routers, prefix cache mechanisms, cache management, offload infrastructure, etc. These are not tested by single turn 8k1k. Read more about this in out AgentX article. + +[AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing?](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) — Since the Claude Code inflection point in November 2025, long-context, multi-turn agentic workloads have grown rapidly. They now dominate traffic for production inferencing. In April 2026, OpenAI’s Enterprise agentic spending overtook ChatGPT spending. + +Second, we believe that comparison to Blackwell is somewhat incomplete and unfair. Jalapeño is really competing against chips like Rubin that also use HBM4. Vera Rubin systems are starting to ship to customers right now, while it will still be some time before OpenAI has anything beyond engineering samples of Jalapeño. + +Thus, performance should really be compared against Rubin, not Blackwell, and in some sense we expect a custom chip like Jalapeño to outperform Blackwell. Vera Rubin NVL72 delivers 5.4x the perf/MW of GB200 NVL72 [as we described in our article analyzing the NVIDIA performance claims in their launch with CoreWeave last month](https://newsletter.semianalysis.com/p/vera-rubin-nvl72-vs-gb200-nvl72-inference). We will compare Jalapeño to Vera Rubin’s July performance figures later below. + +[Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis](https://newsletter.semianalysis.com/p/vera-rubin-nvl72-vs-gb200-nvl72-inference) — Vera Rubin NVL72 is the second generation of Nvidia’s rack-scale Oberon architecture, and its gains on inference come from extreme co-design. Early results from engineering samples are encouraging. Vera Rubin NVL72 running DeepSeek R1 delivers 5.4x performance per MW and 5x performance per dollar over GB200 NVL72 today, and the gap is even wider against GB200 NVL72 during its early bringup in 2025. Vera Rubin is still in the early bringup stage now, so we expect the gap to continue widen. Rubin's inference performance will keep improving as software matures, the same pattern we demonstrated for Blackwell in our + +Third, the models being tested are not on the open frontier. NVIDIA and AMD have published results on larger models such as DeepSeek V4 Pro and Kimi K3, using AgentX. The larger the model and the more recent the release, the more complicated it is to bring up on a new chip. With that said the models OpenAI has working on Jalapeno aren’t exactly small either. + +## Performance Analysis + +OpenAI designs for perf/W. The reason is simple: OpenAI is currently limited by datacenter power, not by budget or floorspace, and thus tokens per MW is paramount. At Computex 2026, Jensen said that perf/W, reliability and long lifetime are the core features of future GPUs. To quote: “If you have 1 gigawatt of power, then throughput per watt is revenue”. He also mentioned that choosing the wrong architecture just because the chips are cheaper doesn’t make sense. + +
+ +This was emphasized by Nvidia during the Vera talk at Hot Chips 2026 while showing the same revenue graph: “The data center is power limited today.” Power matters and drives revenue. + +Operators cannot simply obtain more MW because adding GPUs and adding grid capacity happen on very different timescales. Datacenter power envelopes have constraints such as their utility interconnection, infrastructure, cooling capacity, and UPS/backup-generation design. Grid delays repeatedly outpace hardware and construction timelines, driving the need for BtM (behind-the-meter) power capacity: gas turbines and on-site generators built and located at the data center itself. This capacity sits behind the utility’s meter rather than being drawn from the public grid. It lets an operator power a facility without waiting on grid interconnection and utility upgrades, which is exactly why xAI’s Colossus 2 relies so heavily on BtM while its actual grid connection lags far behind. Find out more in our [Energy model](https://semianalysis.com/energy-model/). + +As we wrote in an X post, tok/s/MW reduces to tokens per joule since a watt is a joule per second. This makes tok/s/MW representative of a system’s efficiency and ability to convert energy into tokens. + +
+ +On this front, even when compared with Rubin, Jalapeño wins. OpenAI’s Jalapeño has STP output token throughput per MW surpassing [Vera Rubin’s MTP results that NVIDIA and CoreWeave published in July. It also far exceeds GB200’s 2025 MTP results.](https://www.coreweave.com/blog/nvidia-vera-rubin-nvl72-on-coreweave-10x-more-tokens-per-megawatt-than-blackwell) As mentioned in our Vera Rubin article, VR was compared to 2025 GB200 results because that was a similar stage of early bring-up, and comparing to GB200 in 2025 holds software maturity constant. Following this logic, we compare Vera Rubin’s latest July 2026 results, GB200 2025 results, and today’s Jalapeño results. This is a very valid comparison as these are the best public Rubin numbers, and OpenAI taped out their chip after Rubin. Both OpenAI and Rubin are still immature thus performance will continue to rise. + +
+ +On perf/TCO, Vera Rubin and Jalapeño are head-to-head, producing almost the same number of output tokens per $. However, as previously mentioned, **Jalapeño’s results are obtained without speculative decoding** and Vera Rubin’s results use speculative decoding. Speculative decoding leads to a ~3-5x reduction in cost per token. When speculative decoding is implemented on Jalapeño, this will enable Jalapeño to serve tokens even more cost effectively. Of course, part of this TCO advantage comes from trading Nvidia’s high margins for Broadcom’s lower (though still high) margins. But this is not all of it. For example, Meta and Microsoft’s AI ASIC programs not getting off the ground despite being at it for much longer shows that cost is only one part of the equation. For Jalapeño’s full TCO breakdown, see the [SemiAnalysis AI Cloud TCO model](https://semianalysis.com/ai-cloud-tco-model/). + +
+ +Architecturally, OpenAI chose not to disaggregate prefill and decode (PD) across separate chip pools. The draft model and main model share the same chips and fabric, a design philosophy that trades some theoretical efficiency for practical operations. The motivation is that the workload mix changes over time, for example the ratio of input to cache write to cache read to output tokens has changed significantly as we have moved through the three eras of models ([knowledge, reasoning, and agentic, as discussed in our recent article](https://newsletter.semianalysis.com/p/are-open-models-catching-up)). Therefore, picking a fixed amount of heterogenous prefill silicon and decode silicon up front can lead to inefficiencies over time. OpenAI chooses a homogenous pool in this architecture and tries to make the chip perform well on everything. + +And it does. On Kimi K2.5 (which Cursor Composer 2.5 is based on), Jalapeño reaches nearly 700tok/s/user and more than 9x the next best performing chip at 100tok/s/user. + +
+ +On GPT-OSS, it’s another bloodbath. Jalapeño’s iso-interactivity throughput per MW is nearly double GB200’s highest throughput point and more than 50x GB200’s concurrency 1 point. The higher concurrency Jalapeño points use EP8. + +
+ +These results are impressive! However, we have to nitpick: they’re just 8k1k, a much easier workload to tune for, and there are no [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) runs yet. As mentioned in our [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) article, multiturn, long context workloads stress much more aspects of the serving stack, such as routers and prefix cache. Many more optimizations are needed to excel in agentic workloads. Read more about this in the [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) article. + +[AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing?](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) — Since the Claude Code inflection point in November 2025, long-context, multi-turn agentic workloads have grown rapidly. They now dominate traffic for production inferencing. In April 2026, OpenAI’s Enterprise agentic spending overtook ChatGPT spending. + +## Digging into the specs and architecture + +All these results were gathered on the A0 stepping of Jalapeño, just 9 months into the program. But there is already a B0 stepping that is currently in the fab! B0 has optimizations that deliver roughly a 25% perf-per-watt improvement over the earlier A0 silicon. Specifically, the B0 stepping delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die that is manufactured on TSMC’s N3P. This compares to 17.5 PFLOPs of dense Rubin NVFP4 for a single Rubin compute die that is similar size and on the same node. + +This is more respectable considering Jalapeño’s TDP is only 700W compared to Rubin’s at 900-1,150W per compute die. As Jalapeño is geared towards inference rather than training, it is understandable that OpenAI doesn’t need to push TDPs higher to maximize FLOPs, but regardless the above shows that Jalapeño delivers respectable peak theoretical FLOPs. + +When compared directly to other accelerators, Jalapeño has the highest HBM bandwidth per watt, and the highest FLOPs per watt, comparable to the 1,800W Rubin Max-Q configuration: + +
+ +Off-package I/O is provided by an N3E I/O chiplet with 32 lanes of 800G SerDes, for the compute fabric, with 24 lanes (600GB/s) being used for local scale-up within the rack, and 8 lanes (200GB/s) for global scale-up which is the 2,048 XPU multi-rack domain. PCIe Gen 5 is used for system I/O to connect to the x86 host CPU. + +Jalapeño will ship with HBM4, making this chip one of the relatively early adopters after Nvidia and AMD, even beating the established TPU and Trainium programs. As one of the key architectural principles behind Jalapeño is getting the most out of HBM bandwidth, settling for anything but the best HBM would run counter to that goal. This results in 15.4TB/s of memory bandwidth per package which bests all the other accelerators shipping that are using HBM3E. The 15.4TB/s bandwidth shows its HBM4 can hit 10Gbps pin speeds, which would give it a slight edge over the 9.6Gbps Nvidia is getting out of its HBM4 in Rubin. The HBM is likely provided by Samsung. + +
+ +OpenAI taped out Jalapeño in November 2025, or more specifically, this was a tape out of the CoWoS design, not just the top die silicon. Within 9 months of that Nov 2025 tapeout, and with only 3 months of bring-up on actual silicon, OpenAI has already delivered very good results with Jalapeño. This is all the more impressive as the team is starting from zero on the software stack. + +Meanwhile, Rubin’s CoWoS tape out was completed in October 2025, a month earlier, and yet the only early results we have seen are from CoreWeave’s engineering samples. Nvidia has not let us test and release benchmarks in the same way that OpenAI has, indicating their chip software is still immature. The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon. + +They are still far from optimized and we can see that generally Jalapeño has delivered better numbers. We don’t think that Nvidia hardware is inferior, but more so that Jalapeño’s software bring-up has progressed more quickly than Nvidia’s. This speaks to the power of hardware/software co-design, which is the main area where a cracked frontier lab ASIC team can excel over more established merchant silicon players. Counterintuitively, starting from scratch may also have benefited OpenAI as it could make clean-sheet architectural decisions without worrying about backwards compatibility or older software versions. + +While OpenAI has engineering samples of Jalapeño, production is currently scheduled to gradually ramp over 2027 with most of the output currently scheduled for the end of next year. [For more details of unit volumes and ASPs, see the SemiAnalysis Accelerator Model](https://semianalysis.com/accelerator-hbm-model/). + +Suffice to say, OpenAI Jalapeno is a real high volume ASIC. + +When compared against Rubin’s timeline, Jalapeño’s is shockingly quick. As shown earlier, Jalapeño’s results beat Rubin’s despite Rubin’s head start. + +
+ +## Jalapeno Architecture + +Digging into the architecture now, the chip’s matrix engine uses MXFP numerical formats and a weight stationary systolic array, similar to TPU. But when compared directly to TPU, it has support for smaller shapes / dimensions, meaning that it doesn’t have weird performance cliffs that get exposed by awkwardly shaped matmuls on bigger systolics. + +It also has 64-bit scalar cores and FP32/INT32 vector cores. OpenAI has also invested in redundancy at the tray level and has yield harvesting built in at the core and channel level. They claim that AI assistance in chip design delivered an 8% reduction in SIMD area and a 10% reduction in matrix-engine area during design. While they did not clarify the exact process/voltage/temperature (PVT) conditions, they also mentioned the AI-assisted blocks improved timing and power over the initial blocks. + +The Jalapeño architecture design focuses on eliminating memory movement of KVCache and weights as well as fixed latencies and overheads in order to make it possible to get closer to the raw peak flops/bandwidth even for small batches or shapes as compared to other accelerators. + +The cores and the HBM are divided into slices, where each core slice has a low-latency local view on its own slice of HBM. Synchronization between slices occurs on a high-bandwidth dedicated collective network. This minimal memory hierarchy already gives Jalapeño a big potential advantage over GPUs, where memory accesses must traverse a complicated memory system, resulting in large latencies that must be amortized or hidden over larger shapes. + +This choice is feasible because with careful placement of weights and KVs, synchronization between cores can be restricted to limited, known high-bandwidth comms such as tensor-parallel communication that can be overlapped with compute. + +
+ +There is also an additional general NoC which is used for general comms and to access the scale-up network. In general OpenAI saves huge power and gets big performance gains with a simplified NOC and memory subsystem vs Nvidia and Google. + +
+ +At the core level, OpenAI describes an out-of-order (OoO) core with an L1 cache. This is a large divergence from the pattern we have seen in other accelerators, all of which instead use software-managed scratchpad commonly paired with some async DMA support. Again, the argument being made here is that this allows Jalapeño to avoid fixed overheads such as barrier latencies, which on other accelerators (such as GPUs) need to be hidden or amortized over with higher work per core, and make it harder to get close to the raw peak bandwidth/flops. + +The tradeoff is that Jalapeño therefore relies on good prefetching to ensure timely arrivals of memory requests, which is less predictable and more difficult to reason about. However, with Codex in a good harness with access to detailed tracing, it is likely that finding the optimal kernel with the best prefetching for a given shape requires little human intervention. We think that is exactly what OpenAI has done to bring up DeepSeek R1, Kimi K2.5, and GPT-OSS so quickly. + +The cores also have support for “small” matrix dimensions, which (depending on how small) should make it more general across different model and batch dimensions less sensitive to matrix dimension alignment, padding overhead, and tiling inefficiency. For instance, TPUs, Trainium, and Etched chips have very large systolic arrays which can require large batches or exactly-divisible model dimensions to avoid tiling inefficiencies. + +With Jalapeño, OpenAI has focused on eliminating fixed latencies in the system to allow for as-close-to-roofline performance as possible across all areas of the pareto curve. In theory, this could give them advantages over the GPU at multiple operating points: + +- Much better upper-bound performance on low-latency/small-batch inference, which on GPUs is limited by many fixed overheads such as launch latencies, barrier latencies, memory system latency +- Some potential to achieve closer to the hardware roofline even for large-batch or long-context + +This comes with the caveat that even if the upper-bound performance is available in theory, it may be more difficult to realize that performance for real kernels. So it seems the approach is: + +1. Design for the highest upper-bound performance across all workload shapes +2. Let Codex do the tedious work of finding the kernels that achieve that upper bound + +Judging by the extremely fast turnaround for the OpenAI team to bring up InferenceX workloads on Jalapeño, we are optimistic about this approach. + +If Jalapeño is a success, it will be a strong signal that the industry’s obsession over programming models and perfect, universal compilers are invalidated by frontier AI models. + +## Software + +OpenAI writes Jalapeño kernels like assembly. Each kernel gets hand-tuned code, some running to ~3,000 lines, backed by correctness checks and a custom sanitizer. Early kernel work was human-in-the-loop rather than fully automated, but this shifted with a more scaled-up, internal version of Codex, one which OpenAI plans to pitch to enterprise customers. The internal serving engine is called “Teacup”. Interestingly, **OpenAI had no internal implementation of MLA kernels until they benchmarked DeepSeek with InferenceX**. The ability for Codex to write functional and efficient kernels so quickly (without any of OpenAI’s kernel engineering team intervening) shows the software pipeline’s developmental ability. + +OpenAI programs Jalapeño with Gluon. Gluon is OpenAI’s kernel programming language. Built on top of Triton, Gluon preserves Triton’s SPMD (Single Program Multiple Data) programming model, but it exposes **low-level programming abstractions**. For example, for NVIDIA GPUs, it offers APIs that map to PTX instructions, including MMA instructions, TMA instructions, mbarrier mechanisms, and many more. The most unique abstraction Gluon provides is **the layout**. Generally speaking, a layout defines a mapping between a hardware resource (e.g. 5th register of warp 9) and a tensor element (e.g. tensor element on row 6 column 7). Gluon’s layout abstraction is based on [Linear Layouts](https://arxiv.org/abs/2505.23819), a type of layout algebra OpenAI invented. Linear Layouts mathematically formalizes what a layout is and provides tools to operate on layouts. This enables many features, such as provably correct layout conversions and optimal memory swizzling. + +In terms of Jalapeño’s programming model, each Gluon program maps to a persistent thread. We believe this hints that Jalapeño suits the **persistent kernel programming pattern**, where each program executes on multiple tiles, and the programmer, rather than the hardware scheduler, assigns the work. OpenAI mentioned TensorInfo, an abstraction that explicitly encodes layouts. This is likely the set of layouts designed for Jalapeño, which will be powered by Linear Layouts. Finally, each core offers data prefetching and decoupled out-of-order units. For example, a user might program a wait on a prefetched data, which is locked behind a semaphore. + +In a weird twist of fate, OpenAI models like GPT 5.6 Sol, which currently run on NVIDIA GPUs, have been used to design a chip that poses a real threat to the CUDA moat - NVIDIA’s own GPUs are helping usher in their potential successor in real time. + +Comparing across time, we can also see Jalapeño’s developmental pace, achieving more than 2x throughput improvements at certain interactivities in less than 2 weeks. Each tarball we get from the Jalapeño team has a world of wonders inside. + +
+ +Not only did kernel performance improve, in the span of 8 days, the Jalapeño team enabled TP32, building on the previous TP8 configs and expanding beyond a single system to get a full rack-scale config running on a large model. This is a really impressive pace of development. + +
+ +To validate performance before committing to real hardware runs, OpenAI also has a simulator “chilisim” accurate to within 5% of measured hardware, using a fixed-width trace bus. Tracing on A0 was limited but has improved substantially on B0, likely with inputs from actual runs on A0 silicon. Engineers have demoed the Codex CLI running an internal model, nicknamed “Raiku” or “5.3 Codex Spark”, at 1.2ms TPOT. + +The team also showed off Codex-written demos running directly on the chip: Doom at 36 FPS, an FP32 fluid-dynamics simulation, and a “Liquid Light” mouse-drag visualization. + +
+ +On the model side, OpenAI’s internal megakernel approach, nicknamed “gigakernel”, is built around a single megakernel that loops on-device to reduce CPU overhead and launch time. The team is also leaning further into test-time compute strategies, with internal interest specifically in how to coherently use 1 million rollouts. + +## To Disagg or Not to Disagg, ‘tis the Question + +We mentioned earlier that OpenAI is not using prefill decode disaggregation on these chips. This came as a surprise to us, as NVIDIA and AMD GPU performance benefits significantly from PDD, even on homogenous hardware. Let’s dig into why the Jalapeño team went this way. + +Prefill-decode disaggregation (PDD) looks attractive when the workload is frozen. Prefill and decode stress hardware differently, so assigning each phase to a separately tuned pool can improve efficiency at one chosen input/output ratio. Production traffic, however, does not stay at that ratio. Input and output sequence lengths, concurrency, cache-hit rates, speculative-acceptance rates, and latency targets all move throughout the day. + +Once devices are divided into prefill and decode pools, too much prefill demand leaves decode chips idle while requests queue. But too much decode demand does the opposite. The operator must continuously predict the right split, provision spare capacity on both sides, and rebalance a system whose ideal ratio is always moving. + +In a unified system, some resources may be underused during a particular phase, but every device remains available to serve the next request. In a disaggregated system, an entire chip can sit idle simply because it belongs to the wrong pool. Local utilization looks better, but global utilization can be bad. + +
+ +Disaggregation also breaks locality. The prefill worker produces a large KV cache that the decode worker immediately needs, so the system must transfer that state across the network before generation can continue. That adds bandwidth consumption, synchronization, queueing, and another failure domain. The cost also rises with input sequence length because KV cache grows. However, avoiding the movement of KVs is largely a power and latency optimization; being willing to move some KVs around can allow for increased hardware utilization at the expense of some power and per-request latency. + +
+ +A fungible fleet shifts capacity between latency-sensitive requests and throughput-oriented batches, while a fixed split strands hardware whenever the traffic mix changes. Moreover, context length changes the balance between attention and FFN work, making any fixed hardware ratio efficient only near its design point. + +
+ +The same constraint applies to speculative decoding. A draft model has to feed candidate tokens to the verifier with extremely low latency. Separating the two across specialized pools turns a tightly coupled decoding loop into a distributed protocol. The extra communication and coordination can consume the latency saved by drafting. Keeping both models on the same devices and low-latency fabric preserves the locality that makes speculation worthwhile in the first place. + +
+ +However, disaggregation can still win where demand is sufficiently large, stable, and predictable, particularly when conventional GPUs need large phase-specific batches to reach good throughput. But it is not free lunch. + +## From Japanese to Indian (Mild to Spicy): Katsu, Vindaloo, and Chana - How are the curry dishes put together into a rack system? + +The Jalapeño System at the rack unit level consists of a CPU host rack and an ASIC rack. The host rack houses 16 host CPU trays named “Katsu,” each corresponding to one of the 16 ASIC trays, named “Vindaloo,” to the right of the Katsu. Each host houses two Turin-class AMD EPYC CPUs with 1.5TB of DRAM, 2x E1.S, and 2x M.2 SSDs per rack. Each tray is also specced with 400G (2x200G) frontend networking. Each Katsu tray connects to each Vindaloo tray via 8 external PCIe DAC cables that run horizontally across the rack at the front. The system level design is done in partnership with Celestica. + +The ASIC rack consists of 16 Vindaloo trays and 8 scale up switch trays (6 for local + 2 for global), named “Chana.” Each Vindaloo tray consists of 8 Jalapeño ASICs, making up a total of 128 Jalapeño ASICs per rack. The ASICs are connected to each of the Chana switch trays via a copper cable backplane, just like that of Nvidia’s Oberon. The scale up topology is split into a local domain of 128 ASICs within the rack and a global domain connecting up to 16 racks or 2,048 ASICs. We will explain the bandwidth and the topology in more detail below. + +Power provisioning to a sidecar host rack draws roughly 50kW provisioned (31kW in production), and the ASIC rack draws 130kW, making the total two rack system roughly 160kW. That’s basically a double-wide GB300 rack in terms of power draw. + +
+ +OpenAI can connect up to 2,048 Jalapeño XPUs within a single scale-up network. The scale-up network consists of two domains, a local domain connecting all 128 XPUs over backplane within the rack, as well as a global domain connecting 2,048 XPUs over 16 racks using a hybrid of copper and optical interconnect. Each rack consists of 8 Chana switch trays. Six Chana switches in the middle are for the local domain, which come with one 102.4T Tomahawk 6 switch ASIC each. Two Chana switches at the top and bottom of the local switches are for the global domain, which we think could consist of 2x 102.4T Tomahawk 6 switches making up to 204.8T per switch tray. + +In the local domain, each of the 128 Jalapeño chips has a per XPU uni-directional bandwidth of 4.8Tb/s and is connected on an all-to-all basis to 6x 102.4Tb/s Tomahawk 6 ASICs. This would amount to 48-differential pair (DP) male and female connector pairs per XPU translating to a total of 6,144DPs worth of passive copper cables per rack used for local scale-up. + +For the global domain, 16 racks totaling 2,048 XPUs are connected together via a combination of copper backplane, electrical 204.8T TH6 switch, 1.6T transceivers, and optical circuit switch. Each XPU has a uni-directional bandwidth of 1.6Tb/s for the global link, which is 16-differential pair (DP) male and female connector pairs per XPU for the backplane between the XPU and the global switch. Bandwidth exiting each global switch tray of 2 ASICs each is split between the backplane and front panel optics. + +Between local domain and global domain, backplane connector count per rack comes up to 64 DPs per XPU and a total of 8,192 DPs worth of passive copper cables per rack. + +The global domain adopts a rail-only architecture consisting of 8-rails across the global domain. We think OpenAI routes optical links in the global domain via Optical Circuit Switches (OCS) installed in every rack. For every XPU, 1.6Tb/s of global bandwidth will travel to the global switch tray over the copper backplane. This then exits the switch through the front panel via 1.6T transceivers, which go to the passive optical switch before exiting the rack. This expands the scale-up world size to 2,048 XPUs combining 16 racks of 128 XPUs each. + +
+ +Because scale-up networking is only about 10% of total system cost, that flexibility buys valuable optionality for future 10–20 trillion parameter models or 2–4 million token context windows. On deployment, OpenAI is partnering with neoclouds and is gathering reliability data with datacenter partners through January while optimizing dock-to-rack rollout time. + +## What’s Next + +Next, we talk about the future of Jalapeño, whose first production token is coming soon. The next goal is 100MW, and the hurdles will mostly be hardware: How much can they produce, how well can they deploy and operate datacenters, how do they handle monitoring, and resiliency, etc. The software is already proven, and with internal models, every software headstart is easily caught up to. Behind the paywall we will discuss implications for NVIDIA, AMD, Cerebras, and other chip companies who have signed deals with OpenAI in the coming years. + +[We also cover production volumes, units, and timelines for the next generation chip in our Accelerator model here](https://semianalysis.com/accelerator-hbm-model/). + +## Direct Implications for NVIDIA, AMD and Cerebras + + + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. + + + +--- + +_This article continues on our Substack. [Subscribe to SemiAnalysis](https://newsletter.semianalysis.com/subscribe) to read the complete article._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "What is OpenAI Jalapeño?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Jalapeño is OpenAI's first self-designed inference ASIC, built with Broadcom from a blank slate for LLM inference and announced at Hot Chips. Design work began in the middle of 2024 and the CoWoS design taped out in November 2025, roughly a 16 month cycle from team hiring to tape-out. The B0 stepping now in the fab delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized N3P compute die at a 700W TDP, with HBM4 providing 15.4 TB/s of memory bandwidth per package." + } + }, + { + "@type": "Question", + "name": "How does Jalapeño compare to NVIDIA Blackwell and Vera Rubin?", + "acceptedAnswer": { + "@type": "Answer", + "text": "On token throughput per all-in utility megawatt, Jalapeño beats Blackwell across almost all scenarios, and its single-token-prediction results surpass the Vera Rubin MTP figures NVIDIA and CoreWeave published in July 2026. On performance per dollar of TCO, Jalapeño and Vera Rubin are roughly head to head. Blackwell is the weaker comparison because Jalapeño competes against HBM4 parts like Rubin, and Jalapeño achieves these results without speculative decoding while the NVIDIA configurations use it." + } + }, + { + "@type": "Question", + "name": "Why did OpenAI skip prefill-decode disaggregation?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Disaggregation is attractive when the workload mix is frozen, but production traffic is not: input and output lengths, concurrency, cache hit rates, speculative acceptance, and latency targets all move through the day. Once chips are split into prefill and decode pools, an entire chip can sit idle because it belongs to the wrong pool, so local utilization improves while global utilization suffers. A unified pool also preserves the locality that speculative decoding depends on, since a draft model separated from its verifier turns a tight decoding loop into a distributed protocol." + } + }, + { + "@type": "Question", + "name": "How is a Jalapeño rack put together?", + "acceptedAnswer": { + "@type": "Answer", + "text": "The system is two racks. A host rack holds 16 CPU trays named Katsu, each with two AMD EPYC Turin-class CPUs and 1.5TB of DRAM, connected by external PCIe DAC cables to the ASIC rack. The ASIC rack holds 16 Vindaloo trays of 8 Jalapeño ASICs each, for 128 ASICs per rack, plus 8 Chana scale-up switch trays built on Tomahawk 6. Scale-up runs as a 128-ASIC local domain over a copper backplane and a global domain reaching 2,048 ASICs across 16 racks. The two-rack system draws roughly 160kW." + } + }, + { + "@type": "Question", + "name": "What software stack runs on Jalapeño?", + "acceptedAnswer": { + "@type": "Answer", + "text": "OpenAI programs the chip with Gluon, its own kernel language built on Triton, which keeps Triton's SPMD model while exposing low-level abstractions. Its distinguishing feature is the layout abstraction, based on Linear Layouts, a layout algebra OpenAI invented that enables provably correct layout conversions and optimal memory swizzling. The internal serving engine is called Teacup, a simulator called chilisim tracks measured hardware to within 5 percent, and a scaled-up internal version of Codex writes much of the kernel code." + } + }, + { + "@type": "Question", + "name": "What are the caveats on these Jalapeño results?", + "acceptedAnswer": { + "@type": "Answer", + "text": "All numbers were provided by OpenAI. SemiAnalysis verified the InferenceX runs in person in the lab but did not run the full benchmark suite, and there are no AgentX results yet. The published runs are 8k1k single-turn, which is easier to tune for than long-context multi-turn traffic: agentic workloads stress routers, prefix caching, cache management, and offload infrastructure that a single-turn 8k1k run never exercises. The models tested are also not the newest frontier open-weight releases." + } + } + ] +}`} diff --git a/packages/app/content/blog/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx b/packages/app/content/blog/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx new file mode 100644 index 000000000..b5c3ac749 --- /dev/null +++ b/packages/app/content/blog/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx @@ -0,0 +1,66 @@ +--- +title: 'Qwen3.5 397B on AgentX: B300 FP4 Delivers 12x the Performance per Dollar of H100' +subtitle: 'What four years of hardware and a 4-bit format buy on a long-context agentic workload' +seoDescription: 'On Qwen3.5 397B AgentX runs, B300 FP4 has 12x better performance per dollar than H100. The comparison spans four hardware generations and two numerical formats on a 256k truncated agentic trace replay.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - qwen + - b300 + - h100 + - h200 + - fp4 + - fp8 + - sglang + - nvidia +--- + +Compared to H100, on Qwen3.5, B300 FP4 has 12x better [performance per dollar](/glossary/performance-per-dollar). + +
+ +## What the number is and is not + +Two things are changing at once here, and both belong in the reading. The first is four years of hardware. The second is numerical format: the comparison spans [FP4](/glossary/fp4) and [FP8](/glossary/fp8) rather than holding precision constant, which is the honest way to compare what an operator can actually deploy today on each part, but not a controlled study of the silicon alone. + +The workload matters as much. This is not 8k1k. Qwen3.5 has a native max context length of 262k tokens, so AgentX replays the truncated 256k agentic dataset against it, which means frequent context compaction and heavy prefix reuse. A ratio measured on a long-context multi-turn replay is a different claim from the same ratio measured on independent short prompts, and it is the one closer to production coding traffic. + +## Read it at your own operating point + +A 12x aggregate ratio is a headline, not a deployment plan. The curve is what carries the decision: pick the [interactivity](/glossary/interactivity) your product needs, read each system there, and check the first-token latency at the same point before concluding anything. + +That is also why the per-model AgentX charts default to showing the whole frontier rather than a single number. Older hardware can remain competitive at low concurrency on performance per dollar while falling away entirely at high throughput, and a single ratio cannot express both halves of that shape. + +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "How much better is B300 than H100 on Qwen3.5 397B?", + "acceptedAnswer": { + "@type": "Answer", + "text": "On the AgentX Qwen3.5 runs, B300 FP4 has 12x better performance per dollar than H100. The comparison spans both a hardware generation gap and a numerical format difference, since it covers FP4 and FP8 rather than holding precision constant." + } + }, + { + "@type": "Question", + "name": "Is the 12x figure measured on a fixed sequence length workload?", + "acceptedAnswer": { + "@type": "Answer", + "text": "No. Qwen3.5 has a native max context length of 262k tokens, so AgentX replays the truncated 256k agentic trace dataset against it. That workload involves frequent context compaction and heavy prefix reuse, which is a different measurement from independent short prompts." + } + } + ] +}`} diff --git a/packages/app/content/blog/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx b/packages/app/content/blog/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx new file mode 100644 index 000000000..03c32b0c4 --- /dev/null +++ b/packages/app/content/blog/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx @@ -0,0 +1,67 @@ +--- +title: 'MI355X versus GB300 NVL72 Inference Performance: 20x Gap on Qwen3.5 SGLang' +subtitle: 'GatedDeltaNet, a 262k native context, and no AMD competition at all on the same engine' +seoDescription: 'Qwen3.5 397B AgentX results. On SGLang against SGLang, NVIDIA is over 20x better at 90 tok/s/user with zero competition from AMD, and NVIDIA SGLang submissions hold much lower p90 TTFT than TRT-LLM.' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - qwen + - sglang + - trtllm + - nvidia + - amd +--- + +Qwen3.5 397B uses GatedDeltaNet instead of vanilla attention for every couple of layers. GatedDeltaNet was invented at MIT and NVIDIA Research and has a theoretically constant state storage requirement instead of vanilla attention's linear storage requirement, which means lower storage than an equivalent dense attention model. Unlike end to end model training research such as the Nemotron disaster, NVIDIA Research is great at fundamental research like GDN and LatentMoE, both of which are used in frontier models. + +One methodology note before the results. This model's native max context length is 262k tokens, so AgentX replays the [truncated 256k dataset](https://inferencex.semianalysis.com/agentx/cc-traces-weka-062126-256k) against it. That simulates a workload on a smaller model where the max context length would be frequently reached with many compactions, which is how users would actually use this model. + +## The gap + +Qwen3.5 397B is a strong hold for NVIDIA on SGLang versus SGLang, with over 20x better performance at 90 tok/s/user. There is currently zero competition from AMD for Qwen3.5 SGLang. + +
+ +Holding the engine constant is what makes this comparison worth stating. A cross-engine result can always be argued about, since the software stacks differ in maturity and in who is paid to optimize them. Here both sides are SGLang, and the gap is still 20x. + +## NVIDIA is trading TTFT for interactivity + +Within the NVIDIA field, we again observe over-optimization for [interactivity](/glossary/interactivity) at the cost of time to first token, especially in the case of TRT-LLM. In the chart above, all of the NVIDIA SGLang submissions have much lower p90 TTFT when compared to TRT-LLM. + +That is a real tradeoff rather than a defect, but it is one a reader has to see. A recipe that streams faster after making the user wait longer will look better on an interactivity axis and worse against a first-token [service level objective](/glossary/service-level-objective), and only reading both together tells you which one fits your product. + +
+ +--- + +_These results are one slice of AgentX 1.0. The full analysis, the replay methodology, and the 70+ upstream PRs the benchmark drove are in [AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?](/blog/agentx-inferencexv3-does-cuda-moat). Every point is explorable on the [free dashboard](https://inferencex.semianalysis.com/)._ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "How much faster is NVIDIA than AMD on Qwen3.5 397B?", + "acceptedAnswer": { + "@type": "Answer", + "text": "On SGLang against SGLang, NVIDIA is over 20x better at 90 tok/s/user, and there is currently zero competition from AMD for Qwen3.5 SGLang. Holding the engine constant on both sides is what makes the comparison meaningful." + } + }, + { + "@type": "Question", + "name": "Why does AgentX replay a truncated dataset for Qwen3.5?", + "acceptedAnswer": { + "@type": "Answer", + "text": "The model has a native max context length of 262k tokens, so AgentX uses the truncated 256k context dataset. That simulates a workload on a smaller model where the max context length is frequently reached with many compactions, which is how users would actually use this model." + } + } + ] +}`} diff --git a/packages/app/content/blog/ultra-high-interactivity-on-nvidia.mdx b/packages/app/content/blog/ultra-high-interactivity-on-nvidia.mdx index ef532f0e5..b6f9f3ef6 100644 --- a/packages/app/content/blog/ultra-high-interactivity-on-nvidia.mdx +++ b/packages/app/content/blog/ultra-high-interactivity-on-nvidia.mdx @@ -3,6 +3,7 @@ title: 'Ultra-High Interactivity on NVIDIA GPUs? TileRT on InferenceX' subtitle: 'Can TileRT software on NVIDIA GPUs compete with Cerebras, Groq LPU, and SambaNova? Batch size 1, disaggregated engine, high-throughput prefill engine, high-interactivity decode engine' date: '2026-08-10' publishDate: '2026-08-10' +modifiedDate: '2026-08-18' tags: - benchmark - gpu @@ -13,6 +14,8 @@ tags: - tilert - vllm - glm5 + - agentx + - agentic --- _Originally published on the [SemiAnalysis newsletter](https://newsletter.semianalysis.com/p/ultra-high-interactivity-on-nvidia) on August 10, 2026._ @@ -235,9 +238,9 @@ Work is being done to simplify this, especially as software development can be a AI coding agents accelerate tuning within known templates, but novel transformations still require expert judgment. A monolithic persistent kernel also reduces the usefulness of conventional per-kernel profiler timelines, making automated feedback loops more difficult. -## Next steps with TileRT and InferenceX +## Next steps with TileRT -We are actively working on moving TileRT benchmarking from InferenceX's single-turn 8k/1k to our new agentic coding benchmark, which we call AgentX. This scenario replays real Claude Code and Codex traces with long-context, multi-turn requests, realistic subagent activity, and dynamic tool-use delays. Its median input length is 140k tokens, while the theoretical median cache-hit rate roofline reaches 99.2%. +AgentX, InferenceX's long-context, multi-turn coding scenario, is now live for supported models and serving stacks. It replays privacy-preserving workload shapes derived from real Claude Code and Codex sessions, with long-context, multi-turn requests, realistic subagent activity, and dynamic tool-use delays. Its median input length is 140k tokens, while the theoretical median cache-hit rate roofline reaches 99.2%. Extending TileRT benchmarking from its current single-turn 8k/1k workload to AgentX remains the next step for this serving stack.
对比实时 AgentX 推理结果 → + +## 什么是 Agentic Inference Benchmark? + +Agentic Inference Benchmark 是面向多轮 agent 流量的系统基准测试。被测系统包括 accelerator、服务器拓扑、推理引擎、模型 checkpoint、精度、并行策略、cache 配置与请求调度器。 + +它回答以下问题: + +- 一套部署能同时服务多少个 agent 会话? +- 每个 agent 需要等待多久才能收到首 token? +- 每个活跃 agent 接收 output token 的速度是多少? +- 在目标延迟下,每颗芯片能提供多少 output 吞吐量? +- 重复 prefix 会命中 HBM、host memory,还是触发重新计算? +- 在可用的响应速度下,每百万 token 的成本是多少? + +“Benchmark”也常用于 agent 任务评估,因此必须说明测试范围。这两类测试回答不同问题: + +| 问题 | 测试类型 | 常见结果 | +| --------------------------- | --------------------------- | ------------------------------------ | +| Agent 能否正确完成任务? | Agent 任务评估 | 成功率、policy 合规率或评分器分数 | +| 推理栈能否承载 agent 流量? | Agentic Inference Benchmark | 延迟、吞吐量、cache 行为、容量与成本 | + +AgentX 对应第二类。任务语义与完成质量需要单独评估。 + +## 为什么固定序列推理基准测试无法代表 Agent 流量? + +固定序列测试仍然适合做受控比较。例如,8K input 和 1K output workload 能隔离常见请求形状,并让不同配置容易复现,但它无法表示持续运行的 agent 会话。 + +| Workload 特征 | 固定序列推理 | Agentic 推理 | +| -------------- | ------------------------------- | ---------------------------------- | +| 请求形状 | 固定 input 与 output 长度 | 每一轮长度都不同 | +| 到达模式 | 按固定速率或 batch 发送独立请求 | 前序请求完成后才触发后续依赖请求 | +| Prefix 复用 | 可选或人工构造 | 大型共享 prefix 随会话不断增长 | +| Tool 时间 | 通常不存在 | 模型调用之间存在不均匀停顿 | +| 并行性 | 请求级并发 | 主 agent 与 subagent 分支可能重叠 | +| Cache 生命周期 | 短且规律 | 长时间状态争用 HBM 与 offload 容量 | +| 有效结论 | 一种请求分布下的性能 | 完整 agent 会话的容量与响应速度 | + +这些差异可能改变系统排序。更大的 HBM 能让长时间存活的 KV cache 保持驻留;更有效的 prefix caching 可以避免重新处理数十万 prompt token;对独立 prompt 影响较小的 scheduler 与 router 开销,也可能随活跃会话的数量和长度增长。 + +## Agent Benchmark 是一张请求依赖图 + +Agent 会话不是扁平的 prompt 列表。主 agent 轮次形成依赖链;某一轮可以启动一个或多个 subagent,主链随后可能等待这些分支完成;辅助调用也可以独立运行而不汇入主链。 + +
+ +AgentX 通过[闭环模式](/zh/glossary/closed-loop-benchmark)回放这张图。每个客户端只有在依赖的前序工作完成后才发送下一个可执行请求。服务器越快,会话越早推进并产生后续工作。当 subagent 扇出时,瞬时 HTTP 请求数可以高于客户端 concurrency;分支完成后,请求数会再次下降。 + +因此,AgentX concurrency 表示**活跃 agent 客户端数量**,并不是固定 request batch。若多个客户端同时存在活跃 subagent,concurrency 50 的运行可能拥有超过 50 个 in-flight 请求。 + +## AgentX 如何构建可复现 Workload? + +AgentX 通过四个阶段,把已记录的会话结构转换为不含原始内容的回放。 + +### 1. 采集请求形状与时间 + +自愿启用的 HTTP proxy 会记录请求到达与完成时间、input 和 output token 数、conversation ID 与 subagent ID。公开数据不包含 prompt、源代码、tool argument 或 tool result。 + +### 2. 保留 Prefix,而不保留内容 + +每段 input 会被转换为以 64-token block 为单位的会话内 hash 链。相同 block ID 会保留不同轮次间的 prefix 关系。回放前,AIPerf 使用确定性的合成编码与 tool-use token 替换这些 block。 + +### 3. 重建依赖关系 + +主 agent 请求形成线性链;subagent 请求形成带有 spawn 和 join 依赖的独立链;记录中的轮次间停顿会重现 tool 执行与等待其他工作的时间。回放保留客户端可见的顺序与重叠关系,但不会推断 provider 内部事件。 + +### 4. 在不同系统上回放同一场景 + +固定 seed 控制会话采样、回放起点与合成 payload。每套软硬件配置都接收相同场景定义,因此曲线差异来自推理系统,而不是人工编写的 prompt batch。 + +AgentX v1.0 从自愿提供的 Claude Code trace 中筛选出 393 个公开会话。每个会话至少包含 20 个请求,并且同时运行的 subagent 不超过 10 个。 + +| AgentX v1.0 特征 | 数值 | +| -------------------------- | ------: | +| 公开会话 | 393 | +| 单请求 input token 中位数 | 142,016 | +| 单请求 output token 中位数 | 444 | +| 包含 subagent 的会话 | 44% | +| Full 数据集最大上下文 | 1M | +| 限制上下文的数据集变体 | 256K | + +
+ +Full 变体保留最高 1M token 的上下文。256K 变体会移除超过上限的请求,同时保留其余请求的时间关系与 subagent 重叠。[AgentX 方法论](/zh/agentx/methodology)公开了数据集规则、图表与来源链接。 + +## 保证比较有效的 Benchmark 控制 + +长时间运行的 agent workload 对初始状态、cache 内容、speculative decoding 与 host memory 十分敏感。AgentX 通过明确规则锁定这些变量。 + +### Cache Warmup 与测量窗口 + +固定 seed 会在每段会话 25% 至 75% 的位置选择起点。`max_tokens=1` 的 primer 会建立活跃主 agent 与 subagent 的 prefix,随后每条回放 lane 再完成 10 个 warmup 请求,之后才开始测量。 + +对外结果只统计随后一小时的 profiling 窗口。每次循环回放都会加入唯一 cache-bust 标记,避免无关会话逐步形成共享 prefix。 + +### 合成 Payload 与 Speculative Decoding + +合成 token 会保留长度和 prefix 结构,却无法自然复现 draft token acceptance。AgentX 因此针对每组模型、speculative 方法、draft length 与 thinking mode,使用 [SPEED-Bench](https://arxiv.org/abs/2604.09557) 编码类别测得的 acceptance length。该数值记录在带版本的 golden 文件中,并通过推理引擎控制项应用。 + +这样可以把推理服务性能与合成文本偶然造成的 speculator acceptance 差异分开。 + +### DRAM 与 KV Cache Offload + +Host memory 容量会影响长 prefix 应该重新加载还是重新计算。没有标准化 DRAM 配置的服务器上限为 3 TB;GB200 NVL72、GB300 NVL72 与 TPUv7 等标准系统可以使用实际装机容量。每种基准测试配置只能按 GPU 占比使用对应的 host memory。 + +回放格式与依赖规则记录在 [AIPerf WEKA trace 指南](https://github.com/ai-dynamo/aiperf/blob/main/docs/tutorials/weka-trace.md)中。 + +## Agentic Benchmark 应该测量哪些指标? + +单个数字无法描述 agent 服务。应该在完整 concurrency sweep 中同时报告吞吐量、延迟与交互性。 + +| 指标 | 说明 | +| -------------------------------- | --------------------------------------------------- | +| 每颗芯片的 output 吞吐量 | 每颗 accelerator 每秒完成的生成 token 数 | +| P90 interactivity | 根据 p90 inter-token latency 得到的单客户端生成速度 | +| 首 token 延迟(TTFT) | Agent 开始收到响应前的等待时间 | +| Inter-token latency(ITL) | 相邻 streamed output token 之间的时间 | +| Input 吞吐量 | 每秒处理的 prompt token 数 | +| Cache 来源分布 | Prompt token 来自 HBM、host memory 还是重新计算 | +| Queue 与 in-flight request depth | 分支启动和汇合时的 scheduler 压力 | +| 每百万 token 成本 | 在明确延迟或交互性要求下的推理服务经济性 | + +百分位数不能省略。如果部分 agent 分支需要在长 prefill 后排队,较低的平均 TTFT 可能与很差的 p90 或 p99 同时出现。总 output 吞吐量上升时,单个客户端接收 token 的速度也可能下降。 + +## 应该读取完整的吞吐量与交互性曲线 + +AgentX 扫描不同的并发 agent 客户端数量,并绘制相应运行点。低并发通常有利于单客户端响应速度;提高并发可以改善 batching 与总吞吐量,直到排队、内存压力或其他瓶颈开始降低交互性。 + +
+ +有效结果是 [Pareto frontier](/zh/glossary/pareto-frontier),而不是唯一的最大吞吐量点。硬件 A 可能在低交互性下给出最高 tok/s/chip,硬件 B 却可能在交互式编码产品所需的响应速度下领先。 + +闭环回放还意味着更快的配置能在相同 profiling 小时内推进到采样会话的更后位置,因此实际请求组合可能略有变化,低并发时尤其明显。结果应包含完整曲线,并使用足够并发降低采样噪声。 + +## Agent 流量会考验推理栈的哪些部分? + +Agentic 推理会暴露固定 prompt batch 容易低估的瓶颈。 + +- **KV-cache 容量:**长上下文与大量活跃会话会让大型 cache 驻留更久。 +- **Prefix-cache 正确性:**错误命中会产生错误状态,漏掉命中则会重复昂贵的 prefill。 +- **Offload 带宽:**从 host memory 移动长 prefix 可能比重新计算更快,但传输粒度与 descriptor 开销会影响结果。 +- **Scheduler 公平性:**并行 subagent 可以提高总吞吐量,同时让某条分支等待过久。 +- **Router 与 frontend CPU:**Prefix matching、过期项清理、序列化与 streamed response 处理都会随活跃会话状态增长。 +- **长上下文 kernel:**Attention、sparse attention、normalization 与 graph capture 路径可能随 sequence length 变化而表现不同。 +- **Speculative decoding:**Acceptance 控制与 scheduler 生命周期会同时影响吞吐量和交互性。 + +[AgentX 优化追踪页](/zh/agentx/optimizations)记录了 vLLM、SGLang、TensorRT-LLM、ATOM、Dynamo 与 KV-cache 系统中由该 workload 暴露的具体改进。 + +## 如何比较 AgentX 结果? + +比较双方应使用同一套场景约束: + +1. 对齐模型、checkpoint、精度与 thinking mode。 +2. 对齐 AgentX 数据集变体与上下文上限。 +3. 记录推理引擎、container image、并行策略与 speculative decoding 方法。 +4. 应用相同的 golden acceptance-length 规则。 +5. 对齐 DRAM 容量、KV-cache offload policy 与按比例分配内存的规则。 +6. 使用相同 seed、warmup 流程、profiling 时长与客户端 concurrency sweep。 +7. 当系统规模不同时,同时比较每颗芯片与整套系统结果。 +8. 在相同 p90 interactivity 或延迟目标下比较。 +9. 保留失败与不稳定配置,而不是只报告成功运行点。 + +相同交互性下的比较通常比峰值吞吐量更有用。它直接回答部署问题:在用户需要的响应速度下,哪套系统每颗芯片或每美元能够服务更多 agent 工作? + +## 这项 Agent Benchmark 能回答什么,不能回答什么? + +AgentX 可以测量 trace 衍生 agent workload 下的推理容量、响应速度、cache 复用、内存压力、吞吐量与成本。它可以显示软件更新是否改善了相同硬件上的长上下文多轮服务,也可以比较不同硬件拓扑如何改变有效 frontier。 + +AgentX 无法判断模型是否写出了正确代码、选择了正确工具、遵守了业务规则,或完成了原始任务。合成 payload 会有意移除这些语义。Agent 质量应使用任务评估,推理性能则使用 AgentX。两类分数描述不同的系统问题,不应混为一个总分。 + + + 查看 AgentX 吞吐量与交互性 frontier → + + +## 常见问题 + +### 什么是面向 LLM 推理的 Agentic Benchmark? + +它是一项系统基准测试,用于衡量推理栈如何服务多轮 agent 流量。测试会回放不断增长的上下文、共享 prefix、tool 等待与 subagent 分支,并报告延迟、吞吐量、交互性、cache 行为、容量与成本。 + +### Agentic Benchmark 与固定序列推理基准测试有何区别? + +固定序列测试会发送 input 和 output 长度明确的独立请求。Agentic Benchmark 回放的是相互依赖的会话,其中请求长度、时间、prefix 复用与并行分支会持续变化,因此更接近真实 agent 产生的服务条件。 + +### AgentX Benchmark 测量什么? + +AgentX 使用从编码 agent trace 衍生的流量,测量完整推理配置。它扫描并发 agent 客户端数量,并报告每颗芯片的 output 吞吐量、p90 interactivity、TTFT、ITL、cache 行为与推理服务成本。 + +### AgentX 为什么使用闭环并发? + +Agent 无法提前发出所有轮次,因为后续动作依赖前序结果。在闭环回放中,每个客户端会在依赖完成后发送下一个可执行请求,从而保留服务器速度与会话产生后续工作的时间关系。 + +### Agent 推理最重要的指标有哪些? + +应结合 output 吞吐量、p90 interactivity、TTFT、ITL、cache 来源分布、queue depth 与成本,并在相同延迟或交互性目标下比较完整 concurrency 曲线。单个最大吞吐量点无法描述交互式部署。 + +### Agent Benchmark 会衡量模型或 Agent 质量吗? + +部分 Benchmark 会,但 AgentX 不会。AgentX 使用保留 workload 形状的合成 payload 来测量推理系统性能。任务正确性、工具选择与 policy 合规性需要单独的 agent 评估。 + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "什么是面向 LLM 推理的 Agentic Benchmark?", + "acceptedAnswer": { + "@type": "Answer", + "text": "它是一项系统基准测试,用于衡量推理栈如何服务多轮 agent 流量。测试会回放不断增长的上下文、共享 prefix、tool 等待与 subagent 分支,并报告延迟、吞吐量、交互性、cache 行为、容量与成本。" + } + }, + { + "@type": "Question", + "name": "Agentic Benchmark 与固定序列推理基准测试有何区别?", + "acceptedAnswer": { + "@type": "Answer", + "text": "固定序列测试会发送 input 和 output 长度明确的独立请求。Agentic Benchmark 回放的是相互依赖的会话,其中请求长度、时间、prefix 复用与并行分支会持续变化。" + } + }, + { + "@type": "Question", + "name": "AgentX Benchmark 测量什么?", + "acceptedAnswer": { + "@type": "Answer", + "text": "AgentX 使用从编码 agent trace 衍生的流量,测量完整推理配置。它扫描并发 agent 客户端数量,并报告每颗芯片的 output 吞吐量、p90 interactivity、TTFT、ITL、cache 行为与推理服务成本。" + } + }, + { + "@type": "Question", + "name": "AgentX 为什么使用闭环并发?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Agent 无法提前发出所有轮次,因为后续动作依赖前序结果。在闭环回放中,每个客户端会在依赖完成后发送下一个可执行请求,从而保留服务器速度与会话产生后续工作的时间关系。" + } + }, + { + "@type": "Question", + "name": "Agent 推理最重要的指标有哪些?", + "acceptedAnswer": { + "@type": "Answer", + "text": "应结合 output 吞吐量、p90 interactivity、TTFT、ITL、cache 来源分布、queue depth 与成本,并在相同延迟或交互性目标下比较完整 concurrency 曲线。" + } + }, + { + "@type": "Question", + "name": "Agent Benchmark 会衡量模型或 Agent 质量吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "部分 Benchmark 会,但 AgentX 不会。AgentX 使用保留 workload 形状的合成 payload 来测量推理系统性能。任务正确性、工具选择与 policy 合规性需要单独评估。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/agentx-inferencexv3-does-cuda-moat.mdx b/packages/app/content/blog/zh/agentx-inferencexv3-does-cuda-moat.mdx new file mode 100644 index 000000000..2ca701034 --- /dev/null +++ b/packages/app/content/blog/zh/agentx-inferencexv3-does-cuda-moat.mdx @@ -0,0 +1,1031 @@ +--- +title: 'AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?' +subtitle: '开源价值 300 万美元的数据集、100 万以上上下文长度、多轮、子智能体、95%+ KVCache 命中率、GB300 NVL72、MI355X、B200' +seoDescription: 'AgentX 1.0 是首个完全开源、1M 上下文的多轮 agentic 编码推理基准测试。文中给出 DeepSeek V4 Pro、Kimi K3、MiniMax M3、Qwen3.5 和 GLM 5.3 在 GB300 NVL72、B300、B200、MI355X 和 H200 上的 agentic 结果,以及该基准测试推动的 70 多个上游 PR。' +date: '2026-08-24' +publishDate: '2026-08-24' +tags: + - agentx + - agentic + - benchmark + - inference + - gpu + - nvidia + - amd + - announcement +--- + +自 2025 年 11 月的 [Claude Code 拐点](https://newsletter.semianalysis.com/p/claude-code-is-the-inflection-point)以来,长上下文、多轮的 agentic 工作负载快速增长,如今已经主导了生产推理的流量。2026 年 4 月,OpenAI 的企业级 agentic 支出超过了 ChatGPT 支出。 + +[Claude Code is the Inflection Point](https://newsletter.semianalysis.com/p/claude-code-is-the-inflection-point) — 当前 GitHub 公开 commit 中有 4% 由 Claude Code 编写。按现在的增长曲线,我们认为到 2026 年底 Claude Code 将占每日全部 commit 的 20% 以上。一眨眼的工夫,AI 已经吞掉了整个软件开发。 + +agentic 工作流已经明确接过了接力棒。**今天,我们发布 AgentX 1.0 —— 全球首个完全开源、1M 上下文的多轮 agentic 编码推理基准测试,以 Apache 2.0 协议发布。[完整仪表板在此。](https://inferencex.semianalysis.com/)** + +
+ +过去,业界大多用固定序列长度的 prefill 和 decode 工作负载来衡量性能,但这种衡量方式并不准确。真实世界是多轮、长上下文、高 prefill 复用的,还伴随子智能体(sub agent)突发、KVCache offload 和大量工具调用。因此,我们的目标是为整个行业建立一套正确衡量 AI 硬件与软件性能的方法。 + +[我们投入了超过 300 万美元构建这个数据集。今天,我们把一切都开源。](https://inferencex.semianalysis.com/)InferenceXv3 在既有的“固定序列长度”场景(8k1k、1k1k、1k8k)之外,新增了 AgentX 这一贴近现实的场景。它用 agentic 编码流量替代了此前 8k 输入、1k 输出 token 的单轮流量,让基准测试场景更接近真实。 + +完整测试矩阵运行在约 2MW 的持续运行算力上,覆盖 1000 多颗芯片和大量 SKU,包括 MI355X、GB300 NVL72、GB200 NVL72、B300、B200、MI325、MI300X、H200 和 RTX Pro 服务器。Rubin 将于本月晚些时候到位,TPU 和 MI455X UALoE72 会在今年内加入。[如果你觉得我们的免费开源工作有价值,欢迎点个 star。](https://github.com/SemiAnalysisAI/InferenceX) + +很高兴看到 NVIDIA 和 AMD 在 agentic 工作负载上都拿出了出色的性能。NVIDIA 在很多前沿模型上表现非常好,AMD 在部分前沿模型的特定对比中同样表现不俗。 + +
+ +AgentX 头几个月产出的最有价值的东西,并不是首批测试结果,而是这个基准测试已经带来的巨大行业影响。**超过 70 个上游 PR** 以 AgentX 作为北极星基准代理,围绕真实生产 agentic 工作负载在 vLLM、SGLang、TensorRT-LLM、ATOM、AITER、Dynamo、LMCache 和 Mooncake 上做优化。这些优化中的大部分都可以迁移到生产流量上。本文后面会逐项深入分析。 + +
+ +开源是 InferenceX 的核心原则,因此我们开放的部分比多数把“开源”挂在嘴边的项目都要多:开放的前端、通过易用 REST API 提供服务的公开数据库(**已有多家一线 AI 实验室的容量规划团队在使用**)、公开的 GitHub Actions CI 溯源信息、[运行日志](https://inferencex.semianalysis.com/inference/agentic/440193?i_seq=agentic-traces&i_xmode=interactivity&view=logs),以及每一个数据点的精度验证。关键在于,我们的测试配置主要跟随 [recipes.vllm.ai](http://recipes.vllm.ai) 和 [SGLang cookbook](https://docs.sglang.io/cookbook/intro) 使用上游镜像,衡量的是客户实际能拿到的性能,而不是针对 benchmark 特调过的镜像。 + +三到四周后,我们会发布一篇 AgentX 更新文章,涵盖针对 agentic 工作负载的进一步优化,以及 AMD 和 NVIDIA 的最新性能结果。需要理解的是,agentic 工作负载的画像变化很快,InferenceX 会持续快速跟进,测试真正有代表性的工作负载。 + +InferenceX 100% 坚持开源,这离不开开源伙伴的贡献与支持。我们要感谢以下为 AgentX 1.0 发布做出巨大贡献的人: + +- **Inferact/vLLM**:Roger Wang、Yifan Qiao、Simon Mo、Jeff Ma 等多位贡献者 +- **RedHat/llm-d**:Michael Goin、Robert Shaw、Tyler Michael Smith +- **RadixArk/SGLang**:Baizhou Zhang、Yuwei An、Mingyi Lu 等多位贡献者 +- **LMCache/TensorMesh**:Samuel Shen +- **Weka**:Callan Fox、Val Bercovici +- **MoonCake 维护者:** Teng Ma、Xu Wenjie、Ke Yang +- **AMD**:Thomas Wang、HaiShaw、Andy Luo、Seungrok Jung、Chun Fang、Parth Panchal、Bill He、Theresa Shan、Hongxia、Fangzhou、Gilbert Lei、Yanfei Wang、Duyi Wang、Peng Sun、Lingpeng Jin、Simon Danielsson、Xiaohu Guo、Haichen Zhang、Chang Liu、Doug Lehr、Poovaiah Palangappa,以及 AMD 上海研发中心的众多同事 +- **NVIDIA**:Xin Li、Anthony Casagrande、Kedar Potdar、Ankur Singh、Ishan Dhanani、Nick Comly、NVIDIA 上海 TensorRT-LLM 团队等多位工程师 +- **Anthropic 团队,感谢他们及时修复了多个 bug,让 AgentX 得以实现** +- **GitHub:** Austen Stone,帮助提升了 AgentX 所依赖的 GitHub Actions 的可靠性 +- 以及其他众多贡献者 + +[此外,我们也感谢所有支持 InferenceX 开源计划的伙伴,包括 Meta、Microsoft、Oracle、OpenAI、MiniMax、月之暗面 Kimi、阿里巴巴 Qwen 和智谱 GLM。](https://inferencex.semianalysis.com/quotes) + +
+ +## agentic 工作负载简介 + +从宏观上看,agentic 工作负载有四个特征: + +1. **多轮:** 一个会话包含大量用户 / 助手交互(几十到上百轮),而聊天机器人场景只有寥寥数轮。多轮、长上下文、高 prefill 复用,并伴随子智能体突发和大量工具调用。 +2. **长上下文:** 系统提示词、工具定义,加上轮次本身很多,上下文会迅速累积。 +3. **高前缀复用:** 会话是线性推进的,第 n-1 轮的输出通常会拼接进第 n 轮,因此大部分上下文可以从 KV cache 读取而不必重算(具体取决于可用于存放 KV 张量的存储容量)。随着 n 增大,已缓存输入与未缓存输入之比通常趋近于 1。 +4. **子智能体突发:** 一个会话会启动多个带全新上下文的短生命周期子智能体,形成突发式的 KVCache 访问模式。 + +
+ +考虑到上述特征,对这类工作负载做基准测试和现有的固定序列长度基准测试有本质区别:agentic 推理本质上是一个*系统问题*。由于前缀复用率极高,KV 张量必须在节点 / rank 之间高效传输(NIXL、MORI-IO、Mooncake)。此外,不同会话应当根据对应前缀所在的位置被路由到不同节点 / rank,以最大化缓存命中率(LLM-d、Dynamo、vLLM/SGLang router)。长上下文会话会挤占 KV cache 的 HBM 容量,需要把 KV 张量 offload 到不同层级的内存(DRAM、SSD),而这个过程本身也必须高效(Mooncake Store、LMCache、vLLM Simple Offloading、SGLang HiCache)。 + +相比之下,固定序列长度的单轮工作负载不涉及前缀复用,推理性能基本反映的是芯片 / kernel 的基础性能。这并不是说 InferenceX 上大量的固定序列长度数据不重要。恰恰相反,剥离 agentic 服务的复杂性之后,底层推理优化的进展会显得格外清晰,这些数据也为 AgentX 结果提供了重要基线。 + +为了让 AgentX 工作负载尽可能贴近真实,我们采集了 [393 条 SemiAnalysis 内部匿名 Claude Code trace](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126) 作为初始语料用于回放。为在保留原始前缀复用模式的同时对内容做匿名化,我们采用了与 [Qwen-Bailian 数据集](https://github.com/alibaba-edu/qwen-bailian-usagetraces-anon)类似的方法,后者是最早的生产 trace 语料之一。随后我们用 [AIPerf](https://github.com/ai-dynamo/aiperf) 在不同并发客户端数量下,按原始请求时序重建这些 trace。为让 AgentX 数据集成为可能,我们与 Anthropic 合作推动了两项 Claude Code 功能上线,在此感谢 Anthropic 团队的帮助。 + +1. [https://github.com/anthropics/claude-code/issues/49207](https://github.com/anthropics/claude-code/issues/49207) +2. [https://github.com/anthropics/claude-code/issues/66761](https://github.com/anthropics/claude-code/issues/66761) + +这段简介应该足以让读者理解下一节的测试结果。后文我们会更深入地介绍测试方法、回放框架和数据集。 + +## agentic 编码推理性能 + +在评估编码推理性能时,OpenAI、Anthropic、xAI 等前沿实验室主要关注三点:每美元性能与交互性(TPOT)的关系、TTFT(首 token 延迟),以及端到端任务完成情况。每兆瓦性能同样重要,因为地面数据中心电力是关键约束(钱是社会构建物,而且看起来实验室们有的是钱;电力在当下则是实打实地难搞)。[我们的数据中心模型给出了逐季度的电力需求与供给爬坡估算。](https://semianalysis.com/datacenter-industry-model/) + +本节我们梳理各前沿模型上 agentic 性能的整体规律。我们强烈建议读者把它当作一份导览,[自己去研究这些结果](https://inferencex.semianalysis.com/)。**所有数据都是开源的,社区完全可以就当下真实推理性能的现状得出自己的结论。** + +## DeepSeek V4 Pro 0813 + +DeepSeek V4 Pro 0813 是一款来自中国、极受欢迎的前沿开放权重模型,总参数约 1.6 万亿,激活参数 490 亿。 + +下图为截至 8 月 21 日、按总拥有成本(TCO)归一化后各 SKU 全部提交结果中的最佳性能。 + +所有 DeepSeek v4 运行中*全部*请求的 ISL/OSL 分布如下:ISL p50=88k、p90=272k、p95=404k、p99=675k;OSL p50=413、p90=2.2k、p95=3.7k、p99=8.6k。 + +
+ +
+ +总体而言,**同时关注每用户每秒 token 数**(TPS,也就是交互性)**和 TTFT 很重要,因为这两者往往此消彼长**。例如上图中,某些 SKU 在还算不错的交互性下拿到了很高的吞吐量,但 TTFT 严重恶化。多高的 p90 TTFT 算“可接受”,很大程度上取决于应用场景。对多数服务 agentic 工作负载的生产系统而言,p90 TTFT 通常在 200–5,000ms 之间。超过 5–10s 就已经在“在线推理”的边界上了。曲线上超高吞吐的那一段仍有实际用途——在延迟不重要、追求系统利用率峰值的场景里(批处理、*超*长时间运行的 agent 等)。 + +单节点性能方面,MI355X 的开源方案(vLLM)落后于厂商自有的 ATOM(AMD 版的 TensorRT-LLM)。我们认为 AMD 用 ATOM 快速推进性能前沿是好事,但仍建议他们把这些改进更优先地上游到 vLLM。 + +
+ +AMD 的分布式推理(DI)团队在过去半年里于 8k1k 场景上取得了很大进展,但要让 DI 成为真实工作负载下可用的方案,还有一段路要走。从每 GPU 吞吐量与交互性的关系看,1xDEP8+1xDEP8 的 disagg 配置只在高吞吐区间取得了小幅提升,在低延迟区间反而更差。 + +更糟的是,中高交互性配置下的吞吐量提升,被 p90 TTFT 的大幅飙升所抵消。原因之一是并发超过 64 时启用了 SGLang 的 `--enable-prefill-delayer` 参数,[它会推迟 prefill 准入,让 DP rank 攒出更满的 batch](https://github.com/sgl-project/sglang/blob/v0.5.17/python/sglang/srt/server_args.py#L3180-L3194)(最多等 30 个 forward pass)。此外,这些点还把 chunked prefill 大小从 8,192 提高到了 65,536。 + +
+ +
+ +在端到端延迟上,ATOM MI355X 胜过 B200 vLLM(但并未胜过 B300 或 B200 SGLang)。问题在于,除了阿里巴巴内部一个小型广告业务部门之外,中国和西方的多数 AI 实验室都不愿在生产中使用 ATOM——它缺失的功能太多。阿里巴巴主力的 Qwen 大模型团队在生产中并不使用 ATOM。 + +
+ +在 2026 年 8 月 21 日之前,AMD 实力很强的 MI355X SGLang 开发团队在端到端(e2e)性能的每美元表现上已经追平了 B200 vLLM。 + +
+ +不过,B300 vLLM 和 B200 SGLang 仍然领先 AMD 的 MI355X。 + +
+ +2026 年 8 月 21 日之后,得益于 Inferact 和 NVIDIA 在 vLLM 上的优化,NVIDIA B200 的每美元性能已经超过 MI355X。这是一场胶着的竞赛,我们很期待接下来几周的性能优化,也会很快发布 AgentX 更新文章。 + +[AMD 已经列出了他们的 DeepSeekv4 vLLM 优化清单,里面有不少值得期待的提升空间。](https://github.com/vllm-project/vllm/issues/52911) + +
+ +再看 NVIDIA。他们最有竞争力的方案是 GB300 Dynamo TRTLLM 和 GB200 Dynamo vLLM,两者都依靠 PD disagg 在合理交互性下取得高吞吐。此外,GB300 配置采用了 wide-EP(DEP32)decode 实例,以在前沿曲线中段获得更高吞吐。 + +注意,2xDEP8+1xDEP12 的 GB200 点与 3xDEP8+1xDEP16 的 GB300 点在 TPS 上比在 TTFT 上接近得多。同样,TTFT 总体上对工作负载的“突发性”更敏感。由于 GB300 那个点的总并发高得多,它会遇到更多子智能体流量,因而有更多冷 prefill。这一点在该点的 TTFT 图上可以看出来: + +
+ +
+ +
+ +按 TCO 归一化后,B300 vLLM 与 B200 vLLM 的聚合式性能相当接近,主要差别在于 B300 凭借比 B200 多 50% 的 HBM 容量,能“挤”出额外的吞吐量。 + +
+ +我们可以用 AgentX 新增的服务端指标可视化进一步观察这个差异。 + +在 384 条并发 agentic trace 的负载下,B300 vLLM DEP8 配 3TB DRAM、使用 vLLM simple offloading,取得了 91% 的 HBM 缓存命中率,外加 1.36% 的 DRAM 缓存命中率。原因是该配置下 **HBM KV cache 的工作集规模**约为 4300 万 token,而任一时刻在途的 token 数只是略微超过这个量。 + +
+ +
+ +在 B200、并发 196(其余参数不变)的情况下,HBM 缓存命中率只有 73%,需要更多依赖 DRAM,offload 缓存命中率接近 20%。此时 HBM KV cache 工作集规模为 2200 万 token,大约是 B300 的一半。 + +DRAM KV offload 通常实现为写穿(write-through)缓存,也就是写入 HBM 缓存的每个前缀同时也会写入 DRAM 缓存。因此,只有当可用于 offload 的 DRAM 显著大于(1.5–3 倍)HBM KV cache 容量时,它才最有效。 + +
+ +
+ +H200 SGLang FP8 能在低并发下服务 DeepSeek v4,从每美元性能看甚至可以和 B200/MI355X SGLang 一较高下。但受限于 HBM 容量,它在高吞吐场景下无法与更新的 SKU 竞争。 + +
+ +此外,随着用户数增加,高并发下对 DRAM KV offload 的依赖会带来无法接受的延迟。 + +
+ +总体来看,MI355X 相对主要竞争对手 B200 和 B300 表现尚可。在曲线上低吞吐 / 低延迟那一段两者最为接近,因为那里只用到张量并行和较为基础的 kernel。AMD 需要优化 MI355X 上的 DEP kernel,才能在高吞吐场景下更有竞争力——尤其是它的 HBM 还是 B200 的 1.5 倍。 + +
+ +
+ +## Kimi K3:2.8 万亿参数 + +Kimi K3 是另一款来自中国的前沿开放权重模型,总参数 2.8 万亿,参数量级与 Claude 的 Mythos/Fable5 模型架构相当。我们把它作为开放权重的架构代理来使用。K3 大到单台 B200 服务器都装不下,必须用 wide EP / wide TP 或流水线并行才能放下全部权重。在 vLLM 上,投机解码 / DSpark 直到最近都完全无法与流水线并行组合,因此 B200 在 Kimi K3 上的表现非常糟糕,被 MI355X 压着打——因为 B200 在流水线并行下用不了投机解码。 + +MI355X vLLM 在短上下文单轮工作负载上 day 0 即开箱可用,但在长上下文多轮工作负载上,MI355X 的 AITER 和 Triton kernel 在第一周就彻底崩了,上游 vLLM 在真实工作负载下对 MI355X 完全不可用。 + +
+ +Hopper 很难承接 Kimi K3 的 AgentX 工作负载:一方面 Kimi 模型体量巨大,另一方面 vLLM 维护者 / NVIDIA 一直没有把精力放在为 K3 优化 Hopper 上。Hopper(SM90)需要为 K3 定制调优 kernel,并针对高交互性服务调优 TP32/EP32 的 shape。 + +
+ +我们认为 AMD 用 ATOM 快速推进 K3 性能是件好事,但仍希望 AMD 进一步优先把这些改进上游到 vLLM。ATOM 目前是 AMD 性能最好的引擎,但对使用上游开源服务栈的客户来说,vLLM 才是更有参考价值的对比对象。 + +
+ +
+ +在端到端延迟 40 到 60 秒的那段曲线上,MI355X ATOM 的每美元性能甚至超过了 GB300 NVL72 vLLM。 + +
+ +## MiniMax M3 + +在 MiniMax M3 432B 上,NVIDIA 完胜所有对手。AMD 的软件性能非常糟糕,长上下文下尤其如此,根源在于 AMD 工程管理层只激励团队针对短上下文单轮工作负载做调优,而忽视了长上下文多轮工作负载。 + +
+ +
+ +B300 TRT-LLM TP2 拿下了 M3 的桂冠。曲线上缺少 DP-attention 的数据点,因为在 M3 上这种做法并不划算——KV cache 局部性会变成路由约束,后文会进一步说明。以并发 40 的 GB200 为例,TP4/EP4/DPA 的吞吐量只有纯 TP4 的 0.60 倍,而 p90 TTFT 高出 3 倍以上。并发 32 时缓存命中率仅 28.8%,理论值则是 96.0%。每个 DP rank 独占缓存池的四分之一,一个 30 万 token 的会话一旦落到错误的 rank 上,就要全部重算。M3 的前沿曲线上没有出现任何带 EP 的 decode 配置,可能是因为并发还不足以让所有专家的负载均衡起来。 + +在 MiniMax M3 上,按 TCO 归一化的吞吐量,B200/B300 也完全胜过对应的机架级方案。在 AgentX 上,机架级的优势不像以往那么明显,因为 Dynamo router 可能成为瓶颈——它的工作量随在线前缀的数量和长度增长。相关优化以及若干带来两位数百分比提升的修复会在后文讨论。此外,wideEP、wide DCP 和 wide TP 都还没有调优良好的 kernel。而 GB200/300 的 TCO 更高,在没有 wide EP / wide DCP 的情况下,每 TCO 性能就显得更差。 + +话虽如此,我们也预期 NVIDIA 会在该 SKU 的机架级方案上做进一步优化,届时会在后续文章中重点介绍。 + +尽管 P90 ISL 达到 317k,目前还没有任何提交使用上下文并行。M3 只有 4 个 KV head,即使 TP8 下 DCP 也最多只能到 2,而且 MSA indexer 还需要单独的上下文并行处理(vLLM 上已有一个 PR)。更多讨论见后文的上下文并行一节。 + +
+ +
+ +NVIDIA 所有 Pareto 最优点在并发 20 以上都启用了 KV offload,而 AMD 的 Pareto 最优点没有一个使用 DRAM KV offload。在其他模型上,AMD 使用 KV offload 的比例也低于 NVIDIA。原因是 AMD vLLM 上 CPU KVCache offload 所需的 GPU 到 CPU 传输效率极低:`hipMemcpyBatchAsync` API 直到 ROCm 7.14 才提供。没有它,vLLM 原生的 Simple CPUOffloading 只能做串行 memcpy,无法把多次拷贝合并成更大的消息。 + +另外值得一提的是,从吞吐量与 p90 交互性的关系看,vLLM 的表现与 TRT-LLM 非常接近;而在吞吐量与 p90 TTFT 的关系上,vLLM 表现更好。 + +
+ +
+ +## Qwen3.5 397B + +Qwen3.5 397B 每隔几层就用 GatedDeltaNet 替代常规注意力。GatedDeltaNet 由 MIT / NVIDIA Research 提出,其状态存储需求在理论上是常数,而常规注意力是线性增长的,因此相比同等规模的稠密注意力模型,它的存储需求更低。与 Nemotron 那样惨败的端到端模型训练研究不同,NVIDIA Research 在 GDN、LatentMoE 这类被前沿模型采用的基础研究上做得非常出色。 + +注意,该模型原生最大上下文长度为 262k token,因此我们使用[截断后的数据集](https://inferencex.semianalysis.com/agentx/cc-traces-weka-062126-256k)。这模拟的是在上下文上限较小的模型上频繁触顶、频繁压缩上下文的工作负载,也就是用户实际使用这类模型的方式。 + +在 SGLang 对 SGLang 的对比中,Qwen3.5 397B 是 NVIDIA 的强势领地:90 tok/s/user 时性能领先 20 倍以上。在 Qwen3.5 SGLang 上,AMD 目前完全没有竞争力。 + +
+ +
+ +我们再次看到 NVIDIA 以牺牲 TTFT 为代价过度优化交互性,TRT-LLM 尤其明显。上图中所有 NVIDIA SGLang 提交的 p90 TTFT 都远低于 TRT-LLM。 + +在 Qwen3.5 上,B300 FP4 的每美元性能是 H100 的 12 倍。 + +
+ +## GLM 5.3 + +GLM 5.3 在 GLM5.2 744B 基础上做了额外的后训练,是一款前沿级别的模型。 + +在开源 SGLang 的性能上,这又是一个 NVIDIA 在真实 agentic 推理性能上胜过 AMD 的模型。在 150 tok/s/user 的 p90 交互性下,NVIDIA 的成本效率最高领先 5 倍。以 AMD 软件当前的状态,在 150 tok/s/user 时 NVIDIA 的性能优势大到这种程度:即便竞品芯片硬件白送(服务商当然仍要承担数据中心托管、电力和其他运营成本),用 NVIDIA 的单 token 成本依然更低。 + +我们期待 AMD 在几周后的 AgentX 更新文章中带来性能优化,那篇文章还会包含其他一些非常有意思的结果。 + +
+ +
+ +看 ATOM 的话,在 p90 E2E 归一化交互性的部分区间内,AMD 的每美元性能优于 GB300 NVL72 SGLang,甚至优于 TRTLLM。这一结果值得肯定,也再次期待 AMD 把这些优化移植到 SGLang。同时我们也期待 NVIDIA 在接下来几周里尽快优化 GB300 NVL72。 + +
+ +这里插入介绍一个我们称之为 **E2E 归一化交互性(E2E Normalized Interactivity)** 的实验性指标。它的目标是在同时考虑 TTFT 和 TPS 的前提下,衡量用户实际感受到的响应速度,定义为 OSL/E2EL。把 E2EL 等于 TTFT 加上 OSL 乘以 TPOT 代入(实际上只解码 OSL - 1 个 token),可得下式。 + +它本质上等于交互性(1/TPOT 那部分)再加上一项与 TTFT 成正比的惩罚。 + +
+ +需要说明的是,这个指标是实验性的,并不完美。例如它对高 TTFT 的惩罚很重,也无法体现 PD 分离等优化的全部细节。AgentX v1.0 的所有提交仍然分别针对常规交互性和 TTFT 做优化。我们会继续打磨能反映现代 agentic 推理各种细节的新北极星指标。 + +## AgentX 的行业影响:面向 agentic 工作负载的优化 + +AgentX 头几个月最有影响力的成果不是产出开源数据集,而是 **AgentX 合作伙伴以 AgentX 为北极星、为优化真实 agentic 工作负载而提交的 50 多个上游 PR** 所带来的行业影响。AgentX 的真实 agentic 流量不只考验原始的 prefill 与 decode kernel,还会检验从 KV cache 生命周期、混合注意力缓存正确性、CPU KV offload、传输进度、路由亲和性,到增量分词、请求序列化和调度器记账在内的完整端到端 token 生成链路。这些环节对每一个生产 agentic 部署都很重要。 + +这只是我们既有使命的延续:帮助生态加速改进,推动软件以极快的速度进步。一个很好的例子是 SemiAnalysis 与 AMD 软件研发团队多年的合作——我们持续提供反馈和建议,帮助他们现代化软件开发方法。这不仅带来了许多加速 AMD 进展的改变,也在推动 AMD 开源方案在 agentic 工作负载上向一流体验靠拢。 + +## 分布式推理生态简介 + +如前所述,agentic 推理本质上是全系统问题,而不只是芯片 / kernel 层面的问题。此外,当大型*分布式系统*要处理数十万条 agentic 请求时,请求调度和 KV cache 管理就不再是小事,会实打实地影响性能。例如子智能体会产生突发式的 KVCache 访问模式,若不加优化,就会错误地把主智能体的缓存挤出去。 + +下图从宏观上展示了这一技术栈。最上层是 router(有时也叫“frontend”),负责把请求分发给不同 worker。例如当服务端启用数据并行注意力时,每个 DP rank 都有各自的 KV cache。为了避免打乱其中任何一个缓存,请求会按不同策略路由,例如[一致性哈希](https://github.com/vllm-project/router/blob/main/src/policies/consistent_hash.rs),让同一会话 / 子智能体的请求按唯一 ID 路由到同一处。 + +
+ +对大多数路由策略来说,各家 router 的实现差异不大。有些是独立组件,例如 [vLLM router](https://github.com/vllm-project/router) 和 [llm-d router](https://github.com/llm-d/llm-d-router);有些则集成在引擎里,例如 [SGLang model gateway](https://github.com/sgl-project/sglang/tree/main/sgl-model-gateway) 和 [ATOM Mesh](https://github.com/ROCm/ATOM/tree/main/atom/mesh)。 + +请求路由完成后,会交给 vLLM、SGLang 等推理引擎的调度器处理。引擎负责实际执行推理并通过 API 返回结果。此外,每个引擎都提供接口,把引擎内部的 KV cache 与外部 KV cache 管理器连接起来,从而形成一个“可插拔”的生态:不同的 KV cache 管理器可以与多种推理引擎集成。 + +当前 AgentX 结果所使用的一种简单部署方式是[在同一节点上把 Mooncake](https://pypi.org/project/mooncake-transfer-engine/) 与 vLLM 一起运行。每个 vLLM worker 内嵌一个 Mooncake Store 客户端,并贡献一部分主机 DRAM 加入外部 KV cache 池。vLLM 通过 MooncakeStoreConnector 接口连接这个池,把可复用的 KV block 载入 GPU 显存,并把新计算出的 block 存回主机内存。Mooncake Store 负责管理外部缓存,包括放置与淘汰;Mooncake Transfer Engine 则负责 GPU 与 CPU 内存之间数据的实际搬运。 + +不同的 KV cache 管理器可能使用不同的传输引擎,在内存层级之间或机器之间(例如 prefill 与 decode worker 之间)实际搬运字节。以 Mooncake Store 为例,它使用 [Mooncake Transfer Engine](https://github.com/kvcache-ai/Mooncake/tree/main/mooncake-transfer-engine) 在 GPU 显存、主机 DRAM 和远端节点之间搬运 KV block。 + +一套部署可以一边用 Mooncake Store 把可复用的 KV block offload 到主机 DRAM,一边用 NIXL 把请求专属的 KV 从 prefill GPU 直接传到 decode GPU。Mooncake TE 负责 Mooncake Store 这条路径的搬运,NIXL 则通过 UCX 和(在支持的平台上)GPUDirect RDMA 负责独立的 prefill-decode 路径。也就是说,同一个推理引擎内部可以并存多条 KV 管理与传输路径。 + +这套生态由许多独立组件构成,包括推理引擎、router、KV cache 管理器、数据传输库和集群控制器。NVIDIA Dynamo、llm-d、AMD Infera 这类平台把其中若干组件“打包”成完整的软件发行版,发布相互兼容的容器镜像、connector、部署清单和编排逻辑,让这些组件可以作为一个系统来部署和运维。最终交付的产品通常是一组协同工作的容器,而不是单体服务(例如 Dynamo、llm-d 和 Infera 通常部署在 k8s 上,用于协调大型分布式系统)。 + +
+ +## 上下文并行 + +长上下文让某些并行技术真正有了用武之地,而固定 8k 的 prompt 很难体现出来——8k 下没什么可切分的,TTFT 本来就很短。此外,TP 和 DP attention 这类并行策略在更长的上下文下并不理想:TP 会让完整 KV 在每个 rank 上都复制一份;DP attention 虽然共享 KV,但在长上下文下容易卡住,因为长上下文工作负载的上下文长度方差也更大。 + +
+ +上下文并行是一种把 query token 切分到多个 GPU 上的并行技术,分为两种形式:用于 prefill 的 PCP 和用于 decode 的 DCP。PCP 中每个 rank 处理自己的 query 分块(KV 以环形方式传递),由于 prefill 通常是计算受限的,这样可以并行化 FLOPs,使 prefill 更快,也不会让某个 rank 单独扛下超大 prompt 的 prefill 尖峰。DCP 中每个 rank 扫描自己那一份 KV 分片,随后以 flash-decode 的方式合并部分注意力结果。由于 decode 是显存带宽受限的,并行读取 KV 可以提高 tok/s。 + +
+ +
+ +这项并行技术部分源自 NVIDIA Research。NVIDIA Research 在这类基础研究上很强,但在端到端训练研究上就相当难堪了——他们的 Nemotron3 Ultra 模型表现糟糕,如今连体量小得多的 Qwen3.8 27B 都能大幅胜过它。DCP/PCP 构成了 CUDA 护城河的一部分,因为 AMD 的 DCP/PCP 实现尚未优化。在 [vLLM 支持矩阵](https://docs.vllm.ai/en/latest/design/attention_backends/#standard-attention-mha-mqa-gqa-backends)里,AMD 的每一个后端都是不支持状态。 + +接下来几节提到的部分改动就聚焦在 DCP/PCP 上。 + +## vLLM 的 agentic 优化 + +我们与来自 Inferact、Red Hat、NVIDIA 和 AMD 的 vLLM 维护者一起,把 AgentX 的真实回放器当作北极星,相关修复最终都进入了上游,其中大部分优化都可以很好地迁移到生产环境。举几个例子: + +vLLM 改进了混合注意力的前缀缓存,使短生命周期的滑动窗口分配不会挤掉有价值的长上下文检查点。[选择性保留](https://github.com/vllm-project/vllm/pull/43447)保住了稀疏回放边界,在 14 条并发请求、上下文最长 100 万 token 的条件下,前缀缓存命中率超过 95%。同样的可达性策略[也应用到了 Mooncake](https://github.com/vllm-project/vllm/pull/44774),并[移除了不可达的滑动窗口查找](https://github.com/vllm-project/vllm/pull/45444)。更早的后续工作还[停止 offload 那些永远不会被复用的滑动窗口 block](https://github.com/vllm-project/vllm/pull/42258),并[把投机解码的 lookahead block 保留在被保留的前缀中](https://github.com/vllm-project/vllm/pull/44082)。 + +
+ +高并发 agent 的 agentic 工作负载需要 offload。得益于前面提到的 AgentX 的推动,vLLM 上已经有工作流在推进:让 CPU KV offload 支持混合模型,而不只是统一的全注意力模型。这个区别之所以重要,是因为统一模型每个 token 只有一种 KV 布局,connector 用单一 block 几何结构就能描述要保存什么;而混合模型同时携带多个 cache group,每组的形状和生命周期都不同,假设单一布局的 connector 无法表达某个 block 属于哪一组。结果就是,最需要 offload 的长会话模型反而用不上它。通用的 [SimpleCPU connector](https://github.com/vllm-project/vllm/pull/37160) 先落地,随后[在 ROCm 上启用](https://github.com/vllm-project/vllm/pull/40549),再[扩展到 DeepSeek-V4 的混合注意力](https://github.com/vllm-project/vllm/pull/42296):相比前缀放不下 HBM 后重新计算,输出吞吐量提高 81.7%,平均端到端延迟降低 46.6%。 + +Mooncake 也获得了[同等的混合内存分配支持](https://github.com/vllm-project/vllm/pull/42828)。同样的布局问题在分离式服务中再次出现:一个[待合入的改动](https://github.com/vllm-project/vllm/pull/51052)会在 1P1D 的 prefill/decode 拆分下,通过 MoRI-IO 把 Kimi-K3 的 conv+ssm 循环状态与注意力 KV 一起传输;没有它,decode 侧会从未初始化的循环状态开始。循环状态槽位复用既有的 remote-block-ids 通道,因此分离式路由器无需任何模型专属改动。该路径已在 MI355X 上端到端验证:每条链路 TP8、跨节点 RDMA,两侧均开启 DSpark 投机解码。 + +在对真实工作负载做性能剖析时,vLLM 维护者发现 offload 期间的开销转移到了 store 路径上——写得太多、太频繁。现在有三处修复解决了这个问题: + +[当相同的传输已在进行中时跳过 store](https://github.com/vllm-project/vllm/pull/41289),这样共享同一前缀的并发会话只需付出一次代价,而不是每个会话各付一次。store 现在[只覆盖新生成的 KV 区间](https://github.com/vllm-project/vllm/pull/46412),因此延续历史的会话只写增量,而不是每轮重写整个前缀。最后,store [不再依赖相同 block 是否仍驻留在 HBM](https://github.com/vllm-project/vllm/pull/46906),因此已排入队列的工作不会因为底下发生淘汰而被丢弃。 + +load 路径则单独做了调优,因为查找发生在每次调度决策时,而不只是在真正搬运数据时。[把调度路径上的查找改为异步](https://github.com/vllm-project/vllm/pull/45659)可以让 connector 离开每步的关键路径,这样一步调度不必等待 CPU 侧的缓存查询就能接纳新工作。随后,[紧凑的零拷贝查找 key](https://github.com/vllm-project/vllm/pull/45969)、[接收端并行加载](https://github.com/vllm-project/vllm/pull/45971)和[预构建的 Mooncake key 字符串](https://github.com/vllm-project/vllm/pull/46188)进一步消除了剩余的 CPU 与传输开销。 + +
+ +长生命周期的混合状态还带来了固定形状请求几乎遇不到的正确性与记账问题。vLLM 现在[按混合 cache group 分别发出 cache 事件](https://github.com/vllm-project/vllm/pull/44103),[正确计算分布式上下文 store 的 stride](https://github.com/vllm-project/vllm/pull/45371),并[在分布式上下文与 prefill 下正确计算查找前缀](https://github.com/vllm-project/vllm/pull/46855)。相关的[上下文并行记账改动](https://github.com/vllm-project/vllm/pull/45340)让缓存归属与分片后的 token 区间对齐。投机状态现在也能[在合并后的 Mooncake group 之间传递](https://github.com/vllm-project/vllm/pull/49069),并[贯穿 SimpleCPU 协调器](https://github.com/vllm-project/vllm/pull/49071),避免重复轮次的缓存悄悄丢失 EAGLE 状态。 + +从 ROCm 侧优化 vLLM 的 agentic 工作负载来看,工作继续下沉到缓存层之下,那里剩余的开销是按层而不是按请求计算的。当前缀既能存活又能及时到达之后,剩下的就是 decode 步骤本身;而一个会话里要跑上千次的 decode 步骤,会为每一次可以避免的拷贝、每一个不匹配的 kernel 付出代价。 + +有三个尚未合入的改动针对这一层: + +- 一个 Kimi-K3 的改动[把 KDA decode 结果直接写入层输出缓冲区](https://github.com/vllm-project/vllm/pull/51183),为每个 KDA 层省下一次设备内拷贝;单看收益很小,但它会在每个解码 token 的每一层上重复发生。 + +- 第二个改动[选用 AITER 的稀疏 MLA decode kernel](https://github.com/vllm-project/vllm/pull/51714) 替代通用路径,AgentX 输出吞吐量提高 5.22%,token 间延迟显著降低。 +- 第三个改动很好地说明了测量形态有多重要。配套改动[把 full-graph 注意力投影路由到调优过的 AITER GEMM](https://github.com/vllm-project/vllm/pull/51713),在低并发的固定序列上取得 2.3% 提升。这说明 kernel 层面的改动可以在统一 shape 上拿到干净的收益,却在 agentic trace 上被缓存与调度带来的波动淹没。一个[待合入的改动](https://github.com/vllm-project/vllm/pull/52882)干脆把这种 shape 敏感性变成了分发依据:它用 gfx950 上的 AITER / native 混合路径取代 DeepSeek V4 C4A selector 的 ROCm top-k 瓶颈,短、中上下文走 AITER,长上下文走 graph-safe 的调优 native 回退路径。据报告,selector 的端到端加速为 1.21x 到 1.76x,在 84 种 shape 的矩阵上 decode kernel 的几何平均加速为 1.2x 到 2.9x。 + +## SGLang 的 agentic 优化 + +AgentX 团队与来自 RadixArk、Meta、NVIDIA 和 AMD 的 SGLang 维护者密切合作,推动了 SGLang 上 agentic 工作负载的优化,使生产推理性能大幅提升。下面详细介绍这些优化。 + +先从分配器一侧说起:SGLang 的滑动窗口工作解决的是与 vLLM 保留策略相同的冲突。窗口页和前缀页取自同一个池子,而窗口是更贪婪的一方——它不断翻转,前缀却静止不动,于是在压力之下,短暂的分配会把持久的分配挤出去。 + +有三处设计改进从不同角度攻击这个问题。其一是[在页离开窗口时主动释放](https://github.com/sgl-project/sglang/pull/26907),而不是等淘汰压力来找它们,让失效的窗口状态不再争抢自己已经用不到的页。其二是[把 compute lock 限制在单个窗口内](https://github.com/sgl-project/sglang/pull/27210),限定一条在途请求最多能同时钉住多少池容量。其三是[清除失去作用后仍然残留的 full-KV 条目](https://github.com/sgl-project/sglang/pull/29369)。 + +不过主动释放有一个与分叉相关的盲区:从共享前缀分叉出来的请求可能仍持有可复用的 full-KV,而分叉点处的窗口状态已被释放,结果为了缺失的那一半廉价数据,整个前缀都要重算。[进行中的工作](https://github.com/sgl-project/sglang/pull/34565)会在这些分叉点保留 SWA 状态,让分叉继承窗口而不是重建它。 + +与之并行的 [ROCm ring-cache 修复](https://github.com/sgl-project/sglang/pull/30339)属于正确性而非容量问题:环形缓冲区在设计上就会复用槽位,而复用一个旧内容仍被引用的槽位,产生的是错误输出,而不是变慢的输出。这些在单个 8k prompt 上都看不出来——窗口不会绕过前缀,池子也不会有争用。但在多轮混合注意力会话中,这些改动决定了昂贵的全注意力历史在下一轮是否还在。 + +HiCache 是 SGLang 内置的一流 offload 机制。它面对与 vLLM connector 相同的混合模型问题,解法是一种不对称处理:[offload 全注意力缓存,回读时再重建那一小段滑动窗口尾部](https://github.com/sgl-project/sglang/pull/29417)。只有昂贵的那一半值得跨总线搬运,廉价的那一半重建比取回更快。在 AMD 上,[分阶段写回](https://github.com/sgl-project/sglang/pull/28534)让这种搬运不会在进行中阻塞引擎。剩下的缺口是循环状态,因为它无法像窗口尾部那样从相邻 token 重建;[FlashInfer GDN 检查点](https://github.com/sgl-project/sglang/pull/29735)让它得以参与前缀复用,在 92.4% 缓存命中率下把吞吐量从 47,771 提升到 53,004 tok/s/GPU。 + +
+ +另有两处改动针对变长流量对 kernel 流水线的影响。AgentX 的真实会话往往以持续变化的上下文长度到达,生产流量也是类似形态。一个按长度做特化的运行时,几乎会为看到的每个请求重新编译一份 kernel。SGLang 维护者的解法是[把上下文长度作为运行时标量传入](https://github.com/sgl-project/sglang/pull/30255),而不是把它固化进某一次编译;这让 AgentX 并发 384 的输出吞吐量提升 26.75%、平均 TTFT 改善 36.25%,收益来自消除编译,而不是把计算做得更快。 + +
+ +同样的思路下,[移除每步一次的 device-to-host 序列长度同步](https://github.com/sgl-project/sglang/pull/30365)消除了一个 decode 气泡——它之所以存在,只是因为 host 想知道一个 device 早就知道的长度。 + +
+ +变长同样会在注意力 kernel 内部造成损耗。在 GB300 上,混合上下文的 decode batch 要付“长尾税”:一次对照 profile 把 9.8 ms decode 步骤差异中的 8.4 ms 归因于注意力,最长的那些请求拖长了持久化 kernel 的共享 wave。[进行中的工作](https://github.com/sgl-project/sglang/pull/34888)把 TRTLLM MHA 的 decode batch 按 KV 长度分组,让短请求不再等待最长的那些请求。 + +decode 也可能在更上一层被饿死——不是在 kernel 里,而是在调度器上。在 DP attention 下,每个 rank 都要参与同一个 MoE 集合通信,但注意力工作是本地调度的;于是某个 rank 一旦被喂进一串 chunked-prefill 续段,就会不断赢下“prefill 优先”的决策,而同伴 rank 正在运行的 batch 只能干等——这是在 AgentX 运行中观察到的现象。[可配置的 decode 间隔](https://github.com/sgl-project/sglang/pull/35017)在 prefill 之后强制插入 decode 轮次;在 AgentX DSv4 Pro 上,输出吞吐量提升 141%,p99 token 间延迟下降 97.3%,代价是 TTFT 中位数从 36.5 秒升到 59 秒——用首 token 等待时间换取输出流的平滑度。 + +
+ +当一条请求不带任何可复用历史时(例如子智能体刚启动),随便哪个 worker 都可以,唯一值得考虑的就是负载均衡。但当请求带着数 MB 的已缓存前缀时,把它发给一个并不持有该前缀的空闲 worker 就是昂贵的选择,router 必须知道状态究竟在哪里。SGLang 为此加入了 [DP 缓存亲和性](https://github.com/sgl-project/sglang/pull/26091),让会话粘在持有其缓存的 rank 上。在这个 PR 中,还实现了 [DP 感知的 prefill 与 decode 路由](https://github.com/sgl-project/sglang/pull/26245),让分离式部署的两侧做出一致的决策;以及[把缓存均衡作为路由信号](https://github.com/sgl-project/sglang/pull/26293),避免亲和性退化成单个热点 worker。router 只能依据它被告知的信息行动,因此混合缓存事件也变得[对 radix cache 可见](https://github.com/sgl-project/sglang/pull/26387)、[对滑动窗口可见](https://github.com/sgl-project/sglang/pull/26579)。 + +投机解码得到了特别关注,因为 MTP 为每条请求引入了第二份、更小的状态,它必须挺过主缓存所经历的一切。SGLang [修复了分离式服务中的 draft window 传输](https://github.com/sgl-project/sglang/pull/30461),让这份状态完整跨越 prefill 到 decode 的边界;[为高并发在线解码加入 overlap 调度](https://github.com/sgl-project/sglang/pull/30497);[移除了一处无实际作用的 EAGLE 重归一化](https://github.com/sgl-project/sglang/pull/31294);并[避免 EAGLE prefill 期间的 host 同步](https://github.com/sgl-project/sglang/pull/33662)。尚未合入的[资源租约调度工作](https://github.com/sgl-project/sglang/pull/32042)和[数据并行 graph metadata 修复](https://github.com/sgl-project/sglang/pull/32196)延续同一目标:当请求可能被撤回再恢复、而不是一路跑到结束时,让 overlap 仍然安全。 + +在 prefill 与 decode 拓扑异构的 agentic 工作负载下,需要前缀感知的暂存,而这正是前缀缓存与分离式服务相互冲突的地方。当两侧的分片方式不一致时,KV 无法作为一整段连续流拷贝过去,必须按传输网格切分,再按 decode 侧期望的偏移重新组装。前缀命中会让这件事更难而不是更容易,因为 prefill worker 此时只发送未缓存的剩余部分,而 decode 侧仍然期待一份完整且位置正确的缓存。[在 staging buffer 中支持 radix cache](https://github.com/sgl-project/sglang/pull/30545) 会按该网格切分已缓存的发送内容,并散布到正确的 decode 偏移上。 + +
+ +然而这也带来了正确性问题。一项 127,500 token 的共享前缀测试,从 128 根针中只答对 2 根变成了 128 根全对——说明缓存此前一直悄悄落在错误的位置,而这种情况在吞吐量基准里只会被记成一个又快又自信的错误答案。在 AgentX 对比中,每用户输出吞吐量中位数还提高了 9.6%,而每 GPU 总吞吐量几乎不变。传输本身也带着无用负载:decode 侧的 PREBUILT batch 从不进入模型前向,但每条传过来的 prompt 仍会被展平并拷进一个 CUDA 输入张量,而第一次 decode 步骤反正会用中继元数据重建它。[去掉这次无用的 prompt 传输](https://github.com/sgl-project/sglang/pull/35070)是单项收益最大的改动,在 AgentX GB300 上带来每用户输出吞吐量 +18.0%、每 GPU decode 吞吐量 +12.7%。一个[后续改动](https://github.com/sgl-project/sglang/pull/35071)把 prefill DP-rank 的引导查询移出 decode 调度器的关键路径,让原本在消费结果时同步付出的一次 HTTP 往返得以重叠,在同一部署上再带来 +1.36% 的每用户输出吞吐量。同一条缝隙上还有进行中的工作:[UMBP 中的多池 DeepSeek-V4 支持](https://github.com/sgl-project/sglang/pull/30762)、[通过 MoRI 承载的统一 KV HiSparse 状态](https://github.com/sgl-project/sglang/pull/32368),以及[在 decode 无可见内容即结束时保留 prefill 侧持有的 token](https://github.com/sgl-project/sglang/pull/34216)。HiSparse 这项工作应当被理解为长上下文的容量与正确性使能,而不是高并发下的吞吐量收益——目前它还不是。 + +
+ +## TensorRT-LLM 的 agentic 优化 + +接下来介绍 TensorRT-LLM 近期的几项优化。首先看 TRTLLM 针对重复对话轮次的前端优化,这项开销完全源自工作负载是多轮的。每一轮对话都会把整段历史再加一点新内容重新发过来,而朴素实现会把这一整段重新分词。按每 KB 计算分词很便宜,但当同样的 10 万 token 每轮都要重新分词一次时,代价就不可接受了。 + +显而易见的修法——只对新增后缀分词——其实是错的,而且很容易被忽略,因为字节对编码(BPE)并不是位置无关的。token 可能跨越拼接点发生合并,因此在边界处切分文本、再把两段 token 序列拼起来,可能得到与整段分词不同的序列,从而与前缀缓存所依据的序列悄然分叉。 + +TRTLLM 实现了[边界感知的增量分词](https://github.com/NVIDIA/TensorRT-LLM/pull/17462):先找出渲染文本的公共前缀,回退一个完整 token 以便重算任何跨拼接点的合并,然后只对变化的后缀分词。在 Qwen3.5 的 AgentX trace 上,它在全部 1,087 次转换中都与完整分词结果一致——这个正确性结论是实测而非假设——并把平均处理时间从 185.1 ms 降到 11.3 ms。 + +
+ +固定的 8k1k 请求没有可复用的上一轮渲染结果,因此这些开销根本不会出现。与之相关地,[chat template 渲染被移入输入处理线程池](https://github.com/NVIDIA/TensorRT-LLM/pull/16231),长 template 不再让主请求循环排在它后面串行执行。 + +
+ +MiniMax-M3 相关的工作聚焦在分离式 KV 搬运上,问题出在粒度。当 prefill 与 decode 对 head 布局理解不一致时,一条逻辑请求的 KV 就不再是几块大的连续区域,而变成数千个小的跨步片段,每一片都要生成自己的传输描述符。搬运的字节数没变,爆炸的是每描述符的开销,而且恰恰在最要紧的长 prompt 上爆得最厉害。[修正多池映射并引入分块 NIXL 中转路径](https://github.com/NVIDIA/TensorRT-LLM/pull/17518),通过一块有界可复用的 arena 把这些碎片合并,用一次额外的暂存拷贝换取数量级更少的描述符。其 AgentX 诊断显示:并发 5 时请求关键路径上的 KV p99 从 26.74 秒降到 125 ms,并发 40 时从 10.15 秒降到 288 ms。 + +
+ +[非阻塞的上下文传输轮询](https://github.com/NVIDIA/TensorRT-LLM/pull/17428)保护了同一条路径,即使调度停滞也会回收已完成的传输,从而打破一个反馈环:已完成的 KV block 一直被钉住,阻止新请求准入。这些停滞本身也有[可以消除的原因](https://github.com/NVIDIA/TensorRT-LLM/pull/16734):在 GB300 分离式 context worker 上,避免 DeepSeek-V4 上下文稀疏注意力元数据中的隐式 device 标量同步,消除了每步 18 次四字节 device 读取,而每一次都会强制触发 cudaStreamSynchronize。修复方式是把 host 侧计数以普通 Python int 传递,这样执行线程不再在一步的大部分时间里持有 GIL,让 KV 传输线程和响应线程排队等待。 + +TensorRT-LLM 还把不规则的长上下文工作迁到了更高效的执行路径上。[MiniMax-M3 的 context graph producer](https://github.com/NVIDIA/TensorRT-LLM/pull/17473) 捕获稳定的稀疏 producer,同时让依赖请求的注意力保持 eager 执行,在其 AgentX 测试中每用户输出吞吐量提升 12.58%。一个进行中的[原生 KV 事件生成改动](https://github.com/NVIDIA/TensorRT-LLM/pull/16876)减少了 KV 感知路由路径上的分配与转换开销。 + +AgentX 还暴露出只有在规模和时长都够大时才会出现的 kernel 选择与调度器生命周期问题。其中两个与选哪个 kernel 有关。MiniMax-M3 [为 MXFP8 自动调优加入了 CuTeDSL 候选](https://github.com/NVIDIA/TensorRT-LLM/pull/17316),扩大候选集,在低并发聚合点上把每 GPU 输出吞吐量提高约 7% 到 10%。反方向上,TensorRT-LLM 在[禁用有缺陷的 split-K MoE tactic](https://github.com/NVIDIA/TensorRT-LLM/pull/17105) 之前,七次 AgentX 运行中有五次崩溃,禁用后七次对照运行零崩溃。一个又快又错的 tactic 比单纯慢的 tactic 更糟,而自动调优器只要它还在候选池里就会兴高采烈地选中它。选择并不是 kernel 出错的唯一方式:MiniMax-M3 针对短 query 的旧稀疏注意力路径,可能把一个非连续、head-major 的 block index 视图交给 SM100 kernel,而后者按连续内存解读,于是选中错误的 KV 页,产生错误或非有限的输出。[在 q_len ≤ 32 时遵循 block index 的 stride](https://github.com/NVIDIA/TensorRT-LLM/pull/17285)修复了索引问题,既不需要物化张量,也不需要新增 kernel;此后在 GB300 上完成的五组完整 AgentX 对照运行零服务错误、无非有限值标记。 + +
+ +另外两个是生命周期 bug,这类问题是长时间运行(而非大规模运行)的典型故障。[序列槽位余量与一致的按槽位索引缓冲区尺寸](https://github.com/NVIDIA/TensorRT-LLM/pull/16279)处理的是一个瞬时重叠:一条即将完成的请求和一条新准入的请求同时需要槽位。持续有请求到达和离开的流量会不断撞上这个窗口,而固定 batch 则永远撞不上。之后的[注意力数据并行 dummy request 修复](https://github.com/NVIDIA/TensorRT-LLM/pull/17278)让九个 Qwen3.5 分离式测试格存活下来,而此前多数测试格都在几分钟内失败——这正是“能跑完基准测试的配置”与“能撑住一整个会话的配置”之间的差别。 + +两个进行中的传输改动针对超长的分离式 prompt,二者合起来展示了一个修复如何制造出下一个瓶颈。默认安排下,decode worker 必须等到整段 prompt 完成 prefill 并传输完毕才能开始,于是两个昂贵阶段串行执行,尽管第一个阶段本来就是增量出结果的。[流水线化的 KV 传输](https://github.com/NVIDIA/TensorRT-LLM/pull/15727)会在每个 prefill 分块完成时就开始发送,让传输与 prefill 计算重叠,只有最后一个分块留在关键路径上。 + +这个改动让分块处理变得频繁,于是暴露出原本只做一次的工作。它的后续改动[每次只取当前分块对应的 block ID](https://github.com/NVIDIA/TensorRT-LLM/pull/17526),而不是每次都取整段 prompt 的 block 列表。对一段 128,000 token、按 1,024 token 分块的 prompt 来说,这就是“构建一次 4,096 项列表”与“为每个层组重建 128 次”的差别。一项随 prompt 总长增长的按分块开销,其增长形态会把流水线刚换来的收益吃掉。 + +
+ +## AMD ATOM 的 agentic 优化 + +AMD 的 ATOM 引擎最初只面向单轮工作负载设计,而不是真实世界的 agentic 多轮生产工作负载,因此要让它较好地支持长上下文多轮场景,核心引擎和 kernel 都需要大量改动。相比 vLLM/SGLang 目前的水平,ATOM 在支持 agentic 工作负载上还有很长的路要走。AgentX 正被用作 ATOM 这轮重构的真实北极星目标。我们要讲的第一项 ATOM 优化,是对 DeepSeek-V4 分页滑动窗口注意力巧妙使用稀疏检查点保留。[已合入的实现](https://github.com/ROCm/ATOM/pull/1640)保留了选定的窗口尾部,让分叉和回放请求能够从有意义的边界处恢复。它的测量把两种效应分得很清楚:在并发 48 的同一条 AgentX trace 上,实际前缀命中率从 5.6% 升到 96.45%,滑动窗口关口处的损失从 91.35% 降到 0.16%。第二个数字是第一个数字背后的机制——十次前缀匹配里有九次是找到了却因为缺少窗口尾部而被丢弃,也就是说缓存并没有失配,而是被否决了。 + +
+ +在这一切能被测量之前,还有两处更早的缓存管理器修复必须先落地,它们都是“缓存自报健康、实则什么也没做”的典型案例。其一[阻止空闲池命中破坏共享缓存条目](https://github.com/ROCm/ATOM/pull/902);其二是[延迟输出的修复](https://github.com/ROCm/ATOM/pull/939),恢复了默认调度模式下的前缀哈希,让重复的长 prompt 从零缓存 token 变为复用每一个完整前缀 block。另一处改动让前缀命中的 prefill [继续走优化过的 sink attention kernel](https://github.com/ROCm/ATOM/pull/1345),而不是回退到通用路径,这样缓存命中不会悄悄吃掉它本该节省的一部分开销。 + +混合模型还带有循环状态或压缩状态,它与普通 KV 有一个决定性区别:无法从周围的 token 重建。窗口尾部可以从相邻上下文重算,但循环状态是此前全部内容累积的结果,一旦丢弃,唯一的办法就是重放整个序列。ATOM [为这类按请求的状态引入了内容寻址的检查点生命周期](https://github.com/ROCm/ATOM/pull/1771),让已生成的轮次留下可复用的恢复点,而无需为它们单独预留受保护的缓存。在一次测试中,一条请求复用了 512 个已生成 token,只计算了两个 token 的后缀。 + +调优细节和功能本身同样重要。无条件发布检查点会让零命中流量损失 17.5% 的吞吐量——这是所有一去不回的会话为帮助那些会回来的会话所付出的代价。按 token 间隔来安排检查点避免了这一惩罚,固定 1k1k 的吞吐量保持在测量噪声范围内,而这正是关键的安全性质:一个面向 agentic 复用的功能,不应该让永远用不到它的工作负载付出代价。 + +ATOM 与 AgentX 相关的 CPU 路径,要从“offload 到底值不值”的算术说起。[独立的 LMCache offload](https://github.com/ROCm/ATOM/pull/1318) 从 CPU 重新载入一段 32,000 token 的前缀约需 0.32 秒,而重算约需 2.5 秒,相差八倍。这说明在这种上下文长度下跨总线是值得的,但对短 prompt 就不成立。 + +这条路径的其余部分关乎归属与索引位置,而不是带宽。ATOM 借鉴了 vLLM 的 [multi-connector](https://github.com/ROCm/ATOM/pull/1406) 设计,让 prefill worker 可以同时把 KV 发给远端 decode worker 并把同一前缀保存到 CPU,在两个消费方都完成之前不释放这些 block;同一批 block 有两个独立读者,这种情况在单轮场景中根本不会出现。 + +[把恢复回来的 block 重新登记进 GPU 前缀索引](https://github.com/ROCm/ATOM/pull/1725)修复了一种更隐蔽的浪费:没有它,从 CPU 载入的前缀用过之后不会被登记为常驻,于是下一轮又要跨总线取一次同样的热前缀,为一份其实已经在 HBM 里的缓存反复付出传输代价。后续工作一并[修复了异步保存顺序、打包 KV 的几何结构、未对齐的交接以及远端请求记账](https://github.com/ROCm/ATOM/pull/1807),在两轮、2,638 条请求的验证中消除了重载损坏。这个 bug 只有在同一批 block 被反复保存、淘汰、恢复很多次时才会显现。 + +
+ +分布式路径的故事在另一个代码库里重演,而这个模式在 SGLang 和 Dynamo 中已经出现过:路由必须知道状态在哪里。ATOM 的 router 叫 ATOMesh,是 SGLang router 的一个删掉了大部分功能的分支。可惜的是,ATOMesh 恰恰需要 SGLang 的缓存感知路由能力,于是这项功能又不得不加回来。ATOM 随后获得了面向缓存感知 router 的 KV 生命周期事件,router 由此才能知道状态在哪里;此外还加入了多节点 prefill 与 decode 路由,以及会话粘性的数据并行路由。这个粘性策略是一个值得明说的双向折中:会话会回到持有其状态的健康 worker,但空闲的分配会过期,避免为已经离开的会话长期打破集群均衡。 + +分离式部署接下来必须搬运模型实际保留的所有状态,而这未必只有一份统一缓存。DeepSeek-V4 需要[同时传输其 FP8 与 BF16 混合缓存布局的两个缓冲区](https://github.com/ROCm/ATOM/pull/1737),EAGLE 分离式部署则要在传输目标缓存之外[搬运 draft 模型独立的 KV cache](https://github.com/ROCm/ATOM/pull/1331)——这正是 TensorRT-LLM 和 SGLang 各自都解决过的“第二份缓存”问题。[远端 KV 准入与背压](https://github.com/ROCm/ATOM/pull/1647)补上了闭环,防止 decode 侧接收超出其安全恢复能力的挂起传输,这是“接了做不完的活”在分离式场景下的形式。 + +在 ATOM 上,[PCP 报告平均 TTFT 降低 35% 到 43%](https://github.com/ROCm/ATOM/pull/1220),在 64,000 token 输入下总吞吐量最高提升约 49%——这种收益随输入长度增长,而不是随 batch 大小增长。要让它在实践中可用,就必须与会话依赖的其他一切组合起来,因此 DCP 被[做到与前缀缓存、chunked prefill 和 FP8 KV 兼容](https://github.com/ROCm/ATOM/pull/1701),随后又[扩展支持 MTP](https://github.com/ROCm/ATOM/pull/1746)。一种无法与前缀缓存共存的并行方式,只是用一项长上下文收益换掉另一项。同样的并行度匮乏也存在于单块 GPU 内部:batch 为 1 的 MLA decode 没有 head 维度或 query 维度可以铺开,只剩 KV 扫描,而硬编码为 16 的 split 预算让这次扫描只用上了 gfx950 的 256 个 CU 中的 16 个。一个[尚未合入的改动](https://github.com/ROCm/ATOM/pull/1911)不再覆盖 kernel 自身推导出的 split 数,让 Aiter 按机器实际的 cluster 数量切分扫描。 + +[分块流水线并行 prefill](https://github.com/ROCm/ATOM/pull/1552) 从内存一侧攻击同一问题,用逐层阶段的流式交接取代反复的张量并行集合通信。它在高负载下的 GLM-5.2 结果是本节中最完整的一组:输出吞吐量翻倍,首 token 延迟中位数从 28.6 秒降到 8.7 秒,每块 prefill GPU 能容纳的 KV block 数是原来的 3.68 倍。最后这个数字最值得先看,因为每块 prefill GPU 的容量决定了在部署撞上 HBM 悬崖之前,究竟能有多少条长会话同时在跑。 + +
+ +## ROCm AITER 的 agentic 优化 + +ATOM / AMD vLLM / AMD SGLang 的长上下文执行依赖底层 AITER kernel 的配套支持,因为引擎层的并行策略只有在 kernel 能表达时才算数。[Prefill 上下文并行进程组](https://github.com/ROCm/aiter/pull/3728)提供了 prefill 上下文并行所需的额外 query 分片维度,同时把融合 kernel 的行索引扩宽到支持 131,000 token 以上的 prompt。[Decode 上下文并行](https://github.com/ROCm/aiter/pull/3267)(DCP)把 KV 分片到已有的张量并行 GPU 上,使更长的序列或更大的 batch 无需在每个 rank 上复制整份缓存即可放下。 + +大容量缓存还暴露出一类短固定请求基本碰不到的故障:地址位宽。在单个缓存池跨过边界之前,32 位偏移完全够用;一旦越界,算术就会回绕,kernel 会在不报任何错误的情况下访问错误的行。AITER 为此加入了[大于 4 GB 时批量 prefill 的运行时 64 位分发](https://github.com/ROCm/aiter/pull/2893)、[大于 2 GB 时的 64 位 MLA 偏移](https://github.com/ROCm/aiter/pull/4474),以及[DeepSeek-V4 统一缓存路径全面使用 64 位寻址](https://github.com/ROCm/aiter/pull/4680),最后一项避免了在约 1.5 亿行的池中静默读写错误的行。 + +
+ +DeepSeek-V4 的 decode 还获得了[面向 64 head 与 128 head MTP 打包的持久化 MLA kernel](https://github.com/ROCm/aiter/pull/3459)。这两个 head 数正是普通解码和投机验证实际产生的形态,因此引擎在常见 shape 上有了专门的长上下文路径,而不是把它们当成为短上下文编写的 kernel 的附带变体。这与前面 vLLM AITER 稀疏 MLA 选择的道理相同:在长上下文下,通用路径不是一个小小的妥协,而是选错了 kernel。 + +## Dynamo 的 agentic 工作负载优化 + +NVIDIA 的相当一部分提交使用了 Dynamo 推理编排与 router 系统。Dynamo 的 AgentX 系列工作表明,一旦引擎 kernel 变快,分布式服务层就可能成为瓶颈。router 的工作量正比于在线前缀的数量和长度,而不是生成的 token 数,因此大量长、相互重叠、生命周期又长的会话给它带来的压力,是固定形状流量永远制造不出来的。第一组 PR 降低了每次路由决策的开销:[减少查找热路径上的工作](https://github.com/ai-dynamo/dynamo/pull/10540)、[消除冗余的后缀失效](https://github.com/ai-dynamo/dynamo/pull/10836),最后是[批量化 KV 匹配、注册、归属和终止解引用](https://github.com/ai-dynamo/dynamo/pull/11095),在并发 512 下报告输出吞吐量中位数提升 22.2%。批量化在这里奏效的原因和它在引擎里奏效的原因一样:每项开销已经盖过了项目本身。 + +
+ +第二组 PR 改变了归属关系的表示方式,这是底下更难的问题。每个被缓存的 block 都要归属到依赖它的请求,既不能在仍被使用时释放,也不能在所有人用完后继续钉住。当数千条并发会话共享相互重叠的前缀时,记账本身就成了显著开销。Dynamo 从[共享 block 链](https://github.com/ai-dynamo/dynamo/pull/11503)转向[arena 级归属计数](https://github.com/ai-dynamo/dynamo/pull/11508),最终走到[后端专属的请求租约](https://github.com/ai-dynamo/dynamo/pull/12329),每一步都在放粗被追踪的粒度。租约设计让 vLLM 后端的 AgentX 回放时间缩短 23.7%、SGLang 缩短 22.0%,同时还降低了峰值内存——这说明问题出在此前的表示方式,而不是流量本身。 + +
+ +进一步的 router profile 消除了同一形态的开销:某个周期性扫描或整体重算之所以过去可以接受,只是因为在线状态一直很小。[分桶式过期清理](https://github.com/ai-dynamo/dynamo/pull/10521)取代了正比于全部被追踪对象的扫描,让高频变动(churn)场景下的 AgentX 吞吐量提升 13.7%。[仅处理增量的后缀清理](https://github.com/ai-dynamo/dynamo/pull/10676)只处理变化的部分,在同样的时间窗口内吸收了约 28 倍的 store 与 remove 事件。[压缩 prompt 路径](https://github.com/ai-dynamo/dynamo/pull/11644)把前端 CPU 占用降低 35.3%,并显著改善了尾部首 token 延迟——这一点很重要,因为这类工作负载中的 prompt 很长且大量重复。过载状态现在也改为[增量跟踪](https://github.com/ai-dynamo/dynamo/pull/10645),而不是每次重算。 + +
+ +有一处路由改动是有意为之的取舍,而非纯粹的收益。Dynamo 现在可以[在路由打分中计入活跃的 decode 请求](https://github.com/ai-dynamo/dynamo/pull/12158),这样一个已经承担了长时间 decode 的 worker,看起来会比单看队列深度更“贵”。在所报告的调优点上,这改善了 AgentX 延迟中位数,代价是少量吞吐量——这种选择只有在请求会长时间占用 worker 时才会显现出来。一个[进行中的后续改动](https://github.com/ai-dynamo/dynamo/pull/13447)把这个取舍打包成新的 agentic router 预设,并在同一方向上走得更远:前缀重叠按 2 计分、prefill 负载按 4 缩放、活跃 decode 请求按 64 加权。在这个调优点上,这笔交易不再牺牲吞吐量:在一次 8xH200 的 AgentX 运行中,该预设相比默认代价函数把固定窗口内完成的输出吞吐量提高 8.26%,运行级 p95 首 token 延迟降低 43.1%、p95 token 间延迟降低 22.6%,并且多完成了一条完整轨迹。 + +接下来优化的是请求平面,因为一条 agentic trace 不是只发一次请求、收一次响应。它会发出许多携带几乎相同 prompt 的相关请求,并把每个 token 作为独立帧流式返回,于是序列化和拷贝的开销要按轮、按 token 反复付出,而不是只付一次。改用 [MessagePack 请求负载](https://github.com/ai-dynamo/dynamo/pull/10437)在其 AgentX 测试中把吞吐量提高 8.1%、平均首 token 延迟降低 9.7%,[直接 Python 转码](https://github.com/ai-dynamo/dynamo/pull/11104)则把这条路径上的中间值树整个去掉了。 + +随后的一系列改动都是去掉一次拷贝,而不是把某次拷贝做得更快:不再拷贝 [MessagePack 事件负载](https://github.com/ai-dynamo/dynamo/pull/11539)、不再拷贝[收到的 ZeroMQ 帧](https://github.com/ai-dynamo/dynamo/pull/11574)、不再为每个 token 支付完整的 [token 间延迟指标开销](https://github.com/ai-dynamo/dynamo/pull/11569)。[聊天流式热路径](https://github.com/ai-dynamo/dynamo/pull/10433)也因同样的理由被缩短。单看每一项都平平无奇,但乘上每条并发会话的每个流式 token,它们决定了前端每秒能撑住多少请求。 + +高并发剖析随后找出了与数据搬运无关的开销。[静态日志过滤](https://github.com/ai-dynamo/dynamo/pull/11820)移除了一个共享的 span matcher 锁——这是争用问题而非数据量问题——把前端吞吐量从 932 提升到 1,133 请求/秒。[更简单的位置 radix 分桶](https://github.com/ai-dynamo/dynamo/pull/12161)在 32 worker 的运行中把 mocker 的峰值内存降低了 5.51 GiB。一个进行中的改动改为[每条响应刷新一次去分词指标](https://github.com/ai-dynamo/dynamo/pull/12999),而不是每个流式分块都更新累计计数器,在对照诊断 profile 中把前端 CPU 时间大约减半。最后这一项是这类问题最清晰的例子:这套埋点单次调用很便宜,但按每 token 一次调用就是灾难。 + +## LMCache 的 agentic 优化 + +LMCache 是一个开源 KV cache 层,位于 vLLM 等推理引擎之下,按前缀哈希把可复用的 KV 分块存放在 CPU DRAM、本地 NVMe 和远端后端(Mooncake、Redis、S3)上。它可以替代 vLLM 原生的 offload connector。 + +
+ +LMCache 的多进程路径针对 agentic 缓存搬运的量级与形态做了改造,起点是一个不是变慢、而是直接卡死的故障。当许多上下文超过 100,000 token 的请求各自在开始前就为整次加载预留 block 时,池子会被一群全在等待、谁也无法推进的请求耗尽。[分块的外部缓存加载](https://github.com/LMCache/LMCache/pull/3382)改为按分块预留,让加载可以交错进行并逐步排空。在并发 32 下,验证完成了 120 条请求,而旧路径在 28 条时就死锁;并发 48 时在 KV 池 98.5% 占满的情况下仍能继续运行。 + +其余改动则减少搬运量以及运行时的干扰次数。只存储 DeepSeek-V4 混合分组中真正有用的部分,把[每 token 的存储量减少了近二十倍](https://github.com/LMCache/LMCache/pull/3635);滑动窗口预取现在[只加载存活窗口](https://github.com/LMCache/LMCache/pull/3869),而不是那些永远不会被读取的窗口状态——这与 vLLM 在 offload 上采用的可达性论证相同,只是换到存储一侧来看。[每个对象组一次原生传输调用](https://github.com/LMCache/LMCache/pull/3908)随后消除了暂存拷贝与 kernel 启动之间反复的 Python 锁交接,这类开销正比于分片数量而不是字节数。 + +有两项 LMCache 改动与 AgentX 尤其相关,但目前仍未合入。[混合锁记账修复](https://github.com/LMCache/LMCache/pull/4524)阻止一条请求释放另一条请求在共享滑动窗口分块或循环状态分块上的读锁。要触发它需要三个条件同时成立:多条请求共享同一批分块、记账必须按分块而非按持有者进行、并且淘汰确实开始发生。带 DRAM offload 的长时间 Kimi-K3 运行同时满足了三者,产生了数万条告警、生成内容损坏,并在淘汰开始后最终导致 GPU 崩溃。只要不是长时间、共享且内存吃紧的运行,这个 bug 就一直潜伏着。 + +另一条并行的 LMCache 工作线让上述一切在 AMD Instinct 硬件上成为可能。CacheBlend 的非前缀复用依赖仅支持 CUDA 的 flashinfer,因此[一个 Triton block-sparse 注意力后端](https://github.com/LMCache/LMCache/pull/3092)重新实现了它所需的三个 kernel:带 CSR 索引和 log-sum-exp 输出的 block-sparse 注意力、causal prefill,以及 log-sum-exp 输出混合。当检测到 ROCm 或缺少 flashinfer 时,会自动路由到这些实现。[ROCm Dockerfile](https://github.com/LMCache/LMCache/pull/3101) 对齐了 CUDA 的构建与轻量镜像。[AMD hipFile 后端](https://github.com/LMCache/LMCache/pull/3843)扩展了此前只能通过 NVIDIA cuFile 访问存储的 GDS L1 slab-file 层,通过 ctypes 绑定 ROCm 的 hipFile 并按 torch.version.hip 分发;cuFile 路径保持不变。 + +最后剩下的缺口是分发。CUDA 用户装的是预编译 wheel,AMD 用户则要从源码构建。我们与 AMD 合作发布了[预编译的 gfx942 与 gfx950 wheel](https://github.com/LMCache/LMCache/pull/4273)来补上这一块。它可以装进上游镜像,并在 MI350X 上通过全部 56 项 KV 传输 kernel 测试;它发布在 GitHub release 而不是 PyPI 上,因此直接 pip install lmcache 装到的仍是 CUDA 构建。[一行的后续修复](https://github.com/LMCache/LMCache/pull/4363)把 bind mount 进来的仓库标记为 git 安全目录,这个问题只在 CI 中出现,因为容器以 root 身份运行在 runner 所属的 checkout 上,setup.py 里的版本探测会拒绝读取它。 + +
+ +[DCP 感知的 CPU offload](https://github.com/LMCache/LMCache/pull/3561) 解决了两个在长上下文下必须同时启用的功能之间的直接冲突。启用 decode 上下文并行后,每个 rank 只持有 KV 的一段 stride,因此任何单个 rank 能保存的内容都不构成可用前缀;修复方式是在保存前汇聚这些跨步分片,加载后再重新分发。没有它,启用上下文并行会悄悄让 CPU 缓存命中失效,而失效的恰恰是催生这两项功能的长前缀。其验证记录了超过 30,000 次 CPU 命中事件,单请求加载量达到数十万 token。 + +## Mooncake 的 agentic 优化 + +Mooncake 承载着月之暗面 Kimi 的生产流量以及多家实验室的生产流量,也是分离式 vLLM 与 SGLang 配置底层的传输引擎。直到不久前,Mooncake 的 AMD 支持既没有 RDMA 注册,也没有可直接安装的软件包。 + +
+ +在 NVIDIA 上为 RDMA 注册 GPU 显存,要么用 nvidia-peermem 内核模块,要么导出 dmabuf 文件描述符。AMD 没有 nvidia-peermem 的对应物,因此 GPU-direct RDMA 完全无路可走,部署只能把 KV 经主机 DRAM 中转。[一个 HIP dmabuf 注册分支](https://github.com/kvcache-ai/Mooncake/pull/2225)加入了与现有 CUDA dmabuf 路径对应的实现,改为通过 ROCm 导出,并先解析出真实的分配基址——因为缓存型分配器会把张量打包在更大一块分配内部的某个偏移处。主机内存仍然直接注册。 + +装不上的支持等于没有支持。Mooncake 发布过 CUDA 和 MUSA 的 wheel,但没有 ROCm 包,因此 AMD 用户只能在每个镜像里从源码构建引擎。[ROCm wheel、CI 与发布流程](https://github.com/kvcache-ai/Mooncake/pull/3184)把 mooncake-transfer-engine-rocm 一并发布到 PyPI。这项工作来自 AMD 工程师 Andy Luo,他在用 AgentX 试用 agentic 工作负载时注意到一个规律:在 ROCm 上从源码构建 MoonCake 并不是一流公民的做法。 + +
+ +传输引擎没有设备 kernel,也不依赖 torch,因此一个与架构无关的 wheel 就能同时覆盖 gfx942 和 gfx950;ROCm 运行时是在加载时绑定而非打包进去的,这意味着同一个 wheel 在上游 vLLM ROCm 镜像和 SGLang ROCm 镜像中都能原样使用。这一点是按完整交叉矩阵验证过的:MI300X 和 MI355X,各自在 vllm/vllm-openai-rocm 与 lmsysorg/sglang 下运行 master 二进制以及带数据校验的 HIP buffer 传输测试。该 PR 还加入了按 tag 触发的发布流程,覆盖 Python 3.10 到 3.13。一个进行中的后续改动[增加了自建的双节点 MI350X 外部 prefill 与 decode 层](https://github.com/kvcache-ai/Mooncake/pull/3338),让 ROCm 分离式路径在真实硬件上得到验证,而不只是能编译通过。 + +综合这些 PR,AMD 上的 AgentX 运行现在可以把传输引擎和 KV cache 层作为已发布产物直接装进原生上游镜像,并在 GPU 显存与网络 fabric 之间直接搬运 KV。 + +## 其他优化 + +上面这些改动针对的是长上下文带来的开销:必须存活的前缀、必须保持正确的混合缓存、必须跟上节奏的传输。但除此之外,还有大量 day-0 使能与正确性 bug,它们让请求失败的程度不亚于百万 token 会话。 + +MiniMax-M3 检验了这些 ROCm 工作能否累积成 day-0 就绪能力,而 [Advancing AI 那篇文章](https://newsletter.semianalysis.com/p/can-amd-break-the-cuda-moat-amd-advancing)直接给出了对比:AMD 首个公开的分离式配置方案 MI355X FP4 于 1 月才进入 InferenceX,比 NVIDIA 晚了数月;而 M3 FP4 的分离式支持是 day 0 就位的。相比 DeepSeek-R1 时期要花几个月才追平,这已经是明显进步。day-0 路径上有三个 vLLM 修复,每一个都是正确性问题而非性能问题。 + +首先被卡住的是分离式部署。NixlConnector 的握手断言 SPLIT 区域的 block_len 会随 prefill 与 decode 的 TP 比例缩放,但 block_len 实际跟随的是每 rank 的 KV head 数。M3 只有 4 个 KV head,因此 TP4 prefill 搭配 TP8 decode 时,两侧都被 GQA 限制为每 rank 一个 head,两个长度相等,而断言却要求相差两倍。握手被拒,KV 没有搬运,decode 只能全部重新生成,gsm8k 得分为 0。[改为按实际 head 比例校验](https://github.com/vllm-project/vllm/pull/45879)修复了这个问题。 + +另外两个是平台差异。M3 的稀疏注意力后端在所有 E4M3 配置下都把以字节存放的 FP8 缓存读作 float8_e4m3fn,但 gfx942 的平台 dtype 是 e4m3fnuz,两种编码并不相同,于是 K 和 V 在被 kernel 消费之前就已经被改变。prefill 和 decode 的 wrapper 也没有把 FNUZ 类型纳入 FP8 检查。[改用平台 dtype 来解释缓存视图](https://github.com/vllm-project/vllm/pull/45720)同时修复了这两处。另外,M3 以 NVIDIA 和 AMD 两个独立模型文件发布,而只有 NVIDIA 那份实现了 EAGLE3 接口,因此在 ROCm 上启用投机解码会在引擎初始化时报“模型不支持”并中止。[把 AMD 模型补齐到同等实现](https://github.com/vllm-project/vllm/pull/45546)恢复了该能力,MI355X 的 gsm8k 分数与未启用 EAGLE3 的 MI355X 运行以及 B200 一致。 + +上面的 TensorRT-LLM 一节涵盖了 M3 上与长上下文相关的工作:分离式 KV 传输中的描述符爆炸、context graph 捕获、稀疏 block stride、自动调优候选,以及必须从候选池中移除的有缺陷 split-K MoE tactic。 + +本地的 AgentX 测试矩阵把会话感知或 KV 感知路由、长且长度多变的对话历史、MTP、混合注意力、聚合式与分离式服务,以及跨越 HBM 容量悬崖的并发扫描组合在一起。它既包含全 GPU 驻留的对比,也包含通过 vLLM SimpleCPU、Mooncake、LMCache 和 SGLang HiCache 的 CPU DRAM offload。正是这种组合激活了上述上游工作。旧的固定序列矩阵通常只创建一个 prompt、做一次 prefill、解码一段固定长度的续写,然后丢弃请求。因此它无法衡量跨轮次的缓存存活、重复分词、会话亲和性、缓存事件流量、offload 抖动、调度停滞期间的传输进度,以及长生命周期的归属记账。 + +允许的优化策略把 CPU KV offload 视为可选项。厂商可以使用 vLLM connector、LMCache、SGLang HiCache、Mooncake、Dynamo KVBM 或其他 CPU DRAM connector,也可以在关闭 offload 后延迟与吞吐更优时选择关闭。NVMe offload 暂缓。CPU DRAM 必须按所用 GPU 比例配置,其中非标准化 DRAM 系统适用 3 TB 上限;标准化 DRAM 系统没有硬性上限,但同样遵循按比例配置的规则。目前本地生成器对所有 runner 一律应用 3 TB 上限,也就是说尚未实现标准化 DRAM 的例外规则。 + +新增的优化面并不只是“注意力更长了”。它是一份不断增长的会话状态在保留、搬运、路由、重建和反复处理上的开销。AgentX 把这些开销放大到足以驱动 vLLM、SGLang、TensorRT-LLM、ATOM、AITER、Dynamo 和 LMCache 上的通用上游改动。我们直接检索 NIXL 和 Mooncake 未发现更多标记为 AgentX 的运行时 PR,因此它们的相关效果仍然通过上述引擎 connector 改动体现。 + +## AgentX 测试方法深入解析 + +AgentX 在开源的真实世界长上下文多轮 agentic trace 回放上是一次重大转变。我们采集了价值超过 300 万美元 token 的 trace,构成自有数据集,内容来自 Claude Code、OpenAI Codex 等真实流量。除数据集外,我们还建立了一套公平回放这些流量模式的完整方法,目标是在尽可能忠实于真实流量的同时,对 GPU 资源需求保持公平。 + +下面我们会深入介绍 agentic trace 数据集、回放方法以及 agentic 行为本身。希望更好理解 agentic 工作负载整体形态、以及各类 harness 在底层如何编排请求的读者,值得读一读这部分。 + +## 价值 300 万美元的 agentic 流量 trace 采集器 + +在最初设计 AgentX 时,我们的北极星目标是让基准测试在 KV 工作负载形态和 KV 复用模式上尽可能真实。我们先尝试回放了一些现有数据集,例如 SWE-bench、Qwen-Bailian 以及 HuggingFace 上的一些 Claude Code trace。当时这些数据集都没有大量使用子智能体、1M 上下文、上下文压缩、动态工作流等近来定义 agentic trace 的特征。在 SemiAnalysis,团队里大多数人都是 AI 重度用户,会用 agent 做各种事情:写代码、分析师研究、Excel 建模、社交媒体运营等等。因此我们判断,最可行也最真实的 trace 只能自己采集。 + +为了采集足够多的 trace,我们做了一个代理来拦截发往 Claude / Codex 的 HTTP 请求。愿意上传 trace 的用户只需把 Claude / Codex 配置中的 base URL 指向这个代理即可。截至撰稿时,我们已采集超过 8,000 个会话、340 万条请求、6,100 亿 token,合计支出超过 300 万美元。[我们为 AgentX v1.0 基准测试开源了其中具有代表性的一个子集。](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126) + +agentic harness 看起来复杂,但归根结底就是在编排一串 HTTP 请求。每条请求都包含系统指令、工具定义和累积对话历史的某种组合。随着会话推进,这段历史不断增长并被反复发回模型,从而形成 AgentX 想要复现的长上下文与高前缀复用。 + +我们的代理会实时记录这些请求和响应,同时提取时间戳、conversation ID、subagent ID 等元数据 / HTTP 头,从而还原对话的*结构*(请求顺序、并发分支,以及每个会话大致的父子关系)。正是这些元数据,让我们能够按照原始 Anthropic API 服务端所看到的样子近似回放这些 trace。 + +为保护员工隐私,回放数据集不包含任何原始 prompt、源代码、工具参数或工具返回结果。我们的做法是先对每条请求的内容分词,再按 64 个 token 分块,最后把每一块替换成一个会话内链式哈希。这样,相同的 prompt 前缀会产生相同的哈希前缀,同时不暴露内容([这篇论文](https://arxiv.org/abs/2506.02634)对该策略有更多讨论)。回放时,这些哈希块可以替换成来自比如某个编码数据集的 token。由此我们保留了原始工作负载中大致的上下文增长曲线和会话 KV 复用模式。 + +需要说明的是,这个过程必然不完美,主要原因是使用前沿模型厂商 API 时,末端 LLM 服务器实际看到的内容有很大一部分是隐藏的。例如 SOTA 模型的思考 / 推理内容如今在 HTTP 请求中是加密的,并被替换成确定性哈希,以阻止蒸馏攻击。不过看起来[这招的效果并不像大实验室预期的那么好……](https://arxiv.org/html/2608.09867v1) + +
+ +此外,虽然我们能拿到用户 / 助手消息、系统提示词、工具使用等全部原始*内容*,但 API 厂商还会在服务端应用额外的、不透明的 chat template。我们也无法观察 Anthropic 的专有分词器,或服务端工具引入的上下文。图片和文档在传输表示与模型实际处理的 token 数之间也没有直接对应关系。我们使用确定性占位符以及按模型经验校准的填充,把重建后的 prompt 长度调整到最接近服务端实际看到的内容。 + +由于信息不完整,我们无法*完全*按照 Anthropic / OpenAI 服务端*实际看到*的样子采集并回放 Claude Code / Codex 的 trace,但可以做到相当接近。下图展示了在所有请求长度和模型上,哈希 token 数(经我们近似 / 处理之后)与 API 厂商真实 token 计数的比值。 + +
+ +总结来说,由于信息不完整,要完全按原始服务端所见的方式采集并回放真实 Claude Code trace 并不容易。但我们掌握的信息足以采集并回放出与原始流量模式、时序、前缀缓存和 DAG 结构高度吻合的 trace。 + +## AgentX 的 300 万美元数据集 + +AgentX v1.0 使用的数据集可在 [HuggingFace](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126) 获取。它是上一节所述 8.3k 会话代理语料中的 393 会话子集。此外我们还做了一些后处理来清理异常,例如: + +- 移除 Claude Code 安全监控(auto mode)请求和标题生成请求,因为它们是 Claude Code 特有的,不一定能代表通用 agentic 流量 +- 移除重建后输入长度超过 990k token 的请求(这些是我们的近似方法高估了) +- 去除重复请求(连接断开时,代理有时会收到完全相同的请求) + +此外,每个对话都被整理成 WEKA trace 格式,该格式由 Callan Fox 在其 [kv-cache-tester](https://github.com/callanjfox/kv-cache-tester) 项目中提出。我们选择这个格式,主要是因为我们与 Callan 密切合作开发了这套基准测试,觉得它用来存放按会话组织的 trace 很直观。总体而言 trace 格式的选择相当随意,我们的代理数据集也可以映射到其他格式,例如 [Mooncake](https://docs.nvidia.com/aiperf/benchmark-modes/trace-replay-with-mooncake-traces)。 + +
+ +应用这些处理后,得到如下数据集。注意各子图的 X 轴并不一致。 + +
+ +ISL/OSL 和轮次间延迟(在 agentic 工作中,这主要是工具调用耗时)的分布大致呈对数正态。ISL 中位数为 142k token,OSL 中位数为 444 token,轮次间延迟中位数为 3.84 秒,只有约 10% 的轮次间延迟超过 1 分钟,这部分很可能是 harness 在等待真人回应的空档。 + +值得一提的是,这些请求分布会随所用 harness 而不同,因为各家注入的上下文数量 / 类型不一样(例如 Pi 以 harness 注入上下文极少著称,Claude Code 则相反)。此外 ISL/OSL 分布还取决于模型,不同模型的分词器产出的 token 数会更多或更少。不过考虑到全球相当大一部分 agentic 编码流量都经过 Claude Code,我们认为这份数据具有代表性。 + +数据集中有 175 个会话至少包含一个子智能体(约占全部会话的 44%),共有 1,697 次子智能体 rollout,每个会话中位数为 4 次。子智能体的墙钟时长中位数(从第一条请求开始到最后一条请求结束)为 2.27 分钟,这一分布同样接近对数正态。 + +
+ +该数据集包含最长 1M 的上下文,用于测试较新的前沿开放权重模型。此外我们还提供[一个截断到 256k 上下文长度的数据集](https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126-256k),用于回放最大上下文长度不超过 256k 的模型。 + +## agentic 流量 trace 回放器 + +我们没有从零造回放方案,而是选择与 [AIPerf](https://github.com/ai-dynamo/aiperf) 合作。它是 NVIDIA 出品、与厂商无关的 HTTP 回放工具,已被 tenstorrent、AWS、AMD 等众多机构采用。虽然我们的目标是把 AgentX 的功能合入上游仓库,但仍维护一个独立的 [fork](https://github.com/SemiAnalysisAI/aiperf) 以保持更强的厂商中立性,从而能自主接纳更多第三方贡献。再次感谢 AIPerf 团队,特别是 Anthony Casagrande,为构建真实且有代表性的 agentic 基准测试所付出的努力。 + +一个 agentic 会话天然可以用有向无环图(DAG)描述。每条请求是一个节点,一条边表示:边头部的请求必须等边尾部的请求完成后才能发出。每条边还带有一个延迟,表示前置条件满足后需要再等多久。 + +最简单的会话是完全线性的:没有子智能体、没有并行请求,每条请求恰好依赖一个前驱。此时图退化成一条线,边所编码的只有轮次间延迟(也就是工具调用时间或“思考”时间),那是客户端本地的工作,不是模型的耗时。 + +
+ +在 agentic trace 中还会派生出子智能体。子智能体是一条拥有独立上下文的请求流,通常用来完成某个聚焦任务。多个子智能体可以并行执行以完成更多工作,某些情况下子智能体也可以与主智能体并行。主智能体随后等待一组子智能体完成,再把它们的输出并入自己的上下文(虽然并非*总是*如此,但这是最常见的模式)。 + +正是这种行为把上面那条线性请求链变成了 DAG,其中某些请求依赖于另一些请求。当识别出属于同一个子智能体的一组请求时,AIPerf 会找到最近的一条主智能体前驱请求,把它定为“派生”请求;类似地,“汇合”请求则是该子智能体组持续时间结束之后的下一条主智能体请求。 + +在下面的例子中,子智能体组只包含一个子智能体(001),它发出两条请求。第一条请求在主智能体的首条请求完成后立即发出;该请求完成后,2.2 秒的轮次间延迟代表工具调用的墙钟时间,随后发出子智能体的第二条请求。主智能体的第二条请求是子智能体 001 的汇合点,它在两个条件同时满足后才发出:距主智能体首次响应已过去至少 17 秒,*并且*子智能体 001 的第二条请求已经完成。 + +一个小的局限是:HTTP 时间戳能反映时序,却未必能反映因果。在下面的例子中,如果子智能体 001 在主智能体请求 2 的记录起始时间之前 7 秒就结束了,我们无法判断这 7 秒是子智能体返回之后才开始的工作,还是原本就在并行进行的独立工作。因此 AIPerf 同时保留两个约束:请求 2 既要等待它记录的主路径延迟,也要等待子智能体 001 完成。这样能复现观察到的时序和工作负载拓扑,但无法复现 harness 内部隐藏的依赖关系。这些是我们希望在后续版本的 AgentX 中改进的地方。 + +
+ +一条请求也可以同时派生出多个子智能体。在下面的例子中,子智能体 001 和 002 的派生父节点都是主智能体请求 1,并在主智能体请求 2 处汇合。 + +AIPerf 还能识别“辅助(auxiliary)”请求,即与流中其他请求不共享上下文的一次性请求。它们从主智能体分叉出去,并且不会汇合回来。实际场景中,这类请求例如 Claude Code 的“总结本次会话”请求,与对话上下文无关;另一个很好的例子是 Claude Code 的 `/btw` 功能。 + +
+ +下面的例子把这些要素放在一起,也是实际数据集中会看到的 trace 片段类型。图中有五条并行流从同一条主智能体请求出发,按各自汇合到哪条主智能体请求被分成不同组。 + +子智能体 001 和 002 分别在第 20 秒和第 23 秒结束,因此其后第一条在第 25 秒发起的主智能体请求就是它们的汇合点。子智能体 003 和 004 在同一个间隙中派生,但运行时间长得多,分别在第 46 秒和第 50 秒结束,因此它们汇合在第 52 秒。AIPerf 用(派生请求,汇合请求)这一组合作为子智能体的键,因此这四条流虽然都从同一节点出发,却被归并为两条分支。分支以其第一个成员命名,这也是汇合边被标为子智能体 001 和子智能体 003 的原因。 + +这也是第一次出现主智能体与自己的子智能体重叠的情况:第 25 秒发出的请求,是在 003 和 004 仍在运行时发出的——主智能体只被与它汇合的那一组阻塞,而不是被所有在途的子智能体阻塞。 + +辅助链会挂在它之前最近的那条主智能体请求上,这里是第 52 秒那条,而不是开启会话的那条。因此该节点同时做了两件事:接收第二组子智能体的汇合,并派生这条一次性请求。当然,辅助请求不会再汇合回来。 + +
+ +## AgentX 进一步解析 + +### Pareto 前沿扫描 + +在 AgentX 工作负载中,为了生成 Pareto 前沿,我们针对单套部署扫描并发 Claude Code 会话数。由于每个对话都带有真实的轮次间延迟和子智能体使用,得到的流量模式更尖锐、也更真实。下面这个例子是对运行 MiniMax M3 的 B200 TP4 vLLM 服务端回放 40 个并发客户端。 + +
+ +
+ +最后值得讨论的是:评估 agentic 工作负载时应该看哪些指标。我们认为交互性(TPS,每秒 token 数)和首 token 延迟(TTFT)依然重要,它们也是评估 SLO 的行业标准。查看 AgentX 结果时,*必须*把 TPS 和 TTFT 放在一起看,因为有些推理优化会以牺牲一者为代价改善另一者。 + +我们正在定义一个能把 TPS 和 TTFT 有意义地结合起来的新指标。它还应当体现这样一个事实:在 agentic 工作负载中,人们往往更关心一个*任务*端到端多快完成,而不是收到 token 的速度或 TTFT 本身。 + +最后要说明的是,端到端延迟在当前形式下参考价值有所下降,因为它与 OSL 直接成正比。因此 P90 端到端延迟会被输出序列长度最长的那 10% 尾部严重影响。它仍可用于整体比较不同配置的综合表现,但我们更建议结合 TPS 和 TTFT 一起看。 + +### warmup、计时、确定性与对话复用 + +AgentX 的目标是测量已经进入稳态的系统。对 agentic 工作负载而言,这意味着性能采集应当从部分上下文轨迹已经被缓存的状态开始。为模拟稳态,也不希望所有对话都从第 0 轮开始,否则会造成“[惊群](https://en.wikipedia.org/wiki/Thundering_herd_problem)”效应。 + +warmup 分两个阶段。第一阶段,AIPerf 用固定的随机 seed 在每个对话 25% 到 75% 之间选取一个墙钟时间点。在该时间点上,它识别出所有活跃的请求流(包括主智能体和任何活跃的子智能体),并为每条流发送该时间点之前最近的一条请求。这些引导请求会重建该时刻的对话状态,并一起发出,AIPerf 会等它们全部完成后再继续。 + +
+ +第二阶段,每条回放通道再前进 10 条请求,让 KV cache 有更多机会成形。所有 warmup 请求都不带轮次间延迟,且最大输出长度为 1 个 token,从而大幅缩短 warmup 时间。 + +
+ +
+ +warmup 结束后开始性能采集,持续一小时,所有指标都严格在这段时间内统计。为保证可复现,AIPerf 接受一个 seed,确保每次运行采样的对话相同、对话起始点相同,且每个对话每次都用相同的合成内容重建。采集期间,我们对每条流设置 5 分钟的空闲上限,避免过长的轮次间空档在整个测试期间“占住”一条 worker 通道。这样做是为了能在 1 小时内完成测试。在后续加入 NVMe offload 的 AgentX 版本中,我们可能会调高这个上限,以便测量更长的 TTL。就目前而言,5 分钟是合理的,因为这也是 [Anthropic 默认的 KV cache TTL]()。 + +这种程度的确定性保证了在推理引擎、硬件、并发和服务端设置相同的情况下,运行结果是可复现的。不过由于 AgentX 是[闭环基准测试](https://notpeerreviewed.com/blog/tail-latency/),不同配置在不同速率下完成的请求数不同,因而各自遇到的工作负载会有一些自然差异。这在低并发下最明显:完成的请求本来就少,工作负载向数据集整体分布收敛的机会也更少。 + +当一个对话在采集期间结束时,它所在的回放通道会从数据集采样器中选取另一个对话。每次回放都会获得一个唯一且确定的 cache-bust 标记,并加在每条独立前缀链的最前面,包括主智能体链,以及任何全新上下文的子智能体链或一次性链。分叉出来的子智能体继承父级的标记。这个标记在一次回放内保持不变,从而保留其 KV 复用模式,但在不同回放之间会变化,以避免出现虚高的缓存命中率。这也使得并发数可以超过数据集中的对话总数(393)。 + +### AgentX 的公平投机解码方法 + +如前所述,数据集在采集时已做匿名化,因此那些 64 token 的哈希块必须在回放前用合成内容填充。AIPerf 的做法是从一个合成的编码 / 工具调用 token 池中确定性地采样。 + +重要的是,KV 复用模式和请求时序都被保留了下来,但合成的请求内容确实带来了一些额外考量:在合成数据上运行投机解码方法,可能导致 speculator 接受 / 拒绝的 token 数与真实数据下明显不同(因为 speculator 并未在合成数据上训练过)。 + +我们在 [InferenceX v2 文章](https://newsletter.semianalysis.com/i/188090866/multi-token-prediction-mtp)中讨论过这个不足,此后改进了方法。我们与社区密切合作,确保大多数开源推理引擎都提供一种机制,让用户可以强制指定从 speculator 接受多少 draft token(即“接受长度”或“接受率”)。随后,对每一种(模型、speculator、draft 长度、思考模式)组合,[我们在 SPEED-Bench 上采集平均 AL](https://github.com/SemiAnalysisAI/InferenceX/tree/main/golden_al_distribution)。[SPEED-Bench](https://huggingface.co/datasets/nvidia/SPEED-Bench) 是一个 agentic 编码数据集,被描述为“用于在多样语义领域和真实服务条件下评估投机解码(SD)的统一基准”。 + +然后在运行时,我们把这些真实的投机解码接受长度应用到 AgentX 上,以确保厂商中立的公平性。 + +
+ +## agentic 工作负载详细遥测数据探索教程 + +AgentX 需要的不只是新的基准测试 harness 和数据集。[我们还花了一些时间重建 InferenceX 可视化的部分模块,让 agentic 结果更易于探索和理解。](https://inferencex.semianalysis.com/inference/agentic/439903)一个 AgentX 数据点代表着数千条请求,跨越不断增长的对话、子智能体、warmup 阶段、缓存状态和动态变化的在途负载。因此,Pareto 曲线上的单个点会掩盖大量有用信息。正如我们反复强调的:推理服务从来没有一刀切的方案。 + +
+ +其中一项重要改动是曲线本身的构建方式。在之前版本的 InferenceX 中,启用和未启用投机解码的配置常常被画成两条独立曲线;现在我们不再这样做。前端会合并允许启用的推理优化,为每个模型、SKU 和推理引擎组合展示可获得的最佳曲线。因此,同一条曲线上的不同点可能使用了不同的优化技术和配置,包括投机解码、分离式服务或 KV cache offload。 + +我们的目标是展示每套软硬件栈在生产中能达到的最佳性能,而不是为每种优化组合都单独画一条曲线。不过,我们仍会公开每个点的底层配置与溯源信息。点击某个点会弹出详细信息,显示究竟是哪套配置产生了它,以及运行元数据、可公开查看的 CI 溯源链接和 AgentX 专属统计。从那里点击“View charts”可以打开完整的点详情页,查看 AgentX 专属统计。 + +
+ +点详情页对所选 AgentX 运行提供了深入得多的视角,包含输入和输出序列长度分布、交互性与 TTFT 随时间的变化、KV cache 利用率、请求队列深度、前缀缓存命中率、输入与 decode 吞吐量、prompt token 来源拆分,以及唯一输入 token 随时间的变化。有了这些指标,就更容易理解为什么两个聚合吞吐量相近的点在整个回放过程中表现却不同。 + +该页面还把 warmup 与采集阶段的数据分开。读者可以在两个阶段之间切换,观察系统在缓存状态建立过程中的行为,以及在用于基准测试的采集期内的行为。 + +
+ +使用 KV cache offload 的点在主图上会多一圈虚线,用来与其他点区分。选中这类点时,详情页会显示 offload 类型、KV offload 引擎、芯片缓存命中率和 CPU 缓存命中率。这样就能看出 KV offload 在最佳曲线中的贡献,而不必为每种 offload 配置单独画一条曲线。 + +另一个新功能是请求时间线。该视图展示所选 AgentX 运行中回放的各条请求,可以按对话或按 worker 组织。对话视图会把子智能体归到其根对话之下,便于看清对话与子智能体何时重叠。warmup 与采集期的请求同样可以分开查看。 + +
+ +时间线中的每条请求都可点击,直接跳转到 InferenceX 数据集页面上对应的对话和轮次。这样,读者可以从 Pareto 曲线上的一个聚合点,一路追到实际被回放的那条匿名化请求。 + +[AgentX 页面](https://inferencex.semianalysis.com/datasets)还提供了火焰图,用于可视化单个对话的结构。每个条形代表一轮,长度相对该对话中最长的一轮缩放,并被拆分为已缓存前缀 token、未缓存输入 token 和生成的输出 token。这直观展示了上下文如何在对话中增长,以及每条请求中有多少可以从 KV cache 复用。 + +
+ +## InferenceX/AgentX 的下一步 + +我们会继续朝着让 AgentX 成为最真实、最有代表性的基准测试这一目标前进。在可预见的一段时间内,我们会继续为当前的 v1.0.x harness [做小修复](https://github.com/SemiAnalysisAI/aiperf/releases),同时计划在 v1.1 harness 上做更大的改动。同一模型的所有提交将始终运行同一个次版本,以保证结果可比。 + +作为快速跟进,我们会加入 SSD/NVMe KV offload。这将支持比 DRAM 更大的 KV cache 工作集,从而打开 Pareto 曲线左侧的高吞吐区间。 + +我们还会采集规模更大、更多样、更新的 agentic trace 数据集,覆盖更多模型和 harness。下一版数据集不再把每条请求表示为一整串连续的哈希 ID,而会保留系统指令、用户与助手消息、工具调用和工具返回结果之间的边界。这样 AgentX 就能评估那些利用 agent harness 已知、但通常对推理引擎不可见的信息的工作负载感知服务技术。例如,router 可以把低复用的工具流量导向专用 prefill worker、在长时间工具调用期间保留 agent 的前缀,或在子智能体分叉时预取并共享前缀。当前格式保留了请求规模和 KV 复用模式,但缺少评估这些优化所需的结构信息。 + +Ishan 提交的这份 [SGLang RFC](https://github.com/sgl-project/sglang/issues/27574) 就是说明这种更丰富 trace 结构为何重要的一个具体例子。它提出了一套由 router 发起的提示接口,利用会话生命周期、共享前缀边界、工具调用时长和子智能体状态等信息,告诉引擎何时应当共享、预取、降级、钉住或保留 KV。捕获这些结构信息,将使未来版本的 AgentX 能够评估这类工作负载感知的缓存策略,而不只是回放扁平的 token 前缀。 + +
+ +最后,我们正在采集细粒度和粗粒度的功耗遥测数据,以便更准确地衡量不同软硬件栈“每单位智能所耗焦耳”的效率。 + +我们手上有大量数据,也有非常多可能的可视化方式。欢迎告诉我们你希望看到哪些可视化,以及更广泛的功能需求! + +[提交前端功能建议!](https://github.com/SemiAnalysisAI/InferenceX-app/issues/new/choose) + +## 模型全生命周期中的性能表现 + +接下来几节我们转向历史单轮数据(8k1k),它覆盖每个模型从发布当天到我们停止对其持续测试为止的表现。这里 NVIDIA 和 AMD 都有亮眼结果,包括若干 MI355X 胜出的固定序列长度配置。 + +AgentX 更能代表当下的 agentic 推理工作负载。不过,历史上的固定序列 InferenceX 结果对追踪性能随时间的变化仍然有用。8k1k、1k1k 这类工作负载剥离了大部分会话级行为,包括前缀复用、持久 KV cache 状态和路由亲和性,因此它们对当前生产流量的代表性较弱,但仍然适合用来观察推理性能如何随软件支持的成熟而提升。 + + + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. + + + +--- + +_本文完整版发布在我们的 Substack 上。[订阅 SemiAnalysis](https://newsletter.semianalysis.com/subscribe) 阅读完整文章。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "什么是 AgentX 1.0?", + "acceptedAnswer": { + "@type": "Answer", + "text": "AgentX 1.0 是全球首个完全开源、1M 上下文的多轮 agentic 编码推理基准测试,以 Apache 2.0 协议发布。它回放真实的匿名化 Claude Code 与 Codex trace,而不是固定序列长度的 prompt;测试运行在约 2MW 的持续运行算力上,覆盖 1000 多颗芯片,包括 MI355X、GB300 NVL72、GB200 NVL72、B300、B200、MI325X、MI300X、H200 和 RTX Pro 服务器。" + } + }, + { + "@type": "Question", + "name": "agentic 工作负载与固定序列长度基准测试有什么区别?", + "acceptedAnswer": { + "@type": "Answer", + "text": "agentic 工作负载是多轮、长上下文的,前缀复用率极高,并会突发启动短生命周期的子智能体。这让 agentic 推理成为系统问题而非 kernel 问题:KV 张量必须跨节点传输,请求必须路由到持有对应前缀的 rank,长上下文还迫使 KV offload 到 DRAM 或 SSD。8k1k 这类固定序列长度工作负载把这些全部剥离,衡量的基本是芯片和 kernel 的基础性能。" + } + }, + { + "@type": "Question", + "name": "CUDA 护城河在 agentic 工作负载上还站得住吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "取决于模型。在 DeepSeek V4 Pro 上竞争非常胶着:MI355X SGLang 在每美元性能上一度追平 B200 vLLM,直到 2026 年 8 月 21 日 Inferact 和 NVIDIA 在 vLLM 上的优化让 B200 反超,而 B300 vLLM 和 B200 SGLang 此前已经领先 MI355X。在 MiniMax M3、Qwen3.5 和 GLM 5.3 上,NVIDIA 大幅领先:Qwen3.5 SGLang 在 90 tok/s/user 下领先 20 倍以上,GLM 5.3 在 150 tok/s/user 下成本效率最高领先 5 倍。上下文并行是护城河中很具体的一环:在 vLLM 的注意力后端支持矩阵中,AMD 的每一个后端都标为不支持。" + } + }, + { + "@type": "Question", + "name": "AgentX 数据集包含什么?", + "acceptedAnswer": { + "@type": "Answer", + "text": "SemiAnalysis 通过拦截发往 Claude 和 Codex 的 HTTP 请求的代理采集 trace,累计超过 8,000 个会话、340 万条请求和 6,100 亿 token,合计支出超过 300 万美元。开源的 AgentX v1.0 子集包含 393 个会话,输入序列长度中位数 142k token,输出序列长度中位数 444 token,轮次间延迟中位数 3.84 秒;其中 175 个会话(约 44%)至少包含一个子智能体,共 1,697 次子智能体 rollout。内容被匿名化为 64 token 分块并替换为会话内链式哈希,因此前缀复用模式得以保留,而原始 prompt 不会保留。" + } + }, + { + "@type": "Question", + "name": "什么是 E2E 归一化交互性?", + "acceptedAnswer": { + "@type": "Answer", + "text": "E2E 归一化交互性是一个实验性指标,定义为输出序列长度除以端到端延迟。由于端到端延迟等于 TTFT 加上输出长度乘以 TPOT,该指标实际等于交互性加上一项与 TTFT 成正比的惩罚,用一个数字同时刻画两个维度的响应性。它并不完美:对高 TTFT 的惩罚很重,也无法体现 prefill-decode 分离等优化的全部细节,因此 AgentX v1.0 的提交仍然分别针对交互性和 TTFT 做优化。" + } + }, + { + "@type": "Question", + "name": "AgentX 目前带来了哪些行业影响?", + "acceptedAnswer": { + "@type": "Answer", + "text": "vLLM、SGLang、TensorRT-LLM、ATOM、AITER、Dynamo、LMCache 和 Mooncake 上已有 70 多个上游 PR 以 AgentX 为北极星基准。例如:vLLM 的混合注意力前缀缓存在 1M 上下文下前缀缓存命中率超过 95%;CPU KV offload 扩展到 DeepSeek V4,输出吞吐量提高 81.7%、平均端到端延迟降低 46.6%;TensorRT-LLM 的边界感知增量分词在 Qwen3.5 上把平均处理时间从 185.1 ms 降到 11.3 ms;AMD ATOM 的稀疏检查点保留把 DeepSeek V4 的前缀命中率从 5.6% 提升到 96.45%。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/brief-overview-of-agentic-workloads.mdx b/packages/app/content/blog/zh/brief-overview-of-agentic-workloads.mdx new file mode 100644 index 000000000..91987ef6d --- /dev/null +++ b/packages/app/content/blog/zh/brief-overview-of-agentic-workloads.mdx @@ -0,0 +1,134 @@ +--- +title: 'Agentic 工作负载简述' +subtitle: '多轮会话、长上下文与近乎完全的 prefix 复用,让 agentic 推理成为一个系统问题,也改变了基准测试需要测量的内容' +seoDescription: 'Agentic 工作负载的四个特征:多轮会话、长上下文、高 prefix 复用与突发式 subagent 扇出,以及为什么对其做基准测试是一个系统问题。' +date: '2026-08-19' +publishDate: '2026-08-19' +tags: + - agentic + - agents + - agentx + - benchmark + - inference +--- + +概括来说,[agentic 工作负载](/zh/glossary/agentic-coding-workload)有四个特征: + +- **多轮。** 一次会话包含几十到几百次用户与助手交互,而 chatbot 场景通常只有寥寥数轮。 +- **长上下文。** system prompt、tool 定义,再加上轮次本身的数量,上下文会迅速累积。 +- **高 [prefix 复用](/zh/glossary/prefix-caching)。** 对话呈线性推进,第 _n-1_ 轮的输出会拼接进第 _n_ 轮的输入,因此大部分上下文可以直接由 [KV cache](/zh/glossary/kv-cache) 提供,而无需重新计算——具体比例取决于可用于存放 KV tensor 的存储容量。随着 _n_ 增大,命中 cache 的输入与未命中输入之比会趋近于 1。 +- **突发式 [subagent](/zh/glossary/subagent) 扇出。** 一轮可能同时启动多个短生命周期的 subagent,由此产生尖峰式的 KV cache 分配与淘汰模式。 + +
+ +## Agentic 推理是一个系统问题 + +考虑到上述特征,对这类工作负载做基准测试,与现有的固定序列长度基准测试有本质区别。 + +由于 prefix 复用比例极高,KV tensor 必须在节点与 rank 之间高效传输,这正是 NIXL、MORI-IO 和 [Mooncake](/zh/agentx/optimizations/mooncake) 这类传输层要解决的问题。随后还需要把不同对话路由到其 prefix 所在的节点或 rank,以最大化 cache 命中率,llm-d、[Dynamo](/zh/agentx/optimizations/dynamo) 以及 vLLM 与 SGLang 的 router 承担的就是这项工作。长上下文对话会给 KV cache 的 HBM 容量带来压力,迫使 KV tensor offload 到 DRAM、SSD 等更慢的存储层级,而这一过程同样需要高效实现——Mooncake Store、[LMCache](/zh/agentx/optimizations/lmcache)、vLLM 的 CPU offloading 与 SGLang HiCache 都在攻克这一层。 + +与之相对的是固定序列长度的单轮工作负载:prefix 复用在其中并不相关,推理性能主要反映芯片与 kernel 的基础表现。这并不意味着 InferenceX 上大量的固定序列长度数据不重要。剥离 agentic 推理服务的复杂性之后,这些数据能清晰呈现底层推理优化的进展,也构成了解读 AgentX 结果时所依据的基线。 + +## 构建真实的回放语料 + +为了让 AgentX 的工作负载尽可能贴近真实场景,我们收集了 393 条 SemiAnalysis 内部的 Claude Code trace 用于回放。为在匿名化内容的同时保留原始的 prefix 复用模式,我们采用了与 [Qwen-Bailian](https://github.com/alibaba-edu/qwen-bailian-usagetraces-anon) 类似的方法——后者是最早公开的生产环境 trace 语料之一。随后我们使用 [AIPerf](https://github.com/ai-dynamo/aiperf) 按照原始的请求时间表,在不同并发客户端数量下重建这些 trace。 + +重建 subagent 结构需要 Claude Code 本身做出改动。我们与 Anthropic 合作推动了两项功能落地,它们都为出站请求补充了 agent 实例身份信息,使并行分支能在 proxy 层被区分开: + +- [anthropics/claude-code #49207](https://github.com/anthropics/claude-code/issues/49207) —— 在 API 请求 header 中加入 agent 实例 ID(`x-claude-code-agent-id`)及其父级 ID(`x-claude-code-parent-agent-id`),让交错的 subagent 请求可被归因,而不是坍缩成单一的会话级流量。 +- [anthropics/claude-code #66761](https://github.com/anthropics/claude-code/issues/66761) —— 把同样的 header 扩展到由 `Workflow` tool 扇出产生的 subagent,此前这些请求只携带父级对话的 session ID。 + +缺少这些 header,一个扇出 N 个并发 subagent 的会话就与 N 个互不相关的对话无法区分,塑造[回放图](/zh/glossary/trace-replay)的 spawn 与 join 依赖关系也就无从还原。 + +完整的数据集规则、分布图与来源链接发布在 [AgentX 方法论](/zh/agentx/methodology)页面;该工作负载已经暴露出的具体引擎改进,则记录在 [AgentX 优化追踪页](/zh/agentx/optimizations)。 + +## 致谢 + +感谢 Anthropic 团队在 Claude Code 中落地 agent 身份标识 header,也感谢 Qwen-Bailian 与 AIPerf 背后的阿里巴巴与 NVIDIA 团队——AgentX 的回放流水线正建立在他们的工作之上。 + +对比实时 AgentX 推理结果 → + +## 常见问题 + +### 什么是 agentic 工作负载? + +Agentic 工作负载指由 AI agent 而非真人在对话框中输入所产生的推理流量。它的特征是几十到几百轮交互的多轮会话、由 system prompt 与 tool 定义堆叠出的长上下文、跨轮次极高的 prefix 复用,以及成批出现的短生命周期 subagent。 + +### 为什么 prefix 复用对 agentic 推理如此关键? + +每一轮的输入都包含上一轮的输入加上其输出,因此绝大部分上下文此前已经处理过。只要该 prefix 的 KV tensor 仍然驻留,prefill 工作就可以跳过。随着会话增长,命中 cache 的输入与未命中输入之比会趋近于 1,于是 cache 容量与命中率主导了推理服务的成本。 + +### 对 agentic 工作负载做基准测试,与固定序列长度基准测试有何不同? + +固定序列长度基准测试发送 input 与 output 长度固定的独立请求,prefix 复用不构成变量,结果主要反映芯片与 kernel 性能。Agentic 基准测试回放的是彼此依赖的会话,其中上下文不断增长、prefix 被共享、subagent 会扇出,因此 KV 传输、prefix 感知路由与 cache 分层 offload 都成为被测量的对象。 + +### 突发式 subagent 扇出会对 KV cache 造成什么影响? + +单轮可能一次性拉起多个 subagent,每个都要分配自己的 KV cache,短暂运行后结束,因此 cache 压力以尖峰而非稳态形式到来。面向均匀请求流调优的调度器与淘汰策略,在这种模式下的表现会明显不同。 + +### AgentX 的回放语料是如何构建的? + +我们收集了 393 条 SemiAnalysis 内部的 Claude Code trace,采用与 Qwen-Bailian 类似的分块哈希方法完成匿名化,在不保留内容的前提下保留 prefix 关系。随后由 AIPerf 按原始请求时间表,在不同并发客户端数量下重建这些 trace。 + +### 固定序列长度基准测试还有意义吗? + +有。固定序列长度结果剥离了 agentic 推理服务的复杂性,单独呈现底层芯片与 kernel 性能,因而是解读 AgentX 结果时所依据的基线。 + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "什么是 agentic 工作负载?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Agentic 工作负载指由 AI agent 而非真人在对话框中输入所产生的推理流量。它的特征是几十到几百轮交互的多轮会话、由 system prompt 与 tool 定义堆叠出的长上下文、跨轮次极高的 prefix 复用,以及成批出现的短生命周期 subagent。" + } + }, + { + "@type": "Question", + "name": "为什么 prefix 复用对 agentic 推理如此关键?", + "acceptedAnswer": { + "@type": "Answer", + "text": "每一轮的输入都包含上一轮的输入加上其输出,因此绝大部分上下文此前已经处理过。只要该 prefix 的 KV tensor 仍然驻留,prefill 工作就可以跳过。随着会话增长,命中 cache 的输入与未命中输入之比会趋近于 1,于是 cache 容量与命中率主导了推理服务的成本。" + } + }, + { + "@type": "Question", + "name": "对 agentic 工作负载做基准测试,与固定序列长度基准测试有何不同?", + "acceptedAnswer": { + "@type": "Answer", + "text": "固定序列长度基准测试发送 input 与 output 长度固定的独立请求,prefix 复用不构成变量,结果主要反映芯片与 kernel 性能。Agentic 基准测试回放的是彼此依赖的会话,其中上下文不断增长、prefix 被共享、subagent 会扇出,因此 KV 传输、prefix 感知路由与 cache 分层 offload 都成为被测量的对象。" + } + }, + { + "@type": "Question", + "name": "突发式 subagent 扇出会对 KV cache 造成什么影响?", + "acceptedAnswer": { + "@type": "Answer", + "text": "单轮可能一次性拉起多个 subagent,每个都要分配自己的 KV cache,短暂运行后结束,因此 cache 压力以尖峰而非稳态形式到来。面向均匀请求流调优的调度器与淘汰策略,在这种模式下的表现会明显不同。" + } + }, + { + "@type": "Question", + "name": "AgentX 的回放语料是如何构建的?", + "acceptedAnswer": { + "@type": "Answer", + "text": "我们收集了 393 条 SemiAnalysis 内部的 Claude Code trace,采用与 Qwen-Bailian 类似的分块哈希方法完成匿名化,在不保留内容的前提下保留 prefix 关系。随后由 AIPerf 按原始请求时间表,在不同并发客户端数量下重建这些 trace。" + } + }, + { + "@type": "Question", + "name": "固定序列长度基准测试还有意义吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "有。固定序列长度结果剥离了 agentic 推理服务的复杂性,单独呈现底层芯片与 kernel 性能,因而是解读 AgentX 结果时所依据的基线。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx new file mode 100644 index 000000000..8af550d2b --- /dev/null +++ b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-b200-vs-b300-kv-working-set.mdx @@ -0,0 +1,86 @@ +--- +title: 'DeepSeek V4 Pro 的 AgentX 结果:B200 对比 B300 与 KV cache 工作集' +subtitle: '多出 50% 的 HBM 能挤出额外吞吐量,逐点遥测数据说明了它究竟从何而来' +seoDescription: 'DeepSeek V4 Pro AgentX 运行中 B200 与 B300 的对比。按 TCO 归一化后的吞吐量相当接近,但遥测数据显示 B300 的 HBM 缓存命中率为 91%、B200 为 73%,KV 工作集分别为 4300 万与 2200 万 token。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - b200 + - b300 + - h200 + - nvidia +--- + +按[总拥有成本](/zh/glossary/total-cost-of-ownership)归一化之后,B300 vLLM 与 B200 vLLM 在 DeepSeek V4 Pro 上的聚合式性能相当接近。主要差别在于 B300 能挤出额外的吞吐量,因为它的 HBM 容量比 B200 多 50%。 + +
+ +真正有意思的不是这个汇总数字,而是 AgentX 服务端遥测在它下面揭示的内容。 + +## 4300 万 token 对 2200 万 token + +在 384 条并发 agentic trace 的负载下,B300 vLLM DEP8 配 3TB DRAM、使用 vLLM simple offloading,取得了 91% 的 HBM 缓存命中率,外加 1.36% 的 DRAM 缓存命中率。原因是该配置下 HBM 中 [KV cache](/zh/glossary/kv-cache) 的工作集规模约为 4300 万 token,而任一时刻在途的 token 数只是略微超过这个量。 + +
+ +换成并发 196 的 B200、其余参数不变,情况就反过来了:HBM 缓存命中率降到 73%,部署明显更依赖 DRAM,[offload](/zh/glossary/kv-cache-offload) 缓存命中率接近 20%。此时 HBM 中 KV cache 的工作集规模为 2200 万 token,大约是 B300 的一半。 + +
+ +## offload 层为什么需要余量 + +DRAM KV offload 通常实现为写穿缓存,也就是写入 HBM 缓存的每个前缀同时也会写入 DRAM 缓存。因此,只有当可用于 offload 的 DRAM 显著大于 HBM 中 KV cache 的容量时(大约 1.5 到 3 倍)它才最有效。 + +正是这一性质,让 B300 额外的 HBM 产生的是叠加效应而不只是简单相加:更大的常驻工作集把更多会话状态留在快速层,慢速层因此只需服务真正的剩余部分,而不必不断吸收被淘汰的数据。 + +## H200 这个反例 + +H200 SGLang FP8 能在低并发下服务 DeepSeek V4,从每美元性能看甚至可以和 B200、MI355X SGLang 一较高下。但它无法在高吞吐场景中与更新的 SKU 竞争,因为 HBM 容量不够。 + +它的失效方式很有教育意义:在更高并发下对 DRAM KV offload 的依赖,会随着用户数增加带来无法接受的延迟。offload 能换来容量,但并不免费;一套需要靠它来弥补结构性显存不足的部署,最终会在尾部延迟上付出代价。 + +
+ +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 DeepSeek V4 Pro 上,B300 比 B200 好多少?", + "acceptedAnswer": { + "@type": "Answer", + "text": "按总拥有成本归一化后,B300 vLLM 与 B200 vLLM 的聚合式性能相当接近。B300 凭借多出 50% 的 HBM 容量挤出了额外吞吐量,这在遥测数据中体现为:并发 384 时 HBM 中 KV cache 工作集约 4300 万 token、缓存命中率 91%;而 B200 在并发 196 时为 2200 万 token、命中率 73%。" + } + }, + { + "@type": "Question", + "name": "H200 为什么在高并发下落后?", + "acceptedAnswer": { + "@type": "Answer", + "text": "H200 SGLang FP8 在低并发下能很好地服务 DeepSeek V4,在该区间的每美元性能上甚至可与 B200、MI355X SGLang 一较高下。但它缺少足以支撑高吞吐场景的 HBM,因此随着并发上升只能更多依赖 DRAM KV offload,而这种依赖会随用户数增长带来无法接受的延迟。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx new file mode 100644 index 000000000..81e1985ff --- /dev/null +++ b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-gb200-vs-gb300-disagg.mdx @@ -0,0 +1,72 @@ +--- +title: 'DeepSeek V4 Pro 的 AgentX 结果:GB200 与 GB300 的机架级分离式部署' +subtitle: '两者都依赖 PD 分离,GB300 额外使用 DEP32 宽专家并行解码,而差距体现在首 token 延迟而非 token 速率上' +seoDescription: 'DeepSeek V4 Pro 在 AgentX 上的 NVIDIA 机架级结果。GB300 Dynamo TRTLLM 与 GB200 Dynamo vLLM 都采用 prefill-decode 分离;GB300 那个点在 TTFT 上落后,因为更高的并发带来更多子智能体流量和更多冷 prefill。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - gb200 + - gb300 + - nvl72 + - disagg + - wide-ep + - nvidia +--- + +在 DeepSeek V4 Pro 上,NVIDIA 最有竞争力的方案是 GB300 Dynamo TRTLLM 和 GB200 Dynamo vLLM。两者都依靠 prefill-decode [分离](/zh/glossary/disaggregated-inference),在合理的[交互性](/zh/glossary/interactivity)下取得高吞吐。GB300 配置还额外采用了 DEP32 的[宽专家并行](/zh/glossary/wide-expert-parallelism)解码实例,以在前沿曲线中段获得更高吞吐。 + +
+ +## 两个点在 token 速率上比在首 token 延迟上更接近 + +以每用户每秒 token 数衡量时,2xDEP8+1xDEP12 的 GB200 点与 3xDEP8+1xDEP16 的 GB300 点相当接近;但以首 token 延迟衡量时,两者差距明显更大。因此只看交互性坐标轴,会低估这两种配置行为上的差异。 + +总体而言,TTFT 对工作负载的「突发性」更敏感。GB300 那个点的总并发要高得多,因而会遇到更多子智能体流量,也就有更多冷 prefill。[子智能体](/zh/glossary/subagent)带着全新上下文启动,没有可复用的前缀,因此它的第一条请求是完整 prefill 而不是缓存命中;当足够多这样的请求同时到达时,[尾部延迟](/zh/glossary/tail-latency)就会被推高。 + +
+ +## 为什么这是 agentic 效应而不是硬件效应 + +在定长工作负载上完全看不到这些。在 8k1k 场景中,每条请求彼此独立且形状相同,因此提高并发只是放大 batch,并不改变已缓存与未缓存工作的构成。而在 AgentX 上,提高并发会改变流量本身的组成:会话更多意味着并行的子智能体分支更多,而这些分支是成簇到达的,不是平滑的请求流。 + +实际的读法是:一套为了在吞吐曲线上更靠右而调优的机架级配置,同时也接受了更具突发性的 prefill 负载;这个选择的代价落在首 token 延迟上,而不是同一张图所展示的 token 速率上。 + +
+ +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 DeepSeek V4 Pro 上,NVIDIA 哪种机架级配置最快?", + "acceptedAnswer": { + "@type": "Answer", + "text": "GB300 Dynamo TRTLLM 和 GB200 Dynamo vLLM 是 NVIDIA 在该模型上最有竞争力的配置,两者都依靠 prefill-decode 分离,在合理交互性下达到高吞吐。GB300 还额外运行 DEP32 的宽专家并行解码实例,以提升前沿曲线中段的吞吐量。" + } + }, + { + "@type": "Question", + "name": "为什么 GB300 那个点的 TTFT 比 GB200 更差?", + "acceptedAnswer": { + "@type": "Answer", + "text": "GB300 那个点的总并发高得多,因而带来更多子智能体流量,也就有更多冷 prefill。子智能体带着全新上下文启动、没有可复用前缀,它的第一条请求是完整 prefill;这类请求成簇到达就会推高尾部延迟。总体而言,TTFT 比每用户 token 速率对工作负载的突发性更敏感。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx new file mode 100644 index 000000000..1b2b14dca --- /dev/null +++ b/packages/app/content/blog/zh/deepseek-v4-pro-agentx-mi355x-vs-b200-august.mdx @@ -0,0 +1,84 @@ +--- +title: 'DeepSeek V4 Pro 的 AgentX 结果:MI355X 对比 B200,以及 8 月 21 日的反转' +subtitle: '在端到端延迟的每美元性能上,AMD 一度追平 B200 vLLM,随后上游 vLLM 的优化改变了格局' +seoDescription: 'DeepSeek V4 Pro 0813 在 MI355X 与 B200 上的 AgentX 1.0 结果。在 2026 年 8 月 21 日之前,MI355X SGLang 在每美元性能上追平了 B200 vLLM;此后来自 Inferact 与 NVIDIA 的 vLLM 优化让 B200 反超。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - deepseek + - mi355x + - b200 + - amd + - nvidia +--- + +DeepSeek V4 Pro 0813 是一款来自中国、极受欢迎的前沿开放权重模型,总参数约 1.6 万亿,激活参数 490 亿。它也是 [AgentX](/zh/glossary/agentx) 上 AMD 与 NVIDIA 竞争最胶着的模型,而且排名在一周之内就发生了变化。 + +工作负载在这里很关键。AgentX 回放的是真实的 agentic 编码会话,而不是固定提示词;在所有 DeepSeek V4 运行中,输入长度分布为 p50 88k、p90 272k、p95 404k、p99 675k token,输出分布为 p50 413、p90 2.2k、p95 3.7k、p99 8.6k。这与 8k1k 是完全不同的系统问题。 + +
+ +在 2026 年 8 月 21 日之前,AMD 实力很强的 MI355X SGLang 开发团队在端到端性能的每美元表现上追平了 B200 vLLM。这是实打实的结果,值得直说。 + +但这并不是全部。同一时期,B300 vLLM 和 B200 SGLang 已经领先 MI355X。 + +
+ +8 月 21 日之后,来自 Inferact 和 NVIDIA 的 vLLM 优化让 B200 的每美元性能超过了 MI355X。这仍是一场胶着的竞赛,而不是已成定局;[AMD 也公布了自己的 DeepSeek V4 vLLM 优化清单](https://github.com/vllm-project/vllm/issues/52911),上面还有不少待办事项。 + +## ATOM 这个前提 + +在端到端延迟上,MI355X 上的 ATOM 胜过 B200 vLLM,但没有胜过 B300 或 B200 SGLang。更重要的前提在于部署:中国和西方的多数 AI 实验室都不愿在生产中使用 ATOM,因此只在厂商引擎下成立的优势,并不能代表使用上游软件栈的客户实际拿到的性能。在单节点上,MI355X 的开源方案(vLLM)依然落后于 ATOM。 + +
+ +## MI355X 真正有竞争力的区间 + +总体来看,MI355X 相对 B200 和 B300 表现尚可。在曲线上低吞吐、低延迟的那一段两者最为接近,因为那里只用到张量并行和较为基础的 kernel。差距出现在高吞吐区间,AMD 需要在那里优化自己的 DEP kernel;而且这个差距是在 MI355X 拥有 B200 的 1.5 倍 HBM 的前提下出现的。 + +
+ +两个坐标轴要一起看。某个配置可能在吞吐量对[交互性](/zh/glossary/interactivity)上表现亮眼,却在 [p90](/zh/glossary/tail-latency) 首 token 延迟上把优势还回去,这也是上图选择以 TTFT 为横轴、而不是只看交互性的原因。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 DeepSeek V4 Pro 上,MI355X 胜过 B200 了吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 2026 年 8 月 21 日之前的一段时间里,MI355X SGLang 在端到端性能的每美元表现上追平了 B200 vLLM,MI355X 上的 ATOM 在端到端延迟上也胜过 B200 vLLM。但同期 B300 vLLM 和 B200 SGLang 都领先 MI355X;8 月 21 日之后,来自 Inferact 和 NVIDIA 的 vLLM 优化让 B200 的每美元性能反超 MI355X。" + } + }, + { + "@type": "Question", + "name": "MI355X 显存更大,为什么在高吞吐区间反而落后?", + "acceptedAnswer": { + "@type": "Answer", + "text": "MI355X 的 HBM 是 B200 的 1.5 倍,但在 AgentX 上,它与 B200、B300 最接近的是曲线上低吞吐、低延迟的那一段,那里只用到张量并行和较为基础的 kernel。高吞吐一端依赖 AMD 仍需优化的 DEP kernel,因此额外的显存容量没有转化为前沿吞吐量。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/deepseekv4-16t-day-0-to-day-43-performance.mdx b/packages/app/content/blog/zh/deepseekv4-16t-day-0-to-day-43-performance.mdx index 7bb0d216f..bceea89d9 100644 --- a/packages/app/content/blog/zh/deepseekv4-16t-day-0-to-day-43-performance.mdx +++ b/packages/app/content/blog/zh/deepseekv4-16t-day-0-to-day-43-performance.mdx @@ -348,10 +348,10 @@ vLLM 的计划在 [DeepSeek V4 路线图 issue (#40902)](https://github.com/vllm - **核心模型支持**:持续 MegaMoE 工作(PR #40833)和 NVFP4 支持。 - **运行时与并行**:Model Runner V2 集成、MTP 优化、prefill/decode (PD) 优化、pipeline parallelism 支持。 - **Kernel 集成**:paged prefill kernel、fast top-k kernel、更多水平融合、DeepEP V2,以及与 DeepSeek 自研 TileKernels 的集成。 -- **KV cache**:KV cache 卸载,涵盖 PD + CPU 卸载(PR #39654)和分布式 KV 卸载。 +- **KV cache**:KV cache offload,涵盖 PD + CPU offload(PR #39654)和分布式 KV offload。 - **硬件支持**:在已完成 Hopper 支持的基础上,SM120 和 AMD 支持仍是关键待办项。 -这里的核心主题聚焦在周边系统:新的模型运行器、pipeline parallelism、KV 卸载,以及更广泛的硬件覆盖。 +这里的核心主题聚焦在周边系统:新的模型运行器、pipeline parallelism、KV offload,以及更广泛的硬件覆盖。 即将到来的 InferenceX 更新——SemiAnalysis 的开源公共 EcosystemX 仪表板——将可视化所有主要 ML 开源库在所有主要 AI 芯片上的软件演进、CI 覆盖率和队列时间:NVIDIA、AMD、TPU、Trainium、Huawei 等。 diff --git a/packages/app/content/blog/zh/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx b/packages/app/content/blog/zh/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx new file mode 100644 index 000000000..d61a5be8c --- /dev/null +++ b/packages/app/content/blog/zh/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72.mdx @@ -0,0 +1,75 @@ +--- +title: 'GLM 5.3 的 AgentX 结果:MI355X ATOM 在部分曲线上胜过 GB300 NVL72' +subtitle: 'AMD 厂商引擎在每美元性能上胜出的区间,以及端到端归一化交互性究竟衡量什么' +seoDescription: '在 GLM 5.3 上,于 p90 端到端归一化交互性的部分区间内,AMD ATOM 的每美元性能优于 GB300 NVL72 SGLang,甚至优于 TRTLLM。本文解释这一实验性指标衡量什么,以及它在哪些地方会产生误导。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - glm5 + - mi355x + - gb300 + - nvl72 + - trtllm + - sglang + - amd + - nvidia +--- + +看 ATOM 的话,在 p90 端到端归一化交互性的部分区间内,AMD 的[每美元性能](/zh/glossary/performance-per-dollar)优于 GB300 NVL72 SGLang,甚至优于 TRTLLM。这一结果值得肯定,AMD 团队做得很好。 + +对两家厂商的建议还是那两条:我们期待 AMD 把这些优化移植到 SGLang,也期待 NVIDIA 在接下来几周里尽快优化 GB300 NVL72。 + +
+ +## 端到端归一化交互性衡量什么 + +这个结果所对应的横轴是一个实验性指标,在据此下结论之前值得先弄清楚它的含义。[端到端归一化交互性](/zh/glossary/e2e-normalized-interactivity)的目标,是在同时考虑首 token 延迟和每秒 token 数的前提下,衡量用户实际感受到的响应速度,其定义为输出序列长度除以端到端延迟。 + +把端到端延迟展开为 TTFT 加上输出长度乘以每输出 token 时间(实际上只解码输出长度减一个 token),可得下式。 + +
+ +它本质上等于[交互性](/zh/glossary/interactivity)(即 1/TPOT 那部分),再加上一项与 TTFT 成正比的惩罚。 + +## 这个指标在哪里会误导 + +需要说明的是,该指标是实验性的,并不完美。例如它对高 TTFT 的惩罚很重,也无法体现 prefill-decode [分离](/zh/glossary/disaggregated-inference)等优化的全部细节。AgentX v1.0 的所有提交仍然分别针对常规交互性和 TTFT 做优化;我们也会继续打磨能反映现代 agentic 推理各种细节的新北极星指标。 + +这个前提在这里对双方都成立。一套用首 token 延迟换取流式速度的配置,在这条坐标轴上一定会被惩罚,无论这种取舍是否契合具体产品;因此仅凭端到端归一化交互性得出的胜负,应当再对照独立的交互性曲线和 TTFT 曲线核实,然后才能作为采购论据。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "什么是端到端归一化交互性?", + "acceptedAnswer": { + "@type": "Answer", + "text": "它是一个实验性指标,定义为输出序列长度除以端到端延迟,目的是在同时考虑 TTFT 和每秒 token 数的前提下衡量用户感受到的响应速度。由于端到端延迟等于 TTFT 加上输出长度乘以每输出 token 时间,该指标实际等于交互性加上一项与 TTFT 成正比的惩罚。" + } + }, + { + "@type": "Question", + "name": "在 GLM 5.3 上,AMD ATOM 在哪里胜过 GB300 NVL72?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 p90 端到端归一化交互性的部分区间内,ATOM 的每美元性能优于 GB300 NVL72 SGLang,甚至优于 TRTLLM。该指标是实验性的,且对高 TTFT 惩罚很重,因此这一结果应当再对照独立的交互性曲线和 TTFT 曲线核实。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx b/packages/app/content/blog/zh/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx new file mode 100644 index 000000000..97065e908 --- /dev/null +++ b/packages/app/content/blog/zh/glm-5-3-agentx-nvidia-vs-amd-sglang-150-toks.mdx @@ -0,0 +1,69 @@ +--- +title: 'GLM 5.3 的 AgentX 结果:150 tok/s/user 下 NVIDIA 单 token 成本最多低 5 倍' +subtitle: '在这个运行点上,即便 AMD 硬件白送也仍然填不平差距' +seoDescription: 'GLM 5.3 在开源 SGLang 上的 AgentX 结果。在 150 tok/s/user 的 p90 交互性下,NVIDIA 的成本效率最高领先 5 倍;差距之大,即便竞品硬件免费,NVIDIA 的单 token 成本依然更低。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - glm5 + - b200 + - b300 + - mi355x + - sglang + - nvidia + - amd +--- + +GLM 5.3 在 GLM 5.2 744B 基础上做了额外的后训练,是一款前沿级别的模型。 + +在开源 SGLang 的性能上,这又是一个 NVIDIA 在真实 agentic 推理性能上胜过 AMD 的模型。在 150 tok/s/user 的 p90 [交互性](/zh/glossary/interactivity)下,NVIDIA 的成本效率最高领先 5 倍。 + +
+ +## 「硬件白送」检验 + +衡量这种差距有一个很实用的方法。以 AMD 软件当前的状态,在 150 tok/s/user 时,NVIDIA 的性能优势大到这种程度:即便竞品芯片硬件白送(服务商当然仍要承担数据中心托管、电力和其他运营成本),用 NVIDIA 的单 token 成本依然更低。 + +这个说法之所以重要,是因为在采购谈判中最常被拿来用的杠杆就是硬件价格;而在这个运行点上,这根杠杆的力量不够。真正的约束是软件,而不是物料成本。 + +
+ +## 这是一次快照,不是最终结论 + +这个差距是相对 AMD 软件当前状态测得的,而在这批结果的其他部分中,这一状态一直在快速变化。我们期待 AMD 在几周后的 AgentX 更新文章中带来性能优化,那篇文章还会包含其他一些非常有意思的结果。 + +范围也要说清楚:这里比较的是开源 SGLang。在同一个模型的部分区间上,AMD 的厂商引擎讲的是另一个故事,我们[另文说明](/zh/blog/glm-5-3-agentx-mi355x-atom-vs-gb300-nvl72)。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 GLM 5.3 上,NVIDIA 比 AMD 便宜多少?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在开源 SGLang、150 tok/s/user 的 p90 交互性下,NVIDIA 的成本效率最高领先 5 倍。以 AMD 软件在该运行点的当前状态,这一优势大到即便竞品硬件免费(服务商仍需承担托管、电力和其他运营成本),NVIDIA 的单 token 成本依然更低。" + } + }, + { + "@type": "Question", + "name": "AMD 在 GLM 5.3 上有胜出的地方吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "有,但不在这组对比里。5 倍这个数字是开源 SGLang 的结果。使用 ATOM 时,在 p90 端到端归一化交互性的部分区间内,AMD 的每美元性能优于 GB300 NVL72 SGLang,甚至优于 TRTLLM。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/inferencex-v2-nvidia-blackwell-vs-amd-vs-hopper.mdx b/packages/app/content/blog/zh/inferencex-v2-nvidia-blackwell-vs-amd-vs-hopper.mdx index 0b71fc1b5..e9051378a 100644 --- a/packages/app/content/blog/zh/inferencex-v2-nvidia-blackwell-vs-amd-vs-hopper.mdx +++ b/packages/app/content/blog/zh/inferencex-v2-nvidia-blackwell-vs-amd-vs-hopper.mdx @@ -188,7 +188,7 @@ B200 SGLang 自我们去年 10 月首次发布以来,FP4 和 FP8 场景均呈 ## 分离式推理框架 -NVIDIA 使用 Dynamo 作为其分离式推理方案。[Dynamo](https://docs.nvidia.com/dynamo/design-docs/overall-architecture) 是一个专为多节点分布式推理设计的推理框架,具备预填充-解码分离、请求路由和 KV 缓存卸载等技术。它与推理引擎无关,允许我们在基准测试中使用 SGLang 和 TRT-LLM 作为后端。对于 AMD,我们使用 SGLang 配合两种不同的 KV 缓存传输框架:MoRI 和 Mooncake。[MoRI](https://github.com/rocm/mori) 是一个高性能通信接口,专注于 RDMA 和 GPU 集成,提供网络集合操作和专家并行内核等应用。Mooncake [最近加入了 PyTorch 生态系统](https://pytorch.org/blog/mooncake-joins-pytorch-ecosystem/),支持预填充-解码分离以及多种容错多节点功能。 +NVIDIA 使用 Dynamo 作为其分离式推理方案。[Dynamo](https://docs.nvidia.com/dynamo/design-docs/overall-architecture) 是一个专为多节点分布式推理设计的推理框架,具备预填充-解码分离、请求路由和 KV 缓存 offload 等技术。它与推理引擎无关,允许我们在基准测试中使用 SGLang 和 TRT-LLM 作为后端。对于 AMD,我们使用 SGLang 配合两种不同的 KV 缓存传输框架:MoRI 和 Mooncake。[MoRI](https://github.com/rocm/mori) 是一个高性能通信接口,专注于 RDMA 和 GPU 集成,提供网络集合操作和专家并行内核等应用。Mooncake [最近加入了 PyTorch 生态系统](https://pytorch.org/blog/mooncake-joins-pytorch-ecosystem/),支持预填充-解码分离以及多种容错多节点功能。 ## DeepSeek Disagg + WideEP 结果深入分析 @@ -493,7 +493,7 @@ AMD 在开源分布式推理、宽专家并行和 FP4 可组合性方面落后 ## AMD ATOM 引擎 -AMD 推出了名为 ATOM 的新推理引擎。ATOM 能提供略好的单节点性能,但在许多关键功能上完全缺失,导致无法用于真实工作负载。例如,它不支持 NVMe 或 CPU KVCache 卸载、工具解析、宽专家并行或分离式服务。这导致生产环境中没有任何客户使用它。与 NVIDIA 的 TRTLLM 不同——后者在 TogetherAI 等公司全球范围内每小时生成数十亿 token,[并且支持工具解析和其他功能](https://nvidia.github.io/TensorRT-LLM/commands/trtllm-serve/trtllm-serve.html#cmdoption-trtllm-serve-serve-tool_parser)——由于上述功能缺失,目前没有任何 token 工厂在使用 ATOM。 +AMD 推出了名为 ATOM 的新推理引擎。ATOM 能提供略好的单节点性能,但在许多关键功能上完全缺失,导致无法用于真实工作负载。例如,它不支持 NVMe 或 CPU KV cache offload、工具解析、宽专家并行或分离式服务。这导致生产环境中没有任何客户使用它。与 NVIDIA 的 TRTLLM 不同——后者在 TogetherAI 等公司全球范围内每小时生成数十亿 token,[并且支持工具解析和其他功能](https://nvidia.github.io/TensorRT-LLM/commands/trtllm-serve/trtllm-serve.html#cmdoption-trtllm-serve-serve-tool_parser)——由于上述功能缺失,目前没有任何 token 工厂在使用 ATOM。 此外,vLLM 等开源推理引擎的维护者对 AMD 感到失望,原因是 AMD 提供的工程和 GPU 资源不足。例如,vLLM 首席维护者 Simon Mo 在 GitHub RFC 中表示,他仍然没有可用的 MI355X 来添加到 vLLM CI 中,因此用户体验不佳。vLLM 上目前没有任何 MI355X 测试,而 NVIDIA 的 B200 在 vLLM 上有大量测试。同样,vLLM 上的 MI300X CI 机器数量仍然不够。上游 vLLM 至少还需要 20 台 MI300 机器、20 台 MI325 机器和 20 台 MI355X 机器才能达到与 CUDA 相同的可用性水平。 @@ -847,7 +847,7 @@ Microsoft/GitHub 自己也意识到了这个问题,已停止在其状态页面 目前,InferenceX 使用完全随机的 token 作为基准测试的输入。然后我们在 [ISL*0.8, ISL] 分布下均匀变化 ISL/OSL,OSL 类似。由于使用随机数据,我们在所有基准测试中禁用了前缀缓存,因为完全随机数据的前缀缓存命中率期望值为 0%。此外,所有随机数据都是单轮的,意味着每个对话只包含一个提示和一个回答。虽然这提供了良好的基线帕累托前沿,但它不是模拟真实世界生产推理工作负载的实际基准测试方案。 -在近期,我们将使用类似 [allenai/WildChat-4.8M](https://huggingface.co/datasets/allenai/WildChat-4.8M) 这样的数据集创建基础多轮基准测试,该数据集记录了真实用户的多轮对话。除了在所有场景中启用前缀缓存外,我们还将启用 KV 缓存 CPU 卸载,因为这是我们在生产工作负载中看到的做法。这将更准确地评估每款芯片的优缺点。例如,MI355X 拥有 288GB HBM3e 而 B200 仅有 192GB。因此,我们预期 MI355X 在高并发多轮场景中表现更好,因为更多内存可以分配给 KV 缓存。另一方面,在 GPU KV 缓存紧张、块被卸载到 CPU 的场景中,我们预期 GB 系列表现更优,因为这些芯片拥有 900 GB/s 双向 CPU-GPU 带宽,相比之下 HGX 使用 PCIe 5.0 和 6.0 分别只有 128 GB/s 和 256 GB/s。此外,我们目前看到 AMD 的 CPU 卸载软件表现不佳,这可能在相同场景中对性能产生负面影响。 +在近期,我们将使用类似 [allenai/WildChat-4.8M](https://huggingface.co/datasets/allenai/WildChat-4.8M) 这样的数据集创建基础多轮基准测试,该数据集记录了真实用户的多轮对话。除了在所有场景中启用前缀缓存外,我们还将启用 KV 缓存 CPU offload,因为这是我们在生产工作负载中看到的做法。这将更准确地评估每款芯片的优缺点。例如,MI355X 拥有 288GB HBM3e 而 B200 仅有 192GB。因此,我们预期 MI355X 在高并发多轮场景中表现更好,因为更多内存可以分配给 KV 缓存。另一方面,在 GPU KV 缓存紧张、块被 offload 到 CPU 的场景中,我们预期 GB 系列表现更优,因为这些芯片拥有 900 GB/s 双向 CPU-GPU 带宽,相比之下 HGX 使用 PCIe 5.0 和 6.0 分别只有 128 GB/s 和 256 GB/s。此外,我们目前看到 AMD 的 CPU offload 软件表现不佳,这可能在相同场景中对性能产生负面影响。 关键是:真实世界的多轮数据集测试了更多 SOTA 推理引擎功能,能够在所有芯片上捕获更细致和可靠的性能数据。 diff --git a/packages/app/content/blog/zh/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx b/packages/app/content/blog/zh/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx new file mode 100644 index 000000000..1e0a4651a --- /dev/null +++ b/packages/app/content/blog/zh/kimi-k3-agentx-mi355x-atom-vs-gb300-nvl72.mdx @@ -0,0 +1,70 @@ +--- +title: 'Kimi K3 的 AgentX 结果:MI355X ATOM 在部分曲线上胜过 GB300 NVL72' +subtitle: 'AMD 的厂商引擎在每美元性能前沿上赢下了实打实的一段,而 Hopper 连服务 K3 都很吃力' +seoDescription: '在 Kimi K3 的 AgentX 运行中,端到端延迟 40 到 60 秒的区间内,MI355X ATOM 在每美元性能上胜过 GB300 NVL72 vLLM;而 Hopper 表现吃力,因为 K3 需要定制的 SM90 kernel 以及调优过的 TP32、EP32 形状。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - kimi + - mi355x + - gb300 + - nvl72 + - h200 + - amd + - nvidia +--- + +在端到端延迟 40 到 60 秒的那段曲线上,MI355X ATOM 的每美元性能甚至胜过 GB300 NVL72 vLLM。在一个 2.8 万亿参数的模型上、面对 NVIDIA 的机架级系统,这是 AMD 实打实赢来的结果。 + +
+ +## 上游这个前提 + +AMD 用 [ATOM](/zh/glossary/atom) 快速推进 K3 性能是件好事。与之相伴的建议是:优先把这些改进上游到 vLLM。ATOM 目前是 AMD 性能最好的引擎,但对使用上游开源服务栈的客户来说,vLLM 才是更有参考价值的对比对象;只在厂商运行时下成立的曲线,并不能代表这些客户实际能部署的性能。 + +## Hopper 基本无法参与 + +Hopper 很难承接 Kimi K3 的 AgentX 工作负载。一方面 Kimi 确实是个庞大的模型,另一方面 vLLM 维护者和 NVIDIA 一直没有把精力放在为 K3 优化 Hopper 上。 + +具体原因是明确的,而非笼统的:Hopper(SM90)需要为 K3 定制调优 kernel,并针对高[交互性](/zh/glossary/interactivity)服务调优 TP32 和 EP32 的 shape。这两者目前都不存在,因此真正的约束与其说是硬件,不如说是没有人在做这件事。 + +
+ +这提醒我们:基准测试图表上的代际差距,有一部分其实是工程投入上的代际差距。Hopper 多年来能很好地服务 DeepSeek R1,是因为那个模型持续获得 kernel 层面的投入;K3 没有得到同等对待,曲线也就如实反映了这一差别。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 Kimi K3 上,AMD 有胜过 NVIDIA 的地方吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "有,在部分曲线上。在端到端延迟 40 到 60 秒的区间内,MI355X ATOM 的每美元性能甚至胜过 GB300 NVL72 vLLM。前提是 ATOM 属于 AMD 的厂商引擎而非上游开源软件栈,因此对多数客户而言 vLLM 仍是更有参考价值的对比对象。" + } + }, + { + "@type": "Question", + "name": "Hopper 在 Kimi K3 上为什么吃力?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Kimi K3 体量巨大,而 vLLM 维护者和 NVIDIA 都没有把精力放在为它优化 Hopper 上。要在 SM90 上以高交互性服务 K3,需要定制调优的 kernel 以及调优过的 TP32、EP32 形状,而这些工作尚未完成。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/kimi-k3-the-manos-the-mythos-the.mdx b/packages/app/content/blog/zh/kimi-k3-the-manos-the-mythos-the.mdx index b6f96556d..2dac3c1c3 100644 --- a/packages/app/content/blog/zh/kimi-k3-the-manos-the-mythos-the.mdx +++ b/packages/app/content/blog/zh/kimi-k3-the-manos-the-mythos-the.mdx @@ -681,12 +681,12 @@ QB 的做法是:在当前批次的截断阈值和路由结果下,求出一 在 InferenceX 上,我们直接基于内部录制的 Claude Code 轨迹(trace)来评测 Kimi K3 的服务性能。我们回放其中达到稳态的一小时轨迹。每轮的输入 token 中位数为 142k,输出 token 中位数为 444,每个会话的轮数中位数为 65。每轮输出 token 偏短,是 agentic harness 类负载的典型特征——agent 频繁调用工具,连编辑操作也是工具调用。 -相比我们此前的 8k1k/1k1k 基准测试,这套基准是一次大的跃进,因为它真实反映了现实中的 agentic 使用场景。从系统角度看,它也更贴近生产系统,能够体现包括 prefix cache 和 KV 向 DRAM 卸载在内的 KV cache 行为。 +相比我们此前的 8k1k/1k1k 基准测试,这套基准是一次大的跃进,因为它真实反映了现实中的 agentic 使用场景。从系统角度看,它也更贴近生产系统,能够体现包括 prefix cache 和 KV 向 DRAM offload 在内的 KV cache 行为。
对 Kimi K3 来说,Day 0 的 bringup 比 DSv4 更轻松,因为权重发布前的文档和准备工作更充分。合适的镜像和投机解码模型与权重同时发布。 @@ -766,7 +766,7 @@ _本文后续内容——B300 并发扫描、AMD MI355X 与 MI455X 的 bringup "name": "InferenceX 如何评测 Kimi K3?", "acceptedAnswer": { "@type": "Answer", - "text": "使用内部录制的 Claude Code 真实轨迹,而非固定的 8k1k/1k1k 合成负载。我们回放一小时达到稳态的轨迹,每轮输入 token 中位数 142k、输出 token 中位数 444,每个会话轮数中位数 65。每轮输出偏短是 agentic harness 的典型特征,因为 agent 频繁调用工具。回放真实轨迹能够复现生产环境中的 KV cache 行为,包括 prefix cache 命中和 KV 向 DRAM 的卸载。" + "text": "使用内部录制的 Claude Code 真实轨迹,而非固定的 8k1k/1k1k 合成负载。我们回放一小时达到稳态的轨迹,每轮输入 token 中位数 142k、输出 token 中位数 444,每个会话轮数中位数 65。每轮输出偏短是 agentic harness 的典型特征,因为 agent 频繁调用工具。回放真实轨迹能够复现生产环境中的 KV cache 行为,包括 prefix cache 命中和 KV 向 DRAM offload。" } }, { diff --git a/packages/app/content/blog/zh/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx b/packages/app/content/blog/zh/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx new file mode 100644 index 000000000..37164b1da --- /dev/null +++ b/packages/app/content/blog/zh/minimax-m3-agentx-b200-b300-vs-rack-scale.mdx @@ -0,0 +1,75 @@ +--- +title: 'MiniMax M3 的 AgentX 结果:B200 与 B300 为何胜过机架级的 GB200 NVL72 与 GB300 NVL72' +subtitle: 'Dynamo router 成为瓶颈,没有提交使用上下文并行,而 AMD 完全没有用上 KV offload' +seoDescription: '在 MiniMax M3 上,B200 与 B300 在按 TCO 归一化的吞吐量上胜过 GB200 与 GB300。Dynamo router 可能成为瓶颈,因为其工作量随在线前缀增长;尽管 p90 输入长度达 317k,仍没有提交使用上下文并行;AMD 的 Pareto 最优点则完全没有使用 KV offload。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - minimax + - b200 + - b300 + - gb200 + - gb300 + - nvl72 + - dynamo + - amd + - rocm +--- + +在 MiniMax M3 上,按 TCO 归一化的吞吐量方面,B200 与 B300 完全胜过各自对应的机架级方案。对于一个由 NVIDIA 横扫的模型来说,这个内部结果颇令人意外,而原因并不在网络。 + +在 AgentX 上,机架级的优势不那么明显,因为 Dynamo router 可能成为瓶颈:它的工作量随在线前缀的数量和长度增长,而不是随生成的 token 数增长。此外,这个模型上还没有针对宽专家并行、宽 DCP 或宽张量并行调优良好的 kernel。而 GB200 与 GB300 的 TCO 更高,在没有宽 EP 或宽 DCP 的情况下,这就直接表现为更差的每 TCO 性能。 + +
+ +我们也预期 NVIDIA 会在该 SKU 的机架级方案上做进一步优化,并会在后续文章中重点介绍。 + +## 没有人在使用上下文并行 + +尽管 p90 输入序列长度达到 317k token,目前仍没有任何提交使用[上下文并行](/zh/glossary/context-parallelism)。这恰恰是该技术存在的意义所在,因此这种缺席需要解释。 + +M3 只有 4 个 KV head,即使在 TP8 下 DCP 也最多只能到 2;而且 MSA indexer 还需要单独的上下文并行处理,相关 vLLM PR 已经提交。因此这个上限是架构层面的,而不是调优疏忽,它也限定了长上下文并行策略在这个模型上能带来多少收益。 + +## AMD 的 offload 缺口 + +NVIDIA 所有 Pareto 最优点在并发 20 以上都启用了 [KV offload](/zh/glossary/kv-cache-offload)。而 AMD 的 Pareto 最优点没有一个使用 DRAM KV offload;在其他模型上,AMD 使用 offload 的比例也低于 NVIDIA。 + +原因很机械:AMD vLLM 上 CPU KV cache offload 所需的 GPU 到 CPU 传输效率极低,因为 hipMemcpyBatchAsync API 直到 ROCm 7.14 才提供。没有它,vLLM 原生的 simple CPU offloading 只能做串行 memcpy,而无法把多次拷贝合并成更大的消息。 + +
+ +缺少一个批量拷贝 API,这种事不会出现在规格书上,在定长基准测试中也根本不会被触发。但在长时间的 agentic 会话中,它决定了整整一层内存是否可用。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 MiniMax M3 上,B200 与 B300 为什么胜过 GB200 与 GB300?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在按 TCO 归一化的吞吐量上,AgentX 中机架级的优势被削弱了:Dynamo router 可能成为瓶颈,因为它的工作量随在线前缀的数量和长度增长,而不是随生成的 token 数增长。此外,该模型上还没有针对宽专家并行、宽 DCP 或宽张量并行调优良好的 kernel,缺少这些能力时,GB200 与 GB300 更高的 TCO 就赚不回来。" + } + }, + { + "@type": "Question", + "name": "AMD 在 MiniMax M3 上为什么没有使用 KV offload?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 AMD vLLM 上,CPU KV cache offload 所需的 GPU 到 CPU 传输效率极低,因为 hipMemcpyBatchAsync API 直到 ROCm 7.14 才提供。没有它,原生的 simple CPU offloading 只能做串行 memcpy 而无法合并成更大的消息;因此 AMD 没有任何 Pareto 最优点使用 DRAM offload,而 NVIDIA 在并发 20 以上的每个最优点都用了。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx b/packages/app/content/blog/zh/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx new file mode 100644 index 000000000..0a7323dc5 --- /dev/null +++ b/packages/app/content/blog/zh/minimax-m3-agentx-b300-trtllm-tp2-vs-the-field.mdx @@ -0,0 +1,78 @@ +--- +title: 'MiniMax M3 的 AgentX 结果:B300 TRT-LLM TP2 拿下桂冠' +subtitle: 'NVIDIA 横扫这个 432B 模型,而缺失的 DP attention 数据点说明了缓存局部性为何变成路由约束' +seoDescription: 'MiniMax M3 432B 的 AgentX 结果。B300 TRT-LLM TP2 领跑前沿曲线;由于 KV cache 局部性变成路由约束,曲线上没有 DP attention 数据点;GB200 上 TP4/EP4/DPA 的吞吐量只有纯 TP4 的 0.60 倍,而 p90 TTFT 高出 3 倍以上。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - minimax + - b300 + - b200 + - gb200 + - trtllm + - vllm + - nvidia +--- + +在 MiniMax M3 432B 上,NVIDIA 完胜所有对手。AMD 的软件性能非常糟糕,长上下文下尤其如此;我们认为根源在于 AMD 工程管理层只激励团队针对短上下文单轮工作负载做调优,而忽视了长上下文多轮工作负载。 + +在 NVIDIA 阵营内部,B300 TRT-LLM 在 TP2 下拿下了 M3 的桂冠。 + +
+ +## 缺失的 DP attention 数据点 + +M3 的前沿曲线上没有 [DP attention](/zh/glossary/dp-attention) 数据点,而这种缺席本身就说明了问题:在 M3 上,数据并行注意力并不划算,因为 KV cache 局部性会变成路由约束。 + +具体数字让机制变得清晰。以并发 40 的 GB200 为例,TP4/EP4/DPA 的吞吐量只有纯 TP4 的 0.60 倍,而 p90 TTFT 高出 3 倍以上;并发 32 时缓存命中率仅 28.8%,理论值则是 96.0%。每个 DP rank 独占缓存池的四分之一,一个 30 万 token 的会话一旦落到错误的 rank 上,就要全部重算。 + +28.8% 实测与 96.0% 理论之间的这道鸿沟,正是数据并行下[前缀复用](/zh/glossary/prefix-cache-hit-rate)的全部症结:缓存本身足够大,只是会话没有被发到持有它的那个 rank 上。 + +M3 的前沿曲线上也没有出现任何带专家并行的 decode 配置,可能是因为并发还不足以让所有专家的负载均衡起来。 + +## vLLM 与 TRT-LLM 的对比 + +值得一提的是,从吞吐量与 p90 交互性的关系看,vLLM 的表现与 TRT-LLM 非常接近;而在吞吐量与 p90 TTFT 的关系上,vLLM 表现更好。 + +
+ +因此更倾向于哪一个,取决于你的产品是按哪条坐标轴来定义的。这也再次说明:应当把交互性和首 token 延迟放在一起读,而不是挑一张更符合预设结论的图。 + +
+ +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 MiniMax M3 上哪种配置最快?", + "acceptedAnswer": { + "@type": "Answer", + "text": "B300 TRT-LLM 在 TP2 下拿下 M3 桂冠。NVIDIA 在该模型上整体领先,而 AMD 的软件性能表现较差,长上下文场景尤其明显。" + } + }, + { + "@type": "Question", + "name": "M3 前沿曲线上为什么没有数据并行注意力的数据点?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 M3 上 DP attention 并不划算,因为 KV cache 局部性会变成路由约束。在并发 40 的 GB200 上,TP4/EP4/DPA 的吞吐量只有纯 TP4 的 0.60 倍,p90 TTFT 高出 3 倍以上;并发 32 时实测缓存命中率为 28.8%,而理论上限为 96.0%,原因是每个 DP rank 只拥有缓存池的四分之一,会话落到错误的 rank 上就要全部重算。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/openai-jalapeno-better-than-nvidia.mdx b/packages/app/content/blog/zh/openai-jalapeno-better-than-nvidia.mdx new file mode 100644 index 000000000..5daf684a3 --- /dev/null +++ b/packages/app/content/blog/zh/openai-jalapeno-better-than-nvidia.mdx @@ -0,0 +1,383 @@ +--- +title: 'OpenAI Jalapeño:比 NVIDIA Blackwell 更强' +subtitle: 'OpenAI 自研 ASIC 对比 Rubin:Jalapeño 的 TCO、每兆瓦吞吐量,以及那些够辣的细节' +seoDescription: 'OpenAI 首代自研推理 ASIC,与 Broadcom 合作打造,并在 InferenceX 上完成基准测试。文中对比它与 Blackwell、Vera Rubin 的每兆瓦性能,介绍 HBM4 与 MXFP4 规格、Gluon 软件栈、OpenAI 为何不做 prefill-decode 分离,以及 Katsu、Vindaloo 与 Chana 组成的机架架构。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - inference + - benchmark + - gpu + - nvidia + - openai + - asic + - rubin +--- + +过去两年,OpenAI 一直在低调打造一款名为「Jalapeño」的推理芯片,如今在 Hot Chips 上正式发布。此前关于它成功流片的传闻已经流传了一段时间,现在细节终于公开。OpenAI 邀请我们实地查看这颗芯片、走进他们的实验室确认它的真实性,并用我们的 [InferenceX](https://inferencex.semianalysis.com/) 套件对它做了[基准测试](https://openai.com/index/jalapeno-first-results/)。 + +今年 6 月,[OpenAI 公布了这项芯片计划](https://openai.com/index/openai-broadcom-jalapeno-inference-chip/),与 Broadcom 合作,从零开始、专为 LLM 推理设计。[设计工作始于 2024 年年中](https://newsletter.semianalysis.com/p/openai-chip-team-is-now-serious),从组建团队到量产流片只用了约 16 个月,这在 ASIC 开发中是极快的节奏。 + +通常第一代芯片并不具备竞争力,但 OpenAI 打破了这一规律:在多个主流开源模型上,它击败了我们测试过的每一款 NVIDIA、AMD 和 Google 芯片,处于业界领先水平。OpenAI 靠的是极致的软硬件协同设计。令人意外的是,他们并没有针对推理的某个特定环节做过度特化,而是专注于做一颗在所有场景下都有高性能的通用芯片。 + +本文将深入介绍 Jalapeño 的架构细节、软件细节,以及它在 InferenceX 上的性能结果。 + +
+ +## 一颗通用的推理芯片 + +外界普遍认为 OpenAI 的芯片是为自家模型特化的,但这是错的:OpenAI 做的是一颗面向 AI 推理的通用芯片。 + +这个时间线堪称疯狂,说明「用 AI 加速芯片设计」的说法确有其事。抛开进度不谈,OpenAI 投入了大量资金、做出了务实的设计取舍,团队本身也很能打,因此这个结果并不令人意外。 + +仅看规格,它就已经是有力的竞争者: + +
+ +而采用 HBM4,使它足以与 NVIDIA 和 AMD 的旗舰 GPU 相提并论: + +
+ +关于这颗芯片的媒体报道,大多沿用了 OpenAI 随口提到的几句话,认为它会以其他芯片做不到的方式为自家模型做优化。这种说法是错的。Jalapeño 是一颗通用推理芯片,能够运行各类模型和各类工作负载,也包括我们的 InferenceX 基准测试——我们正是与 OpenAI 工程师一起在实验室里跑的。他们还开玩笑似地展示了在这颗芯片上运行 Doom,而移植工作只用 Codex 提示词就完成了。 + +下面是我们的核心 perf/W 结果,看的是每兆瓦全量供电所能产出的 token 吞吐量。**Jalapeño 碾压了其他所有芯片**。而且这一切都是在不使用多 token 预测(MTP)的前提下取得的,图中其他芯片则都是各自 SKU 上开启 MTP 的最佳配置。 + +
+ +在几乎所有场景下,Jalapeño 的 perf/W 都胜过 Blackwell,而且并没有针对曲线上的某个点做专门调优。它不仅在低延迟场景表现出色,在高吞吐场景同样如此。更对等的比较是与单 token 预测的结果对比,此时它把所有竞争对手都甩在身后。在低并发场景下,Jalapeño 展现出惊人的交互性:在 DeepSeek R1 上,并发为 1 时超过 700 tok/s/user。 + +更惊人的是,这一切都是在单 token 预测(STP)下完成的,没有投机解码,也没有 prefill 与 decode 分离。除 DeepSeek R1 外,我们还看到了其他一些模型,包括 Kimi-K2.5 和 GPT-OSS,后者约为 1,400 tok/s/user。对所有这些模型,我们确认 Jalapeño 的 GSM8k 评估结果与 NVIDIA 芯片相当。 + +有几点需要说明。第一,所有数据均由 OpenAI 提供。我们在实验室现场验证了 InferenceX 的运行过程,但没有跑完整套 [InferenceX](https://inferencex.semianalysis.com/) 基准测试,也没有看到 [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) 结果。AgentX 是我们更倾向用于比较芯片性能的套件,因为其数据集具备长上下文和多轮特征,能反映真实生产流程中的缓存行为。在 8k1k 上表现良好的框架,在 AgentX 上可能明显变差,因为真实生产负载会给 router、前缀缓存机制、缓存管理、offload 基础设施等组件施加压力,而单轮 8k1k 根本考察不到这些。相关内容可参见我们的 AgentX 文章。 + +[AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing?](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) — 自 2025 年 11 月的 Claude Code 拐点以来,长上下文、多轮的 agentic 工作负载快速增长,如今已经主导生产推理的流量。2026 年 4 月,OpenAI 的企业级 agentic 支出超过了 ChatGPT 支出。 + +第二,我们认为拿它和 Blackwell 比并不完整,也不够公平。Jalapeño 真正的对手是同样使用 HBM4 的 Rubin 这类芯片。Vera Rubin 系统现在已经开始向客户发货,而 OpenAI 手上除了工程样片之外还需要一段时间才能有更多东西。 + +因此,性能比较的对象应该是 Rubin 而不是 Blackwell;从某种意义上说,我们本就预期 Jalapeño 这样的定制芯片会胜过 Blackwell。Vera Rubin NVL72 的 perf/MW 是 GB200 NVL72 的 5.4 倍,[我们在分析 NVIDIA 上月与 CoreWeave 联合发布时所给出性能数据的文章中已有说明](https://newsletter.semianalysis.com/p/vera-rubin-nvl72-vs-gb200-nvl72-inference)。下文我们会把 Jalapeño 与 Vera Rubin 7 月的性能数据做对比。 + +[Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis](https://newsletter.semianalysis.com/p/vera-rubin-nvl72-vs-gb200-nvl72-inference) — Vera Rubin NVL72 是 NVIDIA 机架级 Oberon 架构的第二代产品,其推理性能提升来自极致的协同设计。工程样片的早期结果令人鼓舞:运行 DeepSeek R1 的 Vera Rubin NVL72 相比 GB200 NVL72 目前有 5.4 倍的每兆瓦性能和 5 倍的每美元性能,而与 2025 年 GB200 NVL72 早期 bring-up 阶段相比差距更大。Vera Rubin 目前仍处于早期 bring-up 阶段,我们预计差距还会继续拉大。随着软件成熟,Rubin 的推理性能会持续提升,这与我们此前在 Blackwell 上观察到的规律一致。 + +第三,被测试的模型并不处于开源前沿。NVIDIA 和 AMD 已经用 AgentX 公布了 DeepSeek V4 Pro、Kimi K3 等更大模型的结果。模型越大、发布越新,在一颗新芯片上完成 bring-up 就越复杂。话虽如此,OpenAI 在 Jalapeño 上跑通的这些模型也算不上小。 + +## 性能分析 + +OpenAI 是按 perf/W 来做设计的。原因很简单:OpenAI 当前的瓶颈是数据中心电力,而不是预算或场地,因此每兆瓦 token 数至关重要。在 Computex 2026 上,黄仁勋表示 perf/W、可靠性和长使用寿命是未来 GPU 的核心特性。用他的原话说:「如果你有 1 吉瓦电力,那么每瓦吞吐量就是营收。」他还提到,仅仅因为芯片更便宜就选择错误的架构是说不通的。 + +
+ +NVIDIA 在 Hot Chips 2026 的 Vera 演讲中展示同一张营收图时也强调了这一点:「今天的数据中心是电力受限的。」电力很重要,而且直接驱动营收。 + +运营方无法简单地获取更多兆瓦,因为增加 GPU 与增加电网容量的时间尺度完全不同。数据中心的电力上限受制于并网接入、基础设施、制冷能力以及 UPS 与备用发电设计。电网审批的延误一再落后于硬件与建设进度,这催生了对 BtM(表后)电力容量的需求:直接建在数据中心现场的燃气轮机和自备发电机。这部分容量位于电网计费表之后,而不是从公共电网取电,因此运营方无需等待并网和电力扩容就能给园区供电——这正是 xAI 的 Colossus 2 高度依赖 BtM、而实际并网进度远远滞后的原因。更多内容参见我们的[能源模型](https://semianalysis.com/energy-model/)。 + +正如我们在 X 上写过的,由于瓦特就是焦耳每秒,tok/s/MW 可以约化为每焦耳 token 数。这使 tok/s/MW 能够代表一套系统把能量转化为 token 的效率与能力。 + +
+ +在这一点上,即便与 Rubin 相比,Jalapeño 也胜出。OpenAI 的 Jalapeño 在 STP 下的每兆瓦输出 token 吞吐量,超过了 [NVIDIA 与 CoreWeave 在 7 月公布的 Vera Rubin MTP 结果,也远超 GB200 在 2025 年的 MTP 结果](https://www.coreweave.com/blog/nvidia-vera-rubin-nvl72-on-coreweave-10x-more-tokens-per-megawatt-than-blackwell)。正如我们在 Vera Rubin 文章中所说,之所以拿 VR 与 2025 年的 GB200 结果对比,是因为二者处于类似的早期 bring-up 阶段,这样可以保持软件成熟度这一变量不变。沿用同样的逻辑,我们把 Vera Rubin 2026 年 7 月的最新结果、GB200 2025 年的结果与今天的 Jalapeño 结果放在一起比较。这是很合理的对比:这些是目前公开的最佳 Rubin 数据,而 OpenAI 的流片时间还在 Rubin 之后。OpenAI 和 Rubin 都还不成熟,性能都会继续提升。 + +
+ +在 perf/TCO 上,Vera Rubin 与 Jalapeño 势均力敌,每美元产出的输出 token 数几乎相同。但如前所述,**Jalapeño 的结果是在不使用投机解码的前提下取得的**,而 Vera Rubin 的结果用了投机解码。投机解码可以把单 token 成本降低约 3 到 5 倍。等 Jalapeño 用上投机解码之后,它的 token 服务成本还会更低。当然,这部分 TCO 优势来自用 Broadcom 相对更低(尽管依然很高)的毛利替换了 NVIDIA 的高毛利,但这并不是全部原因。举例来说,Meta 和 Microsoft 的 AI ASIC 项目投入时间更久却始终没能真正跑起来,这说明成本只是方程式的一部分。Jalapeño 的完整 TCO 拆解见 [SemiAnalysis AI Cloud TCO 模型](https://semianalysis.com/ai-cloud-tco-model/)。 + +
+ +在架构上,OpenAI 选择不把 prefill 与 decode(PD)分离到不同的芯片池。draft 模型与主模型共用同一批芯片和同一张网络,这种设计理念用一部分理论效率换取了运维上的实用性。理由在于工作负载的构成会随时间变化:随着我们走过三个模型时代([知识、推理和 agentic,见我们近期的文章](https://newsletter.semianalysis.com/p/are-open-models-catching-up)),输入 token、缓存写入、缓存读取与输出 token 之间的比例已经发生了显著变化。因此,事先固定异构的 prefill 硅片与 decode 硅片比例,长期来看会带来低效。OpenAI 在这套架构中选择了同构资源池,并努力让芯片在所有场景下都表现良好。 + +事实也确实如此。在 Kimi K2.5(Cursor Composer 2.5 基于该模型)上,Jalapeño 达到接近 700 tok/s/user,在 100 tok/s/user 处更是次优芯片的 9 倍以上。 + +
+ +在 GPT-OSS 上又是一边倒。Jalapeño 在等交互性下的每兆瓦吞吐量接近 GB200 最高吞吐点的两倍,是 GB200 并发为 1 时的 50 倍以上。Jalapeño 并发较高的那些点使用了 EP8。 + +
+ +这些结果令人印象深刻!不过我们还是要挑刺:它们只是 8k1k,是一个明显更容易调优的工作负载,而且目前还没有 [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) 运行结果。正如我们在 [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) 文章中所说,多轮长上下文工作负载会给服务栈的更多环节施加压力,例如 router 和前缀缓存。要在 agentic 工作负载上取得优异表现,还需要大量优化。更多内容请见 [AgentX](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) 文章。 + +[AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing?](https://newsletter.semianalysis.com/p/agentx-inferencexv3-does-cuda-moat) — 自 2025 年 11 月的 Claude Code 拐点以来,长上下文、多轮的 agentic 工作负载快速增长,如今已经主导生产推理的流量。2026 年 4 月,OpenAI 的企业级 agentic 支出超过了 ChatGPT 支出。 + +## 深入规格与架构 + +上述所有结果都是在 Jalapeño 的 A0 步进上取得的,距离项目启动仅 9 个月。而 B0 步进已经在 fab 中流片,其优化相比早期的 A0 硅片带来约 25% 的每瓦性能提升。具体来说,B0 步进在采用台积电 N3P 工艺、单颗满掩模尺寸的计算 die 上提供 13.4 PFLOPs 的 MXFP4 算力。作为对比,尺寸相近、同一工艺节点的单颗 Rubin 计算 die 提供 17.5 PFLOPs 的稠密 NVFP4 算力。 + +考虑到 Jalapeño 的 TDP 只有 700W,而 Rubin 每颗计算 die 为 900 至 1,150W,这个数字就相当可观了。由于 Jalapeño 面向推理而非训练,OpenAI 不需要把 TDP 推高来榨取 FLOPs,这可以理解;但无论如何,上述数据说明 Jalapeño 的峰值理论算力是拿得出手的。 + +与其他加速器直接对比时,Jalapeño 拥有最高的每瓦 HBM 带宽和最高的每瓦 FLOPs,与 1,800W 的 Rubin Max-Q 配置相当: + +
+ +片外 I/O 由一颗 N3E I/O chiplet 提供,具备 32 条 800G SerDes 通道用于计算网络,其中 24 条(600GB/s)用于机架内的本地 scale-up,8 条(200GB/s)用于全局 scale-up,即 2,048 颗 XPU 的多机架域。系统 I/O 使用 PCIe Gen 5 连接 x86 主机 CPU。 + +Jalapeño 将搭载 HBM4,使它成为继 NVIDIA 和 AMD 之后相对较早的采用者,甚至领先于已经成熟的 TPU 和 Trainium 项目。既然 Jalapeño 的核心架构原则之一是把 HBM 带宽用到极致,那么选择次优的 HBM 就与这一目标背道而驰。最终它实现了每封装 15.4TB/s 的显存带宽,超过所有在售的 HBM3E 加速器。15.4TB/s 说明其 HBM4 达到了 10Gbps 的引脚速率,略高于 NVIDIA 在 Rubin 上跑出的 9.6Gbps。这批 HBM 很可能由三星提供。 + +
+ +OpenAI 在 2025 年 11 月完成了 Jalapeño 的流片,更准确地说,这是 CoWoS 设计的流片,而不只是顶层 die 的硅片流片。在那次流片之后的 9 个月内、且只有 3 个月的实际硅片 bring-up 时间里,OpenAI 就用 Jalapeño 拿出了非常好的结果。考虑到团队的软件栈是从零起步,这就更令人印象深刻了。 + +与之相对,Rubin 的 CoWoS 流片在 2025 年 10 月完成,早了一个月,但我们目前看到的早期结果只有 CoreWeave 的工程样片。NVIDIA 并没有像 OpenAI 这样让我们测试并发布基准结果,这说明他们的芯片软件仍不成熟。考虑到 OpenAI 在自家硅片上 bring-up 新模型的速度,CUDA 护城河有可能已经不复存在。 + +它们都还远未优化到位,但总体上可以看到 Jalapeño 交出了更好的数据。我们并不认为 NVIDIA 的硬件更差,更多是 Jalapeño 的软件 bring-up 推进得比 NVIDIA 更快。这体现了软硬件协同设计的威力,而这正是一支能打的前沿实验室 ASIC 团队相比更成熟的商用芯片厂商最可能胜出的地方。反直觉的是,从零开始或许也帮了 OpenAI:他们可以做出干净利落的架构决策,而不必顾虑向后兼容或旧版软件。 + +虽然 OpenAI 目前只有 Jalapeño 的工程样片,但量产计划将在 2027 年逐步爬坡,大部分产出目前排在明年年底。[关于出货量与 ASP 的更多细节,请见 SemiAnalysis 加速器模型](https://semianalysis.com/accelerator-hbm-model/)。 + +可以说,OpenAI Jalapeno 是一款真正会大批量出货的 ASIC。 + +与 Rubin 的时间线相比,Jalapeño 的推进速度快得惊人。如前所示,尽管 Rubin 起步更早,Jalapeño 的结果依然胜出。 + +
+ +## Jalapeno 架构 + +接下来深入架构。这颗芯片的矩阵引擎采用 MXFP 数值格式和权重驻留(weight stationary)的脉动阵列,与 TPU 类似。但与 TPU 直接对比时,它支持更小的形状与维度,这意味着不会像更大的脉动阵列那样,在遇到形状不规整的矩阵乘法时出现奇怪的性能悬崖。 + +它还配有 64 位标量核心以及 FP32/INT32 向量核心。OpenAI 在托盘级别投入了冗余设计,并在核心与通道级别内置了良率收割机制。他们表示,AI 辅助设计在设计阶段使 SIMD 面积减少 8%、矩阵引擎面积减少 10%。虽然没有说明具体的工艺、电压、温度(PVT)条件,但他们也提到 AI 辅助生成的模块在时序和功耗上都优于最初版本。 + +Jalapeño 的架构设计重点在于消除 KVCache 与权重的内存搬运,以及固定延迟和各类开销,从而即便在小 batch 或小形状下,也能比其他加速器更接近原始峰值算力与带宽。 + +核心与 HBM 被划分为多个 slice,每个核心 slice 对自己那一份 HBM 具有低延迟的本地视图。slice 之间的同步走一张专用的高带宽集合通信网络。这种极简的内存层级结构,已经让 Jalapeño 相对 GPU 具备了很大的潜在优势:在 GPU 上,内存访问必须穿过复杂的内存系统,产生的高延迟只能靠更大的形状去摊薄或掩盖。 + +之所以可行,是因为只要仔细安排权重和 KV 的放置,核心之间的同步就可以被限制在有限且已知的高带宽通信上,例如可以与计算重叠的张量并行通信。 + +
+ +此外还有一张通用 NoC,用于常规通信以及访问 scale-up 网络。总体而言,相比 NVIDIA 和 Google,OpenAI 依靠简化的 NoC 与内存子系统节省了大量功耗,并获得了可观的性能提升。 + +
+ +在核心层面,OpenAI 描述的是带 L1 缓存的乱序(OoO)核心。这与我们在其他加速器上看到的模式差异很大:后者普遍采用软件管理的 scratchpad,并搭配某种异步 DMA 支持。同样,这里的论点是它能让 Jalapeño 避免诸如 barrier 延迟这类固定开销;而在 GPU 等其他加速器上,这些开销必须靠给每个核心分配更多工作来掩盖或摊薄,也因此更难逼近原始的峰值带宽与算力。 + +代价是 Jalapeño 因而更依赖良好的预取来保证内存请求及时到达,而这一点可预测性更差、也更难推理。不过,只要 Codex 处在一个好的 harness 中并能拿到详细 trace,为给定形状找到预取最优的 kernel 很可能几乎不需要人工介入。我们认为 OpenAI 正是这样做的,才得以如此迅速地跑通 DeepSeek R1、Kimi K2.5 和 GPT-OSS。 + +这些核心还支持「小」矩阵维度。视其小到什么程度,这应当能让它在不同模型和 batch 维度上更通用,也更不易受矩阵维度对齐、padding 开销和 tiling 低效的影响。相比之下,TPU、Trainium 和 Etched 的芯片都采用很大的脉动阵列,往往需要较大 batch 或恰好可整除的模型维度,才能避免 tiling 带来的浪费。 + +在 Jalapeño 上,OpenAI 专注于消除系统中的固定延迟,以便在 Pareto 曲线的各个区间都尽可能逼近 roofline 性能。理论上,这可能在多个运行点上带来相对 GPU 的优势: + +- 在低延迟 / 小 batch 推理上有高得多的性能上界,而 GPU 在这里受制于诸多固定开销,例如启动延迟、barrier 延迟和内存系统延迟 +- 在大 batch 或长上下文场景下,也有一定机会更接近硬件 roofline + +需要说明的是,即便理论上存在这样的性能上界,对真实 kernel 而言未必容易兑现。因此这套思路大致是: + +1. 面向所有工作负载形状,设计出尽可能高的性能上界 +2. 让 Codex 去做那些繁琐的工作,找出能够逼近该上界的 kernel + +从 OpenAI 团队在 Jalapeño 上跑通 InferenceX 工作负载的极快速度来看,我们对这条路线持乐观态度。 + +如果 Jalapeño 成功,那将是一个强烈信号:业界对编程模型和完美通用编译器的执念,正在被前沿 AI 模型所瓦解。 + +## 软件 + +OpenAI 写 Jalapeño 的 kernel 就像写汇编。每个 kernel 都是手工调优的代码,有些长达约 3,000 行,并配有正确性检查和自研 sanitizer。早期的 kernel 工作以人在回路为主而非完全自动化,但随着内部一个规模更大的 Codex 版本投入使用,这一点发生了变化——OpenAI 计划把该版本推向企业客户。内部的服务引擎名为「Teacup」。有意思的是,**在用 InferenceX 对 DeepSeek 做基准测试之前,OpenAI 内部并没有 MLA kernel 的实现**。Codex 能在 OpenAI kernel 工程团队完全不介入的情况下这么快写出可用且高效的 kernel,足见其软件流水线的开发能力。 + +OpenAI 用 Gluon 为 Jalapeño 编程。Gluon 是 OpenAI 的 kernel 编程语言,构建在 Triton 之上,保留了 Triton 的 SPMD(单程序多数据)编程模型,同时暴露出**更底层的编程抽象**。例如在 NVIDIA GPU 上,它提供映射到 PTX 指令的 API,包括 MMA 指令、TMA 指令、mbarrier 机制等等。Gluon 最独特的抽象是 **layout**。一般来说,layout 定义的是硬件资源(例如 warp 9 的第 5 个寄存器)与张量元素(例如第 6 行第 7 列的元素)之间的映射。Gluon 的 layout 抽象基于 [Linear Layouts](https://arxiv.org/abs/2505.23819),这是 OpenAI 提出的一套 layout 代数,用数学方式形式化了 layout 的定义,并提供了在 layout 上做运算的工具。由此可以实现许多能力,例如可证明正确的 layout 转换和最优的内存 swizzling。 + +就 Jalapeño 的编程模型而言,每个 Gluon 程序对应一个常驻线程。我们认为这暗示 Jalapeño 适合**常驻 kernel 编程模式**:每个程序处理多个 tile,且由程序员而非硬件调度器来分配工作。OpenAI 提到了 TensorInfo,这是一种显式编码 layout 的抽象,很可能就是为 Jalapeño 设计、并由 Linear Layouts 驱动的那套 layout。最后,每个核心都提供数据预取和解耦的乱序单元,例如用户可以对已预取的数据编写等待逻辑,由信号量控制解锁。 + +命运的奇妙之处在于:像 GPT 5.6 Sol 这样目前跑在 NVIDIA GPU 上的 OpenAI 模型,被用来设计了一颗真正威胁 CUDA 护城河的芯片——NVIDIA 自己的 GPU 正在实时地帮助催生它潜在的接班者。 + +把时间轴拉开看,还能看到 Jalapeño 的开发节奏:在不到两周的时间里,某些交互性区间的吞吐量提升超过 2 倍。Jalapeño 团队每次发来的压缩包里都藏着惊喜。 + +
+ +不只是 kernel 性能提升。在 8 天之内,Jalapeño 团队在此前 TP8 配置的基础上启用了 TP32,从单机扩展到完整的机架级配置来运行大模型。这样的开发速度确实令人印象深刻。 + +
+ +为了在投入真实硬件运行之前验证性能,OpenAI 还有一个名为「chilisim」的模拟器,采用定宽 trace 总线,与实测硬件的误差在 5% 以内。A0 上的 tracing 能力有限,但在 B0 上有了大幅改进,很可能吸收了 A0 实际运行的反馈。工程师还演示了 Codex CLI 运行一个内部模型(代号「Raiku」或「5.3 Codex Spark」),TPOT 为 1.2ms。 + +团队还展示了由 Codex 编写、直接在芯片上运行的演示:36 FPS 的 Doom、一个 FP32 流体力学模拟,以及一个名为「Liquid Light」的鼠标拖拽可视化。 + +
+ +在模型侧,OpenAI 内部的 megakernel 方案(代号「gigakernel」)围绕单个 megakernel 构建,在设备上循环执行,以减少 CPU 开销和启动时间。团队还在进一步押注测试时计算策略,内部尤其关注如何连贯地使用 100 万次 rollout。 + +## 分离还是不分离,这是个问题 + +前面提到,OpenAI 在这些芯片上没有使用 prefill decode 分离。这让我们相当意外,因为 NVIDIA 和 AMD 的 GPU 即便在同构硬件上也能从 PDD 中获得显著收益。下面我们来看看 Jalapeño 团队为什么这样选。 + +当工作负载固定不变时,prefill-decode 分离(PDD)看起来很有吸引力。prefill 与 decode 对硬件的压力方式不同,因此把两个阶段分别放到各自调优过的资源池中,可以在某个选定的输入输出比例下提升效率。但生产流量并不会停在那个比例上:输入与输出序列长度、并发、缓存命中率、投机接受率和延迟目标,都会在一天之内不断变化。 + +一旦设备被划分为 prefill 池和 decode 池,prefill 需求过多时 decode 芯片就会闲置而请求排队;decode 需求过多时则反过来。运营方必须持续预测正确的比例、在两侧都预留冗余容量,并不断重新平衡一个理想比例始终在移动的系统。 + +在统一的系统中,某些资源在特定阶段可能利用不足,但每台设备都随时可以服务下一条请求。而在分离式系统中,一颗芯片可能仅仅因为归属于错误的资源池就完全闲置:局部利用率看起来更好,全局利用率却可能很差。 + +
+ +分离还会破坏局部性。prefill worker 产生的大量 KV cache 是 decode worker 立刻需要的,因此系统必须先把这份状态通过网络传过去,生成才能继续。这会带来额外的带宽消耗、同步、排队,以及又一个故障域。开销还会随输入序列长度上升,因为 KV cache 会变大。不过,避免搬运 KV 主要是一项功耗与延迟优化;愿意搬运一部分 KV,可以用一些功耗和单请求延迟换来更高的硬件利用率。 + +
+ +可调配的统一集群能够在延迟敏感请求和吞吐导向的批处理之间转移容量,而固定的划分方式则会在流量构成变化时让硬件被闲置。此外,上下文长度会改变注意力与 FFN 之间的工作量平衡,使任何固定的硬件比例都只在其设计点附近才高效。 + +
+ +同样的约束也适用于投机解码。draft 模型必须以极低延迟把候选 token 送给验证方。把两者拆到各自专用的资源池中,会把一个紧耦合的解码循环变成分布式协议,额外的通信与协调开销可能把起草省下的延迟又吃回去。让两个模型留在同一批设备和同一张低延迟网络上,才能保住让投机解码值得做的那份局部性。 + +
+ +不过,当需求足够大、足够稳定且可预测时,分离仍然可以胜出,尤其是当常规 GPU 需要较大的分阶段 batch 才能获得良好吞吐时。但它并不是免费的午餐。 + +## 从日式到印式(由淡到辣):Katsu、Vindaloo 与 Chana —— 这些咖喱菜如何组成一套机架系统 + +Jalapeño 系统在机架层面由一个 CPU 主机机架和一个 ASIC 机架组成。主机机架容纳 16 个名为「Katsu」的主机 CPU 托盘,每个对应右侧一个名为「Vindaloo」的 ASIC 托盘。每台主机配备两颗 Turin 级 AMD EPYC CPU 和 1.5TB DRAM,每机架配 2 个 E1.S 和 2 个 M.2 SSD。每个托盘还配有 400G(2x200G)前端网络。每个 Katsu 托盘通过 8 条外部 PCIe DAC 线缆与对应的 Vindaloo 托盘相连,这些线缆在机架前部横向走线。系统级设计由 Celestica 合作完成。 + +ASIC 机架由 16 个 Vindaloo 托盘和 8 个名为「Chana」的 scale-up 交换托盘(6 个用于本地域、2 个用于全局域)组成。每个 Vindaloo 托盘装有 8 颗 Jalapeño ASIC,每机架合计 128 颗。ASIC 通过铜缆背板连接到各个 Chana 交换托盘,与 NVIDIA 的 Oberon 类似。scale-up 拓扑分为机架内 128 颗 ASIC 的本地域,以及最多连接 16 个机架、2,048 颗 ASIC 的全局域。带宽与拓扑细节见下文。 + +边车式主机机架的供电约为 50kW 规划容量(生产环境 31kW),ASIC 机架为 130kW,两机架系统合计约 160kW,基本相当于一个双倍宽度的 GB300 机架的功耗。 + +
+ +OpenAI 可以在单个 scale-up 网络内连接多达 2,048 颗 Jalapeño XPU。该 scale-up 网络由两个域组成:本地域通过背板连接机架内全部 128 颗 XPU;全局域采用铜缆与光互连混合方案,跨 16 个机架连接 2,048 颗 XPU。每个机架有 8 个 Chana 交换托盘,中间 6 个用于本地域,各配一颗 102.4T 的 Tomahawk 6 交换 ASIC;位于本地交换托盘上下方的 2 个用于全局域,我们认为它们可能各由 2 颗 102.4T Tomahawk 6 组成,单托盘可达 204.8T。 + +在本地域中,128 颗 Jalapeño 芯片每颗具备 4.8Tb/s 的单向带宽,以 all-to-all 方式连接到 6 颗 102.4Tb/s 的 Tomahawk 6 ASIC。这相当于每颗 XPU 需要 48 对差分(DP)公母连接器,每机架用于本地 scale-up 的无源铜缆合计 6,144 DP。 + +在全局域中,16 个机架共 2,048 颗 XPU 通过铜缆背板、204.8T TH6 电交换、1.6T 光模块和光路交换的组合连接在一起。每颗 XPU 的全局链路具备 1.6Tb/s 单向带宽,对应 XPU 与全局交换之间背板上每颗 XPU 16 对差分连接器。每个全局交换托盘(含 2 颗 ASIC)的出向带宽在背板与前面板光口之间分配。 + +把本地域与全局域合并计算,每机架每颗 XPU 的背板连接器数量为 64 DP,全机架用于无源铜缆的连接器合计 8,192 DP。 + +全局域采用 rail-only 架构,跨全局域共 8 条 rail。我们认为 OpenAI 通过安装在每个机架中的光路交换机(OCS)来疏导全局域的光链路。对每颗 XPU 而言,1.6Tb/s 的全局带宽先经铜缆背板到达全局交换托盘,再通过前面板的 1.6T 光模块出口,进入无源光交换后离开机架。由此把 scale-up world size 扩展到 16 个机架、每机架 128 颗、合计 2,048 颗 XPU。 + +
+ +由于 scale-up 网络只占系统总成本约 10%,这种灵活性为未来 10 到 20 万亿参数的模型、或 200 万到 400 万 token 的上下文窗口买到了宝贵的可选性。在部署方面,OpenAI 正与 neocloud 合作,并在与数据中心伙伴一起采集可靠性数据直到 1 月,同时优化从到货到上架的部署时间。 + +## 下一步 + +接下来我们谈谈 Jalapeño 的未来,它的第一个生产 token 即将到来。下一个目标是 100MW,而障碍主要在硬件侧:能造出多少、数据中心能部署和运营得多好、监控与韧性如何处理等等。软件已经得到验证,而有了内部模型,任何软件上的先发优势都很容易被追上。付费部分我们会讨论这对 NVIDIA、AMD、Cerebras,以及未来几年与 OpenAI 签约的其他芯片公司意味着什么。 + +[我们在加速器模型中还覆盖了下一代芯片的产量、出货单位与时间线](https://semianalysis.com/accelerator-hbm-model/)。 + +## 对 NVIDIA、AMD 和 Cerebras 的直接影响 + + + +Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. + + + +--- + +_本文完整版发布在我们的 Substack 上。[订阅 SemiAnalysis](https://newsletter.semianalysis.com/subscribe) 阅读完整文章。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "什么是 OpenAI Jalapeño?", + "acceptedAnswer": { + "@type": "Answer", + "text": "Jalapeño 是 OpenAI 首款自研推理 ASIC,与 Broadcom 合作、从零开始专为 LLM 推理设计,在 Hot Chips 上正式发布。设计工作始于 2024 年年中,CoWoS 设计于 2025 年 11 月流片,从组建团队到流片约 16 个月。目前在 fab 中的 B0 步进采用台积电 N3P 工艺,在单颗满掩模尺寸的计算 die 上提供 13.4 PFLOPs 的 MXFP4 算力,TDP 为 700W,并通过 HBM4 实现每封装 15.4 TB/s 的显存带宽。" + } + }, + { + "@type": "Question", + "name": "Jalapeño 与 NVIDIA Blackwell、Vera Rubin 相比如何?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在每兆瓦全量供电的 token 吞吐量上,Jalapeño 在几乎所有场景中都胜过 Blackwell;其单 token 预测(STP)结果还超过了 NVIDIA 与 CoreWeave 在 2026 年 7 月公布的 Vera Rubin MTP 数据。在每美元 TCO 性能上,Jalapeño 与 Vera Rubin 大致势均力敌。与 Blackwell 比较的参考价值较弱,因为 Jalapeño 真正的对手是 Rubin 这类 HBM4 产品;而且 Jalapeño 取得这些结果时并未使用投机解码,NVIDIA 的对应配置则使用了。" + } + }, + { + "@type": "Question", + "name": "OpenAI 为什么没有采用 prefill-decode 分离?", + "acceptedAnswer": { + "@type": "Answer", + "text": "当工作负载构成固定时,分离很有吸引力,但生产流量并非如此:输入与输出长度、并发、缓存命中率、投机接受率和延迟目标都会在一天之内变化。一旦把芯片划分为 prefill 池和 decode 池,一颗芯片可能仅因归属错误的资源池而完全闲置,于是局部利用率提升、全局利用率反而下降。统一资源池还能保住投机解码所依赖的局部性:一旦把 draft 模型与验证方分开,紧耦合的解码循环就会变成分布式协议。" + } + }, + { + "@type": "Question", + "name": "Jalapeño 的机架是如何组成的?", + "acceptedAnswer": { + "@type": "Answer", + "text": "该系统由两个机架组成。主机机架容纳 16 个名为 Katsu 的 CPU 托盘,每个配备两颗 Turin 级 AMD EPYC CPU 和 1.5TB DRAM,通过外部 PCIe DAC 线缆连接到 ASIC 机架。ASIC 机架容纳 16 个 Vindaloo 托盘,每个装有 8 颗 Jalapeño ASIC,每机架合计 128 颗,另有 8 个基于 Tomahawk 6 的 Chana scale-up 交换托盘。scale-up 分为通过铜缆背板连接的 128 颗 ASIC 本地域,以及跨 16 个机架、连接 2,048 颗 ASIC 的全局域。两机架系统功耗约 160kW。" + } + }, + { + "@type": "Question", + "name": "Jalapeño 上运行的是什么软件栈?", + "acceptedAnswer": { + "@type": "Answer", + "text": "OpenAI 使用自研 kernel 语言 Gluon 为该芯片编程。Gluon 构建在 Triton 之上,保留 Triton 的 SPMD 模型,同时暴露更底层的抽象。它最具特色的能力是 layout 抽象,基于 OpenAI 提出的 layout 代数 Linear Layouts,可实现可证明正确的 layout 转换和最优的内存 swizzling。内部服务引擎名为 Teacup,模拟器 chilisim 与实测硬件的误差在 5% 以内,而大量 kernel 代码由内部规模更大的 Codex 版本编写。" + } + }, + { + "@type": "Question", + "name": "这些 Jalapeño 结果有哪些需要注意的前提?", + "acceptedAnswer": { + "@type": "Answer", + "text": "所有数据均由 OpenAI 提供。SemiAnalysis 在实验室现场验证了 InferenceX 的运行过程,但没有跑完整套基准测试,目前也没有 AgentX 结果。公布的运行均为 8k1k 单轮场景,比长上下文多轮流量更容易调优:agentic 工作负载会给 router、前缀缓存、缓存管理和 offload 基础设施施加压力,而单轮 8k1k 完全考察不到这些。此外,所测试的模型也不是最新的前沿开放权重模型。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx b/packages/app/content/blog/zh/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx new file mode 100644 index 000000000..08107acff --- /dev/null +++ b/packages/app/content/blog/zh/qwen3-5-397b-agentx-b300-fp4-vs-h100.mdx @@ -0,0 +1,66 @@ +--- +title: 'Qwen3.5 397B 的 AgentX 结果:B300 FP4 的每美元性能是 H100 的 12 倍' +subtitle: '在长上下文 agentic 工作负载上,四年硬件迭代加上 4 位格式能换来什么' +seoDescription: '在 Qwen3.5 397B 的 AgentX 运行中,B300 FP4 的每美元性能是 H100 的 12 倍。该对比跨越四代硬件与两种数值格式,工作负载为截断到 256k 的 agentic trace 回放。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - qwen + - b300 + - h100 + - h200 + - fp4 + - fp8 + - sglang + - nvidia +--- + +在 Qwen3.5 上,B300 FP4 的[每美元性能](/zh/glossary/performance-per-dollar)是 H100 的 12 倍。 + +
+ +## 这个数字代表什么,又不代表什么 + +这里同时有两个变量在变,读数时都要考虑。第一是四年的硬件迭代;第二是数值格式:该对比跨越 [FP4](/zh/glossary/fp4) 与 [FP8](/zh/glossary/fp8),并没有固定精度。这样比较反映的是运营方今天在各款硬件上实际能部署的方案,是诚实的口径,但它并不是只针对硅片本身的受控实验。 + +工作负载同样重要。这不是 8k1k。Qwen3.5 的原生最大上下文长度为 262k token,因此 AgentX 对它回放的是截断到 256k 的 agentic 数据集,这意味着频繁的上下文压缩和大量前缀复用。在长上下文多轮回放上测得的比值,与在互相独立的短提示词上测得的同一比值,是两种不同的结论,而前者更接近真实的生产编码流量。 + +## 请按你自己的运行点来读 + +12 倍的总体比值是一个标题,而不是部署方案。真正承载决策的是曲线:先确定你的产品需要的[交互性](/zh/glossary/interactivity),在该点读取每套系统的表现,并在同一点核对首 token 延迟,然后再下结论。 + +这也是各模型的 AgentX 图表默认展示完整前沿、而不是单个数字的原因。较老的硬件可能在低并发下于每美元性能上仍有竞争力,却在高吞吐区间完全掉队;单一比值无法同时表达这条曲线的两端。 + +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 Qwen3.5 397B 上,B300 比 H100 好多少?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 AgentX 的 Qwen3.5 运行中,B300 FP4 的每美元性能是 H100 的 12 倍。该对比同时跨越硬件代际差异和数值格式差异,因为它覆盖 FP4 与 FP8,并未固定精度。" + } + }, + { + "@type": "Question", + "name": "这个 12 倍是在定长序列工作负载上测得的吗?", + "acceptedAnswer": { + "@type": "Answer", + "text": "不是。Qwen3.5 的原生最大上下文长度为 262k token,因此 AgentX 对它回放的是截断到 256k 的 agentic trace 数据集。该工作负载包含频繁的上下文压缩和大量前缀复用,与互相独立的短提示词是完全不同的测量。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx b/packages/app/content/blog/zh/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx new file mode 100644 index 000000000..20f07bfcf --- /dev/null +++ b/packages/app/content/blog/zh/qwen3-5-397b-agentx-nvidia-vs-amd-sglang.mdx @@ -0,0 +1,67 @@ +--- +title: 'MI355X 对比 GB300 NVL72 推理性能:Qwen3.5 SGLang 上 20 倍差距' +subtitle: 'GatedDeltaNet、262k 原生上下文,以及同一引擎上 AMD 完全缺席的竞争' +seoDescription: 'Qwen3.5 397B 的 AgentX 结果。在 SGLang 对 SGLang 的比较中,NVIDIA 在 90 tok/s/user 下领先 20 倍以上,AMD 完全没有竞争力;NVIDIA 的 SGLang 提交在 p90 TTFT 上也远低于 TRT-LLM。' +date: '2026-08-25' +publishDate: '2026-08-25' +tags: + - agentx + - agentic + - benchmark + - inference + - qwen + - sglang + - trtllm + - nvidia + - amd +--- + +Qwen3.5 397B 每隔几层就用 GatedDeltaNet 替代常规注意力。GatedDeltaNet 由 MIT 与 NVIDIA Research 提出,其状态存储需求在理论上是常数,而常规注意力是线性增长的,因此相比同等规模的稠密注意力模型,它的存储需求更低。与 Nemotron 那样惨败的端到端模型训练研究不同,NVIDIA Research 在 GDN、LatentMoE 这类被前沿模型采用的基础研究上做得非常出色。 + +在看结果之前先说明一点测试方法:该模型原生最大上下文长度为 262k token,因此 AgentX 对它回放的是[截断到 256k 的数据集](https://inferencex.semianalysis.com/agentx/cc-traces-weka-062126-256k)。这模拟的是在上下文上限较小的模型上频繁触顶、频繁压缩上下文的工作负载,也就是用户实际使用这类模型的方式。 + +## 差距 + +在 SGLang 对 SGLang 的比较中,Qwen3.5 397B 是 NVIDIA 的强势领地:90 tok/s/user 时性能领先 20 倍以上。在 Qwen3.5 SGLang 上,AMD 目前完全没有竞争力。 + +
+ +固定引擎这一变量,正是这组对比值得一提的原因。跨引擎的结果总有争论空间,因为不同软件栈的成熟度不同,投入优化的人力也不同;而这里两边都是 SGLang,差距依然是 20 倍。 + +## NVIDIA 在用 TTFT 换交互性 + +在 NVIDIA 阵营内部,我们再次看到以牺牲首 token 延迟为代价过度优化[交互性](/zh/glossary/interactivity)的现象,TRT-LLM 尤其明显。上图中所有 NVIDIA SGLang 提交的 p90 TTFT 都远低于 TRT-LLM。 + +这是真实的取舍而非缺陷,但读者必须看得到它。一套让用户先等更久、之后流式输出更快的方案,在交互性坐标轴上会更好看,在首 token 的[服务级目标](/zh/glossary/service-level-objective)下则更差;只有两者一起读,才知道哪一种适合你的产品。 + +
+ +--- + +_本文结果只是 AgentX 1.0 的一个切面。完整分析、回放方法,以及该基准测试推动的 70 多个上游 PR,请见 [AgentX - InferenceXv3:CUDA 护城河在 agentic 推理中还站得住吗?](/zh/blog/agentx-inferencexv3-does-cuda-moat)。每个数据点都可以在[免费仪表板](https://inferencex.semianalysis.com/)上自行探索。_ + +{`{ + "@context": "https://schema.org", + "@type": "FAQPage", + "mainEntity": [ + { + "@type": "Question", + "name": "在 Qwen3.5 397B 上,NVIDIA 比 AMD 快多少?", + "acceptedAnswer": { + "@type": "Answer", + "text": "在 SGLang 对 SGLang 的比较中,NVIDIA 在 90 tok/s/user 下领先 20 倍以上;在 Qwen3.5 SGLang 上,AMD 目前完全没有竞争力。两边都固定使用同一引擎,正是这组对比有意义的前提。" + } + }, + { + "@type": "Question", + "name": "AgentX 为什么对 Qwen3.5 回放截断后的数据集?", + "acceptedAnswer": { + "@type": "Answer", + "text": "该模型原生最大上下文长度为 262k token,因此 AgentX 使用截断到 256k 的上下文数据集。这模拟的是在上下文上限较小的模型上频繁触顶、频繁压缩上下文的工作负载,也就是用户实际使用这类模型的方式。" + } + } + ] +}`} diff --git a/packages/app/content/blog/zh/ultra-high-interactivity-on-nvidia.mdx b/packages/app/content/blog/zh/ultra-high-interactivity-on-nvidia.mdx index 041ee64b3..45cdc7a95 100644 --- a/packages/app/content/blog/zh/ultra-high-interactivity-on-nvidia.mdx +++ b/packages/app/content/blog/zh/ultra-high-interactivity-on-nvidia.mdx @@ -3,6 +3,7 @@ title: '在 NVIDIA GPU 上实现超高交互性?TileRT 登陆 InferenceX' subtitle: '运行在 NVIDIA GPU 上的 TileRT 软件能否与 Cerebras、Groq LPU、SambaNova 竞争?批大小为 1、分离式引擎、高吞吐量预填充引擎、高交互性解码引擎' date: '2026-08-10' publishDate: '2026-08-10' +modifiedDate: '2026-08-18' tags: - benchmark - gpu @@ -13,6 +14,8 @@ tags: - tilert - vllm - glm5 + - agentx + - agentic --- _本文最初于 2026 年 8 月 10 日发布在 [SemiAnalysis 通讯](https://newsletter.semianalysis.com/p/ultra-high-interactivity-on-nvidia)。_ @@ -235,9 +238,9 @@ TileRT 继承了 ASIC 厂商最大的弱点。静态提前编译意味着模型 AI 编码智能体能在已知模板内加速调优,但新颖的变换仍需专家判断。此外,单体式持久化 kernel 也削弱了传统逐 kernel 性能剖析时间线的作用,使自动化反馈闭环更加困难。 -## TileRT 与 InferenceX 的下一步 +## TileRT 的下一步 -我们正在积极推进,将 TileRT 的基准测试从 InferenceX 的单轮 8k/1k 场景扩展到我们新的智能体编码基准——AgentX。该场景回放真实的 Claude Code 与 Codex 轨迹,包含长上下文、多轮请求、真实的子智能体活动与动态工具调用延迟。其输入长度中位数为 140k tokens,理论缓存命中率中位数屋顶线可达 99.2%。 +InferenceX 的长上下文多轮编码场景 AgentX 现已在具备对应数据的模型与服务栈上正式上线。它回放从真实 Claude Code 与 Codex 会话衍生出的隐私保护工作负载形态,包含长上下文、多轮请求、真实的 subagent 活动与动态工具调用延迟。其输入长度中位数为 140k tokens,理论缓存命中率中位数屋顶线可达 99.2%。对 TileRT 而言,下一步仍是将目前的单轮 8k/1k 基准测试扩展到 AgentX。
-该工作负载将检验 TileRT 与 vLLM 的整套系统,而不仅是解码速度,涵盖增量 KV 传输、前缀缓存复用、缓存保留与卸载、路由与调度。关键问题在于:TileRT 能否在轮次之间只传输新增上下文,同时保持其超高交互性优势。 +该工作负载将检验 TileRT 与 vLLM 的整套系统,而不仅是解码速度,涵盖增量 KV 传输、前缀缓存复用、缓存保留与 offload、路由与调度。关键问题在于:TileRT 能否在轮次之间只传输新增上下文,同时保持其超高交互性优势。
\n` to force thinking mode; the official web/app temperature is 0.6 ([DeepSeek-R1-0528 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). + +The underlying technique lineage is DeepSeek-R1's reinforcement-learning recipe: the R1 paper argues that LLM reasoning ability can be incentivized through pure RL without human-labeled reasoning trajectories, producing emergent self-reflection, verification and dynamic strategy adaptation ([DeepSeek-R1 paper, arXiv:2501.12948](https://arxiv.org/abs/2501.12948)). R1 (671B total / 37B active MoE) is trained on top of DeepSeek-V3-Base ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1)), whose architecture adopts Multi-head Latent Attention (MLA) and DeepSeekMoE with an auxiliary-loss-free load-balancing strategy and a multi-token-prediction objective usable for speculative decoding ([DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3)). Both the code repository and model use are under the MIT License, and the R1 series supports commercial use and distillation ([DeepSeek-R1-0528 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). DeepSeek also distilled R1-0528's chain-of-thought into Qwen3-8B Base to produce DeepSeek-R1-0528-Qwen3-8B ([DeepSeek-R1-0528 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). + +## Architecture + +- Total parameters: 671B; activated per token: 37B (DeepSeek-R1 series table) ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1)); the Hugging Face repo metadata lists 685B params, which for DeepSeek-V3 is explained as 671B main weights + 14B MTP module weights ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1), [DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3)) +- Architecture family: Mixture-of-Experts; `architectures: ["DeepseekV3ForCausalLM"]`, `model_type: deepseek_v3` ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1), [config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Base model: DeepSeek-R1 and R1-Zero are trained based on DeepSeek-V3-Base ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1)) +- Attention: Multi-head Latent Attention (MLA), adopted from the DeepSeek-V3/V2 architecture ([DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3)); the R1-0528 config exposes MLA ranks `kv_lora_rank: 512`, `q_lora_rank: 1536`, `qk_nope_head_dim: 128`, `qk_rope_head_dim: 64`, `v_head_dim: 128` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)). Note: the R1-0528 card itself does not name the attention mechanism ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)) +- Layers: `num_hidden_layers: 61`, with `first_k_dense_replace: 3` (first three layers dense rather than MoE) and `num_nextn_predict_layers: 1` (MTP block) ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Attention heads: `num_attention_heads: 128`, `num_key_value_heads: 128`, `hidden_size: 7168`, dense `intermediate_size: 18432` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- MoE experts: `n_routed_experts: 256`, `n_shared_experts: 1`, `num_experts_per_tok: 8`, `moe_intermediate_size: 2048`, `moe_layer_freq: 1`, grouped routing with `n_group: 8` / `topk_group: 4`, `scoring_func: sigmoid`, `topk_method: noaux_tc`, `routed_scaling_factor: 2.5` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Load balancing: auxiliary-loss-free load-balancing strategy (DeepSeek-V3 lineage; `noaux_tc` in config) ([DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3), [config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Context window: the DeepSeek-R1 series table lists 128K ([DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1)); the R1-0528 config sets `max_position_embeddings: 163840` with YaRN rope scaling (`factor: 40` over `original_max_position_embeddings: 4096`, `rope_theta: 10000`) ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Precision / quantization: `torch_dtype: bfloat16` with an FP8 quantization config (`quant_method: fp8`, `fmt: e4m3`, dynamic activation scheme, `weight_block_size: [128,128]`); repo tensor types for R1 are listed as BF16, F8_E4M3, F32 ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json), [DeepSeek-R1 model card](https://huggingface.co/deepseek-ai/DeepSeek-R1)) +- Vocabulary: `vocab_size: 129280`, `hidden_act: silu`, `rms_norm_eps: 1e-06`, `tie_word_embeddings: false` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Multi-token prediction: MTP training objective from the V3 lineage, also usable for speculative decoding ([DeepSeek-V3 model card](https://huggingface.co/deepseek-ai/DeepSeek-V3)); `num_nextn_predict_layers: 1` in the R1-0528 config ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)) +- Reasoning budget: maximum generation length used in DeepSeek's own evals is 64K tokens; average AIME reasoning length ~23K tokens per question (up from ~12K in R1) ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)) +- Distilled sibling: DeepSeek-R1-0528-Qwen3-8B has an architecture identical to Qwen3-8B but shares R1-0528's tokenizer configuration ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)) +- License: MIT (`license: mit` in the card front matter); commercial use and distillation supported ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)) + +## Official vendor eval scores + +All values are DeepSeek-reported for DeepSeek-R1-0528 on its official Hugging Face model card. Eval settings: max generation length 64K, temperature 0.6, top-p 0.95, 16 samples per query for pass@1; SWE-Verified uses the Agentless framework; HLE is evaluated on text-only prompts; GPT-4.1 plays the user role in Tau-bench ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| -------------------------------- | ------------------------------------------------------------------------------- | --------------- | -------- | --------------------------------------------------------------------------------------- | +| MMLU-Pro (EM) | 85.0 | 86.8 | -2.1% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| MMLU-Redux (EM) | 93.4 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| GPQA-Diamond (Pass@1) | 81.0 | 87.0 (no tools) | -6.9% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| Humanity's Last Exam (Pass@1) | 17.7 | 26.7 (no tools) | -33.7% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| SimpleQA (Correct) | 27.8 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| FRAMES (Acc.) | 83.0 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| LiveCodeBench 2408–2505 (Pass@1) | 73.3 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| Codeforces-Div1 (Rating) | 1930 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| SWE-bench Verified (Resolved) | 57.6 | 70.7 | -18.5% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| Aider-Polyglot (Acc.) | 71.6 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| AIME 2024 (Pass@1) | 91.4 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| AIME 2025 (Pass@1) | 87.5 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| HMMT 2025 (Pass@1) | 79.4 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| CNMO 2024 (Pass@1) | 86.9 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| BFCL_v3_MultiTurn (Acc) | 37.0 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| Tau-Bench (Pass@1) | 53.5 (Airline) / 63.9 (Retail) | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| Terminal-Bench | n.a. — not reported by DeepSeek for R1-0528 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| tau2-bench | n.a. — DeepSeek reports the original Tau-Bench (Airline/Retail), not tau2-bench | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | +| BrowseComp | n.a. — not reported by DeepSeek for R1-0528 | 44.4 | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. + +## Benchmark explanations + +- **MMLU-Pro** — a harder, cleaned multi-subject multiple-choice exam suite measuring broad academic and professional knowledge and reasoning, scored as exact match ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **MMLU-Redux** — a relabeled/error-corrected version of MMLU, measuring the same broad knowledge with cleaner ground truth ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **GPQA-Diamond** — the hardest subset of graduate-level physics, chemistry and biology questions written to resist simple web lookup; measures expert scientific reasoning ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **Humanity's Last Exam (HLE)** — an extremely difficult expert-written exam across many disciplines used as a frontier-difficulty ceiling; DeepSeek evaluated text-only prompts ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **SimpleQA** — short factual questions testing parametric factual recall and calibration; measures the share answered correctly ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **FRAMES** — multi-document retrieval-and-reasoning questions requiring synthesis of several sources into one factual answer ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **LiveCodeBench (2408–2505)** — competitive-programming problems drawn from contests in a fixed recent window to limit contamination; measures pass@1 code correctness ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **Codeforces-Div1** — an Elo-style rating estimated from Division-1 competitive-programming contest performance rather than a percentage ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **SWE-bench Verified** — human-validated real GitHub issues where the model must generate a patch passing the repo's tests; evaluated here with the Agentless framework ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **Aider-Polyglot** — multi-language code-editing tasks in the Aider harness, measuring the ability to apply correct edits across programming languages ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **AIME 2024 / AIME 2025** — American Invitational Mathematics Examination problem sets: hard short-answer competition math, a core reasoning-depth test ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **HMMT 2025** — the Harvard-MIT Math Tournament 2025 problem set, olympiad-style short-answer math ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **CNMO 2024** — the 2024 Chinese National High School Mathematics Olympiad, measuring olympiad math in a Chinese-language setting ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **BFCL_v3_MultiTurn** — the Berkeley Function-Calling Leaderboard v3 multi-turn split, measuring correct tool/function invocation across a dialogue ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **Tau-Bench (Airline / Retail)** — simulated customer-service agent tasks requiring tool use and policy adherence over multi-turn dialogue; GPT-4.1 played the user ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **Terminal-Bench** — multi-step shell/terminal agent tasks; not reported by DeepSeek for R1-0528 ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **tau2-bench** — the successor to Tau-Bench with additional domains and a dual-control user simulator; not reported by DeepSeek for R1-0528 ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). +- **BrowseComp** — hard-to-find information-seeking questions requiring persistent web browsing; not reported by DeepSeek for R1-0528 ([model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/README.md)). diff --git a/packages/app/content/models/deepseek-v4.mdx b/packages/app/content/models/deepseek-v4.mdx new file mode 100644 index 000000000..c4bcc7075 --- /dev/null +++ b/packages/app/content/models/deepseek-v4.mdx @@ -0,0 +1,84 @@ +--- +title: 'DeepSeek V4 Pro' +developer: 'DeepSeek' +releaseDate: 'April 24, 2026 (preview); August 13, 2026 (V4-Pro-0813 GA)' +description: "DeepSeek's 1.6T-parameter / 49B-active MoE flagship with hybrid CSA+HCA sparse attention, a 1M-token context window, and MIT-licensed open weights." +--- + +## Overview + +DeepSeek-V4-Pro is the flagship open-weights Mixture-of-Experts (MoE) language model of the DeepSeek-V4 series from DeepSeek-AI, with 1.6T total parameters and 49B activated per token and a native 1M-token context length ([DeepSeek-V4-Pro model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). It was announced as "DeepSeek-V4 Preview … officially live & open-sourced" alongside the smaller DeepSeek-V4-Flash (284B total / 13B active) on DeepSeek's own API news page dated April 24, 2026, which also states that 1M context is the default across all official DeepSeek services and that both models support dual Thinking / Non-Thinking modes ([DeepSeek V4 Preview Release](https://api-docs.deepseek.com/news/news260424)). The accompanying technical report, "DeepSeek-V4: Towards Highly Efficient Million-Token …", was submitted to arXiv on 26 Apr 2026 and presents the two MoE models as a preview of the V4 series ([arXiv:2606.19348](https://arxiv.org/abs/2606.19348)). The weights are released under the MIT License ([DeepSeek-V4-Pro model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). + +Positioning is explicitly agentic and coding-first: DeepSeek describes V4-Pro as having enhanced agentic capabilities, as open-source SOTA on agentic-coding benchmarks, as leading all current open models while trailing only Gemini-3.1-Pro, and as being used for in-house agentic coding at DeepSeek, with integrations for Claude Code, OpenClaw and OpenCode ([DeepSeek V4 Preview Release](https://api-docs.deepseek.com/news/news260424)). The maximum reasoning-effort mode, DeepSeek-V4-Pro-Max, is described in the technical report as redefining the state of the art for open models ([arXiv:2606.19348](https://arxiv.org/abs/2606.19348)). + +The headline techniques are efficiency-oriented. V4-Pro uses a hybrid attention architecture combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), Manifold-Constrained Hyper-Connections (mHC) in place of conventional residual connections, and the Muon optimizer, pre-trained on more than 32T tokens; in the 1M-token setting it needs 27% of the single-token inference FLOPs and 10% of the KV cache of DeepSeek-V3.2 ([DeepSeek-V4-Pro model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)). A production checkpoint, DeepSeek-V4-Pro-0813, superseded the preview: it is built on the preview structure with a DSpark speculative decoding module attached and reports 1.7T params, with "greatly enhanced agentic capabilities" ([DeepSeek-V4-Pro-0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)); DeepSeek's changelog dated 2026-08-13 states the GA release rolled out on app, web and API with low/high/max reasoning effort and native OpenAI Responses API support ([DeepSeek-V4-Pro GA Release](https://api-docs.deepseek.com/news/news260813/), [DeepSeek API Change Log](https://api-docs.deepseek.com/updates/)). + +## Architecture + +- Total parameters: 1.6T (preview checkpoint) ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)); the GA card lists 1.7T params for DeepSeek-V4-Pro-0813 ([0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)) +- Activated parameters per token: 49B ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)) +- Architecture family: Mixture-of-Experts decoder LM, `DeepseekV4ForCausalLM`, `model_type: deepseek_v4` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- Layers: `num_hidden_layers: 61`, plus `num_nextn_predict_layers: 1` (MTP block) ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)); the 61-layer stack is described as layers 0–1 HCA and layers 2–60 alternating CSA/HCA, with the final MTP block using sliding-window attention only ([Hugging Face DeepSeek-V4 blog](https://huggingface.co/blog/deepseekv4)) +- Attention: hybrid architecture combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)) +- CSA details: compresses KV 4× along the sequence, softmax-gated pooling with learned positional bias, an FP4 "lightning indexer" selecting top-k compressed blocks, plus a sliding-window branch for recent tokens ([Hugging Face DeepSeek-V4 blog](https://huggingface.co/blog/deepseekv4)) +- HCA details: compresses KV 128×, dense (no sparse selection) attention over compressed blocks with a sliding-window recency branch ([Hugging Face DeepSeek-V4 blog](https://huggingface.co/blog/deepseekv4)); the config's `compress_ratios` list alternates 128 and 4 across layers, with `index_topk: 1024`, `index_n_heads: 64`, `index_head_dim: 128`, `sliding_window: 128` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- Attention shape: `num_attention_heads: 128`, `num_key_value_heads: 1`, `head_dim: 512`, `q_lora_rank: 1536`, `o_lora_rank: 1024`, `qk_rope_head_dim: 64`, `hidden_size: 7168` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- MoE experts: `n_routed_experts: 384`, `n_shared_experts: 1`, `num_experts_per_tok: 6`, `moe_intermediate_size: 3072`, routing `scoring_func: sqrtsoftplus` with `topk_method: noaux_tc` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)); the model card itself does not state expert counts ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)) +- Residual connections: replaced by Manifold-Constrained Hyper-Connections (mHC) ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)); config exposes `hc_mult: 4`, `hc_sinkhorn_iters: 20`, `hc_eps: 1e-06`, `num_hash_layers: 3` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- Context window: 1M tokens; `max_position_embeddings: 1048576` with YaRN rope scaling factor 16 over `original_max_position_embeddings: 65536` ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- Precision / quantization: distributed as FP4 + FP8 Mixed — MoE expert parameters in FP4, most other parameters in FP8 (Base checkpoints are FP8 Mixed); HF tensor types listed as BF16, I64, F32, F8_E4M3, I8 ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). Config shows `quant_method: fp8`, `fmt: e4m3`, `scale_fmt: ue8m0`, `weight_block_size: [128,128]`, and the 0813 config adds `expert_dtype: fp4` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json), [0813 config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813/raw/main/config.json)) +- KV/FLOP efficiency: at 1M context, 27% of the single-token inference FLOPs and 10% of the KV cache of DeepSeek-V3.2 ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)) +- Optimizer and pre-training data: Muon optimizer; more than 32T diverse, high-quality tokens ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [arXiv:2606.19348](https://arxiv.org/abs/2606.19348)) +- Vocabulary: `vocab_size: 129280`, `hidden_act: silu`, `swiglu_limit: 10.0` ([config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/raw/main/config.json)) +- Speculative decoding (GA checkpoint only): DSpark module attached, `dspark_target_layer_ids: [58,59,60]`, `dspark_block_size: 5`, `dspark_markov_rank: 512` ([0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813), [0813 config.json](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813/raw/main/config.json)) +- License: MIT ([model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)) +- Modalities: text in / text out; the preview API announcement lists no image/audio input for Pro (a separate experimental `deepseek-v4-flash-vision-exp` accepts images) ([DeepSeek API docs](https://api-docs.deepseek.com/)) + +## Official vendor eval scores + +All rows below are vendor-reported. Unless noted, values are DeepSeek-V4-Pro-**Max** (maximum reasoning effort) from the official Hugging Face model card. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ----------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| MMLU-Pro (EM) | 87.5 | 86.8 | +0.8% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| GPQA Diamond (Pass@1) | 90.1 | 87.0 (no tools) | +3.6% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| HLE (Humanity's Last Exam, Pass@1) | 37.7 (preview Max); 42.7 without tools / 60.0 with tools for V4-Pro-0813 | 26.7 (no tools) | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro); [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) | +| HLE with tools (Pass@1) | 48.2 (preview Max) | 37.4 (with tools) | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| LiveCodeBench (Pass@1) | 93.5 | 89.0 (v6) | +5.1% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| Codeforces (Rating) | 3206 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| HMMT 2026 Feb (Pass@1) | 95.2 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| IMOAnswerBench (Pass@1) | 89.8 | 88.6 (no tools) | +1.4% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| SWE-bench Verified (Resolved) | 80.6 | 70.7 | +14.0% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| SWE-bench Pro (Resolved) | 55.4 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| SWE-bench Multilingual (Resolved) | 76.2 | 67.7 | +12.6% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| Terminal-Bench 2.0 (Acc) | 67.9 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| Terminal-Bench 2.1 (V4-Pro-0813) | 87.9 | 56.4 | +55.9% | [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) | +| BrowseComp (Pass@1) | 83.4 | 44.4 | +87.8% | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| Toolathlon (Pass@1) | 51.8 (preview Max); Toolathlon-Verified 74.1 for V4-Pro-0813 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro); [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) | +| MRCR 1M (MMR) | 83.5 | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| DeepSWE (V4-Pro-0813) | 62.7 | n.a. | — | [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) | +| AIME 2024 / AIME 2025 | n.a. — not reported by DeepSeek for V4-Pro; math reasoning is reported via HMMT 2026 Feb, IMOAnswerBench and Apex instead | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro) | +| tau2-bench | n.a. — not present in DeepSeek's V4-Pro or V4-Pro-0813 result tables | n.a. | — | [HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Additional vendor-reported values on the same cards, for reference: Apex 38.3, Apex Shortlist 90.2, SimpleQA-Verified 57.9, Chinese-SimpleQA 84.4, CorpusQA 1M 62.0, MCPAtlas Public 73.6, GDPval-AA Elo 1554 ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)); and for V4-Pro-0813: NL2Repo 61.5, Cybergym 83.3, Agents' Last Exam 25.7, AutomationBench (Public) 31.8, plus internal DSBench-FullStack 71.1 and DSBench-Hard 67.2 ([0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)). + +## Benchmark explanations + +- **MMLU-Pro** — a harder, cleaned multiple-choice exam suite across many academic and professional subjects, measuring broad knowledge and reasoning; scored as exact match ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **GPQA Diamond** — the hardest subset of graduate-level, "Google-proof" physics, chemistry and biology questions written by domain PhDs; measures expert scientific reasoning ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **HLE (Humanity's Last Exam)** — an extremely difficult, expert-authored exam spanning many fields, used as a frontier-difficulty knowledge and reasoning ceiling; DeepSeek also reports a tool-augmented variant ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **LiveCodeBench** — competitive-programming problems collected continuously from live contests to limit contamination; measures code generation correctness at pass@1 ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **Codeforces** — an Elo-style rating estimated from competitive-programming contest performance rather than a percentage accuracy ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **HMMT 2026 Feb** — the February 2026 Harvard-MIT Math Tournament problem set, a hard olympiad-style short-answer math benchmark ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **IMOAnswerBench** — International Mathematical Olympiad-style problems graded on final answers, measuring olympiad math ability ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **SWE-bench Verified** — human-validated real GitHub issues where the model must produce a patch that passes the repository's tests; measures agentic software-engineering ability ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **SWE-bench Pro** — a harder, more realistic variant of the SWE-bench issue-resolution task ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **SWE-bench Multilingual** — the same issue-resolution setup extended beyond Python to repositories in multiple programming languages ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **Terminal-Bench 2.0 / 2.1** — multi-step shell and systems tasks executed in a real terminal environment, measuring end-to-end agentic tool use ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)). +- **BrowseComp** — hard-to-find information-seeking questions that require an agent to browse the web persistently to locate the answer ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **Toolathlon / Toolathlon-Verified** — broad tool-calling agent tasks across many tools and APIs; the Verified split is a human-checked subset ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro), [0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)). +- **MRCR 1M** — multi-round coreference resolution at one-million-token context, measuring long-context retrieval and tracking ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **DeepSWE** — a software-engineering agent evaluation reported by DeepSeek for the 0813 production checkpoint ([0813 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813)). +- **AIME** — the American Invitational Mathematics Examination, a short-answer competition math benchmark; not reported by DeepSeek for V4-Pro ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). +- **tau2-bench** — a customer-service-style tool-use and policy-following agent benchmark; not reported by DeepSeek for V4-Pro ([HF model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)). diff --git a/packages/app/content/models/glm-5-1.mdx b/packages/app/content/models/glm-5-1.mdx new file mode 100644 index 000000000..34d434052 --- /dev/null +++ b/packages/app/content/models/glm-5-1.mdx @@ -0,0 +1,88 @@ +--- +title: 'GLM-5 / GLM-5.1' +developer: 'Z.ai (Zhipu AI)' +releaseDate: 'February 11, 2026 (GLM-5); April 7, 2026 (GLM-5.1)' +description: "Z.ai's 744B-parameter / 40B-active MoE family with DeepSeek Sparse Attention (DSA), trained on 28.5T tokens and released under the MIT license." +--- + +## Overview + +GLM-5 is the flagship open-weights large language model of Zhipu AI (operating internationally as Z.ai, Hugging Face org `zai-org`), launched "targeting complex systems engineering and long-horizon agentic tasks" ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). The GLM-5 weight repository was created on Hugging Face on 2026-02-11 ([Hugging Face model API for zai-org/GLM-5](https://huggingface.co/api/models/zai-org/GLM-5)), and secondary coverage dates the public release to February 11, 2026 ([NYU Shanghai RITS](https://rits.shanghai.nyu.edu/ai/glm-5-zhipu-ai-ships-a-744b-open-weight-frontier-model/)). Compared with GLM-4.5, GLM-5 scales from 355B parameters (32B active) to 744B parameters (40B active) and raises pre-training data from 23T to 28.5T tokens ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [Z.ai GLM-5 blog](https://z.ai/blog/glm-5)). Weights are distributed under the MIT license ([GLM-5 README](https://huggingface.co/zai-org/GLM-5/raw/main/README.md); [Hugging Face model API](https://huggingface.co/api/models/zai-org/GLM-5)), on Hugging Face and ModelScope, with API access on api.z.ai and BigModel.cn ([Z.ai GLM-5 blog](https://z.ai/blog/glm-5)). + +GLM-5.1 is the follow-up point release on the same architecture, described by Z.ai as "our next-generation flagship model for agentic engineering, with significantly stronger coding capabilities than its predecessor," reaching state-of-the-art performance on SWE-Bench Pro and leading GLM-5 by a wide margin on NL2Repo and Terminal-Bench 2.0 ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). Its distinguishing claim is long-horizon durability: where earlier models "exhaust their repertoire early," GLM-5.1 "sustains optimization over hundreds of rounds and thousands of tool calls" ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). The GLM-5.1 repository was created 2026-04-03 ([Hugging Face model API for zai-org/GLM-5.1](https://huggingface.co/api/models/zai-org/GLM-5.1)) and third-party documentation gives the public release date as April 7, 2026 ([Clore.ai GLM-5.1 guide](https://docs.clore.ai/guides/language-models/glm-5-1)); it is also MIT-licensed ([GLM-5.1 README](https://huggingface.co/zai-org/GLM-5.1/raw/main/README.md); [Z.ai GLM-5.1 blog](https://z.ai/blog/glm-5.1)). Both models are served through the Z.ai API with a 200K context window and 128K maximum output ([Z.ai docs: GLM-5](https://docs.z.ai/guides/llm/glm-5); [Z.ai docs: GLM-5.1](https://docs.z.ai/guides/llm/glm-5.1)). + +Notable techniques: GLM-5 integrates DeepSeek Sparse Attention (DSA), "largely reducing deployment cost while preserving long-context capacity" ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)), and the arXiv technical report _GLM-5: from Vibe Coding to Agentic Engineering_ (arXiv:2602.15763, submitted 17 Feb 2026) states that GLM-5 "adopts DSA" to cut training and inference cost while maintaining long-context fidelity, adds an asynchronous RL infrastructure that decouples generation from training, and proposes novel asynchronous agent RL algorithms ([arXiv:2602.15763](https://arxiv.org/abs/2602.15763)). That infrastructure is named **slime**, "a novel asynchronous RL infrastructure that substantially improves training throughput and efficiency, enabling more fine-grained post-training iterations" ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). GLM-5.1 cites the same technical report as GLM-5, consistent with an architecture-sharing point release ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). Beyond NVIDIA, Z.ai reports support on Huawei Ascend, Moore Threads, Cambricon, Kunlun Chip, MetaX, Enflame and Hygon accelerators ([Z.ai GLM-5 blog](https://z.ai/blog/glm-5)). + +## Architecture + +Values below come from the published `config.json` of each repository; GLM-5 and GLM-5.1 configs are byte-for-byte identical apart from the `transformers_version` field, confirming a shared architecture. + +- **Total parameters (vendor figure):** 744B ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [Z.ai docs: GLM-5](https://docs.z.ai/guides/llm/glm-5)) +- **Total parameters (checkpoint tensor count):** 753,864,139,008 (~754B) for both GLM-5 and GLM-5.1 ([GLM-5 HF API](https://huggingface.co/api/models/zai-org/GLM-5); [GLM-5.1 HF API](https://huggingface.co/api/models/zai-org/GLM-5.1)) +- **Active parameters per token:** 40B ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [Z.ai docs: GLM-5](https://docs.z.ai/guides/llm/glm-5)) +- **Model class / architecture string:** `GlmMoeDsaForCausalLM`, `model_type: glm_moe_dsa` ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json); [GLM-5.1 config.json](https://huggingface.co/zai-org/GLM-5.1/raw/main/config.json)) +- **Layers:** 78 hidden layers, with the first 3 layers dense (`first_k_dense_replace: 3`) and MoE in every subsequent layer (`moe_layer_freq: 1`) ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)) +- **Hidden size / dense FFN:** hidden_size 6144, intermediate_size 12288 ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)) +- **MoE experts:** 256 routed experts + 1 shared expert, 8 experts activated per token, expert FFN width 2048, sigmoid scoring with `noaux_tc` top-k routing and routed scaling factor 2.5 ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json); [GLM-5.1 config.json](https://huggingface.co/zai-org/GLM-5.1/raw/main/config.json)) +- **Attention:** DeepSeek Sparse Attention (DSA) ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [arXiv:2602.15763](https://arxiv.org/abs/2602.15763)); MLA-style low-rank projections with `q_lora_rank: 2048`, `kv_lora_rank: 512`, 64 attention heads, `qk_head_dim: 256` (192 nope + 64 rope), `v_head_dim: 256` ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)) +- **DSA indexer:** 32 indexer heads, `index_head_dim: 128`, `index_topk: 2048` selected tokens per query, interleaved indexer RoPE ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)); in DSA a lightweight "lightning indexer" picks the top-k relevant tokens, cutting core attention from O(L²) to O(Lk) ([IndexCache paper, arXiv:2603.12201](https://arxiv.org/abs/2603.12201)) +- **Multi-token prediction (MTP):** `num_nextn_predict_layers: 1`, used for speculative decoding (vLLM launch uses `--speculative-config.method mtp`, SGLang uses EAGLE) ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5)) +- **Context window:** `max_position_embeddings: 202752` in the checkpoint ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)); Z.ai API lists a 200K context window and 128K max output for both GLM-5 and GLM-5.1 ([Z.ai docs: GLM-5](https://docs.z.ai/guides/llm/glm-5); [Z.ai docs: GLM-5.1](https://docs.z.ai/guides/llm/glm-5.1)) +- **RoPE:** default RoPE with theta 1,000,000, interleaved ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)) +- **Precision:** BF16 checkpoint (`dtype: bfloat16`), plus an FP8 sibling repository `zai-org/GLM-5.1-FP8` listing `F8_E4M3` tensors at 754B parameters for cheaper serving ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json); [zai-org/GLM-5.1-FP8](https://huggingface.co/zai-org/GLM-5.1-FP8)); GLM-5.1 tensors are listed as BF16/F32 ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)) +- **Vocabulary:** 154,880 tokens, untied embeddings ([GLM-5 config.json](https://huggingface.co/zai-org/GLM-5/raw/main/config.json)) +- **Pre-training data:** 28.5T tokens (up from 23T for GLM-4.5) ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)) +- **License:** MIT ([GLM-5 README](https://huggingface.co/zai-org/GLM-5/raw/main/README.md); [GLM-5.1 README](https://huggingface.co/zai-org/GLM-5.1/raw/main/README.md)) +- **Serving stacks:** vLLM v0.19.0+, SGLang v0.5.10+, KTransformers v0.5.3+, Transformers, xLLM v0.8.0+ ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)) + +## Official vendor eval scores + +Vendor-reported only (Hugging Face model cards and Z.ai blog). GLM-5 column values are from the GLM-5 card/blog; GLM-5.1 values from the GLM-5.1 card/blog. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| -------------------------------------------------------- | ---------------------------------------------------- | --------------- | -------- | ---------------------------------------------------------------------------------------------------------------------- | +| SWE-bench Verified (GLM-5) | 77.8 | 70.7 | +10.0% | [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) / [Z.ai GLM-5 blog](https://z.ai/blog/glm-5) | +| SWE-bench Multilingual (GLM-5) | 73.3 | 67.7 | +8.3% | [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| SWE-Bench Pro (GLM-5.1 / GLM-5) | 58.4 / 55.1 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| NL2Repo (GLM-5.1 / GLM-5) | 42.7 / 35.9 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| Terminal-Bench 2.0, Terminus-2 (GLM-5.1 / GLM-5) | 63.5 / 56.2 (60.7 on verified set) | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| Terminal-Bench 2.0, best self-reported harness (GLM-5.1) | 69.0 (Claude Code) | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| τ²-Bench (GLM-5) | 89.7 | n.a. | — | [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| τ³-Bench (GLM-5.1 / GLM-5) | 70.6 / 69.2 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| BrowseComp (GLM-5.1 / GLM-5) | 68.0 / 62.0; with context management 79.3 / 75.9 | 44.4 | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| BrowseComp-Zh (GLM-5) | 72.7 | n.a. | — | [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| AIME 2026 (GLM-5.1 / GLM-5) | 95.3 / 95.4 (GLM-5 card reports 92.7 on AIME 2026 I) | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| HMMT Nov. 2025 (GLM-5.1 / GLM-5) | 94.0 / 96.9 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| HMMT Feb. 2026 (GLM-5.1 / GLM-5) | 82.6 / 82.8 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| GPQA-Diamond (GLM-5.1 / GLM-5) | 86.2 / 86.0 | 87.0 (no tools) | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| IMOAnswerBench (GLM-5.1 / GLM-5) | 83.8 / 82.5 | 88.6 (no tools) | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| Humanity's Last Exam (GLM-5.1 / GLM-5) | 31.0 / 30.5 text-only; 52.3 / 50.4 with tools | 26.7 (no tools) | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| CyberGym (GLM-5.1 / GLM-5) | 68.7 / 48.3 (GLM-5 card reports 43.2) | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| MCP-Atlas, public 500-task set (GLM-5.1 / GLM-5) | 71.8 / 69.2 (GLM-5 card reports 67.8) | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5) | +| Tool-Decathlon (GLM-5.1 / GLM-5) | 40.7 / 38.0 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| Vending Bench 2 (GLM-5.1 / GLM-5) | $5,634.41 / $4,432.12 | n.a. | — | [GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1) | +| MMLU-Pro | n.a. (not reported on any fetched vendor page) | 86.8 | — | — | +| LiveCodeBench | n.a. (not reported on any fetched vendor page) | 89.0 (v6) | — | — | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Notes on evaluation harnesses (vendor-stated): SWE-bench runs use OpenHands with a 200K context window; Terminal-Bench 2.0 Claude Code scores are averaged over 5 runs; CyberGym is single-run Pass@1 over 1,507 tasks; Vending Bench 2 runs were conducted independently by Andon Labs ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). GLM-5's τ²-Bench airline domain uses the fixes from the Claude Opus 4.5 system card ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). Where the GLM-5 and GLM-5.1 cards disagree on a GLM-5 number (CyberGym, MCP-Atlas, AIME, Tool-Decathlon), both figures are shown above. + +## Benchmark explanations + +- **SWE-bench Verified** — human-validated GitHub issues that the model must resolve by patching a real repository; measures end-to-end software bug-fixing ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **SWE-bench Multilingual** — the same issue-resolution task across repositories in multiple programming languages, testing generality beyond Python ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **SWE-Bench Pro** — a harder, contamination-resistant issue-resolution suite (hosted by Scale AI) run here with the OpenHands agent under a 400K/200K context ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **NL2Repo** — repository generation: build a working codebase from a natural-language specification ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **Terminal-Bench 2.0** — real-world terminal tasks executed in a sandboxed shell under CPU/RAM limits, scored by task completion ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1); [GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **τ²-Bench / τ³-Bench** — multi-turn tool-use and customer-service style agent dialogues in Retail, Telecom and Airline domains, testing policy compliance with a simulated user ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **BrowseComp / BrowseComp-Zh** — hard web-browsing research questions requiring many search-and-read hops; the Zh variant is the Chinese-language version, with and without context-management strategies ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **AIME 2026** — American Invitational Mathematics Examination problems, short-answer competition math ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **HMMT (Nov. 2025 / Feb. 2026)** — Harvard-MIT Mathematics Tournament problem sets, competition-level math reasoning ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **IMOAnswerBench** — International-Mathematical-Olympiad-style problems graded on final answers ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **GPQA-Diamond** — the hardest subset of graduate-level, Google-proof physics/chemistry/biology multiple-choice questions ([GLM-5.1 model card](https://huggingface.co/zai-org/GLM-5.1)). +- **Humanity's Last Exam (HLE)** — a very broad expert-written exam of frontier knowledge and reasoning; reported text-only by default and separately with tool access ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **CyberGym** — starts from white-box source code and tests whether the model can identify and validate vulnerabilities by triggering faults ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)); evaluated as single-run Pass@1 over 1,507 tasks ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **MCP-Atlas** — 500-task public subset of tool-calling tasks over Model Context Protocol servers, LLM-judged ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5)). +- **Tool-Decathlon** — a ten-domain battery of tool-use/agent tasks run through an official evaluation service ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **Vending Bench 2** — a simulated vending-machine business run over a one-year horizon, scored by final account balance; measures long-term planning and resource management ([Z.ai GLM-5 blog](https://z.ai/blog/glm-5)). +- **VectorDBBench / KernelBench (GLM-5.1 case studies)** — building a high-QPS approximate-nearest-neighbour database in Rust (GLM-5.1: 21.5k QPS over 600+ iterations and 6,000+ tool calls) and generating faster GPU kernels from PyTorch references (GLM-5.1: 3.6× geometric-mean speedup over 50 problems) ([Z.ai GLM-5.1 blog](https://z.ai/blog/glm-5.1)). diff --git a/packages/app/content/models/glm-5-2.mdx b/packages/app/content/models/glm-5-2.mdx new file mode 100644 index 000000000..f7b0d24b8 --- /dev/null +++ b/packages/app/content/models/glm-5-2.mdx @@ -0,0 +1,102 @@ +--- +title: 'GLM-5.2 / GLM-5.3' +developer: 'Z.ai (Zhipu AI)' +releaseDate: 'June 16, 2026 (GLM-5.2); August 14, 2026 (GLM-5.3)' +description: "Z.ai's 744B-class MoE line extending sparse attention with IndexShare for 1M-token contexts, with GLM-5.3 adding a major agentic post-training pass." +--- + +## Overview + +GLM-5.2 is Z.ai's (Zhipu AI's) flagship long-horizon model in the GLM-5 family, introduced as "our latest flagship model for long-horizon tasks," marking "a substantial leap in long-horizon task capability over its predecessor GLM-5.1" and, for the first time in the family, delivering it on "a solid 1M-token context" ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). The weight repository was created on Hugging Face on 2026-06-16 ([Hugging Face model API for zai-org/GLM-5.2](https://huggingface.co/api/models/zai-org/GLM-5.2)), matching Z.ai's own launch announcement of "GLM-5.2: Frontier Intelligence, Open Weights" with "MIT-licensed open weights" and "same API pricing as GLM-5.1" ([Z.ai announcement](https://x.com/Zai_org/status/2066938937344495629)). Its headline features are the solid 1M context, "advanced coding with flexible effort" (multiple thinking-effort levels), the **IndexShare** architecture change, an improved MTP layer for speculative decoding, and "Pure Open: an MIT open-source license — no regional limits, technical access without borders" ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2); [GLM-5.2 README](https://huggingface.co/zai-org/GLM-5.2/raw/main/README.md)). Z.ai positions it as the highest-ranked open-source model across three long-horizon coding benchmarks (FrontierSWE, PostTrainBench, SWE-Marathon) and the strongest open-source model on standard coding benchmarks ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)). + +GLM-5.3, announced on August 14, 2026 ([Yicai Global](https://www.yicaiglobal.com/news/chinas-zai-unveils-glm-53-says-it-leads-open-source-ai-models-in-key-benchmarks)), is a post-training-only release: "It uses the same base model as GLM-5.2 — every gain comes from post-training" ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)). Z.ai frames it as "Frontier Coding with Emergent Cyber Capabilities": a ~50% coding gain over GLM-5.2 on the in-house Z.ai Code Bench, claimed open-source SOTA on Terminal-Bench 3.0 and Agents' Last Exam (CLI), and unexpectedly fast growth in vulnerability discovery and exploitation ability ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)). Working with security teams in China, Z.ai reports the model surfaced 2,436 vulnerabilities across 269 projects, including 1,097 medium-to-high severity issues, with an average latent lifetime of 26.6 years and the oldest flaw introduced in 1981 ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). GLM-5.3 is text-only, always reasoning, with a 1M-token context, 128K max output, and three effort levels (`low`, `high`, `max`, default `max`); disabling thinking is no longer supported ([Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)). + +Licensing/availability differs between the two. GLM-5.2 ships as MIT open weights on Hugging Face and ModelScope ([Hugging Face model API](https://huggingface.co/api/models/zai-org/GLM-5.2); [Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)). GLM-5.3 weights are not yet public at the time of writing: Z.ai states "The model weights of GLM-5.3 will be publicly available soon in two weeks," with access initially via the Z.ai API, the GLM Coding Plan, and ZCode ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)). Notable techniques across the pair: IndexShare for cheap long-context sparse attention, SAO (with compaction) for RL on long-horizon tasks, and **slime**, Z.ai's open-source asynchronous RL post-training framework with Megatron for training and SGLang for rollout — GLM-5.3 reports >2.3× end-to-end RL throughput gains from system-level optimizations and training-rollout logprob agreement controlled to the 1e-7 level ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). Independent evaluation from Artificial Analysis placed GLM-5.3 at 60 on its Intelligence Index on August 18, 2026, level with Kimi K3 ([Unite.AI](https://www.unite.ai/glm-5-3-scores-60-on-artificial-analysis-intelligence-index-matching-kimi-k3/)). + +## Architecture + +GLM-5.3 shares GLM-5.2's base model exactly ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)), so the GLM-5.2 checkpoint config is the authoritative architecture source for both. + +- **Total parameters (checkpoint tensor count):** 753,329,940,480 (~753B) ([Hugging Face model API for zai-org/GLM-5.2](https://huggingface.co/api/models/zai-org/GLM-5.2)); the family's vendor-stated headline figure is 744B total / 40B active, stated for GLM-5 on the same architecture lineage ([GLM-5 model card](https://huggingface.co/zai-org/GLM-5); [Z.ai docs: GLM-5](https://docs.z.ai/guides/llm/glm-5)) +- **Active parameters per token:** n.a. for GLM-5.2/5.3 specifically — not stated on any fetched vendor page ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2); [GLM-5.2 README](https://huggingface.co/zai-org/GLM-5.2/raw/main/README.md)) +- **Model class:** `GlmMoeDsaForCausalLM`, `model_type: glm_moe_dsa` ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **Layers:** 78 hidden layers; `mlp_layer_types` shows the first 3 layers dense and the remaining 75 sparse (MoE) ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **Hidden size / dense FFN:** hidden_size 6144, intermediate_size 12288 ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **MoE experts:** 256 routed experts + 1 shared expert, 8 activated per token, expert FFN width 2048, sigmoid scoring with `noaux_tc` routing, routed scaling factor 2.5 ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **Attention:** DSA-based sparse attention with MLA-style low-rank projections — 64 attention heads, `q_lora_rank: 2048`, `kv_lora_rank: 512`, `qk_head_dim: 256` (192 nope + 64 rope), `v_head_dim: 256`, `head_dim: 192` ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)); DSA's lightning indexer selects top-k tokens per query, reducing core attention from O(L²) to O(Lk) while the indexer itself stays O(L²) ([IndexCache paper, arXiv:2603.12201](https://arxiv.org/abs/2603.12201)) +- **IndexShare (novel component):** "reuses the same indexer across every four sparse attention layers, reducing per-token FLOPs by 2.9× at a 1M context length" ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)); the indexer sits at the first of each group of 4 layers and its top-k indices are reused for all 4, removing indexer dot-product and top-k work in 3/4 of layers; trained with IndexShare from mid-training at 128K sequence length ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)) +- **IndexShare in config:** `index_topk_freq: 4`, `index_skip_topk_offset: 3`, `index_share_for_mtp_iteration: true`, and an `indexer_types` list alternating one `full` layer followed by three `shared` layers across all 78 layers ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **DSA indexer sizing:** 32 indexer heads, `index_head_dim: 128`, `index_topk: 2048`, interleaved indexer RoPE ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **Multi-token prediction (MTP):** `num_nextn_predict_layers: 1`; the MTP layer was improved for speculative decoding, raising acceptance length by up to 20% — ablation: baseline 4.56 → +IndexShare+KV Share 5.10 → +rejection sampling 5.29 → +end-to-end TV loss 5.47; MTP step parameters are shared as in GLM-5.1, and in multi-step MTP the indexer runs on the first step with its top-k indices reused for later steps ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json); [Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)) +- **Context window:** `max_position_embeddings: 1048576` (1M) in the checkpoint ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)); Z.ai API lists 1M context / 128K max output for GLM-5.2 and for GLM-5.3 ([Z.ai docs: GLM-5.2](https://docs.z.ai/guides/llm/glm-5.2); [Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)). GLM-5.1's context was 200K, so 5.2 is a 5× extension ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)) +- **RoPE:** default RoPE with theta raised to 8,000,000 (vs 1,000,000 in GLM-5/5.1), interleaved ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json); [GLM-5.1 config.json](https://huggingface.co/zai-org/GLM-5.1/raw/main/config.json)) +- **Precision:** BF16 checkpoint (`dtype: bfloat16`); FP8 KV-cache is used in Z.ai's large-scale agentic RL training infrastructure ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json); [Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)) +- **Vocabulary:** 154,880 tokens, untied embeddings ([GLM-5.2 config.json](https://huggingface.co/zai-org/GLM-5.2/raw/main/config.json)) +- **Effort control:** GLM-5.2 exposes High and Max thinking effort ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)); GLM-5.3 exposes `low`/`high`/`max` with reasoning always enabled ([Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3)) +- **License:** MIT for GLM-5.2 ([Hugging Face model API](https://huggingface.co/api/models/zai-org/GLM-5.2); [GLM-5.2 README](https://huggingface.co/zai-org/GLM-5.2/raw/main/README.md)); GLM-5.3 weights pending release ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)) +- **Serving stacks:** SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, KTransformers v0.5.12+, Unsloth v0.1.47-beta+, plus vLLM-Ascend/xLLM/SGLang on Ascend NPU ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)); inference optimization builds on LayerSplit, with a throughput advantage that grows with context length ([Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2)) + +## Official vendor eval scores + +Vendor-reported only. GLM-5.3 figures come from the Z.ai GLM-5.3 blog table (Z.ai ran the comparisons itself); GLM-5.2 figures from the GLM-5.2 Hugging Face card / Z.ai blog. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| -------------------------------------- | -------------------------------------------------------------------------------------------------- | ----------------- | -------- | ------------------------------------------------------------------------------------------------------------ | +| Terminal-Bench 3.0 | GLM-5.3: 28.3 · GLM-5.2: 4.6 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| Terminal-Bench 2.1 | GLM-5.3: 88.2 · GLM-5.2: 81.0 (Terminus-2), 82.7 (best harness, Claude Code) | 56.4 | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| DeepSWE v1.1 | GLM-5.3: 66.9 · GLM-5.2: 46.2 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| SWE-bench Pro | GLM-5.2: 62.1 (GLM-5.1: 58.4) — not reported for GLM-5.3 | n.a. | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| SWE-Marathon v1.1 | GLM-5.3: 42.5 · GLM-5.2: 19.4 (GLM-5.2 card reports 13.0 for SWE-Marathon) | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| FrontierSWE | GLM-5.3: 78.1 · GLM-5.2: 67.5 (GLM-5.2 card reports 74.4 dominance as of 2026-06-16) | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| NL2Repo | GLM-5.3: 58.0 · GLM-5.2: 48.9 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| ProgramBench (Almost Solved) | GLM-5.3: 19.0 · GLM-5.2: 9.5 (GLM-5.2 card reports 63.7 on ProgramBench) | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| PostTrainBench | GLM-5.3: 39.8 · GLM-5.2: 31.7 (GLM-5.2 card reports 34.3) | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| Agents' Last Exam (ALE-CLI) | GLM-5.3: 28.5 · GLM-5.2: 23.8 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| Toolathlon Verified | GLM-5.3: 73.0 · GLM-5.2: 59.9 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| AutomationBench v1.0.6 | GLM-5.3: 48.2 · GLM-5.2: 26.2 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| CyberGym | GLM-5.3: 84.5 · GLM-5.2: 77.2 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| ExploitBench | GLM-5.3: 54.4 · GLM-5.2: 24.4 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| ExploitGym (2h / 6h tasks completed) | GLM-5.3: 105 / 130 · GLM-5.2: 29 / 39 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| Humanity's Last Exam (with tools) | GLM-5.3: 62.5 · GLM-5.2: 54.7 (GLM-5.2 text-only: 40.5) | 37.4 (with tools) | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| GDPval-AA v2 (Elo) | GLM-5.3: 1769 · GLM-5.2: 1508 | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| AIME 2026 | GLM-5.2: 99.2 — not reported for GLM-5.3 | n.a. | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| HMMT Nov. 2025 / Feb. 2026 | GLM-5.2: 94.4 / 92.5 — not reported for GLM-5.3 | n.a. | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| GPQA-Diamond | GLM-5.2: 91.2 — not reported for GLM-5.3 | 87.0 (no tools) | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| IMOAnswerBench | GLM-5.2: 91.0 — not reported for GLM-5.3 | 88.6 (no tools) | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| CritPt | GLM-5.2: 20.9 (blog) / 16.7 (model card) | 3.1 (no tools) | — | [Z.ai GLM-5.2 blog](https://z.ai/blog/glm-5.2); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| MCP-Atlas (public set) | GLM-5.2: 76.8 — not reported for GLM-5.3 | n.a. | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| Tool-Decathlon | GLM-5.2: 48.2 — not reported for GLM-5.3 | n.a. | — | [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2) | +| Z.ai Code Bench (private, High effort) | GLM-5.3: 31.4% at ~50K output tokens/task (Max effort: 34.5% at ~75K); GLM-5.2: 23.4% at 96K (Max) | n.a. | — | [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3) | +| SWE-bench Verified | n.a. — not reported for GLM-5.2 or GLM-5.3 on any fetched vendor page | 70.7 | — | — | +| τ²-bench / τ³-Bench | n.a. — not reported for GLM-5.2 or GLM-5.3 on any fetched vendor page | n.a. | — | — | +| BrowseComp | n.a. — not reported for GLM-5.2 or GLM-5.3 on any fetched vendor page | 44.4 | — | — | +| LiveCodeBench | n.a. — not reported on any fetched vendor page | 89.0 (v6) | — | — | +| MMLU-Pro | n.a. — not reported on any fetched vendor page | 86.8 | — | — | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Vendor harness notes: GLM-5.2 reasoning evals use temperature 1.0 / top_p 0.95, 163,840-token generation limit, GPT-5.5 (medium) as judge, text-only subsets by default; SWE-Bench Pro uses OpenHands under a 400K context; DeepSWE uses the official pier framework with mini-swe-agent; Terminal-Bench 2.1 Claude Code scores are averaged over 5 runs; FrontierSWE was evaluated by Proximal, PostTrainBench by PostTrainBench, and SWE-Marathon by Abundant AI, all at 1M context and max effort ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). GLM-5.3's HLE-with-tools run used a 300K context with context management and GPT-5.6-luna (medium) as judge ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). + +## Benchmark explanations + +- **Terminal-Bench 2.1 / 3.0** — real-world command-line tasks solved autonomously in a sandboxed terminal; 3.0 is the harder successor generation, described by Z.ai's docs as measuring complex task completion in a real terminal environment ([Z.ai docs: GLM-5.3](https://docs.z.ai/guides/llm/glm-5.3); [GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **DeepSWE v1.1** — long-horizon software-engineering task suite run with the official pier framework and mini-swe-agent harness, each task in an isolated no-internet container ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **SWE-bench Pro** — contamination-resistant GitHub issue resolution requiring repo-level patches, run with OpenHands under a 400K context ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **SWE-Marathon** — very long-horizon software engineering evaluated by Abundant AI at 1M context, testing sustained multi-hour work ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **FrontierSWE** — frontier-difficulty software-engineering tasks scored as a "dominance" rate, evaluated by Proximal at 1M context and max effort ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **NL2Repo** — generate an entire working repository from a natural-language specification, with rule- and LLM-based anti-cheat checks ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **ProgramBench** — 200 hard programming instances run through Claude Code with up to 2,000 turns and 6-hour timeouts; "Almost Solved" is the stricter partial-credit tier ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2); [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **PostTrainBench** — machine-learning post-training research tasks (run the experiment, improve a model) evaluated by the benchmark authors at 1M context ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **Agents' Last Exam (ALE-CLI)** — real-world professional agent scenarios emphasizing cross-tool collaboration and long-horizon execution, scored pass@1 in a CLI harness ([Yicai Global](https://www.yicaiglobal.com/news/chinas-zai-unveils-glm-53-says-it-leads-open-source-ai-models-in-key-benchmarks); [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **Toolathlon Verified** — verified multi-tool "chores" requiring correct orchestration of many external tools ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **AutomationBench v1.0.6** — end-to-end automation of real workflow tasks by an agent ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **CyberGym** — starts from white-box source code and tests whether the model can identify and validate vulnerabilities by triggering faults ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **ExploitBench** — requires deeper reasoning about real vulnerabilities and producing working exploitation, i.e. further up the exploitation chain than CyberGym ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **ExploitGym** — counts how many exploitation tasks a model completes under time-normalized budgets (2-hour and 6-hour), normalized by per-model throughput ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **Humanity's Last Exam (HLE)** — broad expert-written exam of frontier knowledge and reasoning; the "with tools" variant permits search/tool use ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2); [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **GDPval-AA v2** — Elo-style evaluation of real-world, economically valuable knowledge work spanning 44 occupations ([Yicai Global](https://www.yicaiglobal.com/news/chinas-zai-unveils-glm-53-says-it-leads-open-source-ai-models-in-key-benchmarks); [Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). +- **AIME 2026 / HMMT / IMOAnswerBench** — competition mathematics (American Invitational Mathematics Examination, Harvard-MIT Mathematics Tournament, IMO-style problems) graded on exact final answers ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **GPQA-Diamond** — hardest subset of graduate-level, Google-proof science multiple-choice questions ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **CritPt** — frontier physics-reasoning research problems, reported alongside HLE in the reasoning block ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **MCP-Atlas** — 500-task public subset of Model Context Protocol tool-calling tasks, LLM-judged (Gemini 3.0 Pro as judge) ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **Tool-Decathlon** — ten-domain agentic tool-use battery run through the official evaluation service with 128K max tokens ([GLM-5.2 model card](https://huggingface.co/zai-org/GLM-5.2)). +- **Z.ai Code Bench** — Z.ai's private benchmark placing coding agents in complex local development environments, scoring end-to-end task completion and checklist accuracy plus output-token efficiency ([Z.ai GLM-5.3 blog](https://z.ai/blog/glm-5.3)). diff --git a/packages/app/content/models/gptoss-120b.mdx b/packages/app/content/models/gptoss-120b.mdx new file mode 100644 index 000000000..f702266d1 --- /dev/null +++ b/packages/app/content/models/gptoss-120b.mdx @@ -0,0 +1,69 @@ +--- +title: 'gpt-oss-120b' +developer: 'OpenAI' +releaseDate: 'August 5, 2025' +description: "OpenAI's 117B-parameter / 5.1B-active open-weight MoE with alternating dense and sliding-window attention and native MXFP4 quantization, under Apache 2.0." +--- + +## Overview + +gpt-oss-120b is an open-weight reasoning language model released by OpenAI on August 5, 2025, alongside the smaller gpt-oss-20b, both distributed under the Apache 2.0 license ([OpenAI, "Introducing gpt-oss"](https://openai.com/index/introducing-gpt-oss/)). The Hugging Face model card confirms the Apache 2.0 license and lists the model at 117B total parameters with 5.1B active parameters ([Hugging Face: openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)). OpenAI's accompanying model card describes both models as open-weight reasoning models trained with large-scale distillation and reinforcement learning, and states that weights, inference implementations, tool environments, and tokenizers are all released under Apache 2.0 ([arXiv:2508.10925 abstract](https://arxiv.org/abs/2508.10925)). + +Positioning: OpenAI reports that gpt-oss-120b reaches near-parity with OpenAI o4-mini on core reasoning benchmarks, outperforms o3-mini and matches or exceeds o4-mini on Codeforces competition coding, MMLU, HLE, and TauBench tool calling, and beats o4-mini on AIME 2024/2025 competition mathematics and HealthBench ([OpenAI, "Introducing gpt-oss"](https://openai.com/index/introducing-gpt-oss/)). The models are optimized for deep research browsing, Python tool use, and developer-provided functions, and use a rendered chat format designed for clear instruction following and role delineation ([arXiv:2508.10925 abstract](https://arxiv.org/abs/2508.10925)). + +Notable techniques: the model is an autoregressive mixture-of-experts transformer that alternates banded-window and fully dense attention, uses Grouped Query Attention with learned attention-sink biases in the softmax denominator, RoPE positional embeddings with YaRN context extension, and gated SwiGLU MoE activations ([OpenAI gpt-oss model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf)). Its MoE weights — more than 90% of all parameters — are natively quantized to MXFP4 at 4.25 bits per parameter, which lets the 120b model fit on a single 80 GB GPU such as an NVIDIA H100 or AMD MI300X ([OpenAI gpt-oss model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [Hugging Face model card](https://huggingface.co/openai/gpt-oss-120b)). + +## Architecture + +- Model type: autoregressive Mixture-of-Experts transformer; HF architecture class `GptOssForCausalLM`, model type `gpt_oss` ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Total parameters: 116.83B (≈117B as listed on the HF card) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [Hugging Face](https://huggingface.co/openai/gpt-oss-120b)) +- Active parameters per token: 5.13B (≈5.1B) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [Hugging Face](https://huggingface.co/openai/gpt-oss-120b)) +- Parameter breakdown: MLP 114.71B, attention 0.96B, embed + unembed 1.16B; checkpoint size 60.8 GiB ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf)) +- Layers: 36 (`num_hidden_layers: 36`) ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Residual stream / hidden size: 2,880 ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Experts: 128 MoE experts per block, top-4 selected per token (softmax-weighted over selected experts via a linear router); `num_local_experts: 128`, `num_experts_per_tok: 4` ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- MoE activation: gated SwiGLU (SwiGLU limit 7.0 in config) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Attention pattern: alternates banded-window (sliding) and fully dense attention; config lists a strict `sliding_attention` / `full_attention` alternation across all 36 layers ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Sliding-window bandwidth: 128 tokens ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Attention heads: 64 query heads per layer, head dim 64, 8 key-value heads — Grouped Query Attention (group size 8) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json); [OpenAI blog](https://openai.com/index/introducing-gpt-oss/)) +- Attention sinks: each attention head has a learned bias in the softmax denominator (off-by-one attention), letting heads attend to nothing ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf)) +- Positional embeddings: RoPE with YaRN scaling (`rope_theta: 150000`, scaling factor 32.0, original max positions 4096) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Context window: 131,072 tokens for dense layers (128k native) ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json); [OpenAI blog](https://openai.com/index/introducing-gpt-oss/)) +- Vocabulary size: 201,088 ([config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json)) +- Quantization: MXFP4 on MoE weights at 4.25 bits/parameter (>90% of parameters); config `quant_method: mxfp4` with self-attention, router, embeddings and lm_head left unquantized; tensor types BF16 · U8 ([model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); [config.json](https://huggingface.co/openai/gpt-oss-120b/raw/main/config.json); [Hugging Face](https://huggingface.co/openai/gpt-oss-120b)) +- Hardware footprint: runs within 80 GB of memory / a single 80GB GPU ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/); [Hugging Face](https://huggingface.co/openai/gpt-oss-120b)) + +## Official vendor eval scores + +All values are OpenAI-reported for gpt-oss-120b. Scores vary by reasoning effort (low / medium / high); the table reports high reasoning effort, with medium shown where the HF card also lists it. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| --------------------------------------- | -------------------------------------------------------------------- | ----------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| AIME 2024 (no tools, high reasoning) | 95.8% (low 56.3%, medium 80.4%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| AIME 2024 (with tools, high) | 96.6% | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| AIME 2025 (no tools, high) | 92.5% (low 50.4%, medium 80.0%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| AIME 2025 (with tools, high) | 97.9% | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| GPQA Diamond (no tools, high) | 80.1% (medium 73.1%) | 87.0 (no tools) | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); medium also on [Hugging Face card](https://huggingface.co/openai/gpt-oss-120b) | +| GPQA Diamond (with tools, high) | 80.9% (medium 73.5%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); medium also on [Hugging Face card](https://huggingface.co/openai/gpt-oss-120b) | +| MMLU (high) | 90.0% (low 85.9%, medium 88.0%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| Humanity's Last Exam (no tools, high) | 14.9% | 26.7 (no tools) | -44.2% | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| Humanity's Last Exam (with tools, high) | 19.0% | 37.4 (with tools) | -49.2% | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| SWE-bench Verified (high) | 62.4% (low 47.9%, medium 52.6%) | 70.7 | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf); medium also on [Hugging Face card](https://huggingface.co/openai/gpt-oss-120b) | +| Codeforces Elo (no tools, high) | 2,463 Elo (low 1,595; medium 2,205) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| Codeforces Elo (with tools, high) | 2,622 Elo (low 1,653; medium 2,365) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| τ-Bench Retail (high) | 67.8% (low 49.4%, medium 62.0%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| HealthBench (high) | 57.6% (Hard 30.0%, Consensus 89.9%) | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | +| BFCL / IFEval | n.a. — not reported in the OpenAI model card or HF card fetched here | n.a. | — | [OpenAI model card PDF](https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. + +## Benchmark explanations + +- **AIME 2024 / AIME 2025** — American Invitational Mathematics Examination problem sets; measures competition-level mathematical reasoning, scored as accuracy with and without tool access ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/)). +- **GPQA Diamond** — the hardest subset of graduate-level, Google-proof science multiple-choice questions (biology, chemistry, physics); measures expert-level scientific reasoning ([Hugging Face card links the GPQA dataset](https://huggingface.co/openai/gpt-oss-120b)). +- **MMLU** — Massive Multitask Language Understanding: broad multiple-choice knowledge and general problem solving across academic and professional subjects ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/)). +- **Humanity's Last Exam (HLE)** — extremely difficult expert-written questions spanning many disciplines; measures frontier general problem solving ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/)). +- **SWE-bench Verified** — human-validated real GitHub issues; measures whether a model can produce patches that resolve software bugs in actual repositories ([Hugging Face card links the SWE-bench Verified dataset](https://huggingface.co/openai/gpt-oss-120b)). +- **Codeforces (Elo)** — competitive programming rating derived from Codeforces contest problems; measures algorithmic coding skill on a rating scale rather than a percentage ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/)). +- **τ-Bench Retail (Tau-Bench)** — agentic tool-calling evaluation in a retail customer-service environment; measures multi-turn function calling and policy-following ([OpenAI blog describes TauBench as a tool-calling / agentic evaluation suite](https://openai.com/index/introducing-gpt-oss/)). +- **HealthBench / Hard / Consensus** — health-conversation evaluations scoring the quality and safety of medical responses, with a harder subset and a physician-consensus variant ([OpenAI blog](https://openai.com/index/introducing-gpt-oss/)). diff --git a/packages/app/content/models/kimi-k26.mdx b/packages/app/content/models/kimi-k26.mdx new file mode 100644 index 000000000..8b2b51dbd --- /dev/null +++ b/packages/app/content/models/kimi-k26.mdx @@ -0,0 +1,94 @@ +--- +title: 'Kimi K2.5 / K2.6 / K2.7-Code' +developer: 'Moonshot AI' +releaseDate: 'January 27, 2026 (K2.5); April 20, 2026 (K2.6); June 12, 2026 (K2.7-Code)' +description: "Moonshot AI's 1T-parameter / 32B-active MoE family sharing a DeepSeek-V3-style MLA backbone with native INT4 quantization and a 256K context window." +--- + +## Overview + +The Kimi K2.5 → K2.6 → K2.7-Code line is Moonshot AI's 1-trillion-parameter open-weights MoE family, all three sharing an identical architecture: 1T total parameters, 32B activated per token, 61 layers, MLA attention, 384 routed experts with 8 selected per token plus 1 shared expert, a 256K context window and a 400M-parameter MoonViT vision encoder ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6); [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). Moonshot states this explicitly on the later cards — "Kimi-K2.6 has the same architecture as Kimi-K2.5, and the deployment method can be directly reused" and "Kimi-K2.7-Code has the same architecture as Kimi-K2.5/Kimi-K2.6" ([Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6); [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). The text backbone is DeepSeek-style: K2.7-Code's `config.json` declares the text architecture as `DeepseekV3ForCausalLM` with `topk_method: noaux_tc`, sigmoid expert scoring, YaRN RoPE scaling and MLA-style `q_lora_rank`/`kv_lora_rank` projections ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)). + +Release timeline, per Moonshot's own channels: **Kimi K2.5** was published 2026-01-27 and is an open-source native multimodal agentic model built by continual pretraining on ~15 trillion mixed visual and text tokens on top of Kimi-K2-Base, adding instant and thinking modes and the Agent Swarm execution scheme ([Kimi Research & Tech Blog](https://www.kimi.ai/blog/); [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)); its card also logs a 2026.1.29 changelog removing the default system prompt and fixing the `<|media_begin|>` token ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)). **Kimi K2.6** was published 2026-04-20 and pushes long-horizon coding, coding-driven design, proactive autonomous execution and swarm orchestration scaling to 300 sub-agents executing 4,000 coordinated steps ([Kimi Research & Tech Blog](https://www.kimi.ai/blog/); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). **Kimi K2.7-Code** was announced on Moonshot's official X account on June 12, 2026 as "our latest coding model … now released and open-sourced," with +21.8% on Kimi Code Bench v2, +11.0% on Program Bench and +31.5% on MLS Bench Lite versus K2.6, 30% lower reasoning-token usage, and availability via the Kimi API and Kimi Code ([@Kimi_Moonshot](https://x.com/Kimi_Moonshot/status/2065377579130142937)). + +Positioning shifts across the family from general multimodal agent to coding specialist. K2.5 is framed around native multimodality, coding from visual specifications, and an agent swarm that decomposes tasks into parallel domain-specific sub-agents ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)). K2.7-Code is explicitly "a coding-focused agentic model built upon Kimi K2.6," strengthening end-to-end completion of complex software-engineering workflows while cutting thinking tokens ~30% versus K2.6; it forces thinking and `preserve_thinking` on (they cannot be disabled), unlike K2.6 where preserve-thinking is off by default and an instant (non-thinking) mode exists ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). All three ship under the Modified MIT License for both code repository and weights ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). Notable techniques include native INT4 quantization ("the same native int4 quantization method as Kimi-K2-Thinking"), interleaved thinking with multi-step tool calls, and Kimi Code CLI as the recommended agent framework; supported inference engines are vLLM, SGLang and KTransformers with `transformers >=4.57.1, <5.0.0` ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). Moonshot's API platform lists dedicated pricing pages for `kimi-k2.7-code` and `kimi-k2.6`, and notes that `kimi-k2.5` and the `moonshot-v1` series are no longer available to newly registered users following the K3 launch ([Kimi API pricing docs](https://platform.kimi.ai/docs/pricing); [Kimi API model list](https://platform.kimi.ai/docs/models)). + +## Architecture + +Shared by K2.5, K2.6 and K2.7-Code unless noted. + +- **Architecture family:** Mixture-of-Experts (MoE), native multimodal (image + video input, text output) ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)) +- **Total parameters:** 1T; Hugging Face reports the K2.7-Code safetensors model size as 1T params ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)) +- **Activated parameters per token:** 32B ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6); [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)) +- **Layers:** 61 including 1 dense layer ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); `num_hidden_layers: 61`, `first_k_dense_replace: 1` ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Attention mechanism:** MLA (Multi-head Latent Attention) ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); config: `q_lora_rank: 1536`, `kv_lora_rank: 512`, `qk_nope_head_dim: 128`, `qk_rope_head_dim: 64`, `v_head_dim: 128` ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Attention hidden dimension / heads:** 7168 hidden, 64 attention heads ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); `hidden_size: 7168`, `num_attention_heads: 64`, `num_key_value_heads: 64` ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Experts:** 384 routed experts, 8 selected per token, 1 shared expert ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); `n_routed_experts: 384`, `num_experts_per_tok: 8`, `n_shared_experts: 1`, `moe_layer_freq: 1` ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **MoE dimensions / routing:** MoE hidden dimension per expert 2048 ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); `moe_intermediate_size: 2048`, dense `intermediate_size: 18432`, sigmoid expert scoring, `topk_method: noaux_tc`, `norm_topk_prob: true`, `routed_scaling_factor: 2.827`, `aux_loss_alpha: 0.001` — the DeepSeek-V3-style MoE recipe ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Text backbone class:** `DeepseekV3ForCausalLM` inside `KimiK25ForConditionalGeneration` (`model_type: kimi_k25`, text `model_type: kimi_k2`) ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Activation function:** SwiGLU per the model cards; `hidden_act: silu` in config ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Context window:** 256K per the cards; `max_position_embeddings: 262144` ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)). Evaluations were run at a 262,144-token context length ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)) +- **Positional encoding:** YaRN RoPE scaling, `rope_theta: 50000.0`, factor 64.0, `original_max_position_embeddings: 4096`, beta_fast 32.0 / beta_slow 1.0 ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Vocabulary:** 160K per the cards; `vocab_size: 163840` ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Vision encoder:** MoonViT, 400M parameters ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); vision config: 27 layers, 16 heads, hidden size 1152, intermediate 4304, `patch_size: 14`, `patchmerger` projector, `video_attn_type: spatial_temporal` ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Precision / quantization:** native INT4 — same method as Kimi-K2-Thinking ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)); config declares `compressed-tensors` `pack-quantized`, `num_bits: 4`, `group_size: 32`, symmetric, with attention, shared experts, dense MLP, `lm_head`, vision tower and mm projector excluded; published tensor types are BF16/F32/I32 ([Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json); [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)) +- **Novel components (KDA, LatentMoE, SiTU-GLU):** n.a. — these are Kimi K3 components and are not present in the K2.x cards or config, which specify MLA + SwiGLU + DeepSeek-style MoE instead ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.7-Code config.json](https://huggingface.co/moonshotai/Kimi-K2.7-Code/raw/main/config.json)) +- **Thinking behavior:** K2.7-Code forces thinking and `preserve_thinking` to True and does not support instant mode; K2.6/K2.5 support instant vs thinking modes with `preserve_thinking` off by default ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)) +- **Recommended sampling:** temperature 1.0, top-p 0.95 in thinking mode (K2.6 instant mode: 0.6) ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code); [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)) +- **Pretraining data (K2.5):** continual pretraining on ~15 trillion mixed visual and text tokens atop Kimi-K2-Base ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)) +- **Token efficiency (K2.7-Code):** ~30% fewer thinking tokens than K2.6 ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)) +- **Serving:** vLLM, SGLang, KTransformers; `transformers >=4.57.1, <5.0.0` ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)) + +## Official vendor eval scores + +Vendor-reported only. K2.7-Code's model card publishes six benchmarks (all with the K2.6 baseline alongside); the broader public suites (SWE-bench Verified, LiveCodeBench, AIME, HMMT, GPQA, MMLU-Pro) are reported by Moonshot for K2.6 and K2.5 but not re-run for K2.7-Code. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ----------------------------------------------------- | ---------------------------------------------------------------- | --------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| Kimi Code Bench v2 (in-house) | K2.7-Code **62.0**; K2.6 50.9 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| Program Bench | K2.7-Code **53.6**; K2.6 48.3 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| MLS Bench Lite | K2.7-Code **35.1**; K2.6 26.7 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| Kimi Claw 24/7 Bench (in-house, long-horizon agentic) | K2.7-Code **46.9**; K2.6 42.9 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| MCP Atlas | K2.7-Code **76.0**; K2.6 69.4 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| MCP Mark Verified | K2.7-Code **81.1**; K2.6 72.8 | n.a. | — | [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code) | +| SWE-Bench Verified | K2.7-Code n.a.; K2.6 **80.2**; K2.5 76.8 | 70.7 | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| SWE-Bench Pro | K2.7-Code n.a.; K2.6 **58.6**; K2.5 50.7 | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| SWE-Bench Multilingual | K2.7-Code n.a.; K2.6 **76.7**; K2.5 73.0 | 67.7 | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| Terminal-Bench 2.0 (Terminus-2) | K2.7-Code n.a.; K2.6 **66.7**; K2.5 50.8 | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| LiveCodeBench (v6) | K2.7-Code n.a.; K2.6 **89.6**; K2.5 85.0 | 89.0 (v6) | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| BrowseComp | K2.7-Code n.a.; K2.6 **83.2** (86.3 with Agent Swarm); K2.5 74.9 | 44.4 | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| HLE-Full (with tools) | K2.7-Code n.a.; K2.6 **54.0**; K2.5 50.2 | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| HLE-Full (no tools) | K2.7-Code n.a.; K2.6 **34.7**; K2.5 30.1 | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| AIME 2026 | K2.7-Code n.a.; K2.6 **96.4**; (K2.5 on AIME 2025: 96.1) | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6); [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5) | +| HMMT 2026 (Feb) | K2.7-Code n.a.; K2.6 **92.7**; (K2.5 on HMMT 2025 Feb: 95.4) | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6); [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5) | +| GPQA-Diamond | K2.7-Code n.a.; K2.6 **90.5**; K2.5 87.6 | 87.0 (no tools) | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| MMLU-Pro | K2.7-Code n.a.; K2.6 n.a.; K2.5 **87.1** | 86.8 | — | [Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5) | +| MMMU-Pro | K2.7-Code n.a.; K2.6 **79.4**; K2.5 78.5 | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | +| tau2-bench | n.a. — not reported by Moonshot for K2.5, K2.6 or K2.7-Code | n.a. | — | [Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Vendor-reported deltas for K2.7-Code versus K2.6, stated in Moonshot's launch post: +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, +31.5% on MLS Bench Lite, and 30% lower reasoning-token usage ([@Kimi_Moonshot](https://x.com/Kimi_Moonshot/status/2065377579130142937)). + +Evaluation conditions: K2.7-Code and K2.6 were tested with thinking mode enabled via Kimi Code CLI at temperature 1.0, top-p 0.95 and 262,144-token context; MCP-Atlas and MCPMark-Verified used a 100 tool-call/step budget with 32k max tokens per step, averaged over 3 runs ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). K2.6's SWE-Bench family results used an in-house harness adapted from SWE-agent, averaged over 10 runs ([Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). + +## Benchmark explanations + +- **Kimi Code Bench v2** — Moonshot's in-house coding-agent benchmark of realistic software-engineering tasks across 10+ mainstream languages and a full production stack (backend, infra, performance, systems, security, frontend, ML/data) ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **Program Bench** — 200 tasks asking an agent to recreate a program's behavior from only a compiled binary plus documentation (no source, decompilation or internet), judged against 248,000+ fuzz-generated behavioral tests ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **MLS Bench Lite** — official 30-task subset of MLS-Bench testing whether an agent can invent generalizable, scalable ML methods (LLM pretraining/post-training, robotics, world models, CV, RL, ML systems), with 5 hours per task ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **Kimi Claw 24/7 Bench** — Moonshot's in-house long-horizon agentic benchmark for persistent, multi-day coworking tasks: 17 professional scenarios and 610 evaluation points, executed through the OpenClaw harness ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **MCP Atlas** — realistic tool-use tasks served through scalable MCP servers, measuring multi-step tool orchestration ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **MCP Mark Verified** — human-verified MCPMark: MCP tool use across five real server environments (Notion, GitHub, Filesystem, Postgres, Playwright) ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **SWE-Bench Verified** — human-validated real GitHub issues where the model must produce a patch that passes hidden tests; measures repository-level bug fixing. +- **SWE-Bench Pro** — harder, contamination-resistant SWE-bench variant with larger, more complex real-world tasks. +- **SWE-Bench Multilingual** — SWE-bench-style issue resolution across repositories in multiple programming languages. +- **Terminal-Bench 2.0** — terminal/command-line agent tasks run under the default Terminus-2 agent framework; measures end-to-end shell task completion ([Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). +- **LiveCodeBench (v6)** — competitive-programming problems drawn from recent contests to limit training contamination; measures algorithmic code generation. +- **BrowseComp** — hard web-search/browsing agent benchmark for locating obscure facts online over many tool calls; K2.6 also reports an Agent Swarm variant ([Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). +- **HLE-Full** — Humanity's Last Exam full set (text and image): extremely hard expert-written questions, reported with and without search/code tools ([Kimi-K2.6 model card](https://huggingface.co/moonshotai/Kimi-K2.6)). +- **AIME 2026 / AIME 2025** — American Invitational Mathematics Examination: olympiad-qualifier math problems, averaged over many runs ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)). +- **HMMT 2026 (Feb) / HMMT 2025 (Feb)** — Harvard-MIT Mathematics Tournament February contest: harder competition math than AIME. +- **GPQA-Diamond** — hardest subset of graduate-level, Google-proof science multiple-choice questions in physics, chemistry and biology. +- **MMLU-Pro** — harder, reasoning-heavier successor to MMLU for broad multiple-choice knowledge. +- **MMMU-Pro** — massive multi-discipline multimodal understanding, "Pro" version: college-level reasoning over images. +- **tau2-bench** — customer-service-style tool-use agent benchmark with domain policies and simulated users (not reported by Moonshot for this family). diff --git a/packages/app/content/models/kimi-k3.mdx b/packages/app/content/models/kimi-k3.mdx new file mode 100644 index 000000000..14f29fd39 --- /dev/null +++ b/packages/app/content/models/kimi-k3.mdx @@ -0,0 +1,106 @@ +--- +title: 'Kimi K3' +developer: 'Moonshot AI' +releaseDate: 'July 16, 2026 (open weights by July 27, 2026)' +description: "Moonshot AI's 2.8T-parameter / 104B-active MoE flagship built on Kimi Delta Attention and Gated MLA, with native vision and a 1M-token context window." +--- + +## Overview + +Kimi K3 is Moonshot AI's flagship open-weights model, described on its Hugging Face model card as "an open-weight, native multimodal agentic model and our most capable model to date" — a 2.8T-parameter Mixture-of-Experts model built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), with native vision and a 1-million-token context window, billed as "the world's first open 3T-class model" ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). Moonshot's own tech blog frames the release the same way: 2.8 trillion parameters, a 1M-token context window, native vision, Stable LatentMoE routing that effectively activates 16 of 896 experts, and roughly a 2.5× improvement in overall scaling efficiency versus Kimi K2 ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)). + +Moonshot's official blog index dates the Kimi K3 post to 2026-07-16 ([Kimi Research & Tech Blog](https://www.kimi.ai/blog/)), and the launch post on Moonshot's official X account, published July 16, 2026, announced K3 as live on Kimi.com, Kimi Work, Kimi Code and the Kimi API, with "Open Weights by July 27, 2026" ([@Kimi_Moonshot](https://x.com/Kimi_Moonshot/status/2077830247446446450)). The tech blog confirms the same commitment: "the full Kimi K3 model weights will be released by July 27, 2026," with further architecture, training and evaluation detail promised in a forthcoming Kimi K3 technical report ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)). The Hugging Face repository (`moonshotai/Kimi-K3`) carries model-card metadata `license: other` / `license_name: kimi-k3`, and the card states that "both the code repository and the model weights are released under the Kimi K3 License" — i.e. a custom Moonshot license, not MIT or Apache ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). + +Positioning is explicitly agentic and coding-first. The card lists long-horizon coding ("sustains long engineering sessions, navigates massive repositories, and orchestrates terminal tools — from GPU kernel optimization and compiler development to vision-in-the-loop game dev, CAD, and even chip design") and agentic knowledge work (deep research, interactive visualizations, dashboards, motion design and video editing) as headline capabilities, and states that K3 "always has thinking enabled" with a `reasoning_effort` field supporting `low`, `high` and `max` (default `max`), plus a preserved-thinking-history requirement for multi-turn and tool-call usage ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). Notable techniques disclosed by Moonshot include KDA and AttnRes, Stable LatentMoE with Quantile Balancing routing, Per-Head Muon optimization, the Sigmoid Tanh Unit (SiTU) activation, and Gated MLA, alongside native MXFP4 quantization-aware training applied from the SFT stage onward ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3); [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). Serving is recommended on vLLM, SGLang and TokenSpeed, with the hosted API exposed as model id `kimi-k3` at platform.kimi.ai ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); vendor API pricing on the tech blog is $3.00/MTok cache-miss input, $0.30/MTok cache-hit input and $15.00/MTok output ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)). + +## Architecture + +- **Architecture family:** Mixture-of-Experts (MoE), autoregressive, native multimodal (text + image) ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)) +- **Total parameters:** 2.8T ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)) +- **Activated parameters per token:** 104B ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)) +- **Layers:** 93 total, of which 1 dense layer ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `num_hidden_layers: 93`, `first_k_dense_replace: 1` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Attention-layer composition:** 69 KDA layers + 24 Gated MLA layers ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). `config.json` lists `full_attn_layers` at layers 4, 8, 12, … 88, 92, 93 — i.e. full (Gated MLA) attention roughly every 4th layer, with the remaining 69 indices in `kda_layers` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Attention mechanism:** KDA (Kimi Delta Attention, hybrid linear attention) & Gated MLA; MLA configured with `mla_use_nope: true`, `mla_use_output_gate: true`, `q_lora_rank: 1536`, `kv_lora_rank: 512`, `qk_nope_head_dim: 128`, `qk_rope_head_dim: 64`, `v_head_dim: 128` ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md); [config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **KDA configuration:** 96 heads, `head_dim: 128`, `short_conv_kernel_size: 4`, `gate_lower_bound: -5.0`, `use_full_rank_gate: true` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Attention Residuals (AttnRes):** `attn_res_block_size: 12` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)); described by Moonshot as selectively retrieving representations across model depth rather than accumulating them uniformly, and as delivering ~25% higher training efficiency at <2% additional cost ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3); [@Kimi_Moonshot](https://x.com/Kimi_Moonshot/status/2077830247446446450)) +- **Attention hidden dimension / heads:** 7168 hidden, 96 attention heads ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `hidden_size: 7168`, `num_attention_heads: 96`, `num_key_value_heads: 96` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Experts:** 896 routed experts, 16 selected per token, 2 shared experts ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `num_experts: 896`, `num_experts_per_token: 16`, `num_shared_experts: 2`, `moe_layer_freq: 1` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **LatentMoE dimensions:** Latent MoE dimension 3584, MoE hidden dimension per expert 3072 ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `routed_expert_hidden_size: 3584`, `moe_intermediate_size: 3072`, `latent_moe_use_norm: true` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Routing:** Stable LatentMoE framework; sigmoid router with `noaux_tc` top-k method and renormalization (`moe_router_activation_func: sigmoid`, `topk_method: noaux_tc`, `moe_renormalize: true`) ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)); Moonshot adds Quantile Balancing, which derives expert allocation from router-score quantiles and removes heuristic updates and the balancing hyperparameter ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)) +- **Activation function:** SiTU-GLU per the model card ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `hidden_act: "situ"` with `activation_situ_beta: 4.0` and `activation_situ_linear_beta: 25.0` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)). The tech blog names the component Sigmoid Tanh Unit (SiTU) ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)) +- **Dense FFN intermediate size:** 33792 ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Context window:** 1,048,576 tokens ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); `max_position_embeddings: 1048576` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Vocabulary:** 160K per the card; `vocab_size: 163840` in config ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md); [config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Vision encoder:** MoonViT-V2, 401M parameters ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); vision tower config: 27 layers, 12 heads, `vt_hidden_size: 1024`, `vt_intermediate_size: 4096`, `patch_size: 14`, `mm_projector_type: patchmergerv2` ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Precision / quantization:** MXFP4 weights with MXFP8 activations, via quantization-aware training from the SFT stage onward ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)); config declares `format: mxfp4-pack-quantized`, `num_bits: 4`, `group_size: 32`, `quant_method: compressed-tensors`, with attention, shared experts, dense MLP, `lm_head`, vision tower and mm projector excluded from quantization ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Model class / implementation:** `KimiK3ForConditionalGeneration` over a `KimiLinearForCausalLM` text backbone (`model_type: kimi_linear`) ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Multi-token prediction:** none configured (`num_nextn_predict_layers: 0`) ([config.json](https://huggingface.co/moonshotai/Kimi-K3/raw/main/config.json)) +- **Optimizer technique:** Per-Head Muon, extending Muon by optimizing attention heads independently ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3)) +- **Decoding efficiency claim:** KDA enables up to 6.3× faster decoding in million-token contexts ([@Kimi_Moonshot](https://x.com/Kimi_Moonshot/status/2077830247446446450)) +- **Recommended deployment scale:** supernode configurations with 64 or more accelerators; vLLM, SGLang and TokenSpeed recipes ([Kimi K3 Tech Blog](https://www.kimi.com/blog/kimi-k3); [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)) +- **Modality:** Text + Image input per the card's model summary; the introduction also states K3 "understands text, images, and video within the same model" ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)) + +## Official vendor eval scores + +All values are the Kimi K3 (max reasoning effort, temperature 1.0) column of Moonshot's own model-card evaluation table ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| --------------------------------------------------------------------- | ---------------------------------------------------------------- | --------------- | -------- | ----------------------------------------------------------------------------------- | +| GPQA Diamond | 93.5 | 87.0 (no tools) | +7.5% | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| HLE-Full (no tools / with tools) | 43.5 / 56.0 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| AA-LCR (long-context reasoning) | 74.7 | 65.4 | +14.2% | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| CritPt | 23.4 | 3.1 (no tools) | +654.8% | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| Terminal-Bench 2.1 (Kimi Code harness) | 88.3 | 56.4 | +56.6% | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| DeepSWE (Kimi Code harness; 67.3 with mini-SWE-agent) | 67.5 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| ProgramBench | 77.8 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| FrontierSWE | 81.2 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| SWE-Marathon | 42.0 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| Kimi Code Bench 2.0 (in-house) | 72.9 (73.7 with Claude Code harness) | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| MLS-Bench-Lite | 48.3 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| SciCode | 58.7 | 44.6 (subtask) | +31.6% | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| BrowseComp (300K-token context compaction; 90.4 with full 1M context) | 91.2 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| DeepSearchQA (F1) | 95.0 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| MCPMark-Verified | 94.5 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| MCP-Atlas (500-task public subset) | 84.2 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| OSWorld-Verified | 84.8 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| τ³-Banking | 33.4 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| GDPval-AA v2 (Elo) | 1686 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| MMMU-Pro (no tools / with Python) | 81.6 / 83.4 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| Video-MME (w. subtitles) | 90.0 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| OmniDocBench | 91.1 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| SWE-bench Verified | n.a. — not reported in Moonshot's K3 model-card table | 70.7 | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| LiveCodeBench | n.a. — not reported for K3 | 89.0 (v6) | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| AIME / HMMT / MMLU-Pro | n.a. — not reported for K3 | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | +| tau2-bench | n.a. — not reported for K3 (Moonshot reports τ³-Banking instead) | n.a. | — | [Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. + +## Benchmark explanations + +- **GPQA Diamond** — hardest subset of graduate-level, Google-proof multiple-choice science questions in physics, chemistry and biology; measures expert-level scientific reasoning. +- **HLE-Full** — Humanity's Last Exam, full set (text and image questions): very hard expert-written questions across many disciplines; Moonshot reports scores without and with tool augmentation ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **AA-LCR** — Artificial Analysis Long Context Reasoning: reasoning over very long inputs; K3's figure is cited from Artificial Analysis as of July 23, 2026 ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **CritPt** — hard research-level physics reasoning benchmark; K3's figure is cited from Artificial Analysis as of July 23, 2026 ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **Terminal-Bench 2.1** — end-to-end terminal/command-line agent tasks: can the model drive a shell to complete real system and engineering jobs. +- **DeepSWE** — agentic software-engineering task suite (v1.1 tasks) measuring repository-level issue resolution with an agent harness ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **ProgramBench** — asks a coding agent to recreate a program's behavior from only a compiled binary plus documentation, judged against fuzz-generated behavioral tests ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **FrontierSWE** — frontier-difficulty software-engineering tasks scored by a dominance metric recomputed with the official evaluation script ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **SWE-Marathon** — very long-horizon software-engineering tasks, including GPU tasks with performance gates and anti-cheat validators ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **Kimi Code Bench 2.0** — Moonshot's in-house coding-agent benchmark spanning realistic engineering tasks across 10+ languages and a full production stack, including cybersecurity/safety tasks ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md); [Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **MLS-Bench-Lite** — 30-task official subset of MLS-Bench, testing whether an agent can invent generalizable, scalable ML methods within a 5-hour budget ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **SciCode** — scientific code-generation tasks derived from research problems; K3's figure is cited from Artificial Analysis ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **BrowseComp** — hard web-browsing/search agent benchmark: finding hard-to-locate facts online across many tool calls. +- **DeepSearchQA (F1)** — deep research question answering with search tools, scored by F1 against reference answers. +- **MCPMark-Verified** — human-verified edition of MCPMark, testing MCP tool use across Notion, GitHub, Filesystem, Postgres and Playwright servers ([Kimi-K2.7-Code model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code)). +- **MCP-Atlas** — realistic tool-use tasks over scalable MCP servers, run here on the 500-task public subset with a 100-turn limit ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **OSWorld-Verified** — computer-use agent benchmark: completing real desktop/OS tasks through GUI interaction. +- **τ³-Banking** — tool-agent benchmark in a banking domain testing multi-turn policy-following and tool calls; K3's figure is cited from Artificial Analysis ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **GDPval-AA v2 (Elo)** — Artificial Analysis Elo rating for economically valuable, real-world professional work tasks ([Kimi-K3 model card](https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md)). +- **MMMU-Pro** — massive multi-discipline multimodal understanding, "Pro" (harder) version: college-level reasoning over images. +- **Video-MME** — video understanding across short, medium and long videos; reported here with subtitles. +- **OmniDocBench** — document parsing/OCR accuracy, scored as (1 − normalized Levenshtein distance) × 100 ([Kimi-K2.5 model card](https://huggingface.co/moonshotai/Kimi-K2.5)). +- **SWE-bench Verified** — human-validated subset of real GitHub issues requiring a patch that passes hidden tests (not reported for K3). +- **LiveCodeBench** — contamination-resistant competitive-programming benchmark using recently published contest problems (not reported for K3). +- **AIME / HMMT** — high-school olympiad-level competition math exams (not reported for K3). +- **MMLU-Pro** — harder, reasoning-focused version of MMLU multiple-choice knowledge testing (not reported for K3). +- **tau2-bench** — customer-service-style tool-use agent benchmark with domain policies (not reported for K3). diff --git a/packages/app/content/models/llama-3-3-70b.mdx b/packages/app/content/models/llama-3-3-70b.mdx new file mode 100644 index 000000000..69ccb27dc --- /dev/null +++ b/packages/app/content/models/llama-3-3-70b.mdx @@ -0,0 +1,63 @@ +--- +title: 'Llama 3.3 70B Instruct' +developer: 'Meta' +releaseDate: 'December 6, 2024' +description: "Meta's 70B dense instruction-tuned transformer with GQA and a 128K context window, released under the Llama 3.3 Community License." +--- + +## Overview + +Llama 3.3 70B Instruct is Meta's instruction-tuned, text-only multilingual large language model, released on December 6, 2024 ([Hugging Face: meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct); [Meta Llama 3.3 model card on GitHub](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). It is an auto-regressive model built on an optimized transformer architecture with 70B parameters (the Hugging Face repo lists 71B params for the checkpoint), a 128k context window, and pretraining on more than 15 trillion tokens with a knowledge cutoff of December 2023 ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)). It takes multilingual text as input and produces multilingual text and code, with official support for English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)). + +Positioning: Meta describes Llama 3.3 70B as offering performance similar to Llama 3.1 405B at a fraction of the serving cost, released "to close out the year" 2024 ([Meta AI blog, "The future of AI: Built with Llama"](https://ai.meta.com/blog/future-of-ai-built-with-llama/)). Meta's own instruction-tuned comparison table supports this: Llama 3.3 70B Instruct matches Llama 3.1 70B on MMLU (86.0) while beating both 3.1 70B and, on several tasks, the 405B model — for example IFEval 92.1 vs. 88.6 for 405B and MATH 77.0 vs. 73.8 for 405B ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)). + +Licensing and techniques: the model is distributed under the custom commercial Llama 3.3 Community License Agreement, which grants royalty-free use, reproduction, modification and distribution but requires "Built with Llama" attribution, prefixing derivative model names with "Llama", a NOTICE file attribution, compliance with the Acceptable Use Policy, and a separate Meta-granted license for licensees exceeding 700 million monthly active users as of the release date ([Hugging Face model card / license text](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct); [license file](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/LICENSE)). Architecturally, all Llama 3 versions use Grouped-Query Attention for improved inference scalability, and the released BF16 checkpoints can additionally be loaded in 8-bit or 4-bit via `bitsandbytes`/`transformers` ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)). + +## Architecture + +- Model type: auto-regressive language model using an optimized transformer architecture; dense (no mixture-of-experts) ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)) +- Total parameters: 70B per Meta's model information table (Hugging Face lists 71B params for the checkpoint); all parameters active per token since the model is dense ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)) +- Active parameters: n.a. as a separately reported figure — Meta reports only the 70B dense parameter count ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)) +- Layers: 80 (`num_hidden_layers: 80`; the Meta/HF cards do not state layer count — value from a mirrored copy of the official config.json, as the `meta-llama` repo is gated) ([config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Hidden size: 8192 ([config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Attention: Grouped-Query Attention (GQA), 64 query heads and 8 key-value heads (8 query heads per KV group) ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct); [config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Experts / MoE: none — dense model, no expert routing reported ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)) +- Context window: 128k tokens (`max_position_embeddings: 131072`) ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Positional embeddings: RoPE with `rope_theta: 500000.0` ([config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Vocabulary size: 128,256 ([config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Precision / quantization: released in BF16 (`torch_dtype: bfloat16`); checkpoints can be run in 8-bit or 4-bit with `bitsandbytes` (`load_in_8bit=True` / `load_in_4bit=True`) ([Hugging Face model card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct); [config.json mirror](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json)) +- Training data: 15T+ pretraining tokens, knowledge cutoff December 2023 ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)) + +## Official vendor eval scores + +Meta-reported results for Llama 3.3 70B Instruct (English text, instruction-tuned evaluation table). + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ----------------------------------------------------- | -------------------------------------------------- | --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| MMLU (CoT, 0-shot, macro_avg/acc) | 86.0 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| MMLU Pro (CoT, 5-shot, macro_avg/acc) | 68.9 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| IFEval (steerability) | 92.1 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| GPQA Diamond (CoT, 0-shot, acc) | 50.5 | 87.0 (no tools) | -42.0% | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| HumanEval (0-shot, pass@1) | 88.4 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| MBPP EvalPlus (base, 0-shot, pass@1) | 87.6 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| MATH (CoT, 0-shot, sympy_intersection_score) | 77.0 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| BFCL v2 (0-shot, overall_ast_summary/macro_avg/valid) | 77.3 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| MGSM (0-shot, exact match, multilingual) | 91.1 | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md); [Hugging Face card](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct) | +| SWE-bench Verified | n.a. — not reported in Meta's Llama 3.3 model card | 70.7 | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md) | +| Codeforces | n.a. — not reported by Meta for this model | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md) | +| τ-bench (Tau-bench) | n.a. — not reported by Meta for this model | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md) | +| AIME | n.a. — not reported by Meta for this model | n.a. | — | [Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. + +## Benchmark explanations + +- **MMLU (CoT)** — Massive Multitask Language Understanding: multiple-choice questions across 57 academic and professional subjects, evaluated with chain-of-thought prompting; measures broad knowledge and reasoning ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **MMLU Pro (CoT)** — a harder, more reasoning-intensive refresh of MMLU with more answer options and cleaner items; measures robust general reasoning ([Hugging Face card links the MMLU-Pro dataset](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)). +- **IFEval** — instruction-following evaluation with verifiable constraints (format, length, keywords); Meta classifies it under "Steerability" ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **GPQA Diamond (CoT)** — hardest subset of graduate-level, Google-proof science questions; Meta classifies it under "Reasoning" and scores accuracy ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **HumanEval** — Python function-completion problems scored pass@1 by unit tests; measures basic code generation correctness ([Meta model card, "Code" category](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **MBPP EvalPlus (base)** — Mostly Basic Python Problems with the EvalPlus expanded test suite, pass@1; measures code correctness under stricter tests ([Meta model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **MATH (CoT)** — competition math word problems graded by symbolic equivalence (sympy); measures multi-step mathematical reasoning ([Meta model card, "Math" category](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **BFCL v2** — Berkeley Function-Calling Leaderboard v2, scored on abstract-syntax-tree validity of generated function calls; measures tool-use/function-calling accuracy ([Meta model card, "Tool Use" category](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). +- **MGSM** — Multilingual Grade School Math, exact-match accuracy on GSM8K-style problems translated into multiple languages; measures multilingual arithmetic reasoning ([Meta model card, "Multilingual" category](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)). diff --git a/packages/app/content/models/minimax-m27.mdx b/packages/app/content/models/minimax-m27.mdx new file mode 100644 index 000000000..6b8ebd50d --- /dev/null +++ b/packages/app/content/models/minimax-m27.mdx @@ -0,0 +1,85 @@ +--- +title: 'MiniMax M2.5 / M2.7' +developer: 'MiniMax' +releaseDate: 'February 12, 2026 (M2.5); March 18, 2026 (M2.7)' +description: "MiniMax's 229B-parameter / ~10B-active MoE family with GQA, FP8 block quantization, and a 192K context window." +--- + +## Overview + +MiniMax-M2.5 and MiniMax-M2.7 are the two most recent open-weights checkpoints of MiniMax's M2 series: sparse mixture-of-experts language models built, in the words of the series technical report, on the thesis that "mini activations can unleash maximum real-world intelligence," with a flagship configuration of 229.9B total parameters and only 9.8B activated per token ([The MiniMax-M2 Series, arXiv:2605.26494](https://arxiv.org/abs/2605.26494)). M2.5 was released on 2026-02-12 — the date MiniMax stamps on its launch announcement ([MiniMax M2.5 announcement](https://www.minimax.io/news/minimax-m25)), matching the Hugging Face repository creation timestamp of 2026-02-12 ([HF API metadata for MiniMax-M2.5](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M2.5)) and the "MiniMax-M2.5 Version Release Date: 2026-02-13" in its model license ([MiniMax-M2.5 LICENSE-MODEL](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/LICENSE-MODEL)). M2.7 followed on 2026-03-18 as a point release of the same series ([MiniMax M2.7 announcement](https://www.minimax.io/news/minimax-m27-en); [MiniMax API release notes](https://platform.minimax.io/docs/release-notes/models)), shipped in two serving variants, MiniMax-M2.7 and M2.7-highspeed ([MiniMax API release notes](https://platform.minimax.io/docs/release-notes/models)). + +Positioning is squarely agentic and coding/office-work oriented. MiniMax describes M2.5 as "extensively trained with reinforcement learning in hundreds of thousands of complex real-world environments" and "SOTA in coding, agentic tool use and search, office work, and a range of other economically valuable tasks," headlined by 80.2% SWE-Bench Verified, 51.3% Multi-SWE-Bench and 76.3% BrowseComp ([MiniMax-M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)). Cost and speed are central to the pitch: M2.5-Lightning sustains 100 tokens/second at $0.30/M input and $2.40/M output, standard M2.5 runs at 50 tokens/second for half that, and MiniMax claims "$1 to run the model continuously for an hour" at 100 tok/s ([MiniMax-M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)). M2.7 is pitched as "our first model deeply participating in its own evolution," able to build complex agent harnesses and complete elaborate productivity tasks using Agent Teams, complex Skills and dynamic tool search; MiniMax reports an internal M2.7 version autonomously optimizing a programming scaffold over 100+ rounds for a 30% performance improvement ([MiniMax-M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)). + +Architecturally the point release is a continuation, not a redesign: the series report treats the post-training pipeline as components that "co-evolve from M2 through M2.5 to the latest M2.7" on the same MoE base — a 62-layer decoder-only Transformer, 256 fine-grained experts with sigmoid gating, full multi-head attention with GQA, a 192K-token native context window and a Multi-Token Prediction module doubling as a speculative-decoding draft path ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). Both released checkpoints report the same 229B model size on Hugging Face and identical `config.json` contents, including FP8 (`float8_e4m3fn`) block-wise quantization ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json); [M2.7 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/raw/main/config.json)). Notable training-side techniques include the Forge agent-native RL framework (with a tree-structured sample-merging strategy MiniMax credits with ~40× training speedup) and the CISPO algorithm for MoE training stability ([MiniMax-M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)), plus windowed-FIFO scheduling and prefix-tree merging in the report's system description ([arXiv:2605.26494](https://arxiv.org/abs/2605.26494)). Licensing differs between the two: M2.5's HF metadata declares `license_name: modified-mit` pointing at the GitHub LICENSE ([HF API metadata for MiniMax-M2.5](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M2.5)) with a "MINIMAX MODEL LICENSE" file in-repo ([M2.5 LICENSE-MODEL](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/LICENSE-MODEL)), whereas M2.7 ships a "NON-COMMERCIAL LICENSE" that permits non-commercial use on MIT-style terms, requires "Built with MiniMax M2.7" attribution for commercial use, and prohibits any commercial use without prior written authorization from MiniMax ([M2.7 LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/raw/main/LICENSE)). + +## Architecture + +Shared by both checkpoints unless noted. M2.5 and M2.7 publish byte-identical architecture fields in `config.json`. + +- **Total parameters:** 229.9B for the series flagship per the technical report ([arXiv:2605.26494](https://arxiv.org/abs/2605.26494)); Hugging Face safetensors metadata reports 228,703,644,928 for M2.5 and 228,689,764,864 for M2.7, both surfaced as "229B params" on the cards ([HF API metadata M2.5](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M2.5); [HF API metadata M2.7](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M2.7); [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)). +- **Activated parameters:** 9.8B per token for the series flagship ([arXiv:2605.26494](https://arxiv.org/abs/2605.26494)); the report describes M2.7 as having "∼10B activated parameters" ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Architecture class:** `MiniMaxM2ForCausalLM`, `model_type: minimax_m2` — i.e. M2.7 reuses the M2 model class ([M2.7 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/raw/main/config.json); [M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)). +- **Layers:** 62 (`num_hidden_layers: 62`), a decoder-only Transformer ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json); [MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Hidden size:** 3,072; expert intermediate size 1,536; no shared expert (`shared_intermediate_size: 0`) ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)). +- **Attention:** grouped-query attention with 48 query heads and 8 key-value heads, head dim 128, full (non-sliding) attention on every layer (`attn_type_list` is all 1s) ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)); the report confirms "full multi-head attention with GQA," 48 query / 8 KV heads ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **QK normalization:** `use_qk_norm: true`, `qk_norm_type: per_layer` ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json); [M2.7 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/raw/main/config.json)). +- **Positional encoding:** RoPE with `rope_theta: 5,000,000` and `rotary_dim: 64` (partial rotary over the 128-dim head) ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)); the report lists Rotary Position Embeddings ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Experts:** 256 routed experts (`num_local_experts: 256`), 8 activated per token (`num_experts_per_tok: 8`), sigmoid scoring with learnable routing bias (`scoring_func: sigmoid`, `use_routing_bias: true`) ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)); the report describes "256 fine-grained experts with sigmoid gating" and expert-specific bias terms ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Multi-token prediction:** `use_mtp: true`, `num_mtp_modules: 3`, `mtp_transformer_layers: 1` ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)). The report details a single MTP module (K=1) in pre-training expanded to three modules (K=3) during the continued-pretraining decay phase, MTP loss weight annealed 0.3 → 0.1, with the three modules used as speculative-decoding draft heads verified by the main model in one forward pass ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Precision / quantization:** FP8 weights — `quant_method: fp8`, `fmt: float8_e4m3fn`, dynamic activation scaling, `weight_block_size: [128, 128]`, with `gate`, `e_score_correction_bias` and `lm_head` excluded from conversion; base dtype bfloat16 ([M2.7 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/raw/main/config.json)). HF lists tensor types F32 · BF16 · F8_E4M3 for both repos ([M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5); [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)). +- **Context window:** `max_position_embeddings: 196608` (192K) ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)); the report states a "192K-token native context window" reached via 8K → 32K → 192K context-extension stages ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Vocabulary:** 200,064 tokens; `tie_word_embeddings: false`; activation `silu`; `rms_norm_eps: 1e-06` ([M2.5 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/raw/main/config.json)). +- **Pre-training scale:** 29.2T tokens total (19.9T constant phase, 9.3T decay-phase budget) ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). +- **Inference settings:** `temperature=1.0`, `top_p=0.95`, `top_k=40`, with per-model default system prompts; served through SGLang, vLLM, Transformers (plus KTransformers for M2.5, NVIDIA NIM for M2.7) ([M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5); [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)). +- **Multimodality:** text-in/text-out (`pipeline_tag: text-generation` for both repos) ([HF API metadata M2.7](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M2.7)); M2.7's MM Claw benchmark is described as an internal multi-modal office benchmark ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). + +## Official vendor eval scores + +Vendor-reported only. Unless noted, both columns come from Table 4 of MiniMax's own M2-series technical report, evaluated with thinking enabled and the interleaved-thinking protocol; AIME 2026, GPQA-Diamond, SciCode, IFBench, AA-LCR, HLE and MMLU-Pro follow the Artificial Analysis Index v4.0 protocol, no tools, pass@1 ([MiniMax-M2 series report HTML](https://arxiv.org/html/2605.26494v2)). + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| --------------------------- | ---------------------------------- | --------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| SWE-Bench Verified | M2.5: 80.2% · M2.7: n.a. | 70.7 | — | [MiniMax-M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) / [MiniMax M2.5 announcement](https://www.minimax.io/news/minimax-m25) (not reported for M2.7 in fetched MiniMax sources) | +| SWE-bench Pro | M2.5: 55.4 · M2.7: 56.2 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.7 56.22% also in [MiniMax M2.7 announcement](https://www.minimax.io/news/minimax-m27-en) | +| SWE-bench Multilingual | M2.5: 74.1 · M2.7: 76.5 | 67.7 | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.7 76.5 also on the [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) | +| Multi-SWE-bench | M2.5: 51.3 · M2.7: 52.7 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.5 51.3% also in [MiniMax M2.5 announcement](https://www.minimax.io/news/minimax-m25) | +| Terminal-Bench 2.0 | M2.5: 51.7 · M2.7: 57.0 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.7 57.0% also in [MiniMax M2.7 announcement](https://www.minimax.io/news/minimax-m27-en) | +| NL2Repo | M2.5: 26.6 · M2.7: 39.8 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2) | +| MLE Bench Lite (medal rate) | M2.5: 51.5 · M2.7: 66.6 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.7 66.6% also on the [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) | +| VIBE-Pro | M2.5: 54.2 · M2.7: 55.6 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2) | +| BrowseComp | M2.5: 76.3 · M2.7: 77.8 | 44.4 | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.5 76.3% (with context management) in [MiniMax M2.5 announcement](https://www.minimax.io/news/minimax-m25) | +| Wide Search | M2.5: 70.3 · M2.7: 75.2 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2) | +| Toolathlon | M2.5: 38.3 · M2.7: 46.3 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.7 46.3% also in [MiniMax M2.7 announcement](https://www.minimax.io/news/minimax-m27-en) | +| GDPval-AA | M2.5: 35.0 · M2.7: 50.0 (ELO 1495) | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); ELO 1495 from the [M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7) | +| AIME 2026 | M2.5: 87.2 · M2.7: 94.2 | n.a. | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2) | +| GPQA-Diamond | M2.5: 85.2 · M2.7: 89.8 | 87.0 (no tools) | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); M2.5 85.2 also in the [M2.5 model card appendix](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) | +| MMLU-Pro | M2.5: 85.2 · M2.7: 81.8 | 86.8 | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2) (as printed; M2.7 is lower than M2.5 in the vendor table) | +| HLE (no tools) | M2.5: 19.0 · M2.7: 28.0 | 26.7 (no tools) | — | [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2); the M2.5 card appendix lists 19.4 for "HLE w/o tools" ([M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)) | +| tau2-bench | n.a. | n.a. | — | Named among agent benchmarks in the report's attention-ablation section but no M2.5/M2.7 score published ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)) | +| LiveCodeBench | n.a. | 89.0 (v6) | — | Not reported for M2.5 or M2.7 in any fetched MiniMax source ([MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2)) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Other vendor-reported figures: M2.5 scored 79.7 on SWE-Bench Verified under Droid scaffolding and 76.1 under OpenCode scaffolding, averaged 3.52M tokens and 22.8 minutes per SWE-Bench Verified task (vs 31.3 minutes for M2.1, a 37% speedup), and achieved a 59.0% average GDPval-MM win rate against mainstream models ([MiniMax M2.5 announcement](https://www.minimax.io/news/minimax-m25)). M2.5's card appendix additionally reports AIME25 86.3, SciCode 44.4, IFBench 70.0 and AA-LCR 69.5 ([M2.5 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)). For M2.7, MiniMax reports MM Claw 62.7%, 97% skill compliance across 40+ complex skills, MEWC v2 63.3, Finance Modeling Pro 57.0, RISE 64.3 and HyperTask 67.6 ([M2.7 model card](https://huggingface.co/MiniMaxAI/MiniMax-M2.7); [MiniMax-M2 series report, Table 4](https://arxiv.org/html/2605.26494v2)). + +## Benchmark explanations + +- **SWE-Bench Verified** — human-validated set of real GitHub issues; scores the share of issues whose generated patch passes the repository's tests ([MiniMax evaluated it with Claude Code scaffolding averaged over 4 runs](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)). +- **SWE-bench Pro** — industry-grade repository repair, a harder successor to SWE-bench aimed at realistic enterprise codebases ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **SWE-bench Multilingual** — cross-language repository-level editing, testing coding competence beyond Python ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **Multi-SWE-bench** — multi-repo task transfer: issue resolution spanning multiple repositories/languages ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **Terminal-Bench 2.0** — terminal and system-operation tasks in a sandbox (8 vCPU / 16 GB, 2-hour timeout, Terminus-2 scaffold in MiniMax's setup), measuring shell-agent reliability ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **NL2Repo** — MiniMax's internal benchmark for synthesizing an entire repository from a natural-language specification ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **MLE Bench Lite** — autonomous machine-learning engineering across 22 Kaggle-style competitions; scored as test-set medal rate after 24-hour runs on a single A30 ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **VIBE-Pro** — MiniMax's internal end-to-end full-stack application development benchmark, verifying interaction logic and visual fidelity in containerized deployments ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **BrowseComp** — hard open-web search benchmark measuring an agent's ability to locate obscure facts and synthesize an answer ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **Wide Search** — broad-coverage web research benchmark requiring many parallel lookups aggregated into one answer, run in the WebExplorer agent framework ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **Toolathlon** — heterogeneous tool-use benchmark exercising many different external tools in one agent workflow ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **GDPval-AA** — the Artificial-Analysis-judged subset of OpenAI's GDPval suite of economically valuable office tasks, scored on deliverable quality ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **AIME 2026** — American Invitational Mathematics Examination problems; competition mathematics accuracy ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **GPQA-Diamond** — hardest split of graduate-level physics/chemistry/biology questions written to be Google-proof ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **MMLU-Pro** — broad multi-domain knowledge benchmark with harder, more distractor-rich multiple-choice items than MMLU ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **HLE (Humanity's Last Exam, no-tool subset)** — frontier-difficulty expert questions across disciplines, answered without tools ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **tau2-bench** — simulated-user tool-use benchmark with domain policies (retail/telecom splits) measuring policy-compliant multi-turn agent behavior; not scored for M2.5/M2.7 (n.a.) ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **LiveCodeBench** — rolling competitive-programming benchmark built from recent contest problems to resist training contamination; not reported by MiniMax for this family (n.a.). +- **RISE / MM Claw / MEWC v2 / Finance Modeling Pro / HyperTask** — MiniMax-internal benchmarks for realistic multi-step browsing with a Playwright browser, multimodal office "claw" tasks, hard Microsoft Excel World Championship problems, expert-rubric-scored Excel financial modeling, and long-horizon multi-requirement "vibe coding" respectively ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). diff --git a/packages/app/content/models/minimax-m3.mdx b/packages/app/content/models/minimax-m3.mdx new file mode 100644 index 000000000..9edceb30d --- /dev/null +++ b/packages/app/content/models/minimax-m3.mdx @@ -0,0 +1,91 @@ +--- +title: 'MiniMax M3' +developer: 'MiniMax' +releaseDate: 'June 1, 2026' +description: "MiniMax's ~428B-parameter / ~23B-active multimodal MoE with GQA attention, a 1M-token context window, and 7 multi-token-prediction modules." +--- + +## Overview + +MiniMax-M3 is the flagship open-weights model of the M-series from Shanghai-based AI company MiniMax, described on its Hugging Face model card as "a native multimodal model with 1M context" with "~428B parameters and ~23B activated parameters" ([MiniMaxAI/MiniMax-M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). MiniMax's API release notes date the launch to "Jun. 1, 2026," describing M3 as "the latest M-series language model for agentic reasoning, tool use, coding, multimodal chat input, and long-context tasks" ([MiniMax API release notes](https://platform.minimax.io/docs/release-notes/models)); the Hugging Face repository itself was created on 2026-06-02 ([HF API metadata for MiniMaxAI/MiniMax-M3](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M3)). The model is a mixture-of-experts vision-language model: it accepts text, image and video input and emits text, and the repository is published under `pipeline_tag: image-text-to-text` ([M3 README source](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/README.md)). + +Positioning is explicitly agentic and coding-first. MiniMax's launch blog markets M3 as delivering "coding & agentic frontier" capability with autonomous task decomposition, tool invocation, multi-step reasoning, autonomous browsing and long-range coding, and calls it the first open-weight model combining frontier coding capability, million-token context and native multimodality ([MiniMax M3 model page](https://www.minimax.io/models/text/m3)). The model card frames three headline capabilities: native multimodality ("mixed-modality training from the very first step"), context scaling through sparse attention, and "frontier-level performance across long-horizon agentic benchmarks, excelling in both coding and cowork" ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). NVIDIA's hosted model card adds concrete target workloads: long-form video understanding up to 30 minutes, long-horizon coding tasks of 8+ hours, and agentic/tool-use workflows, with two switchable reasoning modes ([NVIDIA build.nvidia.com M3 model card](https://build.nvidia.com/minimaxai/minimax-m3/modelcard)); the card itself exposes three modes via the `thinking` parameter — `enabled`, `adaptive`, `disabled` ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). + +The defining technical contribution is **MiniMax Sparse Attention (MSA)**, "a high-performance sparse attention operator designed for million-token contexts" that, compared with GQA, "dramatically reduces the attention compute and memory footprint while preserving model quality," yielding "9× prefill and 15× decode speedups compared to M2 at 1M context, reducing per-token compute to 1/20" ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). The accompanying technical report describes MSA as blockwise sparse attention built on top of GQA, with a lightweight Index Branch that scores KV blocks and independently selects a Top-k subset per GQA group, a Main Branch performing exact block-sparse attention over selected blocks, exp-free Top-k selection, and a KV-outer sparse attention kernel; on a 109B-parameter native-multimodal testbed it reports 28.4× lower per-token attention compute at 1M context and 14.2× prefill / 7.6× decoding wall-clock speedups on H800 while performing "on par with GQA" ([MiniMax Sparse Attention, arXiv:2606.13392](https://arxiv.org/abs/2606.13392)). Licensing is **not** MIT: the repository ships the "MINIMAX COMMUNITY LICENSE," which grants rights for non-commercial purposes, requires prominent "Built with MiniMax M3" attribution for commercial use, requires a one-time notice to api@minimax.io below US$20M annual revenue, and requires separate prior written authorization above US$20M ([M3 LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/LICENSE)); HF card metadata tags it `license: other`, `license_name: minimax-community` ([M3 README source](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/README.md)). + +## Architecture + +- **Total parameters:** ~428B per the model card ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)); Hugging Face safetensors metadata reports 427,040,140,160 parameters exactly ([HF API metadata](https://huggingface.co/api/models/MiniMaxAI/MiniMax-M3)). +- **Activated parameters:** ~23B per token ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). +- **Architecture class / model type:** `MiniMaxM3SparseForConditionalGeneration`, `model_type: minimax_m3_vl`; the text tower is `MiniMaxM3SparseForCausalLM` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Layers:** 60 (`num_hidden_layers: 60`) ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Hidden size:** 6,144; MoE expert intermediate size 3,072; dense-layer intermediate size 12,288; shared-expert intermediate size 3,072 ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Attention layout:** 64 query heads, 4 key-value heads (GQA), head dim 128 ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)); MSA is explicitly "blockwise sparse attention built upon GQA" ([arXiv:2606.13392](https://arxiv.org/abs/2606.13392)). +- **QK normalization:** `use_qk_norm: true` with `qk_norm_type: per_head` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Partial RoPE:** `rotary_dim: 64` with `partial_rotary_factor: 0.5` and `rope_theta: 5,000,000` — i.e. RoPE applied to half of each 128-dim head ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Sparse attention (MSA) configuration:** `use_sparse_attention: true`, `sparse_index_dim: 128`, `sparse_num_index_heads: 4`, `sparse_topk_blocks: 16`, `sparse_block_size: 128`, `sparse_score_type: max`, `sparse_local_block: 1`; sparse attention is disabled in the first 3 layers and enabled in layers 4–60 via the `sparse_attention_freq` list ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **MSA mechanism:** lightweight Index Branch scores KV blocks and selects a Top-k subset independently per GQA group (group-specific sparse retrieval); Main Branch runs exact block-sparse attention over only those blocks; exp-free Top-k selection; KV-outer sparse attention kernel for tensor-core utilization ([arXiv:2606.13392](https://arxiv.org/abs/2606.13392)). MiniMax states MSA partitions KV into blocks more precisely than DSA/MoBA-style approaches and is "more than 4× faster than Flash-Sparse-Attention and flash-moba" ([MiniMax M3 launch blog](https://www.minimax.io/blog/minimax-m3)). +- **Experts:** 128 routed experts (`num_local_experts: 128`), top-4 routed per token (`num_experts_per_tok: 4`), plus 1 shared expert (`n_shared_experts: 1`); sigmoid routing scores with routing bias and `routed_scaling_factor: 2.0`; first 3 layers dense, layers 4–60 MoE per `moe_layer_freq` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Activation / norms:** `hidden_act: swigluoai` with `swiglu_alpha: 1.702`, `swiglu_limit: 7.0`; RMSNorm with `use_gemma_norm: true`, `rms_norm_eps: 1e-06` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Multi-token prediction:** `num_mtp_modules: 7`, `num_nextn_predict_layers: 1` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Context window:** `max_position_embeddings: 1,048,576` (1M tokens) ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)); MiniMax markets "up to 1M tokens" with a guaranteed minimum of 512K ([MiniMax M3 model page](https://www.minimax.io/models/text/m3)). +- **Vocabulary:** 200,064 tokens; image token index 200025, video token index 200026 ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Vision tower:** CLIP-style vision encoder, 32 layers, hidden size 1,280, 16 heads, patch size 14, image size 2016, 3D RoPE position embeddings, projection to 6,144 dims, patch-merge token compression (spatial merge 2, temporal patch 2) ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)). +- **Precision:** `torch_dtype: bfloat16` ([M3 config.json](https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/config.json)); the HF card lists tensor types BF16 · F32 ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). MiniMax also publishes an MXFP8 checkpoint referenced in ATOM/MXFP4-MXFP8 usage guidance ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). +- **Recommended inference settings:** `temperature=1.0`, `top_p=0.95`; served via SGLang, vLLM, Transformers, KTransformers, unsloth, ATOM ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). +- **Reasoning modes:** `thinking` parameter with `enabled` / `adaptive` / `disabled` ([M3 model card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). + +## Official vendor eval scores + +All rows below are MiniMax-reported scores taken from the official benchmark table published in the MiniMax-M3 Hugging Face repository (`figures/benchmark.jpeg`, referenced from the model card) or from MiniMax's own launch blog / model page. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ------------------ | ---------------------- | --------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| SWE-Bench Verified | 80.5 | 70.7 | +13.9% | [MiniMax M3 vendor benchmark table (HF repo)](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| SWE-Bench Pro | 59.0 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); also 59.0% in the [M3 launch blog](https://www.minimax.io/blog/minimax-m3) | +| Terminal Bench 2.1 | 66.0 | 56.4 | +17.0% | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); also 66.0% in the [M3 launch blog](https://www.minimax.io/blog/minimax-m3) | +| SWE-fficiency | 34.8 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); also 34.8% in the [M3 launch blog](https://www.minimax.io/blog/minimax-m3) | +| NL2Repo | 42.1 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| LiveSQLBench | 40.2 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| KernelBench Hard | 28.8 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); also 28.8% in the [M3 launch blog](https://www.minimax.io/blog/minimax-m3) | +| PostTrainBench | 37.1 (rank #3 overall) | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); [MiniMax M3 model page](https://www.minimax.io/models/text/m3) | +| BrowseComp | 83.5 | 44.4 | +88.1% | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); [MiniMax M3 model page](https://www.minimax.io/models/text/m3) | +| MCP Atlas | 74.2 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg); also 74.2% in the [M3 launch blog](https://www.minimax.io/blog/minimax-m3) | +| Apex-Agents | 27.7 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| OSWorld-Verified | 75.2 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| MMMU-Pro | 78.1 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| Video-MMMU | 84.6 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| USAMO 2026 | 36 / 42 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| IMO 2025 | 35 / 42 | n.a. | — | [MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg) | +| tau2-bench | n.a. | n.a. | — | Not reported for M3 in any MiniMax source fetched ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg), [model page](https://www.minimax.io/models/text/m3)) | +| LiveCodeBench | n.a. | 89.0 (v6) | — | Not reported for M3 in any MiniMax source fetched ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)) | +| AIME | n.a. | n.a. | — | M3's reasoning block reports IMO 2025 and USAMO 2026 instead ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)) | +| GPQA Diamond | n.a. | 87.0 (no tools) | — | Not reported for M3 in any MiniMax source fetched ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)) | +| MMLU-Pro | n.a. | 86.8 | — | Not reported for M3 in any MiniMax source fetched ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)) | +| HLE | n.a. | 26.7 (no tools) | — | Not reported for M3 in any MiniMax source fetched ([vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Additional vendor-reported figures from the same table (comparison context): the vendor chart also lists SWE Atlas-QnA 37.9, SWE Atlas-Test Writing 30.8, CL-bench 20.5, VIBE-V2 50.1, SVG-Bench 63.7, PaperBench 52.6, DRACO 73.2, GDPval rubrics 74.8, BankerToolBench 76.1, OfficeQA Pro 45.1, SpreadSheetBench-v1 89.4, YC-Bench 2.1M, LOCA-Bench (256k) 49.3, Claw-Eval 74.5, OmniDocBench 91.6 and VideoMME (w/ sub) 85.4 for M3 ([MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). + +Vendor evaluation methodology notes (from the same chart): SWE-Bench Verified was run on internal infrastructure with Claude Code as scaffolding, default system prompt overridden, 4 runs averaged; Terminal-bench 2.1 used an 8C16G sandbox, 2-hour timeout, 128K max output tokens and Terminus 2 scaffolding; SWE-fficiency used the open-source dataset/workflow in a 1C2G sandbox with a 2-hour timeout ([MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). + +## Benchmark explanations + +- **SWE-Bench Verified** — human-validated subset of real GitHub issues; measures whether the model's patch resolves the issue and passes the repo's tests ([MiniMax reports it as an agentic coding benchmark run with Claude Code scaffolding](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). +- **SWE-Bench Pro** — harder, industry-grade repository-repair benchmark for coding agents ([described as "industry-grade repository repair" in the MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **Terminal Bench 2.1** — terminal/system-operation tasks executed in a sandboxed shell, measuring command-line agent competence ([MiniMax describes Terminal-Bench as terminal and system-operation tasks](https://arxiv.org/html/2605.26494v2)). +- **SWE-fficiency** — measures a coding agent's ability to make repository changes that improve software efficiency, using the open-source SWE-fficiency dataset and workflow ([evaluation methodology in MiniMax's M3 chart](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). +- **NL2Repo** — MiniMax's internal natural-language-to-repository synthesis benchmark: build a whole repo from a specification ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **LiveSQLBench** — text-to-SQL agent benchmark (600 questions across 22 PostgreSQL databases in MiniMax's setup), scoring correct query results ([M3 chart methodology](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). +- **KernelBench Hard** — hardest split of GPU-kernel generation tasks: write correct, fast CUDA-style kernels ([reported by MiniMax as a coding/agentic frontier result](https://www.minimax.io/blog/minimax-m3)). +- **PostTrainBench** — autonomous post-training benchmark: given pretrain-only base models, the agent must synthesize data, train, evaluate and iterate without human help ([MiniMax M3 model page](https://www.minimax.io/models/text/m3)). +- **BrowseComp** — hard web-browsing benchmark measuring whether an agent can find difficult-to-locate facts online ([used by MiniMax as a search/deep-research benchmark](https://arxiv.org/html/2605.26494v2)). +- **MCP Atlas** — tool-use benchmark over Model Context Protocol tool servers, measuring multi-tool orchestration ([reported in MiniMax's M3 agentic results](https://www.minimax.io/blog/minimax-m3)). +- **Apex-Agents** — Mercor's agentic evaluation suite of expert-designed real-work tasks ([listed in MiniMax's M3 cowork block and as a leaderboard on the M3 card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). +- **OSWorld-Verified** — computer-use benchmark: complete real desktop GUI tasks in a virtual OS environment ([reported as a GUI benchmark with task completion rate in MiniMax's M3 materials](https://www.minimax.io/blog/minimax-m3)). +- **MMMU-Pro** — multimodal college-level multiple-choice reasoning across images and diagrams, a harder variant of MMMU ([listed as a multimodal benchmark on the M3 card](https://huggingface.co/MiniMaxAI/MiniMax-M3)). +- **Video-MMMU / VideoMME** — long-form video understanding benchmarks; MiniMax reports Video-MME at 512 frames, measuring comprehension of extended video input ([M3 launch blog](https://www.minimax.io/blog/minimax-m3)). +- **USAMO 2026 / IMO 2025** — olympiad proof-based mathematics contests, scored out of 42 points, measuring frontier mathematical reasoning ([MiniMax M3 vendor benchmark table](https://huggingface.co/MiniMaxAI/MiniMax-M3/resolve/main/figures/benchmark.jpeg)). +- **tau2-bench** — customer-service style tool-use benchmark with simulated users and domain policies (retail/telecom splits); named among agent benchmarks in MiniMax's M-series report but not scored for M3 ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). +- **LiveCodeBench** — contamination-resistant competitive-programming benchmark using recently published contest problems; not reported by MiniMax for M3 (n.a.). +- **AIME / GPQA Diamond / MMLU-Pro / HLE** — competition math, graduate-level science QA, broad multi-domain knowledge, and frontier-difficulty open knowledge respectively; MiniMax defines them this way in the M-series report but publishes no M3 numbers for them ([MiniMax-M2 series report](https://arxiv.org/html/2605.26494v2)). diff --git a/packages/app/content/models/qwen-3-5.mdx b/packages/app/content/models/qwen-3-5.mdx new file mode 100644 index 000000000..da34f1c27 --- /dev/null +++ b/packages/app/content/models/qwen-3-5.mdx @@ -0,0 +1,81 @@ +--- +title: 'Qwen3.5-397B-A17B' +developer: 'Alibaba (Qwen)' +releaseDate: 'February 2026' +description: "Alibaba's 397B-parameter / 17B-active MoE with a 3:1 Gated DeltaNet to Gated Attention hybrid stack, 262K native context, and Apache 2.0 weights." +--- + +## Overview + +Qwen3.5-397B-A17B is the first open-weights model released in Alibaba's Qwen3.5 series, announced by the Qwen team as a **native vision-language foundation model** with 397B total parameters and 17B activated per forward pass ([Alibaba Cloud "Qwen3.5: Towards Native Multimodal Agents"](https://www.alibabacloud.com/blog/602894)). The official announcement blog is dated February 2026 ([Qwen blog](https://qwen.ai/blog?id=qwen3.5)), with third-party trackers placing the public release in mid-February 2026 — February 16, 2026 per [OpenRouter](https://openrouter.ai/qwen/qwen3.5-397b-a17b) and February 17, 2026 per [Artificial Analysis](https://artificialanalysis.ai/articles/qwen3-5-397b-a17b-everything-you-need-to-know). Weights are published on Hugging Face as a post-trained checkpoint compatible with Transformers, vLLM, SGLang and KTransformers, with a hosted counterpart (**Qwen3.5-Plus**) offering a 1M-token default context, built-in tools and adaptive tool use ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). The repository ships the **Apache License, Version 2.0** ([LICENSE file on Hugging Face](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/blob/main/LICENSE)), also reported as Apache 2.0 by [Artificial Analysis](https://artificialanalysis.ai/articles/qwen3-5-397b-a17b-everything-you-need-to-know). + +Positioning is explicitly agentic and multimodal: the vendor frames the model as targeting "reasoning, coding, agent capabilities, and multimodal understanding" ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)), and the distribution description highlights language understanding, logical reasoning, code generation, agent-based tasks, image and video understanding, and GUI interaction, with "robust code-generation and agent capabilities" and strong generalization across agent tasks ([OpenRouter](https://openrouter.ai/qwen/qwen3.5-397b-a17b)). Qwen reports that early-fusion multimodal training achieves cross-generational parity with Qwen3 while outperforming the Qwen3-VL line on reasoning, coding, agents and visual understanding ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). Independent evaluation placed it at 45 on the Artificial Analysis Intelligence Index — #3 among open-weights models at publication — with a GDPval-AA agentic ELO of 1,221 versus 860 for Qwen3 235B ([Artificial Analysis](https://artificialanalysis.ai/articles/qwen3-5-397b-a17b-everything-you-need-to-know)). + +On technique, the model unifies thinking and non-thinking behaviour in one checkpoint: it "operate[s] in thinking mode by default," emitting `...` content before the answer, and unlike Qwen3 it drops the `/think` and `/nothink` soft switches, so non-thinking responses are obtained via API parameters instead ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)); Artificial Analysis notes this reverses Qwen3's split instruct/thinking variants ([Artificial Analysis](https://artificialanalysis.ai/articles/qwen3-5-397b-a17b-everything-you-need-to-know)). The checkpoint is trained with **multi-token prediction (MTP)** across multiple steps ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)), listed by Qwen as an efficiency feature of the underlying Qwen3-Next architecture ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)), and serving stacks expose it directly as MTP speculative decoding, e.g. `{"method": "qwen3_5_mtp", "num_speculative_tokens": 3}` ([vLLM Ascend docs](https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.5-397B-A17B.html)). Training used a native FP8 pipeline over activations, MoE routing and GEMMs with BF16 preserved in sensitive layers (~50% activation-memory reduction, >10% speedup), plus a fully disaggregated asynchronous RL framework with FP8 end-to-end training, rollout router replay, speculative decoding and multi-turn rollout locking for a 3×–5× end-to-end speedup ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). + +## Architecture + +- **Total / active parameters:** 397B total, 17B activated ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Type:** Causal language model with vision encoder (native vision-language, early text-vision fusion) ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Base architecture:** Qwen3-Next — hybrid linear attention (Gated Delta Networks) + high-sparsity sparse MoE ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Layers:** 60 (`num_hidden_layers: 60`) ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Hidden layout:** `15 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))`; config shows three `linear_attention` layers per one `full_attention` layer, `full_attention_interval: 4` ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Hidden dimension:** 4096 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)) +- **Gated DeltaNet (linear attention):** 64 value heads, 16 QK heads, head dim 128, conv kernel dim 4 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Gated Attention (full attention):** 32 Q heads, 2 KV heads, head dim 256, output gating enabled (`attn_output_gate: true`), no attention bias ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Rotary position embedding dimension:** 64 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)) +- **MoE experts:** 512 experts, 10 routed + 1 shared activated per token, expert intermediate dimension 1024 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Vocabulary / embeddings:** token embedding and LM output 248,320 (padded); Qwen describes the vocabulary as expanded to 250k from 150k, giving 10–60% encoding efficiency gains ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Context window:** 262,144 tokens native (`max_position_embeddings: 262144`), extensible to 1,010,000 tokens; Qwen3.5-Plus defaults to 1M ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json)) +- **Precision:** released weights `dtype: bfloat16` (tensor types BF16/F32); an FP8-quantized repo uses fine-grained FP8 with block size 128; training used a native FP8 pipeline with BF16 retained in sensitive layers ([config.json](https://huggingface.co/Qwen/Qwen3.5-397B-A17B/raw/main/config.json); [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [Qwen3.5-397B-A17B-FP8 card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B-FP8); [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Novel components:** MTP trained with multi-steps, exposed for speculative decoding in serving (`qwen3_5_mtp`) ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B); [vLLM Ascend docs](https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.5-397B-A17B.html)) +- **Multilingual coverage:** 201 languages and dialects (up from 119) ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Reported efficiency:** decoding throughput 8.6× / 19.0× that of Qwen3-Max at 32k / 256k context; 3.5× / 7.2× versus Qwen3-235B-A22B ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)) +- **Thinking mode:** thinking on by default with `...` traces; no `/think` `/nothink` soft switch; recommended thinking sampling `temperature=0.6, top_p=0.95, top_k=20` ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)) + +## Official vendor eval scores + +All values below are Qwen-reported (thinking mode, post-trained checkpoint) as published on the Hugging Face model card and the official Qwen3.5 announcement blog. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ---------------------- | ------------------------------------------- | --------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| SWE-bench Verified | 76.4 | 70.7 | +8.1% | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| SWE-bench Multilingual | 69.3 | 67.7 | +2.4% | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| Terminal Bench 2 | 52.5 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| TAU2-Bench (τ²-bench) | 86.7 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| BFCL-V4 | 72.9 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| BrowseComp | 69.0 (context-folding) / 78.6 (discard-all) | 44.4 | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| LiveCodeBench v6 | 83.6 | 89.0 (v6) | -6.1% | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| AIME 2026 | 91.3 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| HMMT Feb 25 | 94.8 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| HMMT Nov 25 | 92.7 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| GPQA (Diamond) | 88.4 | 87.0 (no tools) | +1.6% | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| MMLU-Pro | 87.8 | 86.8 | +1.2% | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| HLE | 28.7 (HLE-Verified: 37.6) | 26.7 (no tools) | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | +| MMMU (multimodal) | 85.0 | n.a. | — | [Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) / [Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. +Vendor scaffold caveats stated by Qwen: τ²-bench follows the official setup except the airline domain, which uses the Claude Opus 4.5 system-card fixes; search-agent benchmarks use a 256k simple context-folding strategy, and BrowseComp's 78.6 uses the discard-all strategy also used by DeepSeek-V3.2 and Kimi K2.5 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). + +## Benchmark explanations + +- **SWE-bench Verified** — human-validated set of real GitHub issues; measures whether the model's agentic patch actually resolves the issue and passes tests ([SWE-bench Verified leaderboard reference on the model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). +- **SWE-bench Multilingual** — same issue-resolution task extended beyond Python to repositories in multiple programming languages ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **Terminal Bench 2** — end-to-end agentic terminal tasks: the model must drive a shell to complete real command-line workloads ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **TAU2-Bench (τ²-bench)** — multi-turn tool-using customer-service style agent tasks across domains (including airline), scoring task completion under policy constraints ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). +- **BFCL-V4** — Berkeley Function Calling Leaderboard v4; accuracy of function/tool call selection and argument construction ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **BrowseComp** — hard web-browsing research questions requiring an agent to search, navigate and synthesize evidence over many hops, here with context-management strategies ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). +- **LiveCodeBench v6** — contamination-controlled competitive-programming problems collected over time; measures code generation correctness ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **AIME 2026** — American Invitational Mathematics Examination 2026 problems; short-answer competition math reasoning ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **HMMT Feb 25 / Nov 25** — Harvard-MIT Mathematics Tournament problem sets from the February and November 2025 contests; olympiad-level math accuracy ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **GPQA (Diamond)** — graduate-level, Google-proof multiple-choice questions in biology, chemistry and physics written by domain PhDs ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). +- **MMLU-Pro** — harder, reasoning-heavy revision of MMLU with expanded answer options, testing broad academic and professional knowledge ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). +- **HLE / HLE-Verified** — Humanity's Last Exam, extremely difficult expert-written cross-domain questions; HLE-Verified is a revised version with a component-wise verification protocol and error taxonomy ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.5-397B-A17B)). +- **MMMU** — massive multi-discipline multimodal understanding: college-level questions requiring joint image and text reasoning ([Alibaba Cloud blog](https://www.alibabacloud.com/blog/602894)). + +## Caveats + +- Neither the Qwen blog nor the model card states an exact calendar release day; only "February 2026" is official. Third-party dates disagree (Feb 15 / 16 / 17, 2026). +- The model card does not carry an explicit license statement in prose; Apache 2.0 is confirmed from the repository LICENSE file. +- Alibaba has not published a formal Qwen3.5 tech report; the architecture facts on this page come from the model card, its `config.json`, and the announcement blog. +- Vendor benchmark numbers use Qwen's own scaffolds and context-management strategies; the two BrowseComp figures illustrate how sensitive agentic scores are to harness choice. diff --git a/packages/app/content/models/qwen-3-8-flash-next.mdx b/packages/app/content/models/qwen-3-8-flash-next.mdx new file mode 100644 index 000000000..1cb875b60 --- /dev/null +++ b/packages/app/content/models/qwen-3-8-flash-next.mdx @@ -0,0 +1,123 @@ +--- +title: 'Qwen3.8-Flash-Next' +developer: 'Alibaba (Qwen)' +releaseDate: 'August 2026' +description: "Alibaba's open-weights preview of the Qwen4 architecture: 176B total across a 125B main model and a 51B n-gram embedding table, 6B active, with Qwen Sparse Attention, gated residuals and a Gated DeltaNet hybrid stack." +--- + +## Overview + +Qwen3.8-Flash-Next is an open-weights preview of the architecture underlying **Qwen4**, published by Alibaba's Qwen team in August 2026 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next); [Qwen blog](https://qwen.ai/blog?id=qwen3.8-flash-next)). Qwen frames the release as deliberately architectural rather than a scaling exercise: the card describes the design as emphasising "architectural innovation for sustainable AGI progress, departing from the scaling-focused approach of simply increasing parameters and context windows" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). The stated purpose of shipping it early is to give the developer community time to prepare for the full Qwen4 family ([TechNode](https://technode.com/2026/08/26/alibabas-qwen-to-open-source-qwen3-8-flash-next-previewing-qwen4-architecture/)). + +The parameter count needs care. The card leads with "125B with 6B activated", but that figure covers the main model only. A **51B n-gram embedding table** brings the total to **176B**, and a separate **4B MTP head** sits outside both numbers ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)). Reading the headline alone understates the checkpoint by roughly a third, which matters for capacity planning even though the n-gram table is a lookup rather than a compute-bearing parameter block. + +Weights ship under the `qwen-community-1.0` license, in BF16 and in an FP8 checkpoint that uses fine-grained quantization with a block size of 128; Qwen states the FP8 build's "performance metrics are nearly identical to those of the original model" ([Qwen3.8-Flash-Next-FP8 card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)). Thinking mode is enabled by default, emitting `...` blocks, and is configurable through `enable_thinking`, `preserve_thinking` and `reasoning_effort`; preserved thinking retains reasoning blocks across conversation history by default ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). + +On InferenceX the model is benchmarked on the [AgentX](/glossary/agentx) agentic-traces scenario rather than fixed sequence lengths, served by SGLang with native NEXTN [multi-token prediction](/glossary/multi-token-prediction). Hopper cannot run the NVFP4 checkpoint — there are no SM100 tensor cores on sm_90 — so H200 serves the FP8 weights while Blackwell parts serve `RadixArk/Qwen3.8-Flash-Next-NVFP4`. + +## Architecture + +- **Total / active parameters:** 176B total — a 125B main model plus a 51B n-gram embedding table — with 6B activated per forward pass, and a further 4B in the MTP head ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Layers:** 48 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Hidden layout:** `12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE))` — a 3:1 linear-to-sparse-attention hybrid ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Hidden dimension:** 2560 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Gated DeltaNet (linear attention):** 48 value heads, 16 QK heads, head dimension 128 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Qwen Sparse Attention (full attention):** 24 Q heads, 2 KV heads, head dimension 256, rotary position embedding dimension 64 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **QSA indexer:** MQA with 4 query heads and 1 shared key head, indexer head dimension 128, budget of "512 blocks or 2048 tokens" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **MoE experts:** 512 experts, 10 routed + 1 shared activated per token, expert intermediate dimension 640 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **N-gram embedding:** 20,000,000 bigram/trigram embeddings applied at layer 2 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Gated residual:** 4 branches, bottleneck rank 320 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **MTP:** 1 layer, trained with multi-steps ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Vocabulary:** 248,320 (padded) ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Context window:** 262,144 tokens native, extensible up to 1,000,000 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **Precision:** BF16 weights, plus an FP8 checkpoint using fine-grained quantization at block size 128 ([Qwen3.8-Flash-Next-FP8 card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)) +- **License:** `qwen-community-1.0` ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)) + +## The Qwen4 architecture innovations + +Four changes distinguish this checkpoint from Qwen3.5 and the Qwen3-Next line it descends from. Each one attacks a different cost in the serving loop, which is why the model is interesting on a benchmark that measures whole systems rather than kernels. + +### N-gram embedding: parameter scaling that is not MoE + +The headline idea is a second axis for adding parameters. Qwen describes the n-gram table as "a unique axis for parameter scaling that requires less computation" than mixture-of-experts, achieving "highly efficient parameter scaling for memory-constrained accelerators" by "indexing with short n-grams" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). + +The mechanics matter for anyone sizing a deployment. Twenty million bigram and trigram embeddings are applied at layer 2, contributing 51B parameters — more than 40% of the checkpoint's non-MTP weight — while adding a table lookup rather than a matrix multiply. MoE buys capacity by activating more experts, which costs both memory bandwidth and all-to-all traffic when experts are sharded; an n-gram table buys capacity by growing a structure that is read, not computed. On an accelerator where HBM capacity binds before FLOPS do, those are very different trades. + +It also changes what "active parameters" means as a proxy for cost. Six billion active parameters implies a very cheap forward pass, and for the MoE and attention stack it is; the n-gram table is capacity you pay for in memory footprint rather than in arithmetic per token. + +### Qwen Sparse Attention: sparsity at the micro-block level + +QSA replaces the Gated Attention layers of Qwen3.5's hybrid with a sparse mechanism that "operates at the micro-block level" rather than per token, which the card says "cuts long-context latency significantly" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). + +The selection machinery is small and explicit: an MQA indexer with 4 query heads and a single shared key head, head dimension 128, choosing under a fixed budget of 512 blocks or 2048 tokens. A bounded budget is the operationally important part. Attention cost stops growing with context length once the budget binds, so the per-token cost of a 200k-token session converges toward the cost of a much shorter one, and long-context [interactivity](/glossary/interactivity) stops degrading the way a dense-attention model's does. + +Sparsity of this kind is a systems claim as much as a modelling one. As InferenceX has repeatedly found with [sparse attention](/glossary/sparse-attention) stacks, theoretical sparsity says little about realised speed: index construction, irregular memory access, kernel fusion and precision support decide what actually arrives, which is why the benchmark pins engine and image versions alongside the model. + +### Gated residual: finer control across a widened residual stream + +The gated residual "modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and per-branch scalar write gate", which Qwen says provides "finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). The configuration is 4 branches at bottleneck rank 320. + +The read gate is element-wise and data-dependent; the write gate is a single scalar per branch. That asymmetry is the design: expressive where it is cheap to be expressive, and deliberately coarse where per-element parameters would add inference cost for little return. + +### The hybrid ratio and MTP, carried forward + +Two elements are inherited rather than new, and both matter for how the model serves. The 3:1 interleave of Gated DeltaNet to full-attention layers is the same ratio Qwen3.5 uses, keeping most layers on [linear attention](/glossary/linear-attention) whose state does not grow with sequence length. And the model ships a 1-layer MTP head trained with multiple steps, exposed by SGLang as native NEXTN [speculative decoding](/glossary/speculative-decoding). + +MTP is the piece with the largest effect on measured throughput, and also the one most sensitive to how it is benchmarked. AgentX replays anonymized traces filled with synthetic tokens, so a speculator would accept an unrepresentative number of draft tokens if left to the content; runs therefore apply a fixed [acceptance length](/glossary/acceptance-length) measured separately, which for this model at MTP=3 was 3.24 with thinking off. + +### Training recipe + +Qwen also reports optimizer-level changes: Muon and AdamW applied to specific weight categories, and the elimination of traditional batch-size warmups in favour of starting directly at the target batch size, which "substantially reduces total optimizer steps" ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). This has no bearing on inference performance, but it is part of what Qwen means by previewing Qwen4 rather than shipping a larger Qwen3. + +## Official vendor eval scores + +All values below are Qwen-reported for Qwen3.8-Flash-Next as published on the Hugging Face model card. The card's own comparison columns name Qwen3.8-27B, Qwen3.7-Plus and DeepSeek-V4-Flash; those are omitted here in favour of the cross-page Nemotron 3 Ultra reference. + +| Benchmark | Score | Nemotron3 Ultra | % better | Source | +| ---------------------------- | --------- | ------------------- | -------- | --------------------------------------------------------------- | +| GPQA Diamond | 91.7 | 87.0 (no tools) | +5.4% | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| LiveCodeBench v6 | 91.9 | 89.0 (v6) | +3.3% | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| SWE-bench Multilingual | 81.0 | 67.7 | +19.6% | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| SWE-bench Pro | 62.5 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| IFBench | 81.3 | 81.7 (prompt loose) | -0.5% | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| DeepSWE 1.1 | 58.7 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| Toolathlon Verified | 73.5 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| CoWorkBench | 73.9 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| JobBench | 55.7 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| Agents' Last Exam (Pass@1) | 25.2 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| ClawEval-MM (Pass@3) | 64.4 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| AndroidWorld | 84.5 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| OSWorld 2.0 (Binary/Partial) | 19.4/52.3 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| Vision2Web | 64.0 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| ERQA | 72.3 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| LVBench | 76.6 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| RealWorldQA | 88.5 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| RecreationBench | 49.9 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| MathVision (Without/With CI) | 90.6/95.7 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | +| CharXiv RQ (Without/With CI) | 84.6/90.6 | n.a. | — | [HF model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) | + +Nemotron 3 Ultra figures are the vendor-reported numbers from the [NVIDIA Nemotron 3 Ultra model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16), shown as a common reference point across model pages. `n.a.` means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. **% better** is `(score − Nemotron) / Nemotron`, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows `—` instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. + +## Benchmark explanations + +- **GPQA Diamond** — graduate-level, Google-proof multiple-choice questions in biology, chemistry and physics written by domain PhDs ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **LiveCodeBench v6** — contamination-controlled competitive-programming problems collected over time; measures code generation correctness ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **SWE-bench Multilingual** — real GitHub issue resolution extended beyond Python to repositories in multiple programming languages ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **SWE-bench Pro** — a harder issue-resolution set than SWE-bench Verified, with longer and more involved patches ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **DeepSWE 1.1** — agentic software-engineering tasks scored on whether the agent's changes resolve the task end to end ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **Toolathlon Verified** — verified subset of multi-tool agentic tasks, scoring correct tool selection and sequencing ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **CoWorkBench / JobBench** — workplace-style agentic task suites measuring completion of realistic multi-step office and job workflows ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **Agents' Last Exam** — very hard agentic tasks designed to remain unsolved by current systems; reported Pass@1 ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **IFBench** — instruction-following accuracy under explicit constraints ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **AndroidWorld / OSWorld 2.0** — GUI-agent benchmarks driving a real Android environment and a real desktop OS respectively; OSWorld reports binary and partial credit ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **Vision2Web / ClawEval-MM / RecreationBench** — multimodal agentic and understanding suites over web pages and visual scenes ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **LVBench / RealWorldQA / ERQA** — long-video understanding, real-world visual question answering, and embodied reasoning question answering ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). +- **MathVision / CharXiv RQ** — visual mathematics and chart-reasoning benchmarks, each reported with and without a code interpreter ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)). + +## Caveats + +- **The 125B figure is the main model, not the checkpoint.** Total parameters are 176B once the 51B n-gram embedding table is counted, with a further 4B in the MTP head ([Hugging Face model card](https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8)). +- **All eval scores are vendor-reported**, produced by Qwen under its own harness. The card's comparison columns are also Qwen's own selection of rivals. +- **Precision differs by chip generation.** Hopper has no NVFP4 path, so H200 results use the FP8 checkpoint while Blackwell results use NVFP4. Compare across chips with that in mind. +- **The acceptance length used for MTP is measured separately**, not derived from the replayed traces, because AgentX content is synthetic. The interim value at the first sweep was 3.24 at MTP=3, collected with thinking off. +- **This is an architecture preview.** Qwen positions the checkpoint as an early look at Qwen4 rather than a finished flagship, and engine support was days old at first benchmark, so results should be expected to move as kernels mature. diff --git a/packages/app/cypress/component/agentic-distribution.cy.tsx b/packages/app/cypress/component/agentic-distribution.cy.tsx new file mode 100644 index 000000000..cf20d02bc --- /dev/null +++ b/packages/app/cypress/component/agentic-distribution.cy.tsx @@ -0,0 +1,94 @@ +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; + +import { Distribution } from '@/components/inference/agentic-point/distribution'; + +// Mounted outside the Next app shell; next-style-loader inserts the global +// stylesheet before this anchor, so it must exist before the import below. +const cssAnchor = document.createElement('noscript'); +cssAnchor.id = '__next_css__DO_NOT_USE__'; +document.head.append(cssAnchor); +require('@/app/globals.css'); + +/** Deterministic lognormal sample so bar counts and the fit are reproducible. */ +function lognormalSample(count: number, mu: number, sigma: number): number[] { + let seed = 987_654; + const uniform = () => { + seed = (seed * 1103515245 + 12345) % 2147483648; + return (seed + 1) / 2147483649; + }; + return Array.from({ length: count }, () => { + const z = Math.sqrt(-2 * Math.log(uniform())) * Math.cos(2 * Math.PI * uniform()); + return Math.round(Math.exp(mu + sigma * z)); + }); +} + +const FIT_STROKE = '#a855f7'; + +function mountDistribution(values: number[], pathname = '/inference/agentic/96255') { + cy.mount( + +
+ +
+
, + ); +} + +describe('Agentic ISL/OSL distribution', () => { + it('bins on a log axis without a fitted overlay', () => { + const values = lognormalSample(1500, 7.3, 0.9); + mountDistribution(values); + + cy.contains('1,500 requests').should('be.visible'); + cy.contains('log scale').should('be.visible'); + // Bars only — no fitted overlay or stats readout. + cy.contains('lognormal fit').should('not.exist'); + cy.contains('μ=').should('not.exist'); + cy.get(`path[stroke="${FIT_STROKE}"]`).should('not.exist'); + cy.get('rect[opacity="0.55"]').should('have.length.greaterThan', 10); + + // Percentile guides remain, one per percentile. + cy.get( + 'line[stroke="#3b82f6"], line[stroke="#22c55e"], line[stroke="#f59e0b"], line[stroke="#ef4444"]', + ).should('have.length', 8); + }); + + it('spreads mass across the axis instead of piling it into the first bins', () => { + // The point of the log axis: on a linear one this sample puts almost + // everything in the leftmost bins. Assert the tallest bar is not the first. + mountDistribution(lognormalSample(1500, 7.3, 0.9)); + cy.get('rect[opacity="0.55"]').then(($bars) => { + const heights = [...$bars].map((bar) => Number(bar.getAttribute('height'))); + const peakIndex = heights.indexOf(Math.max(...heights)); + expect(peakIndex).to.be.greaterThan(1); + expect(peakIndex).to.be.lessThan(heights.length - 2); + }); + }); + + it('reports zero-token requests that a log axis cannot place', () => { + const values = [...lognormalSample(400, 6, 0.8), 0, 0, 0]; + mountDistribution(values); + cy.contains('400 requests').should('be.visible'); + cy.contains('3 requests with 0 tokens excluded from the log axis').should('be.visible'); + }); + + it('still draws the histogram when every request is the same length', () => { + mountDistribution(Array.from({ length: 50 }, () => 512)); + cy.contains('50 requests').should('be.visible'); + cy.get('rect[opacity="0.55"]').should('have.length.greaterThan', 0); + }); + + it('shows the empty placeholder when nothing can be plotted', () => { + mountDistribution([]); + cy.contains('No data').should('be.visible'); + cy.get(`path[stroke="${FIT_STROKE}"]`).should('not.exist'); + }); + + it('renders in Simplified Chinese on the /zh route', () => { + mountDistribution(lognormalSample(600, 7, 0.8), '/zh/inference/agentic/96255'); + cy.contains('600 个请求').should('be.visible'); + cy.contains('对数刻度').should('be.visible'); + cy.contains('数值(tokens,对数刻度)').should('be.visible'); + cy.contains('requests').should('not.exist'); + }); +}); diff --git a/packages/app/cypress/component/chart-legend.cy.tsx b/packages/app/cypress/component/chart-legend.cy.tsx index 15b9ab100..b14e78c4c 100644 --- a/packages/app/cypress/component/chart-legend.cy.tsx +++ b/packages/app/cypress/component/chart-legend.cy.tsx @@ -1,4 +1,4 @@ -import { useState } from 'react'; +import { Profiler, useState } from 'react'; import LegendPointsDialog from '@/components/inference/ui/LegendPointsDialog'; import { OffloadHaloLegendKey } from '@/components/inference/ui/OffloadHaloLegendKey'; @@ -81,25 +81,47 @@ describe('ChartLegend (sidebar variant)', () => { cy.get('.sidebar-legend label').should('have.length', 4); }); - it('legend items have colored dots', () => { - cy.get('.sidebar-legend label').first().find('span').first().should('exist'); + it('derives long unofficial labels without a nested update', () => { + const branch = 'qwen3.5-fp4-gb200-dynamo-sglang-agentic-mtp-pareto-refresh'; + const onRender = cy.spy().as('legendRender'); + + cy.mount( + + {}} + variant="sidebar" + /> + , + ); + + cy.get('[data-testid="chart-legend"]').should('contain.text', branch); + cy.get('@legendRender').should((renderSpy) => { + // Cypress loses the Sinon spy type when resolving an alias. + const profilerSpy = renderSpy as unknown as { + getCalls: () => { args: unknown[] }[]; + }; + const calls = profilerSpy.getCalls(); + expect( + calls.map(({ args }) => args[1]), + 'React render phases', + ).not.to.include('nested-update'); + }); }); - it('search input filters legend items by hiding non-matches', () => { - cy.get('.sidebar-legend input[placeholder="Search..."]').should('exist'); - cy.get('.sidebar-legend input[placeholder="Search..."]').clear().type('MI300'); - // Non-matching items are hidden via overflow-hidden class, not removed from DOM - cy.get('.sidebar-legend li.overflow-hidden').should('have.length', 3); - cy.get('.sidebar-legend li:not(.overflow-hidden)').should('have.length', 1); - cy.get('.sidebar-legend li:not(.overflow-hidden)').should('contain.text', 'AMD MI300X'); + it('legend items have colored dots', () => { + cy.get('.sidebar-legend label').first().find('span').first().should('exist'); }); - it('search clear button resets search', () => { - cy.get('.sidebar-legend input[placeholder="Search..."]').type('test'); - cy.get('button[aria-label="Clear search"]').should('be.visible'); - cy.get('button[aria-label="Clear search"]').click(); - cy.get('.sidebar-legend input[placeholder="Search..."]').should('have.value', ''); - cy.get('button[aria-label="Clear search"]').should('not.exist'); + it('renders no search input (removed from the sidebar panel)', () => { + cy.get('.sidebar-legend input[type="text"]').should('not.exist'); }); it('clicking a legend item toggles its active state', () => { @@ -115,13 +137,14 @@ describe('ChartLegend (sidebar variant)', () => { cy.contains('Reset filter').should('not.exist'); }); - it('expand/collapse button toggles legend state', () => { - cy.get('.sidebar-legend').should('have.class', 'bg-accent'); - cy.get('.sidebar-legend button') - .filter(':contains("Collapse"), :contains("Expand")') - .first() - .click(); - cy.get('.sidebar-legend').should('not.have.class', 'bg-accent'); + it('close button hides the panel and the reopen button restores it', () => { + cy.get('.sidebar-legend').should('exist'); + cy.get('[data-testid="legend-close-button"]').click(); + cy.get('.sidebar-legend').should('not.exist'); + cy.get('[data-testid="legend-open-button"]').should('be.visible'); + cy.get('[data-testid="legend-open-button"]').click(); + cy.get('.sidebar-legend').should('exist'); + cy.get('[data-testid="legend-open-button"]').should('not.exist'); }); it('renders no points-table icon when items have no onShowPoints handler', () => { @@ -232,7 +255,7 @@ function LegendWithPointsTable() { if (!open) setOpenSeries(null); }} title={isOverlay ? '✕ my-branch' : 'B300 (vLLM)'} - subtitle="DeepSeek V4 Pro · Agentic Traces" + subtitle="DeepSeek V4 Pro · Agentic" accentColor={isOverlay ? '#dc2626' : '#2b83ba'} rows={buildLegendPointsRows(isOverlay ? OVERLAY_POINTS : OFFICIAL_POINTS, isOverlay)} isOverlay={isOverlay} @@ -258,7 +281,7 @@ describe('ChartLegend points-table icon + dialog', () => { cy.get('[data-testid="legend-points-dialog"]').should('contain.text', 'B300 (vLLM)'); cy.get('[data-testid="legend-points-dialog"]').should( 'contain.text', - 'DeepSeek V4 Pro · Agentic Traces', + 'DeepSeek V4 Pro · Agentic', ); // Two rows, conc ascending, linked to the agentic detail pages cy.get('[data-testid="legend-points-row"]').should('have.length', 2); diff --git a/packages/app/cypress/component/chart-selectors.cy.tsx b/packages/app/cypress/component/chart-selectors.cy.tsx index fd3c16d27..490ef6546 100644 --- a/packages/app/cypress/component/chart-selectors.cy.tsx +++ b/packages/app/cypress/component/chart-selectors.cy.tsx @@ -1,12 +1,14 @@ import { useState } from 'react'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; import { ModelSelector, + ScenarioSelector, SequenceSelector, PrecisionSelector, } from '@/components/ui/chart-selectors'; import { TooltipProvider } from '@/components/ui/tooltip'; -import { Model } from '@/lib/data-mappings'; +import { Model, Sequence } from '@/lib/data-mappings'; function ModelSelectorHarness() { const [value, setValue] = useState('DeepSeek-R1-0528'); @@ -42,6 +44,20 @@ function SequenceSelectorHarness() { ); } +function ScenarioSelectorHarness({ initial = Sequence.AgenticTraces }: { initial?: Sequence }) { + const [value, setValue] = useState(initial); + return ( + + + + ); +} + function PrecisionSelectorHarness() { const [value, setValue] = useState(['FP8']); return ( @@ -93,6 +109,22 @@ describe('Chart Selectors', () => { 'be.visible', ); }); + + it('localizes model category labels and reasons on Chinese routes', () => { + cy.mount( + + + , + ); + cy.get('[data-testid="model-selector"]').click(); + cy.contains('维护模式').should('be.visible'); + cy.contains('已弃用').should('be.visible'); + cy.contains('Maintenance Mode').should('not.exist'); + cy.get('[data-testid="selector-category-maintenance-mode-info"]').trigger('pointermove', { + pointerType: 'mouse', + }); + cy.contains('这些模型的相关性较低,因此以较低优先级更新。').should('be.visible'); + }); }); describe('SequenceSelector', () => { @@ -112,6 +144,85 @@ describe('Chart Selectors', () => { }); }); + describe('ScenarioSelector', () => { + it('labels the agentic scenario "Agentic"', () => { + cy.mount(); + cy.get('[data-testid="scenario-selector"]').should('have.text', 'Agentic'); + cy.get('[data-testid="scenario-selector"]').click(); + cy.contains('[role="option"]', 'Agentic').should('be.visible'); + cy.contains('[role="option"]', 'Agentic Traces').should('not.exist'); + // The lone agentic entry needs no "Agentic" heading above it. + cy.get('[data-slot="select-content"]') + .find('[data-slot="select-label"]') + .should('not.contain.text', 'Agentic'); + }); + + it('explains the agentic workload in a tooltip that links to /agentx', () => { + cy.mount(); + cy.get('[data-testid="scenario-agentic-info"]').trigger('pointermove', { + pointerType: 'mouse', + }); + + cy.contains('Realistic Long Context Multi Turn Agentic Workload with Sub Agents.').should( + 'be.visible', + ); + cy.get('[data-testid="scenario-agentic-info-link"]') + .should('be.visible') + .and('have.attr', 'href', '/agentx'); + }); + + it('renders nothing when the only scenario is fixed-sequence', () => { + cy.mount( + +
+ {}} + availableSequences={[Sequence.EightK_OneK]} + data-testid="scenario-selector" + /> +
+
, + ); + cy.get('[data-testid="selector-host"]').should('exist'); + cy.get('[data-testid="scenario-selector"]').should('not.exist'); + cy.contains('Scenario').should('not.exist'); + }); + + it('renders nothing when agentic is the only scenario', () => { + // A static "Scenario: Agentic" readout is as redundant as a one-option + // dropdown — the whole control disappears for single-scenario models. + cy.mount( + +
+ {}} + availableSequences={[Sequence.AgenticTraces]} + data-testid="scenario-selector" + /> +
+
, + ); + cy.get('[data-testid="selector-host"]').should('exist'); + cy.get('[data-testid="scenario-selector"]').should('not.exist'); + cy.contains('Scenario').should('not.exist'); + cy.get('[data-testid="scenario-static-value"]').should('not.exist'); + cy.get('[data-testid="scenario-agentic-info"]').should('not.exist'); + }); + + it('hides the agentic explainer on fixed-sequence scenarios', () => { + cy.mount(); + cy.get('[data-testid="scenario-selector"]').should('contain.text', '8K / 1K'); + cy.get('[data-testid="scenario-agentic-info"]').should('not.exist'); + + // ...and appears as soon as the user picks the agentic scenario. + cy.get('[data-testid="scenario-selector"]').click(); + cy.contains('[role="option"]', 'Agentic').click(); + cy.get('[data-testid="scenario-agentic-info"]').should('exist'); + }); + }); + describe('PrecisionSelector', () => { beforeEach(() => { cy.mount(); @@ -120,5 +231,23 @@ describe('Chart Selectors', () => { it('shows current selection', () => { cy.get('[data-testid="precision-multiselect"]').should('contain', 'FP8'); }); + + it('renders nothing when only one precision is available', () => { + cy.mount( + +
+ {}} + availablePrecisions={['FP8']} + data-testid="precision-multiselect" + /> +
+
, + ); + cy.get('[data-testid="selector-host"]').should('exist'); + cy.get('[data-testid="precision-multiselect"]').should('not.exist'); + cy.contains('Precision').should('not.exist'); + }); }); }); diff --git a/packages/app/cypress/component/dataset-list.cy.tsx b/packages/app/cypress/component/dataset-list.cy.tsx index f7cfcb9a7..ce62d6fa6 100644 --- a/packages/app/cypress/component/dataset-list.cy.tsx +++ b/packages/app/cypress/component/dataset-list.cy.tsx @@ -3,6 +3,7 @@ import { AppRouterContext } from 'next/dist/shared/lib/app-router-context.shared import { DatasetList } from '@/components/datasets/dataset-list'; import type { DatasetRecord } from '@/hooks/api/use-datasets'; +import { createMockRouter } from '../support/mock-router'; const datasets: DatasetRecord[] = [ { @@ -43,17 +44,6 @@ const datasets: DatasetRecord[] = [ }, ]; -function createMockRouter() { - return { - push: cy.stub(), - replace: cy.stub(), - refresh: cy.stub(), - back: cy.stub(), - forward: cy.stub(), - prefetch: cy.stub().resolves(), - }; -} - function mountList() { const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } }); cy.mount( @@ -74,7 +64,7 @@ describe('DatasetList', () => { cy.contains('cc-traces-weka (256k)').should('be.visible'); cy.contains('1,234').should('be.visible'); // conversation_count, localized cy.contains('82%').should('be.visible'); // cachedPct - cy.get('a[href="/datasets/cc-traces-weka-full"]').should('exist'); + cy.get('a[href="/agentx/cc-traces-weka-full"]').should('exist'); }); it('shows the empty state when no datasets are ingested', () => { diff --git a/packages/app/cypress/component/date-picker.cy.tsx b/packages/app/cypress/component/date-picker.cy.tsx index 68465586b..631f03c50 100644 --- a/packages/app/cypress/component/date-picker.cy.tsx +++ b/packages/app/cypress/component/date-picker.cy.tsx @@ -1,4 +1,5 @@ import { useState } from 'react'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; import { DatePicker } from '@/components/ui/date-picker'; @@ -16,9 +17,11 @@ const AVAILABLE_DATES = [ function DatePickerWrapper({ initialDate, availableDates = AVAILABLE_DATES, + useDefaults = false, }: { initialDate?: string; availableDates?: string[]; + useDefaults?: boolean; }) { const [date, setDate] = useState(initialDate); return ( @@ -28,7 +31,7 @@ function DatePickerWrapper({ availableDates={availableDates} minDate={availableDates[0]} maxDate={availableDates.at(-1)} - placeholder="Select date" + placeholder={useDefaults ? undefined : 'Select date'} /> ); } @@ -118,4 +121,23 @@ describe('DatePicker', () => { // Original date should still be shown cy.contains('Nov 15, 2025').should('be.visible'); }); + + it('localizes the complete dialog chrome on Chinese routes', () => { + cy.mount( + + + , + ); + cy.contains('运行日期:').click(); + cy.contains('选择运行日期').should('be.visible'); + cy.contains('选择运行日期以查看该次运行的性能数据。').should('be.visible'); + cy.contains('button', '前往最新日期').should('be.visible'); + cy.contains('button', '取消').should('be.visible'); + cy.contains('button', '应用').should('be.visible'); + cy.get('h3.font-semibold').should('contain', '2025年11月'); + cy.get('[role="dialog"]') + .should('contain.text', '关闭') + .and('not.contain.text', 'Cancel') + .and('not.contain.text', 'Close'); + }); }); diff --git a/packages/app/cypress/component/date-range-picker.cy.tsx b/packages/app/cypress/component/date-range-picker.cy.tsx index 9fd7c4ed2..61aeb7d86 100644 --- a/packages/app/cypress/component/date-range-picker.cy.tsx +++ b/packages/app/cypress/component/date-range-picker.cy.tsx @@ -1,13 +1,16 @@ import { useState } from 'react'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; import { type DateRange, DateRangePicker } from '@/components/ui/date-range-picker'; function DateRangePickerHarness({ initialRange = { startDate: '', endDate: '' }, availableDates, + useDefaults = false, }: { initialRange?: DateRange; availableDates?: string[]; + useDefaults?: boolean; }) { const [range, setRange] = useState(initialRange); return ( @@ -16,7 +19,7 @@ function DateRangePickerHarness({ dateRange={range} onChange={setRange} availableDates={availableDates} - placeholder="Select date range" + placeholder={useDefaults ? undefined : 'Select date range'} />
{range.startDate && range.endDate ? `${range.startDate} to ${range.endDate}` : 'no range'} @@ -86,4 +89,19 @@ describe('DateRangePicker', () => { cy.get('[role="dialog"]').should('not.exist'); cy.get('[data-testid="date-range-output"]').should('contain', '2026-01-01 to 2026-01-01'); }); + + it('localizes status, quick actions, and dialog actions on Chinese routes', () => { + const dates = ['2025-12-01', '2026-01-01', '2026-02-01']; + cy.mount( + + + , + ); + cy.contains('选择日期范围').click(); + cy.contains('请选择开始和结束日期以定义日期范围。').should('be.visible'); + cy.contains('button', '最大范围').should('be.visible'); + cy.contains('button', '取消').should('be.visible'); + cy.contains('button', '应用').should('be.visible'); + cy.get('[role="dialog"]').should('not.contain.text', 'Max Range'); + }); }); diff --git a/packages/app/cypress/component/faq-question-link.cy.tsx b/packages/app/cypress/component/faq-question-link.cy.tsx new file mode 100644 index 000000000..df2502c9e --- /dev/null +++ b/packages/app/cypress/component/faq-question-link.cy.tsx @@ -0,0 +1,55 @@ +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; + +import { FaqQuestionLink } from '@/components/about/faq-question-link'; + +const ID = 'faq-normalized-interactivity'; +const QUESTION = 'What is the difference between normalized interactivity and interactivity?'; + +function mountFaqQuestionLink(pathname = '/about') { + cy.mount( + +
+ +
+
, + ); +} + +describe('FaqQuestionLink', () => { + it('shows an icon-only control and copies the direct URL', () => { + cy.window().then((win) => { + cy.stub(win.navigator.clipboard, 'writeText').as('writeFaqLink').resolves(); + }); + mountFaqQuestionLink(); + + cy.get(`a[href="#${ID}"]`).should('have.text', QUESTION); + cy.get(`[data-testid="faq-copy-link-${ID}"]`) + .should('be.visible') + .and('have.text', '') + .and('have.attr', 'title', 'Copy link') + .and('have.attr', 'aria-label', `Copy link: ${QUESTION}`) + .find('svg.lucide-link') + .should('be.visible'); + cy.get(`[data-testid="faq-copy-link-${ID}"]`) + .click() + .should('have.attr', 'title', 'Copied') + .and('have.attr', 'aria-label', `Copied: ${QUESTION}`) + .find('svg.lucide-check') + .should('be.visible'); + cy.get('@writeFaqLink').should( + 'have.been.calledOnceWith', + Cypress.sinon.match(new RegExp(`#${ID}$`, 'u')), + ); + }); + + it('uses concise Chinese button copy on the Chinese route', () => { + mountFaqQuestionLink('/zh/about'); + + cy.get(`[data-testid="faq-copy-link-${ID}"]`) + .should('be.visible') + .and('have.attr', 'aria-label', `复制链接: ${QUESTION}`) + .and('have.attr', 'title', '复制链接') + .find('svg.lucide-link') + .should('be.visible'); + }); +}); diff --git a/packages/app/cypress/component/footer.cy.tsx b/packages/app/cypress/component/footer.cy.tsx index 6e7916779..4224d9f01 100644 --- a/packages/app/cypress/component/footer.cy.tsx +++ b/packages/app/cypress/component/footer.cy.tsx @@ -49,8 +49,13 @@ describe('Footer', () => { cy.get('[data-testid="footer-link-benchmarks"]') .should('have.attr', 'href') .and('include', 'github.com/SemiAnalysisAI/InferenceX'); - cy.get('[data-testid="footer-link-frontend"]') - .should('have.attr', 'href') + cy.get('[data-testid="footer-link-agentx-harness"]') + .should('contain.text', 'AgentX Harness') + .and('have.attr', 'href') + .and('include', 'github.com/SemiAnalysisAI/agentx-harness'); + cy.get('[data-testid="footer-link-visualization"]') + .should('contain.text', 'Visualization') + .and('have.attr', 'href') .and('include', 'github.com/SemiAnalysisAI/InferenceX-app'); }); @@ -58,9 +63,9 @@ describe('Footer', () => { cy.get('[data-testid="footer-link-supporters"]') .should('contain.text', 'Supporters') .and('have.attr', 'href', '/quotes'); - cy.get('[data-testid="footer-link-datasets"]') - .should('contain.text', 'Datasets') - .and('have.attr', 'href', '/datasets'); + cy.get('[data-testid="footer-link-agentx"]') + .should('contain.text', 'AgentX') + .and('have.attr', 'href', '/agentx'); cy.get('[data-testid="footer-link-articles"]') .should('contain.text', 'Articles') .and('have.attr', 'href', '/blog'); diff --git a/packages/app/cypress/component/gpu-graph.cy.tsx b/packages/app/cypress/component/gpu-graph.cy.tsx index 0634ffff6..4589327fd 100644 --- a/packages/app/cypress/component/gpu-graph.cy.tsx +++ b/packages/app/cypress/component/gpu-graph.cy.tsx @@ -185,12 +185,12 @@ describe('GPUGraph', () => { cy.get('#test-gpu-agentic-decorations [data-testid="spec-decode-marker-key"]').should( 'not.exist', ); - cy.get('#test-gpu-agentic-decorations [data-testid="offload-halo-key"]').should('exist'); - cy.get('#test-gpu-agentic-decorations [data-testid="agentic-optimization-note"]') - .should('contain.text', 'Inference optimizations enabled') - .find('button') - .focus(); - cy.contains('Each configuration may use inference optimizations').should('be.visible'); + // The KV-offload key and optimization note moved to the axis-metric info + // footer rendered by ChartDisplay, so the chart itself carries neither. + cy.get('#test-gpu-agentic-decorations [data-testid="offload-halo-key"]').should('not.exist'); + cy.get('#test-gpu-agentic-decorations [data-testid="agentic-optimization-note"]').should( + 'not.exist', + ); cy.get('#test-gpu-agentic-decorations svg .dot-group').first().trigger('mouseenter'); cy.get('[data-chart-tooltip]').should('contain.text', 'Speculative Decoding'); cy.get('[data-chart-tooltip]').should('contain.text', 'MTP'); diff --git a/packages/app/cypress/component/header.cy.tsx b/packages/app/cypress/component/header.cy.tsx index 8589264ab..6d85f6247 100644 --- a/packages/app/cypress/component/header.cy.tsx +++ b/packages/app/cypress/component/header.cy.tsx @@ -4,6 +4,7 @@ import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.share import { Header } from '@/components/header/header'; import { ThemeProvider } from '@/components/ui/theme-provider'; +import { createMockRouter } from '../support/mock-router'; // Mounted outside the Next app shell; next-style-loader inserts the global // stylesheet before this anchor, so it must exist before the import below. @@ -21,17 +22,6 @@ const MIN_TOUCH_PX = 44; /** Tolerance for sub-pixel layout rounding. */ const EPSILON = 0.5; -function createMockRouter() { - return { - push: cy.stub(), - replace: cy.stub(), - refresh: cy.stub(), - back: cy.stub(), - forward: cy.stub(), - prefetch: cy.stub().resolves(), - }; -} - function rectOf(selector: string) { return cy.get(selector).then(($el) => $el[0].getBoundingClientRect()); } @@ -80,6 +70,20 @@ describe('Header', () => { cy.wrap(mockRouter.push).should('have.been.calledTwice'); }); + it('retains resilient locale navigation outside Overview', () => { + cy.clock(); + mountHeader('/inference'); + cy.get('[data-testid="language-toggle"]') + .invoke('attr', 'href') + .then((href) => { + expect(href).to.include('/zh/inference'); + cy.get('[data-testid="language-toggle"]').click(); + cy.wrap(mockRouter.push).should('have.been.calledOnceWith', href); + cy.tick(250); + cy.wrap(mockRouter.push).should('have.been.calledTwice'); + }); + }); + it('makes a click on the tab already showing a no-op', () => { mountHeader('/overview'); cy.get('[data-testid="nav-link-overview"]').click(); @@ -104,10 +108,50 @@ describe('Header', () => { cy.get('[data-testid="nav-link-compare"]').should('have.attr', 'href', '/compare'); }); + it('shows AgentX as a top-level nav link and highlights AgentX child pages', () => { + cy.get('[data-testid="nav-link-agentx"]') + .should('be.visible') + .and('have.attr', 'href', '/agentx') + .find('[data-nav-badge="agentx"]') + .should('have.text', 'NEW'); + + mountHeader('/agentx/claude-code-traces'); + cy.get('[data-testid="nav-link-agentx"]').should('have.class', 'text-brand'); + }); + + it('keeps AgentX in the Chinese navigation tree', () => { + mountHeader('/zh/agentx'); + cy.get('[data-testid="nav-link-agentx"]') + .should('be.visible') + .and('contain.text', 'AgentX') + .and('have.attr', 'href', '/zh/agentx') + .and('have.class', 'text-brand'); + cy.get('[data-testid="nav-link-agentx"]') + .find('[data-nav-badge="agentx"]') + .should('have.text', '新'); + }); + + it('orders the nav with Home first and AgentX second', () => { + const expected = [ + 'Home', + 'AgentX', + 'Overview', + 'Dashboard', + 'Comparisons', + 'Articles', + 'About', + ]; + cy.get('[data-testid^="nav-link-"]').then(($links) => { + // Strip the NEW badge so the comparison is against the label alone. + const labels = [...$links].map((link) => (link.textContent ?? '').replace('NEW', '').trim()); + expect(labels).to.deep.equal(expected); + }); + }); + it('keeps footer destinations out of the primary nav', () => { cy.get('[data-testid="nav-link-supporters"]').should('not.exist'); - cy.get('[data-testid="nav-link-datasets"]').should('not.exist'); - cy.get('[data-testid="nav-link-blog"]').should('not.exist'); + // Telemetry moved to the footer; Articles took its place in the nav. + cy.get('[data-testid="nav-link-telemetry"]').should('not.exist'); }); it('shows the GitHub stars button linking to the correct repo', () => { @@ -133,9 +177,14 @@ describe('Header', () => { cy.contains('a', 'Overview').should('be.visible').and('have.attr', 'href', '/overview'); cy.contains('a', 'Dashboard').should('be.visible').and('have.attr', 'href', '/inference'); cy.contains('a', 'Comparisons').should('be.visible').and('have.attr', 'href', '/compare'); + cy.contains('a', 'AgentX') + .should('be.visible') + .and('have.attr', 'href', '/agentx') + .find('[data-nav-badge="agentx"]') + .should('have.text', 'NEW'); + cy.contains('a', 'Articles').should('be.visible').and('have.attr', 'href', '/blog'); cy.contains('a', 'Supporters').should('not.exist'); - cy.contains('a', 'Datasets').should('not.exist'); - cy.contains('a', 'Articles').should('not.exist'); + cy.contains('a', 'Telemetry').should('not.exist'); }); }); @@ -149,6 +198,40 @@ describe('Header', () => { cy.wrap(mockRouter.push).should('have.been.calledTwice'); }); + it('uses the hamburger without horizontal overflow from 1009 through 1024 CSS pixels', () => { + [1009, 1012, 1020, 1024].forEach((width) => { + cy.viewport(width, 720); + cy.get('[data-testid="nav-link-dashboard"]').should('not.be.visible'); + cy.get('[data-testid="mobile-menu-toggle"]').should('be.visible'); + cy.document().then((doc) => { + expect(doc.documentElement.scrollWidth, `${width}px document scrollWidth`).to.be.at.most( + doc.documentElement.clientWidth, + ); + }); + cy.get('[data-testid="header"]').then(($header) => { + const header = $header[0]; + expect(header.scrollWidth, `${width}px header scrollWidth`).to.be.at.most( + header.clientWidth, + ); + }); + }); + }); + + it('keeps every primary link inside the header at the xl desktop breakpoint', () => { + cy.viewport(1280, 720); + cy.get('[data-testid="header"]').then(($header) => { + const header = $header[0]; + const bounds = header.getBoundingClientRect(); + expect(header.scrollWidth, 'header scrollWidth').to.be.at.most(header.clientWidth); + + cy.get('[data-testid^="nav-link-"]:visible').each(($link) => { + const rect = $link[0].getBoundingClientRect(); + expect(rect.left, `${$link.text()} left edge`).to.be.at.least(bounds.left - EPSILON); + expect(rect.right, `${$link.text()} right edge`).to.be.at.most(bounds.right + EPSILON); + }); + }); + }); + describe('at 320x700', () => { beforeEach(() => { cy.viewport(320, 700); @@ -204,10 +287,12 @@ describe('Header', () => { cy.get('[data-testid="mobile-menu-toggle"]').click(); cy.get('[data-testid="mobile-menu"]').should('be.visible'); cy.get('[data-testid="mobile-menu"]').within(() => { - ['Home', 'Overview', 'Dashboard', 'Comparisons', 'About'].forEach((label) => { - cy.contains('a', label).should('be.visible'); - }); - ['Supporters', 'Datasets', 'Articles'].forEach((label) => { + ['Home', 'Overview', 'Dashboard', 'Comparisons', 'Articles', 'AgentX', 'About'].forEach( + (label) => { + cy.contains('a', label).should('be.visible'); + }, + ); + ['Supporters', 'Telemetry'].forEach((label) => { cy.contains('a', label).should('not.exist'); }); }); diff --git a/packages/app/cypress/component/inference-chart-controls.cy.tsx b/packages/app/cypress/component/inference-chart-controls.cy.tsx index f6db52302..3884388fa 100644 --- a/packages/app/cypress/component/inference-chart-controls.cy.tsx +++ b/packages/app/cypress/component/inference-chart-controls.cy.tsx @@ -26,6 +26,7 @@ describe('Inference ChartControls', () => { it('renders the Y-axis metric selector', () => { cy.get('[data-testid="yaxis-metric-selector"]').should('be.visible'); + cy.get('[data-testid="cost-display-selector"]').should('not.exist'); }); it('Y-axis metric selector shows grouped options', () => { @@ -41,6 +42,36 @@ describe('Inference ChartControls', () => { cy.get('@setSelectedYAxisMetric').should('have.been.calledOnce'); }); + it('lists and selects the schema-v2 derived axes in the Measured Energy group', () => { + const options = [ + { + key: 'y_measuredJPerSuccessfulQuery', + label: 'Measured Joules per Successful Query', + }, + { + key: 'y_measuredWhPerSuccessfulQuery', + label: 'Measured Watt-hours per Successful Query', + }, + { + key: 'y_measuredPowerPercentTdp', + label: 'Measured Average Power as Percent of TDP', + }, + ]; + + for (const option of options) { + cy.get('[data-testid="yaxis-metric-selector"]').click(); + cy.contains('Measured Energy') + .parent() + .within(() => { + cy.contains('[role="option"]', option.label) + .scrollIntoView() + .should('be.visible') + .click(); + }); + cy.get('@setSelectedYAxisMetric').should('have.been.calledWith', option.key); + } + }); + it('hides the GPU comparison section when no GPUs are selected', () => { // Default mock: selectedGPUs = [] — GPU date range pickers should not render cy.contains('Comparison Date Range').should('not.exist'); @@ -54,6 +85,31 @@ describe('Inference ChartControls', () => { }); }); +describe('Inference ChartControls cost metrics', () => { + beforeEach(() => { + mountWithProviders(, { + inference: { selectedYAxisMetric: 'y_costh' }, + }); + }); + + it('shows cost per million and tokens per dollar as separate Y-axis options', () => { + cy.get('[data-testid="yaxis-metric-selector"]').click(); + cy.contains('[role="option"]', 'Cost per Million Total Tokens (Owning - Hyperscaler)').should( + 'exist', + ); + cy.contains('[role="option"]', 'Total Tokens per $1 USD (Owning - Hyperscaler)').should( + 'exist', + ); + cy.get('[data-testid="cost-display-selector"]').should('not.exist'); + }); + + it('selects tokens per dollar through the Y-axis metric control', () => { + cy.get('[data-testid="yaxis-metric-selector"]').click(); + cy.contains('[role="option"]', 'Total Tokens per $1 USD (Owning - Hyperscaler)').click(); + cy.get('@setSelectedYAxisMetric').should('have.been.calledWith', 'y_tokensPerDollarH'); + }); +}); + describe('Inference ChartControls with GPUs selected', () => { it('shows the date range picker when GPUs are selected', () => { mountWithProviders(, { @@ -66,7 +122,7 @@ describe('Inference ChartControls with GPUs selected', () => { cy.contains('Comparison Date Range').should('be.visible'); }); - it('flags the date range when nothing has been picked to compare against', () => { + it('leaves the optional date range unflagged for a selected current config', () => { mountWithProviders(, { inference: { selectedGPUs: ['h100'], @@ -75,7 +131,9 @@ describe('Inference ChartControls with GPUs selected', () => { }, }); - cy.contains('button', 'Select date range').should('have.class', 'animate-pulse'); + cy.contains('button', 'Select date range') + .should('not.have.class', 'animate-pulse') + .and('not.have.class', 'border-red-500'); }); it('leaves the date range unflagged when exact comparison entries are pinned', () => { diff --git a/packages/app/cypress/component/multi-date-picker.cy.tsx b/packages/app/cypress/component/multi-date-picker.cy.tsx deleted file mode 100644 index 9f80ff427..000000000 --- a/packages/app/cypress/component/multi-date-picker.cy.tsx +++ /dev/null @@ -1,128 +0,0 @@ -import { useState } from 'react'; - -import { MultiDatePicker } from '@/components/ui/multi-date-picker'; - -const AVAILABLE_DATES = [ - '2025-10-15', - '2025-11-02', - '2025-11-05', - '2025-11-10', - '2025-11-15', - '2025-11-20', - '2025-12-01', - '2025-12-05', - '2025-12-15', -]; - -function MultiDatePickerWrapper({ - initialDates = [], - maxDates = 3, -}: { - initialDates?: string[]; - maxDates?: number; -}) { - const [dates, setDates] = useState(initialDates); - return ( - - ); -} - -describe('MultiDatePicker', () => { - it('renders placeholder when no dates selected', () => { - cy.mount(); - cy.contains('Select dates').should('be.visible'); - }); - - it('shows formatted text for selected dates', () => { - // 1 date: shows the formatted date - cy.mount(); - cy.contains('Nov 15, 2025').should('be.visible'); - }); - - it('shows "vs" text for 2 selected dates', () => { - cy.mount(); - cy.contains('Nov 10, 2025 vs Nov 20, 2025').should('be.visible'); - }); - - it('shows count text for 3+ selected dates', () => { - cy.mount(); - cy.contains('3 dates selected').should('be.visible'); - }); - - it('calendar opens on click', () => { - cy.mount(); - cy.contains('Select dates').click(); - cy.contains('Select Comparison Dates').should('be.visible'); - }); - - it('can select multiple dates', () => { - // Start with a date in November so calendar opens to November - cy.mount(); - cy.contains('Nov 2, 2025').click(); - // Calendar opens to November. Available within range: Nov 2, 5, 10, 15, 20. - cy.get('.grid-cols-7 button').contains(/^5$/u).click(); - cy.get('.grid-cols-7 button').contains('10').click(); - // Should see the selected dates as pills - cy.contains('Selected Dates:').should('be.visible'); - cy.contains('Nov 2, 2025').should('be.visible'); - cy.contains('Nov 5, 2025').should('be.visible'); - cy.contains('Nov 10, 2025').should('be.visible'); - }); - - it('cannot exceed maxDates', () => { - // Start with 1 date already selected in November, calendar opens to Nov - cy.mount(); - cy.contains('Nov 2, 2025').click(); - // Calendar opens to November (month of selected date). Select one more. - cy.get('.grid-cols-7 button').contains(/^5$/u).click(); - // Now at maxDates=2, third date (10) should be disabled - cy.get('.grid-cols-7 button').contains('10').should('be.disabled'); - }); - - it('selected dates show as removable pills', () => { - // Start with a date so calendar opens to its month - cy.mount(); - cy.contains('Nov 5, 2025').click(); - // Calendar opens to November. Add another date. - cy.get('.grid-cols-7 button').contains('10').click(); - // Pill should have a remove button - cy.get('[aria-label="Remove Nov 10, 2025"]').should('be.visible'); - }); - - it('remove button on pill deselects date', () => { - cy.mount(); - cy.contains('Nov 5, 2025').click(); - // Calendar opens to November. Select another date. - cy.get('.grid-cols-7 button').contains('10').click(); - // Remove the first pill (Nov 5) - cy.get('[aria-label="Remove Nov 5, 2025"]').click(); - // Only Nov 10 should remain as a pill in the selected dates section - cy.contains('Selected Dates:') - .parents('.rounded-md') - .first() - .within(() => { - cy.contains('Nov 5, 2025').should('not.exist'); - cy.contains('Nov 10, 2025').should('be.visible'); - }); - }); - - it('Clear All removes all selected dates', () => { - cy.mount(); - cy.contains('Nov 5, 2025').click(); - // Calendar opens to November. Select another date. - cy.get('.grid-cols-7 button').contains('10').click(); - cy.contains('Selected Dates:').should('be.visible'); - // Click Clear All - cy.contains('Clear All').click(); - // Selected Dates section should disappear (no pills) - cy.contains('Selected Dates:').should('not.exist'); - }); -}); diff --git a/packages/app/cypress/component/multi-select.cy.tsx b/packages/app/cypress/component/multi-select.cy.tsx index 76d1ffdfe..e20bf0442 100644 --- a/packages/app/cypress/component/multi-select.cy.tsx +++ b/packages/app/cypress/component/multi-select.cy.tsx @@ -1,4 +1,5 @@ import { useState } from 'react'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; import { MultiSelect, type MultiSelectOption } from '@/components/ui/multi-select'; @@ -15,11 +16,13 @@ function MultiSelectWrapper({ maxSelections, minSelections, searchable = true, + useDefaults = false, }: { initial?: string[]; maxSelections?: number; minSelections?: number; searchable?: boolean; + useDefaults?: boolean; }) { const [value, setValue] = useState(initial); return ( @@ -27,7 +30,7 @@ function MultiSelectWrapper({ options={OPTIONS} value={value} onChange={setValue} - placeholder="Select Chips..." + placeholder={useDefaults ? undefined : 'Select Chips...'} maxSelections={maxSelections} minSelections={minSelections} searchable={searchable} @@ -115,6 +118,23 @@ describe('MultiSelect', () => { cy.contains('Select Chips...').should('be.visible'); }); + it('localizes visible and accessible defaults on Chinese routes', () => { + cy.mount( + + + , + ); + + cy.get('[aria-label="移除 NVIDIA H100 SXM"]').should('exist'); + cy.get('[aria-label="清除所有选择"]').should('exist'); + cy.get('[data-slot="select-trigger"]').click(); + cy.get('input[placeholder="搜索..."]') + .should('have.attr', 'aria-label', '搜索选项') + .type('不存在'); + cy.contains('没有结果').should('be.visible'); + cy.contains('1 / 2 项已选择').should('be.visible'); + }); + it('click outside closes dropdown', () => { cy.mount(
diff --git a/packages/app/cypress/component/normalized-interactivity-help-link.cy.tsx b/packages/app/cypress/component/normalized-interactivity-help-link.cy.tsx new file mode 100644 index 000000000..0774c8f81 --- /dev/null +++ b/packages/app/cypress/component/normalized-interactivity-help-link.cy.tsx @@ -0,0 +1,35 @@ +import { NormalizedInteractivityHelpLink } from '@/components/inference/ui/NormalizedInteractivityHelpLink'; + +describe('NormalizedInteractivityHelpLink', () => { + it('renders a subtle icon-only link to the English FAQ', () => { + cy.mount( +
+ +
, + ); + + cy.get('[data-testid="normalized-interactivity-faq-link"]') + .should('be.visible') + .and('have.text', '') + .and('have.attr', 'href', '/about#faq-normalized-interactivity') + .and('have.attr', 'title', 'What does E2E Normalized Interactivity mean?') + .and('have.attr', 'aria-label', 'What does E2E Normalized Interactivity mean?') + .and('have.class', 'no-export') + .find('svg[aria-hidden="true"]') + .should('be.visible') + .and('have.attr', 'width', '24'); + }); + + it('links the Chinese control to the Chinese FAQ', () => { + cy.mount( +
+ +
, + ); + + cy.get('[data-testid="normalized-interactivity-faq-link"]') + .should('have.attr', 'href', '/zh/about#faq-normalized-interactivity') + .and('have.attr', 'title', '什么是端到端归一化交互性?') + .and('have.attr', 'aria-label', '什么是端到端归一化交互性?'); + }); +}); diff --git a/packages/app/cypress/component/quick-filters-dialog.cy.tsx b/packages/app/cypress/component/quick-filters-dialog.cy.tsx new file mode 100644 index 000000000..86cf90525 --- /dev/null +++ b/packages/app/cypress/component/quick-filters-dialog.cy.tsx @@ -0,0 +1,61 @@ +import { QuickFiltersDialog } from '@/components/inference/ui/QuickFiltersDialog'; +import type { QuickFilters } from '@/components/inference/types'; +import { Sequence } from '@/lib/data-mappings'; +import { mountWithProviders } from '../support/test-utils'; + +const availableQuickFilters: QuickFilters = { + vendors: ['NVIDIA', 'AMD'], + frameworks: ['vllm', 'sglang'], + deployment: ['single-node', 'multi-node', 'disagg'], + spec: ['mtp', 'stp'], +}; + +describe('QuickFiltersDialog', () => { + it('shows every filter group for fixed-sequence charts', () => { + mountWithProviders(, { + inference: { availableQuickFilters }, + }); + + cy.get('[data-testid="quick-filters-dialog"]').should('be.visible'); + cy.get('[data-testid="quick-filter-vendor-NVIDIA"]').should('exist'); + cy.get('[data-testid="quick-filter-framework-vllm"]').should('exist'); + cy.get('[data-testid="quick-filter-deployment-disagg"]').should('exist'); + cy.get('[data-testid="quick-filter-spec-mtp"]').should('exist'); + }); + + it('removes speculative decoding from agentic charts and toggles supported filters', () => { + mountWithProviders(, { + inference: { + selectedSequence: Sequence.AgenticTraces, + availableQuickFilters, + }, + }); + + cy.get('[data-testid="quick-filter-vendor-NVIDIA"]').click(); + cy.get('@setQuickFilterVendors').should('have.been.calledWith', ['NVIDIA']); + cy.get('[data-testid="quick-filter-framework-vllm"]').should('exist'); + cy.get('[data-testid="quick-filter-deployment-disagg"]').should('exist'); + cy.get('[data-testid^="quick-filter-spec-"]').should('not.exist'); + cy.contains('Spec Decoding').should('not.exist'); + }); + + it('clears every filter category from one action', () => { + mountWithProviders(, { + inference: { + quickFilters: { + vendors: ['AMD'], + frameworks: ['sglang'], + deployment: ['disagg'], + spec: ['mtp'], + }, + availableQuickFilters, + }, + }); + + cy.contains('button', 'Clear filters').click(); + cy.get('@setQuickFilterVendors').should('have.been.calledWith', []); + cy.get('@setQuickFilterFrameworks').should('have.been.calledWith', []); + cy.get('@setQuickFilterDeployment').should('have.been.calledWith', []); + cy.get('@setQuickFilterSpec').should('have.been.calledWith', []); + }); +}); diff --git a/packages/app/cypress/component/quotes-content.cy.tsx b/packages/app/cypress/component/quotes-content.cy.tsx index d32d13c5d..34101ab8e 100644 --- a/packages/app/cypress/component/quotes-content.cy.tsx +++ b/packages/app/cypress/component/quotes-content.cy.tsx @@ -32,6 +32,26 @@ describe('QuotesContent', () => { cy.get('button[aria-label^="Jump to"]').should('have.length', uniqueOrgsWithLogo.length); }); + it('renders the Mooncake team quote with the KVCache.AI identity', () => { + cy.get('#quote-mooncake').within(() => { + cy.contains('Mooncake makes KV cache a first-class system resource').should('be.visible'); + cy.contains('a', 'Mooncake Team') + .should('have.attr', 'href', 'https://kvcache.ai/') + .and('have.attr', 'target', '_blank'); + cy.get('img[alt="Mooncake"]') + .should('have.attr', 'src', '/logos/kvcache-ai.svg') + .and(($logo) => { + expect(($logo[0] as HTMLImageElement).naturalWidth).to.be.greaterThan(0); + }); + }); + + cy.mount(); + cy.get('#quote-mooncake').should( + 'contain.text', + 'Mooncake 将 KV cache 作为分离式 LLM 推理中的一等系统资源', + ); + }); + it('scrolls to the matching quote when a logo is clicked', () => { const targetOrg = uniqueOrgsWithLogo[0]; diff --git a/packages/app/cypress/component/scatter-graph.cy.tsx b/packages/app/cypress/component/scatter-graph.cy.tsx index 3ee91849d..504c69178 100644 --- a/packages/app/cypress/component/scatter-graph.cy.tsx +++ b/packages/app/cypress/component/scatter-graph.cy.tsx @@ -1,16 +1,20 @@ -import { useState } from 'react'; -import { GlobalFilterContext } from '@/components/GlobalFilterContext'; -import { InferenceContext } from '@/components/inference/InferenceContext'; -import { UnofficialRunContext } from '@/components/unofficial-run-provider'; +import { useReducer, useState } from 'react'; +import { GlobalFilterSelectionContext } from '@/components/GlobalFilterContext'; +import { InferenceContextsProvider } from '@/components/inference/InferenceContext'; +import { + overlaySelectionReducer, + UnofficialRunContext, +} from '@/components/unofficial-run-provider'; import ScatterGraph from '@/components/inference/ui/ScatterGraph'; import ChartDisplay from '@/components/inference/ui/ChartDisplay'; import { mountWithProviders } from '../support/test-utils'; +import { expandLegendAdvanced } from '../support/legend-advanced'; import { createMockInferenceData, createMockChartDefinition, createMockHardwareConfig, - createMockGlobalFilterContext, - createMockInferenceContext, + createMockGlobalFilterContexts, + createMockInferenceContextValues, createMockUnofficialRunContext, } from '../support/mock-data'; import { Model, Precision, Sequence } from '@/lib/data-mappings'; @@ -138,7 +142,7 @@ describe('ScatterGraph', () => { }), ), ); - const baseInference = createMockInferenceContext(); + const baseInference = createMockInferenceContextValues(); function OfficialScopeHarness() { const [secondScope, setSecondScope] = useState(false); @@ -155,7 +159,12 @@ describe('ScatterGraph', () => { }; return ( - + @@ -169,7 +178,7 @@ describe('ScatterGraph', () => { chartDefinition={chartDefinition} />
- + ); } @@ -204,7 +213,7 @@ describe('ScatterGraph', () => { chartType: 'interactivity', y_tpPerGpu_roofline: 'upper_left', }); - const baseInference = createMockInferenceContext(); + const baseInference = createMockInferenceContextValues(); const baseUnofficial = createMockUnofficialRunContext(); function DelayedOfficialScopeHarness() { @@ -256,15 +265,22 @@ describe('ScatterGraph', () => { ...baseUnofficial, isUnofficialRun: true, activeOverlayHwTypes: activeOverlayKeys, - setActiveOverlayHwTypes: setActiveOverlayKeys, + setUnifiedOverlaySelection: (official: Set, overlay: Set) => { + setOfficialOverride(official); + setActiveOverlayKeys(overlay); + }, allOverlayHwTypes: new Set(['h100_vllm', 'b200_vllm']), localOfficialOverride: officialOverride, - setLocalOfficialOverride: setOfficialOverride, }; return ( - + @@ -291,7 +307,7 @@ describe('ScatterGraph', () => { overlayData={overlayData} />
- + ); } @@ -346,6 +362,9 @@ describe('ScatterGraph', () => { chartType: 'interactivity', y_tpPerGpu_roofline: 'upper_left', }); + const runId = 32177976542; + const runBranch = 'qwen3.5-fp4-gb200-dynamo-sglang-agentic-mtp-pareto-refresh'; + const runUrl = `https://github.com/SemiAnalysisAI/InferenceX/actions/runs/${runId}`; const officialData = [ createMockInferenceData({ hwKey: 'h100', x: 8, y: 240, precision: Precision.FP4 }), createMockInferenceData({ hwKey: 'h100', x: 16, y: 200, precision: Precision.FP4 }), @@ -358,26 +377,26 @@ describe('ScatterGraph', () => { x: 8, y: 320, precision: Precision.FP4, - run_url: 'https://github.com/x/y/actions/runs/12345', + run_url: runUrl, }), createMockInferenceData({ hwKey: 'b200_trt', x: 16, y: 280, precision: Precision.FP4, - run_url: 'https://github.com/x/y/actions/runs/12345', + run_url: runUrl, }), createMockInferenceData({ hwKey: 'b200_trt', x: 32, y: 220, precision: Precision.FP4, - run_url: 'https://github.com/x/y/actions/runs/12345', + run_url: runUrl, }), ], hardwareConfig: hwConfig, - label: 'feature-branch', - runUrl: 'https://github.com/x/y/actions/runs/12345', + label: runBranch, + runUrl, }; mountWithProviders( @@ -403,15 +422,15 @@ describe('ScatterGraph', () => { unofficial: { activeOverlayHwTypes: new Set(['b200_trt']), allOverlayHwTypes: new Set(['b200_trt']), - runIndexByUrl: { 'https://github.com/x/y/actions/runs/12345': 0, '12345': 0 }, + runIndexByUrl: { [runUrl]: 0, [String(runId)]: 0 }, unofficialRunInfos: [ { - id: 12345, + id: runId, name: 'CI run', - branch: 'feature-branch', - sha: 'abc123', - createdAt: '2026-05-01T00:00:00Z', - url: 'https://github.com/x/y/actions/runs/12345', + branch: runBranch, + sha: '7a4a06b', + createdAt: '2026-08-18T19:40:51Z', + url: runUrl, conclusion: 'success', status: 'completed', isNonMainBranch: true, @@ -435,11 +454,104 @@ describe('ScatterGraph', () => { cy.get('#test-scatter-overlay-labels svg .line-label') .filter('[data-line-key]:not([data-line-key^="overlay-"])') .should('have.length.greaterThan', 0); - // Overlay label text is the run's branch name (matching the overlay legend), - // not the hw label. + // The exact branch that crashed the production page remains visible in the + // overlay line label and legend after ScatterGraph's render-time updates. cy.get('#test-scatter-overlay-labels svg .line-label[data-line-key^="overlay-"]') .find('text') - .should('contain.text', 'feature-branch'); + .should('contain.text', runBranch); + cy.get( + '#test-scatter-overlay-labels svg .line-label[data-line-key^="overlay-"] .ll-gpu', + ).should('not.exist'); + cy.get('#test-scatter-overlay-labels [data-testid="chart-legend"]').should( + 'contain.text', + runBranch, + ); + }); + + it('places precision between the GPU and engine in multi-precision labels', () => { + const runUrl = 'https://github.com/x/y/actions/runs/precision-order'; + const chartDefinition = createMockChartDefinition({ + chartType: 'interactivity', + y_tpPerGpu_roofline: 'upper_left', + }); + const data = [Precision.FP4, Precision.FP8].flatMap((precision, precisionIndex) => + [8, 16].map((x, index) => + createMockInferenceData({ + hwKey: 'b200_trt', + x, + y: 320 - precisionIndex * 20 - index * 40, + precision, + }), + ), + ); + const overlayData = { + data: [Precision.FP4, Precision.FP8].flatMap((precision, precisionIndex) => + [8, 16].map((x, index) => + createMockInferenceData({ + hwKey: 'h100_vllm', + x, + y: 260 - precisionIndex * 20 - index * 40, + precision, + run_url: runUrl, + }), + ), + ), + hardwareConfig: hwConfig, + label: '', + runUrl, + }; + + mountWithProviders( +
+ +
, + { + inference: { + hardwareConfig: hwConfig, + activeHwTypes: new Set(['b200_trt']), + hwTypesWithData: new Set(['b200_trt']), + selectedPrecisions: [Precision.FP4, Precision.FP8], + showLineLabels: true, + }, + unofficial: { + activeOverlayHwTypes: new Set(['h100_vllm']), + allOverlayHwTypes: new Set(['h100_vllm']), + runIndexByUrl: { [runUrl]: 0, 'precision-order': 0 }, + // No run metadata: exercise the hardware-label fallback path. + unofficialRunInfos: [], + }, + }, + ); + + cy.get('#test-scatter-precision-order svg .line-label[data-hw-key="b200_trt"] .ll-text') + .should('have.length', 2) + .then(($labels) => { + expect($labels.toArray().map((label) => label.textContent)).to.have.members([ + 'B200 FP4 (TRTLLM)', + 'B200 FP8 (TRTLLM)', + ]); + for (const label of $labels) { + expect( + [...label.querySelectorAll('tspan')].map((segment) => segment.className.baseVal), + ).to.deep.equal(['ll-gpu', 'll-precision', 'll-engine']); + } + }); + cy.get('#test-scatter-precision-order svg .line-label[data-line-key^="overlay-"] .ll-text') + .should('have.length', 2) + .then(($labels) => { + expect($labels.toArray().map((label) => label.textContent)).to.have.members([ + 'H100 FP4 (vLLM)', + 'H100 FP8 (vLLM)', + ]); + }); }); it('renders a line label for a singleton unofficial overlay series', () => { @@ -548,21 +660,25 @@ describe('ScatterGraph', () => { createMockInferenceData({ hwKey: 'h100', x: 300, y: 190, precision: Precision.FP8 }), createMockInferenceData({ hwKey: 'h100', x: 340, y: 150, precision: Precision.FP8 }), ]; - const baseInference = createMockInferenceContext(); + const baseInference = createMockInferenceContextValues(); function IngestedSingletonLabelHarness() { const [showLineLabels, setShowLineLabels] = useState(true); + const inference = { + ...baseInference, + hardwareConfig: hwConfig, + activeHwTypes: new Set(['b200_tilert_mtp', 'h100']), + hwTypesWithData: new Set(['b200_tilert_mtp', 'h100']), + selectedPrecisions: [Precision.FP8], + showLineLabels, + setShowLineLabels, + }; return ( -
{ chartDefinition={interactivityChartDef} />
-
+ ); } @@ -585,7 +701,18 @@ describe('ScatterGraph', () => { .should('have.css', 'opacity', '1') .find('text') .should('have.text', 'B200 (TileRT, MTP)'); + cy.get( + '#test-scatter-ingested-singleton-label svg .line-label[data-hw-key="b200_tilert_mtp"] .ll-gpu', + ) + .should('have.text', 'B200') + .and('have.attr', 'font-weight', '700'); + cy.get( + '#test-scatter-ingested-singleton-label svg .line-label[data-hw-key="b200_tilert_mtp"] .ll-engine', + ) + .should('have.text', ' (TileRT, MTP)') + .and('have.attr', 'fill', '#d1d5db'); + expandLegendAdvanced(); cy.get('#scatter-line-labels').click(); cy.get('#test-scatter-ingested-singleton-label svg .line-label').should('not.exist'); cy.get('#scatter-line-labels').click(); @@ -676,6 +803,12 @@ describe('ScatterGraph', () => { cy.get('#test-scatter-m3-eagle svg .line-label[data-line-key^="overlay-"]') .find('text') .should('contain.text', 'EAGLE'); + cy.get('#test-scatter-m3-eagle svg .line-label[data-line-key^="overlay-"] .ll-gpu') + .should('have.text', 'B200') + .and('have.attr', 'font-weight', '700'); + cy.get('#test-scatter-m3-eagle svg .line-label[data-line-key^="overlay-"] .ll-engine') + .should('contain.text', 'EAGLE') + .and('have.attr', 'fill', '#d1d5db'); // No label should show the generic MTP token for M3. cy.get('#test-scatter-m3-eagle svg .line-label text').should('not.contain.text', 'MTP'); }); @@ -767,15 +900,84 @@ describe('ScatterGraph', () => { ).should('have.css', 'opacity', '1'); // The exclusion resolver must be bypassed rather than resolving in favor of // either the official or overlay engine family. - cy.get('@setActiveOverlayHwTypes').should('not.have.been.called'); + cy.get('@setUnifiedOverlaySelection').should('not.have.been.called'); // An additional official engine can also be selected while the preview is // loaded; the production-only conflict toggle must not be consulted. cy.get('label[for="checkbox-h100_vllm"]').click(); cy.get('@blockedComparisonToggle').should('not.have.been.called'); - cy.get('@setLocalOfficialOverride').should((setOverride) => { - const selection = setOverride.lastCall.args[0] as Set; - expect([...selection]).to.have.members(['b200_sglang', 'h100_vllm']); + cy.get('@setUnifiedOverlaySelection').should((setSelection) => { + const official = setSelection.lastCall.args[0] as Set; + expect([...official]).to.have.members(['b200_sglang', 'h100_vllm']); + }); + }); + + it('keeps the unofficial overlay active when soloing an official hardware series', () => { + const chartDefinition = createMockChartDefinition({ + chartType: 'interactivity', + y_tpPerGpu_roofline: 'upper_left', + }); + const officialData = ['b200_sglang', 'h100_vllm'].flatMap((hwKey, hwIndex) => + [8, 16, 32].map((x, index) => + createMockInferenceData({ + hwKey, + x, + y: 320 - hwIndex * 20 - index * 40, + precision: Precision.FP4, + }), + ), + ); + const runUrl = 'https://github.com/x/y/actions/runs/official-solo'; + const overlayData = { + data: [8, 16, 32].map((x, index) => + createMockInferenceData({ + hwKey: 'h100_vllm', + x, + y: 260 - index * 40, + precision: Precision.FP4, + run_url: runUrl, + }), + ), + hardwareConfig: hwConfig, + label: 'official-solo', + runUrl, + }; + + mountWithProviders( +
+ +
, + { + inference: { + hardwareConfig: hwConfig, + activeHwTypes: new Set(['b200_sglang', 'h100_vllm']), + hwTypesWithData: new Set(['b200_sglang', 'h100_vllm']), + selectedModel: Model.DeepSeek_V4_Pro, + selectedSequence: Sequence.AgenticTraces, + selectedPrecisions: [Precision.FP4], + }, + unofficial: { + activeOverlayHwTypes: new Set(['h100_vllm']), + allOverlayHwTypes: new Set(['h100_vllm']), + }, + }, + ); + + cy.get('#test-scatter-official-solo svg .overlay-roofline-path').should('exist'); + cy.get('label[for="checkbox-h100_vllm"]').click(); + cy.get('@setUnifiedOverlaySelection').should((setSelection) => { + const official = setSelection.lastCall.args[0] as Set; + const overlay = setSelection.lastCall.args[1] as Set; + expect([...official]).to.deep.equal(['h100_vllm']); + expect([...overlay]).to.deep.equal(['h100_vllm']); }); }); @@ -938,7 +1140,7 @@ describe('ScatterGraph', () => { }); describe('ChartDisplay engine comparison guard', () => { - it('includes cost-clipped official and unofficial points in table mode', () => { + it('includes explicitly clipped official and unofficial points in table mode', () => { const chartDefinition = createMockChartDefinition({ chartType: 'interactivity', x: 'median_intvty', @@ -1044,8 +1246,8 @@ describe('ChartDisplay engine comparison guard', () => { it('keeps official table rows synchronized with legend state after a scope change', () => { const chartDefinition = createMockChartDefinition({ chartType: 'interactivity' }); - const baseInference = createMockInferenceContext(); - const baseGlobalFilters = createMockGlobalFilterContext(); + const baseInference = createMockInferenceContextValues(); + const baseGlobalFilters = createMockGlobalFilterContexts().selection; function OfficialRowsScopeHarness() { const [secondScope, setSecondScope] = useState(false); @@ -1082,8 +1284,13 @@ describe('ChartDisplay engine comparison guard', () => { }; return ( - - + + @@ -1094,8 +1301,8 @@ describe('ChartDisplay engine comparison guard', () => { Select vLLM - - + + ); } @@ -1111,7 +1318,62 @@ describe('ChartDisplay engine comparison guard', () => { cy.get('[data-testid="inference-results-table"] tbody tr').should('have.length', 1); cy.get('[data-testid="inference-results-table"] tbody').contains('vLLM').should('exist'); cy.get('[data-testid="inference-results-table"] tbody').contains('SGLang').should('not.exist'); - cy.get('@setLocalOfficialOverride').should('not.have.been.called'); + cy.get('@setUnifiedOverlaySelection').should('not.have.been.called'); + }); + + it('renders the table columns without the median interactivity or TTFT columns', () => { + // Mirror the real interactivity chart: x IS interactivity, which is what + // made the separate median column a duplicate. + const chartDefinition = createMockChartDefinition({ + chartType: 'interactivity', + x: 'median_intvty', + x_label: 'Interactivity (tok/s/user)', + }); + const row = createMockInferenceData({ + hwKey: 'b200_sglang', + hw: 'Official SGLang', + model: Model.DeepSeek_V4_Pro, + precision: Precision.FP4, + }); + + mountWithProviders(, { + inference: { + graphs: [ + { + model: Model.DeepSeek_V4_Pro, + sequence: Sequence.AgenticTraces, + chartDefinition, + data: [row], + }, + ], + selectedModel: Model.DeepSeek_V4_Pro, + selectedSequence: Sequence.AgenticTraces, + selectedXAxisMode: 'interactivity', + activeHwTypes: new Set(['b200_sglang']), + hwTypesWithData: new Set(['b200_sglang']), + }, + globalFilters: { + selectedModel: Model.DeepSeek_V4_Pro, + selectedSequence: Sequence.AgenticTraces, + effectiveSequence: Sequence.AgenticTraces, + }, + unofficial: {}, + }); + + cy.get('[data-testid="inference-table-view-btn"]').click(); + cy.get('[data-testid="inference-results-table"] thead th').then(($headers) => { + const headers = [...$headers].map((th) => (th.textContent ?? '').trim()); + // Interactivity is already the x-axis column on the interactivity chart, + // so the median column duplicated it. Assert both that the column is gone + // and that exactly one interactivity column remains, so a rename cannot + // quietly reintroduce the duplicate. + expect(headers).to.not.include('Median Interactivity (tok/s)'); + expect(headers.filter((h) => h.toLowerCase().includes('interactivity'))).to.have.length(1); + // Median TTFT was dropped too; unlike interactivity it is not duplicated + // by the x-axis column, so nothing else in the table should carry it. + expect(headers).to.not.include('Median TTFT (ms)'); + expect(headers.filter((h) => h.toLowerCase().includes('ttft'))).to.have.length(0); + }); }); it('keeps same-hardware cross-engine AgentX STP rows out of table mode', () => { @@ -1249,8 +1511,10 @@ describe('ChartDisplay engine comparison guard', () => { cy.get('[data-testid="inference-results-table"] tbody tr').should('have.length', 2); cy.get('[data-testid="inference-results-table"] tbody').contains('vLLM').should('exist'); cy.get('[data-testid="inference-results-table"] tbody').contains('SGLang').should('exist'); - // The reconciliation effect must not strip the run's hw types from the provider. - cy.get('@setActiveOverlayHwTypes').should('not.have.been.called'); + cy.get('@reconcileOverlayScope').should((reconcile) => { + const scope = reconcile.lastCall.args[0] as { overlayHwTypes: Set }; + expect([...scope.overlayHwTypes]).to.have.members(['h100_vllm']); + }); }); it('keeps an explicitly empty official legend out of table mode', () => { @@ -1320,15 +1584,24 @@ describe('ChartDisplay engine comparison guard', () => { status: 'completed', isNonMainBranch: true, }; - const baseInference = createMockInferenceContext(); - const baseGlobalFilters = createMockGlobalFilterContext(); + const baseInference = createMockInferenceContextValues(); + const baseGlobalFilters = createMockGlobalFilterContexts().selection; const baseUnofficial = createMockUnofficialRunContext(); function OverlayScopeHarness() { const [secondScope, setSecondScope] = useState(false); const [secondScopeLoaded, setSecondScopeLoaded] = useState(false); - const [activeOverlayKeys, setActiveOverlayKeys] = useState(new Set(['h100_sglang'])); - const [officialOverride, setOfficialOverride] = useState | null>(null); + const [selection, dispatchSelection] = useReducer(overlaySelectionReducer, { + availabilityKey: String(runInfo.id), + activeOverlayHwTypes: new Set(['h100_sglang']), + availableOverlayHwTypes: new Set(['h100_sglang', 'h200_sglang', 'b200_vllm']), + localOfficialOverride: null, + scopeKey: `${Model.DeepSeek_V4_Pro}|${Sequence.AgenticTraces}|${Precision.FP4}|${runInfo.url}|official:b200_sglang|overlay:h100_sglang,h200_sglang`, + scopeOverlayHwTypes: new Set(['h100_sglang', 'h200_sglang']), + scopeReady: true, + bestSelectionKey: '', + bestPerSku: false, + }); const [, setRenderVersion] = useState(0); const model = secondScope ? Model.DeepSeek_R1 : Model.DeepSeek_V4_Pro; const officialKeys = secondScope ? ['h100_vllm', 'b200_sglang'] : ['b200_sglang']; @@ -1387,17 +1660,25 @@ describe('ChartDisplay engine comparison guard', () => { unofficialRunInfos: [runInfo], runIndexByUrl: { [runInfo.url]: 0, [String(runInfo.id)]: 0 }, getOverlayData: () => ({ data: overlayRows, hardwareConfig: hwConfig }), - activeOverlayHwTypes: activeOverlayKeys, - setActiveOverlayHwTypes: setActiveOverlayKeys, - allOverlayHwTypes: new Set(['h100_sglang', 'h200_sglang', 'b200_vllm']), - localOfficialOverride: officialOverride, - setLocalOfficialOverride: setOfficialOverride, + activeOverlayHwTypes: selection.activeOverlayHwTypes, + reconcileOverlayScope: ( + input: Parameters[0], + ) => dispatchSelection({ type: 'scope', input }), + setUnifiedOverlaySelection: (official: Set, overlay: Set) => + dispatchSelection({ type: 'selection', official, overlay }), + allOverlayHwTypes: selection.availableOverlayHwTypes, + localOfficialOverride: selection.localOfficialOverride, }; return ( - + - + @@ -1406,7 +1687,13 @@ describe('ChartDisplay engine comparison guard', () => { @@ -1417,9 +1704,9 @@ describe('ChartDisplay engine comparison guard', () => { Rerender - + - + ); } diff --git a/packages/app/cypress/component/scatter-label-overlap.cy.tsx b/packages/app/cypress/component/scatter-label-overlap.cy.tsx new file mode 100644 index 000000000..6740152fa --- /dev/null +++ b/packages/app/cypress/component/scatter-label-overlap.cy.tsx @@ -0,0 +1,152 @@ +import ScatterGraph from '@/components/inference/ui/ScatterGraph'; +import { Precision } from '@/lib/data-mappings'; + +import { + createMockChartDefinition, + createMockHardwareConfig, + createMockInferenceData, +} from '../support/mock-data'; +import { mountWithProviders } from '../support/test-utils'; + +const hwConfig = createMockHardwareConfig(); + +const CHART_ID = 'test-scatter-label-overlap'; + +interface Box { + left: number; + right: number; + top: number; + bottom: number; +} + +/** Overlap depth of two boxes along each axis; negative means no overlap. */ +function overlap(a: Box, b: Box): { x: number; y: number } { + return { + x: Math.min(a.right, b.right) - Math.max(a.left, b.left), + y: Math.min(a.bottom, b.bottom) - Math.max(a.top, b.top), + }; +} + +/** + * Both axes must exceed this for a collision to count. The placement engine + * models a text block with fixed ascent/descent constants, so a glyph's real + * ink box can stick out by about a pixel without anything looking wrong. + */ +const TOLERANCE_PX = 1; + +/** A curve dense enough that its run-name pill has point labels to land on. */ +function curve(hwKey: string, ys: number[]) { + return ys.map((y, i) => + createMockInferenceData({ + hwKey, + x: 2 ** (i + 3), + y, + conc: 2 ** (i + 3), + precision: Precision.FP4, + // Widens the label in advanced mode (a disagg prefill/decode split), + // which is the shape that overlapped in practice. + disagg: true, + dp_attention: true, + prefill_tp: 8, + prefill_ep: 8, + prefill_dp_attention: true, + prefill_num_workers: 2, + decode_tp: 16, + decode_ep: 16, + decode_dp_attention: true, + decode_num_workers: 1, + }), + ); +} + +function mountChart(useAdvancedLabels: boolean) { + mountWithProviders( +
+ +
, + { + inference: { + hardwareConfig: hwConfig, + activeHwTypes: new Set(['h100', 'b200_trt']), + hwTypesWithData: new Set(['h100', 'b200_trt']), + selectedPrecisions: [Precision.FP4], + showLineLabels: true, + showPointLabels: true, + useAdvancedLabels, + }, + unofficial: {}, + }, + ); +} + +/** Asserts no visible point label is drawn through a visible pill. */ +function expectNoPillOverlap() { + cy.get(`#${CHART_ID} svg .ll-bg`).should('exist'); + + cy.get(`#${CHART_ID} svg`).then(($svg) => { + const svg = $svg[0]; + + const pills = [...svg.querySelectorAll('.line-label, .parallelism-label')] + .filter((group) => group.style.opacity !== '0') + .map((group) => ({ + label: group.textContent ?? '', + box: (group.querySelector('.ll-bg, .pl-bg') as SVGRectElement).getBoundingClientRect(), + })); + + const points = [...svg.querySelectorAll('.point-label')] + .filter((text) => { + const group = text.closest('.dot-group'); + return text.style.opacity !== '0' && group?.style.opacity !== '0'; + }) + .map((text) => ({ label: text.textContent ?? '', box: text.getBoundingClientRect() })); + + // Without these the assertion below would pass on an empty chart. + expect(pills.length, 'visible run-name pills').to.be.greaterThan(0); + expect(points.length, 'visible point labels').to.be.greaterThan(3); + + const collisions: string[] = []; + for (const pill of pills) { + for (const point of points) { + const { x, y } = overlap(pill.box, point.box); + if (x > TOLERANCE_PX && y > TOLERANCE_PX) { + collisions.push( + `"${point.label}" overlaps pill "${pill.label}" by ${x.toFixed(1)}x${y.toFixed(1)}px`, + ); + } + } + } + + expect(collisions, collisions.join('; ')).to.have.length(0); + }); +} + +describe('Scatter label overlap', () => { + it('draws both label kinds (guards the overlap assertions below)', () => { + mountChart(false); + cy.get(`#${CHART_ID} svg .line-label`).should('have.length.greaterThan', 0); + cy.get(`#${CHART_ID} svg .point-label`).should('have.length.greaterThan', 3); + }); + + it('keeps point labels clear of the run-name pills', () => { + mountChart(false); + expectNoPillOverlap(); + }); + + it('keeps the wider parallelism labels clear of the run-name pills', () => { + mountChart(true); + expectNoPillOverlap(); + }); +}); diff --git a/packages/app/cypress/component/tab-nav.cy.tsx b/packages/app/cypress/component/tab-nav.cy.tsx index 9d29cd7b8..24f86098f 100644 --- a/packages/app/cypress/component/tab-nav.cy.tsx +++ b/packages/app/cypress/component/tab-nav.cy.tsx @@ -4,17 +4,7 @@ import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.share import { TabNav } from '@/components/tab-nav'; import { UnofficialRunContext } from '@/components/unofficial-run-provider'; import { createMockUnofficialRunContext } from '../support/mock-data'; - -function createMockRouter() { - return { - push: cy.stub(), - replace: cy.stub(), - refresh: cy.stub(), - back: cy.stub(), - forward: cy.stub(), - prefetch: cy.stub().resolves(), - }; -} +import { createMockRouter } from '../support/mock-router'; /** * Mount TabNav with the provided URL search string written into the window diff --git a/packages/app/cypress/e2e/agentic-catalog.cy.ts b/packages/app/cypress/e2e/agentic-catalog.cy.ts new file mode 100644 index 000000000..1ff5ada12 --- /dev/null +++ b/packages/app/cypress/e2e/agentic-catalog.cy.ts @@ -0,0 +1,74 @@ +import { unlockAgenticGate } from '../support/e2e'; + +/** + * `/inference/agentic` is the telemetry catalog: a hero drawn from the AgentX + * telemetry tutorial, then one card per (model, SKU, engine, precision) config + * that has stored per-request telemetry. + */ +describe('AgentX telemetry catalog', () => { + beforeEach(() => { + cy.window().then((win) => { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }); + }); + + it('leads with the telemetry hero and links to the full tutorial', () => { + cy.visit('/inference/agentic', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentic-catalog-hero"]').within(() => { + cy.get('h1').should('have.text', 'Exploring Agentic Workloads: Detailed Telemetry'); + // Highlight counts come from the tutorial guide, which agentx-telemetry.test.ts + // pins to what the detail page actually renders. + cy.get('dl div').should('have.length', 4); + cy.contains('11').should('be.visible'); + cy.get('[data-testid="agentic-catalog-tutorial-link"]') + .should('contain.text', 'Read the telemetry tutorial') + .and('have.attr', 'href', '/agentx/telemetry'); + }); + }); + + it('lists configurations grouped by model, each opening a point-detail page', () => { + cy.visit('/inference/agentic', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentic-catalog-summary"]') + .should('contain.text', 'AgentX runs with stored telemetry') + .and('contain.text', 'serving configurations across'); + + cy.get('[data-testid^="agentic-catalog-model-"]').should('have.length.greaterThan', 0); + cy.get('[data-testid^="agentic-catalog-card-"]') + .should('have.length.greaterThan', 0) + .first() + .should('have.attr', 'href') + .and('match', /^\/inference\/agentic\/\d+$/u); + }); + + it('reaches the catalog from the footer and highlights Dashboard in the header nav', () => { + cy.visit('/inference/agentic', { onBeforeLoad: unlockAgenticGate }); + + // Telemetry is a footer destination now, so the catalog sits under the + // Dashboard nav entry rather than owning one of its own. + cy.get('[data-testid="nav-link-telemetry"]').should('not.exist'); + cy.get('[data-testid="nav-link-dashboard"]').should('have.class', 'text-brand'); + cy.get('[data-testid="footer-link-telemetry"]').should( + 'have.attr', + 'href', + '/inference/agentic', + ); + }); + + it('ships the Simplified Chinese catalog', () => { + cy.visit('/zh/inference/agentic', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentic-catalog-summary"]').should( + 'contain.text', + '已存储遥测数据的 AgentX 运行', + ); + cy.get('[data-testid="agentic-catalog-tutorial-link"]') + .should('contain.text', '阅读遥测数据教程') + .and('have.attr', 'href', '/zh/agentx/telemetry'); + cy.get('[data-testid^="agentic-catalog-card-"]') + .first() + .should('have.attr', 'href') + .and('match', /^\/zh\/inference\/agentic\/\d+$/u); + }); +}); diff --git a/packages/app/cypress/e2e/agentic-detail-back-nav.cy.ts b/packages/app/cypress/e2e/agentic-detail-back-nav.cy.ts new file mode 100644 index 000000000..9ce372500 --- /dev/null +++ b/packages/app/cypress/e2e/agentic-detail-back-nav.cy.ts @@ -0,0 +1,322 @@ +import { interceptDerivedAgenticMetrics } from '../support/e2e'; + +// --------------------------------------------------------------------------- +// Regression: returning from an agentic point-detail page reset the chart. +// +// Chart filters only ever reach an in-memory store — `url-state.ts` strips the +// share params from the address bar after load — and the point-detail links are +// plain `` (they must support open-in-new-tab), so following one is a +// full-document navigation that destroys that store. Back therefore returned to +// a bare `/inference` and the chart rebuilt from defaults: DeepSeek V4 Pro with +// every legend series on, no matter what the reader had been looking at. +// +// The shared fixtures carry ZERO agentic_traces rows, so this spec injects its +// own agentic coverage (same pattern as gpu-compare-agentic-detail.cy.ts). +// --------------------------------------------------------------------------- + +const AGENTIC_DATE = '2026-06-12'; +const DEFAULT_MODEL_DB_KEY = 'dsv4'; // DeepSeek-V4-Pro, the dashboard default +const TARGET_MODEL_DB_KEY = 'kimik3'; // Kimi-K3, the non-default the reader picks +const TARGET_MODEL_LABEL = 'Kimi K3'; + +const AGENTIC_HARDWARE = [ + { hardware: 'b200', framework: 'vllm' }, + { hardware: 'b300', framework: 'vllm' }, + { hardware: 'h200', framework: 'vllm' }, +]; + +/** The series the reader solos — must differ from the "all on" default. */ +const SOLO_HW_KEY = 'b300_vllm'; +const SOLO_HW_LABEL = 'B300 (vLLM)'; + +const availabilityFor = (model: string) => [ + ...AGENTIC_HARDWARE.map((g) => ({ + model, + isl: null, + osl: null, + precision: 'fp4', + hardware: g.hardware, + framework: g.framework, + spec_method: 'none', + disagg: false, + benchmark_type: 'agentic_traces', + date: AGENTIC_DATE, + })), + // Fixed-seq rows alongside, so the scenario selector sees the "both exist" + // signal it needs to settle confidently on agentic. + ...AGENTIC_HARDWARE.map((g) => ({ + model, + isl: 8192, + osl: 1024, + precision: 'fp4', + hardware: g.hardware, + framework: g.framework, + spec_method: 'none', + disagg: false, + benchmark_type: 'single_turn', + date: AGENTIC_DATE, + })), +]; + +const availability = [ + ...availabilityFor(DEFAULT_MODEL_DB_KEY), + ...availabilityFor(TARGET_MODEL_DB_KEY), +]; + +const percentileLadder = (prefix: string, base: number): Record => ({ + [`median_${prefix}`]: base, + [`p75_${prefix}`]: base * 1.2, + [`p90_${prefix}`]: base * 1.5, + [`p95_${prefix}`]: base * 1.7, + [`p99_${prefix}`]: base * 2.2, + [`std_${prefix}`]: base * 0.3, +}); + +const agenticMetrics = (conc: number): Record => { + const scale = conc / 16; + const itl = 0.011 * scale; + return { + ...percentileLadder('ttft', 0.4 * scale), + ...percentileLadder('tpot', 0.012 * scale), + ...percentileLadder('itl', itl), + ...percentileLadder('e2el', 8 * scale), + median_intvty: 1 / itl, + p75_intvty: 1 / (itl * 1.2), + p90_intvty: 1 / (itl * 1.5), + p99_intvty: 1 / (itl * 2.2), + std_intvty: (1 / itl) * 0.1, + tput_per_gpu: 950 / Math.sqrt(scale), + output_tput_per_gpu: 210, + input_tput_per_gpu: 740, + total_tput_tps: 7600 * conc * 0.05, + }; +}; + +let benchIdCursor = 900100; +const benchmarksFor = (model: string) => + AGENTIC_HARDWARE.flatMap((g) => + [16, 64, 128].map((conc) => ({ + id: benchIdCursor++, + hardware: g.hardware, + framework: g.framework, + model, + precision: 'fp4', + spec_method: 'none', + disagg: false, + is_multinode: false, + prefill_tp: 8, + prefill_ep: 1, + prefill_dp_attention: false, + prefill_num_workers: 0, + decode_tp: 8, + decode_ep: 1, + decode_dp_attention: false, + decode_num_workers: 0, + num_prefill_gpu: 8, + num_decode_gpu: 8, + isl: null, + osl: null, + conc, + offload_mode: 'off', + benchmark_type: 'agentic_traces', + image: 'vllm/vllm-openai:v0.9.0', + metrics: agenticMetrics(conc), + workers: null, + date: AGENTIC_DATE, + run_url: null, + })), + ); + +const benchmarks = [...benchmarksFor(DEFAULT_MODEL_DB_KEY), ...benchmarksFor(TARGET_MODEL_DB_KEY)]; +const tracedIds = new Set(benchmarks.map((b) => b.id)); + +/** Minimal detail-page payloads — this spec only cares about its nav controls. */ +function interceptDetailPage(): void { + cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: null }); + cy.intercept('GET', '/api/v1/trace-histograms*', { body: {} }); + cy.intercept('GET', '/api/v1/request-timeline*', { body: null }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { body: null }); + cy.intercept('GET', '/api/v1/agentic-aggregates*', { body: null }); +} + +function interceptChart(): void { + cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability'); + cy.intercept('GET', '/api/v1/benchmarks*', { body: benchmarks }).as('benchmarks'); + cy.intercept('GET', '/api/v1/trace-availability*', (request) => { + const ids = new URL(request.url).searchParams.get('ids')?.split(',') ?? []; + request.reply({ + body: Object.fromEntries( + ids.filter((id) => tracedIds.has(Number(id))).map((id) => [id, true]), + ), + }); + }); + interceptDerivedAgenticMetrics(); + interceptDetailPage(); +} + +function visitChart(path: string): void { + interceptChart(); + cy.visit(path, { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + cy.get('[data-testid="inference-chart-display"]').should('exist'); +} + +/** Pick the non-default model through the dropdown, exactly as the report does. */ +function selectTargetModel(): void { + cy.get('[data-testid="model-selector"]').should('contain.text', 'DeepSeek V4 Pro'); + cy.get('[data-testid="model-selector"]').click(); + cy.contains('[role="option"]', TARGET_MODEL_LABEL).click(); + cy.get('[data-testid="model-selector"]').should('contain.text', TARGET_MODEL_LABEL); +} + +/** Solo one legend series so the restored state is distinguishable from "all on". */ +function soloLegendSeries(): void { + cy.get('[data-testid="chart-legend"] ul input[type="checkbox"]:checked').should( + 'have.length.greaterThan', + 1, + ); + cy.contains('[data-testid="chart-legend"] label', SOLO_HW_LABEL).click(); + cy.get('[data-testid="chart-legend"] ul input[type="checkbox"]:checked').should('have.length', 1); +} + +/** Pin the tooltip on a visible point of the soloed series and open its detail page. */ +function openPointDetail(): void { + cy.get(`[data-testid="legend-points-${SOLO_HW_KEY}"]`).should('exist'); + cy.get('[data-testid="inference-chart-display"] svg .dot-group') + .should('have.length.greaterThan', 0) + .then(($dots) => { + const target = [...$dots].find((node) => { + const datum = (node as unknown as { __data__?: { hwKey?: string; id?: number } }).__data__; + return datum?.hwKey === SOLO_HW_KEY && tracedIds.has(Number(datum?.id)); + }); + expect(target, `a traced ${SOLO_HW_KEY} point`).to.not.equal(undefined); + cy.wrap(target).find('.visible-shape').click({ force: true }); + }); + cy.get('[data-chart-tooltip]:visible [data-action="view-charts"]').should('be.visible').click(); +} + +/** + * The detail page's two nav controls, addressed by their visible label so this + * spec fails on the restoration assertion rather than on a missing hook when + * run against a build without the fix. + */ +const backControl = (locale: 'en' | 'zh') => + cy.contains('button', locale === 'zh' ? '返回' : 'Back'); +const chartLink = (locale: 'en' | 'zh') => + cy.contains('a', locale === 'zh' ? '推理图表' : 'Inference chart'); + +/** The state the reader left behind, as it must come back. */ +function assertRestoredChart(): void { + cy.get('[data-testid="model-selector"]').should('contain.text', TARGET_MODEL_LABEL); + cy.get('[data-testid="chart-legend"] ul input[type="checkbox"]:checked') + .should('have.length', 1) + .closest('li') + .should('contain.text', SOLO_HW_LABEL); +} + +describe('Agentic point detail — returning to the chart', () => { + it('carries the chart state into the detail link instead of dropping it', () => { + visitChart('/inference'); + selectTargetModel(); + soloLegendSeries(); + + cy.get('[data-testid="inference-chart-display"] svg .dot-group') + .should('have.length.greaterThan', 0) + .then(($dots) => { + const target = [...$dots].find((node) => { + const datum = (node as unknown as { __data__?: { hwKey?: string; id?: number } }) + .__data__; + return datum?.hwKey === SOLO_HW_KEY && tracedIds.has(Number(datum?.id)); + }); + cy.wrap(target).find('.visible-shape').click({ force: true }); + }); + + cy.get('[data-chart-tooltip]:visible [data-action="view-charts"]') + .should('be.visible') + .then(($link) => { + const href = $link.attr('href') ?? ''; + expect(href, 'detail href').to.match(/^\/inference\/agentic\/\d+\?/u); + const params = new URLSearchParams(href.slice(href.indexOf('?'))); + expect(params.get('g_model'), 'g_model carried into the detail link').to.equal('Kimi-K3'); + expect(params.get('i_active'), 'legend selection carried').to.equal(SOLO_HW_KEY); + }); + }); + + it('restores the model and legend when the reader clicks Back', () => { + visitChart('/inference'); + selectTargetModel(); + soloLegendSeries(); + openPointDetail(); + + cy.location('pathname').should('match', /^\/inference\/agentic\/\d+$/u); + backControl('en').click(); + + // Picking Kimi K3 moved the chart's history entry onto its indexable + // subroute, so Back returns there — the path itself now pins the model. + cy.location('pathname').should('eq', '/inference/kimi-k3'); + assertRestoredChart(); + }); + + it('restores the model and legend through the "Inference chart" link', () => { + visitChart('/inference'); + selectTargetModel(); + soloLegendSeries(); + openPointDetail(); + + chartLink('en') + .should('have.attr', 'href') + .and('match', /^\/inference\?/u); + chartLink('en').click(); + + cy.location('pathname').should('eq', '/inference'); + assertRestoredChart(); + }); + + it('keeps a Chinese reader on /zh for the whole round trip', () => { + visitChart('/zh/inference'); + selectTargetModel(); + soloLegendSeries(); + openPointDetail(); + + cy.location('pathname').should('match', /^\/zh\/inference\/agentic\/\d+$/u); + chartLink('zh') + .should('have.attr', 'href') + .and('match', /^\/zh\/inference\?/u); + chartLink('zh').click(); + + cy.location('pathname').should('eq', '/zh/inference'); + assertRestoredChart(); + }); + + it('does not drop the unofficialruns overlay param on the round trip', () => { + visitChart('/inference?unofficialruns=987654321'); + selectTargetModel(); + soloLegendSeries(); + + cy.get('[data-testid="inference-chart-display"] svg .dot-group') + .should('have.length.greaterThan', 0) + .then(($dots) => { + const target = [...$dots].find((node) => { + const datum = (node as unknown as { __data__?: { hwKey?: string; id?: number } }) + .__data__; + return datum?.hwKey === SOLO_HW_KEY && tracedIds.has(Number(datum?.id)); + }); + cy.wrap(target).find('.visible-shape').click({ force: true }); + }); + + cy.get('[data-chart-tooltip]:visible [data-action="view-charts"]') + .should('be.visible') + .then(($link) => { + const href = $link.attr('href') ?? ''; + const params = new URLSearchParams(href.slice(href.indexOf('?'))); + expect(params.get('unofficialruns'), 'overlay run carried forward').to.equal('987654321'); + }); + + cy.get('[data-chart-tooltip]:visible [data-action="view-charts"]').click(); + cy.location('search').should('contain', 'unofficialruns=987654321'); + chartLink('en').should('have.attr', 'href').and('contain', 'unofficialruns=987654321'); + }); +}); diff --git a/packages/app/cypress/e2e/agentic-point-coach-mark.cy.ts b/packages/app/cypress/e2e/agentic-point-coach-mark.cy.ts new file mode 100644 index 000000000..57c1b1dd0 --- /dev/null +++ b/packages/app/cypress/e2e/agentic-point-coach-mark.cy.ts @@ -0,0 +1,289 @@ +/** + * First-visit coach mark on the agentic inference chart. + * + * The affordance it teaches is invisible: clicking an agentic scatter point + * pins its tooltip, and the pinned tooltip carries a "View charts" link to the + * per-point server metrics and logs. The callout draws a pointer to a real, + * currently-visible point and goes away for good once the user either + * dismisses it or clicks a point. + */ +import { + interceptDerivedAgenticMetrics, + keepAgenticCoachMark, + unlockAgenticGate, +} from '../support/e2e'; +import { plotBounds } from '@/lib/d3-chart/plot-bounds'; +import { OVERLAY_RUN_ID, interceptOverlayRun } from '../support/overlay-fixtures'; + +// This spec owns coach-mark state, so it opts out of the global suppression in +// cypress/support/e2e.ts rather than fighting it on every visit (that hook also +// runs on cy.reload(), where onBeforeLoad cannot reach). +keepAgenticCoachMark(); + +const STORAGE_KEY = 'inferencex-agentic-point-coach-mark-dismissed'; +const COACH_MARK = '[data-testid="agentic-point-coach-mark"]'; +const AGENTIC_POINTS = + '[data-testid="scatter-graph"] .dot-group[data-benchmark-type="agentic_traces"]'; +// The pointer targets the marker inside the group, not the group's +// label-inflated bounding box. +const AGENTIC_MARKERS = `${AGENTIC_POINTS} .visible-shape`; + +/** + * Competing dashboard toasts land bottom-right, in the *overlay* slot rather + * than the coach mark's own slot — suppressed anyway so nothing else is + * animating while positions are measured. + */ +function freshCoachMarkState(win: Cypress.AUTWindow) { + unlockAgenticGate(win); + win.localStorage.removeItem(STORAGE_KEY); + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + win.sessionStorage.setItem('inferencex-reproducibility-nudge-shown', '1'); + win.sessionStorage.setItem('inferencex-star-nudge-shown', '1'); + win.localStorage.setItem('inferencex-filter-hint-nudge-dismissed', '1'); +} + +/** + * The callout only appears once there is a point on screen to point at, and at + * this viewport the chart starts below the fold — so bring it into view, the + * same thing a real first-time visitor does before they can see any of this. + */ +function visitAgenticChart(url = '/inference?i_seq=agentic-traces') { + interceptOverlayRun(); + interceptDerivedAgenticMetrics(); + cy.visit(url, { onBeforeLoad: freshCoachMarkState }); + cy.get(AGENTIC_POINTS).should('have.length.greaterThan', 0); + cy.get('[data-testid="scatter-graph"]').scrollIntoView(); +} + +function centreOf(element: Element) { + const rect = element.getBoundingClientRect(); + return { x: rect.left + rect.width / 2, y: rect.top + rect.height / 2 }; +} + +beforeEach(() => { + cy.clearAllLocalStorage(); + cy.clearAllSessionStorage(); + cy.intercept('GET', '/api/v1/trace-availability?ids=*', (request) => { + const ids = new URL(request.url).searchParams.get('ids')?.split(',').filter(Boolean) ?? []; + request.reply({ + body: Object.fromEntries(ids.map((id) => [id, true])), + }); + }); +}); +describe('Agentic point coach mark', () => { + it('points at a real, visible agentic point on the first visit', () => { + visitAgenticChart(); + + cy.get(COACH_MARK) + .should('be.visible') + .and('contain.text', 'Every point has a story') + .and('contain.text', 'Click any point to view server metrics and logs'); + + // Non-modal by construction: a focus trap or a scrim would block the very + // click the tip is asking for. + cy.get(COACH_MARK).should('have.attr', 'role', 'dialog'); + cy.get(COACH_MARK).should('have.attr', 'aria-modal', 'false'); + cy.get('[data-testid="agentic-point-coach-mark-layer"]').should( + 'have.css', + 'pointer-events', + 'none', + ); + + // The callout must target an actual point, not empty plot area. The + // highlight ring sits exactly on the anchor (the pointer line deliberately + // stops short so its arrowhead doesn't cover the dot). + cy.get('[data-testid="agentic-point-coach-mark-pointer"]').should('exist'); + cy.get('[data-testid="agentic-point-coach-mark-target"]').then(($ring) => { + const tipX = Number($ring.attr('cx')); + const tipY = Number($ring.attr('cy')); + + cy.get(AGENTIC_MARKERS).then(($points) => { + const hit = [...$points].some((point) => { + const { x, y } = centreOf(point); + return Math.abs(x - tipX) < 1 && Math.abs(y - tipY) < 1; + }); + expect(hit, 'pointer ends on an agentic point').to.eq(true); + }); + }); + }); + + it('waits for the chart to be on screen before pointing at anything', () => { + interceptOverlayRun(); + interceptDerivedAgenticMetrics(); + cy.visit('/inference?i_seq=agentic-traces', { onBeforeLoad: freshCoachMarkState }); + cy.get(AGENTIC_POINTS).should('have.length.greaterThan', 0); + + // Chart below the fold: no anchor, so no stranded arrow — and no dismissal + // burned either, since the tip has not been shown. + cy.window().its('scrollY').should('eq', 0); + cy.wait(2500); + cy.get(COACH_MARK).should('not.exist'); + cy.window().then((win) => { + expect(win.localStorage.getItem(STORAGE_KEY)).to.eq(null); + }); + + cy.get('[data-testid="scatter-graph"]').scrollIntoView(); + cy.get(COACH_MARK).should('be.visible'); + }); + + it('targets the clipped plot area, not the axis gutters', () => { + // `plotBounds` reconstructs the clip region from the live chart DOM. If the + // skeleton it reads ever changes shape it returns null and the resolver + // silently falls back to the SVG box — which includes the 60px axis + // gutters, where a zoomed point is invisible. Assert against the real DOM + // so that fallback cannot go unnoticed. + visitAgenticChart(); + cy.get(COACH_MARK).should('be.visible'); + + cy.get('[data-testid="scatter-graph"] [data-testid="d3-chart-svg"]').then(($svg) => { + const svg = $svg[0]; + const bounds = plotBounds(svg); + expect(bounds, 'clip region resolves against the real chart').to.not.eq(null); + + const box = svg.getBoundingClientRect(); + expect(bounds!.left, 'left gutter excluded').to.be.greaterThan(box.left); + expect(bounds!.bottom, 'bottom gutter excluded').to.be.lessThan(box.bottom); + + cy.get('[data-testid="agentic-point-coach-mark-target"]').then(($ring) => { + const cx = Number($ring.attr('cx')); + const cy = Number($ring.attr('cy')); + expect(cx).to.be.within(bounds!.left, bounds!.right); + expect(cy).to.be.within(bounds!.top, bounds!.bottom); + }); + }); + }); + + it('keeps the card fully on screen', () => { + visitAgenticChart(); + cy.get(COACH_MARK).should('be.visible'); + + cy.window().then((win) => { + const rect = win.document.querySelector(COACH_MARK)!.getBoundingClientRect(); + const { clientWidth, clientHeight } = win.document.documentElement; + expect(rect.left, 'left edge').to.be.at.least(0); + expect(rect.top, 'top edge').to.be.at.least(0); + expect(rect.right, 'right edge').to.be.at.most(clientWidth); + expect(rect.bottom, 'bottom edge').to.be.at.most(clientHeight); + }); + }); + + it('follows its point when the chart is zoomed', () => { + visitAgenticChart(); + cy.get('[data-testid="agentic-point-coach-mark-target"]').should('exist'); + + // Shift+wheel is the chart's zoom gesture (see the chart instructions). + cy.get('[data-testid="d3-chart-svg"]') + .first() + .trigger('wheel', { deltaY: -400, shiftKey: true, bubbles: true }); + + cy.get('[data-testid="agentic-point-coach-mark-target"]').should(($ring) => { + const tipX = Number($ring.attr('cx')); + const tipY = Number($ring.attr('cy')); + // Wherever the point ended up, the callout is still on top of one — + // no stranded arrow pointing at empty plot area. + const points = Cypress.$(AGENTIC_MARKERS) + .filter((_i, el) => getComputedStyle(el.parentElement!).opacity !== '0') + .toArray(); + const hit = points.some((point) => { + const { x, y } = centreOf(point); + return Math.abs(x - tipX) < 1.5 && Math.abs(y - tipY) < 1.5; + }); + expect(hit, 'callout still targets an agentic point after zoom').to.eq(true); + }); + }); + + it('dismisses from the close button and stays gone after reload', () => { + visitAgenticChart(); + cy.get(COACH_MARK).should('be.visible'); + + cy.get('[data-testid="agentic-point-coach-mark-dismiss"]').click(); + cy.get(COACH_MARK).should('not.exist'); + cy.window().then((win) => { + expect(win.localStorage.getItem(STORAGE_KEY)).to.eq('1'); + }); + + interceptOverlayRun(); + interceptDerivedAgenticMetrics(); + cy.reload(); + cy.get(AGENTIC_POINTS).should('have.length.greaterThan', 0); + cy.wait(1500); + cy.get(COACH_MARK).should('not.exist'); + }); + + it('dismisses when the user actually clicks a point', () => { + visitAgenticChart(); + cy.get(COACH_MARK).should('be.visible'); + + cy.get(AGENTIC_POINTS).first().find('.visible-shape').click({ force: true }); + + cy.get(COACH_MARK).should('not.exist'); + // Clicking a point is engagement, so it persists the dismissal too. + cy.window().then((win) => { + expect(win.localStorage.getItem(STORAGE_KEY)).to.eq('1'); + }); + // …and the interaction it was teaching still happened. + cy.get('[data-chart-tooltip]:visible').should('have.length', 1); + }); + + it('is dismissable from the keyboard', () => { + visitAgenticChart(); + cy.get(COACH_MARK).should('be.visible'); + + cy.get('body').type('{esc}'); + cy.get(COACH_MARK).should('not.exist'); + cy.window().then((win) => { + expect(win.localStorage.getItem(STORAGE_KEY)).to.eq('1'); + }); + }); + + it('never anchors to an unofficial-run overlay marker', () => { + // Overlay runs have no stored trace, so their tooltip offers no + // "View charts" link — anchoring there would teach a dead end. + visitAgenticChart(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces`); + cy.wait('@unofficialRun'); + cy.get('[data-testid="scatter-graph"] .unofficial-overlay-pt').should( + 'have.length.greaterThan', + 0, + ); + + cy.get(COACH_MARK).should('be.visible'); + cy.get('[data-testid="agentic-point-coach-mark-target"]').then(($ring) => { + const tipX = Number($ring.attr('cx')); + const tipY = Number($ring.attr('cy')); + const onPoint = (element: Element) => { + const { x, y } = centreOf(element); + return Math.abs(x - tipX) < 1 && Math.abs(y - tipY) < 1; + }; + + cy.get('[data-testid="scatter-graph"] .unofficial-overlay-pt').then(($overlay) => { + expect([...$overlay].some(onPoint), 'pointer avoids overlay markers').to.eq(false); + }); + cy.get(AGENTIC_MARKERS).then(($official) => { + expect([...$official].some(onPoint), 'pointer lands on an official point').to.eq(true); + }); + }); + }); + + it('does not appear on the agentic detail page it links to', () => { + cy.visit('/inference/agentic/206885', { + onBeforeLoad: freshCoachMarkState, + failOnStatusCode: false, + }); + cy.wait(2000); + cy.get(COACH_MARK).should('not.exist'); + }); + + it('renders Simplified Chinese copy on /zh', () => { + visitAgenticChart('/zh/inference?i_seq=agentic-traces'); + + cy.get(COACH_MARK) + .should('be.visible') + .and('contain.text', '每个数据点背后都有细节') + .and('contain.text', '点击任意数据点即可查看服务端指标与日志'); + cy.get('[data-testid="agentic-point-coach-mark-dismiss"]').should( + 'have.attr', + 'aria-label', + '关闭提示', + ); + }); +}); diff --git a/packages/app/cypress/e2e/agentic-point-time-series.cy.ts b/packages/app/cypress/e2e/agentic-point-time-series.cy.ts index ebb7bba3c..d63920401 100644 --- a/packages/app/cypress/e2e/agentic-point-time-series.cy.ts +++ b/packages/app/cypress/e2e/agentic-point-time-series.cy.ts @@ -7,6 +7,7 @@ const timelineRequest = ( overrides: Record = {}, ) => ({ cid: 'conversation-1', + ri: 0, ti: index, wid: 'worker-1', ad: 0, @@ -23,49 +24,344 @@ const timelineRequest = ( ...overrides, }); +const requestChartPayload = (requests: ReturnType[]) => { + const cids = [...new Set(requests.map((request) => request.cid))]; + const phases = [...new Set(requests.map((request) => request.phase))]; + return { + version: 602, + timelineVersion: 6, + startNs: 0, + endNs: 7_000_000_000, + durationS: 7, + cids, + phases, + requests: requests.map((request) => [ + cids.indexOf(request.cid), + phases.indexOf(request.phase), + Math.round(request.start / 1_000), + Math.round(request.end / 1_000), + request.ttftMs, + request.tpotMs, + request.isl, + request.osl, + request.cancelled ? 1 : 0, + ]), + }; +}; + +const benchmarkSiblings = { + sku: { + hardware: 'b300', + framework: 'vllm', + model: 'kimik3', + precision: 'fp4', + spec_method: 'mtp', + benchmark_type: 'agentic_traces', + github_run_id: 31893747354, + date: '2026-08-15', + dataset_slug: null, + }, + siblings: [ + { + id: 206885, + conc: 8, + offload_mode: 'off', + decode_tp: 8, + decode_ep: 1, + decode_pp: 1, + decode_dcp_size: 8, + decode_pcp_size: 1, + decode_dp_attention: false, + decode_num_workers: 0, + prefill_tp: 8, + prefill_ep: 1, + prefill_pp: 1, + prefill_dcp_size: 8, + prefill_pcp_size: 1, + prefill_dp_attention: false, + prefill_num_workers: 0, + num_prefill_gpu: 8, + num_decode_gpu: 8, + disagg: false, + is_multinode: false, + tput_per_gpu: 1.2, + total_requests: 395, + is_current: true, + has_trace: true, + }, + ], +}; + describe('Agentic point request metric time series', () => { - before(() => { + beforeEach(() => { + const requests = [ + timelineRequest(0, 100, 10), + timelineRequest(1, 200, 20), + timelineRequest(2, 400, 25), + timelineRequest(3, 800, 40), + timelineRequest(4, 1600, 80, { ri: 1 }), + timelineRequest(5, 3200, 160, { phase: 'warmup' }), + timelineRequest(6, 6400, 320, { cancelled: true }), + timelineRequest(7, 0, 0, { + cid: 'conversation-1::sa:subagent_001_abcd', + credit: 1_100_000_000, + start: 1_100_000_000, + end: 1_900_000_000, + ttftMs: null, + tpotMs: null, + isl: null, + osl: null, + }), + timelineRequest(8, 0, 0, { + cid: 'conversation-1::sa:subagent_001_abcd:aux:011', + credit: 1_200_000_000, + start: 1_200_000_000, + end: 1_800_000_000, + ttftMs: null, + tpotMs: null, + isl: null, + osl: null, + }), + ]; cy.intercept('GET', '/api/v1/trace-histograms*', { body: {} }); cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: null }); - cy.intercept('GET', '/api/v1/benchmark-siblings*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { body: benchmarkSiblings }); + cy.intercept('GET', '/api/v1/request-chart-data*', { + body: requestChartPayload(requests), + }).as('requestChartData'); cy.intercept('GET', '/api/v1/request-timeline*', { body: { - version: 3, + version: 6, startNs: 0, endNs: 7_000_000_000, durationS: 7, - requests: [ - timelineRequest(0, 100, 10), - timelineRequest(1, 200, 20), - timelineRequest(2, 400, 25), - timelineRequest(3, 800, 40), - timelineRequest(4, 1600, 80), - timelineRequest(5, 3200, 160, { phase: 'warmup' }), - timelineRequest(6, 6400, 320, { cancelled: true }), - timelineRequest(7, 0, 0, { - cid: 'conversation-1::sa:subagent_001_abcd', - credit: 1_100_000_000, - start: 1_100_000_000, - end: 1_900_000_000, - ttftMs: null, - tpotMs: null, - isl: null, - osl: null, - }), - timelineRequest(8, 0, 0, { - cid: 'conversation-1::sa:subagent_001_abcd:aux:011', - credit: 1_200_000_000, - start: 1_200_000_000, - end: 1_800_000_000, - ttftMs: null, - tpotMs: null, - isl: null, - osl: null, - }), - ], + requests, }, }); cy.visit('/inference/agentic/206885', { onBeforeLoad: unlockAgenticGate }); + // The segmented toggle is present in the server-rendered HTML. Waiting for + // this client-only query proves hydration has attached its click handlers, + // avoiding a Firefox race where an early click leaves `?view=` unchanged. + cy.wait('@requestChartData'); + }); + + it('uses the shared topology label for the active agentic point', () => { + cy.contains('button', 'TP8/DCP8 • c=8').should('be.visible'); + }); + + it('opens the stored server log and loads it incrementally', () => { + const longServerLogPath = + 'agentic/conc_1152/aiperf_artifacts/logs/aiperf/2026-08-20/server.log'; + const firstChunk = `INFO server ready\n${'trace line\n'.repeat(4_000)}`; + const searchMatchOffset = 70_027; + const searchJumpOffset = searchMatchOffset - 16 * 1024; + const routerJumpChunk = `${'context line\n'.repeat(1_260)}${'x'.repeat( + searchMatchOffset - searchJumpOffset - 'context line\n'.length * 1_260, + )}router ready for requests\n`; + cy.intercept( + { method: 'GET', pathname: '/api/v1/server-log-files' }, + { + body: [longServerLogPath, 'results/benchmark.log', 'results/router.log'], + }, + ).as('serverLogFiles'); + cy.intercept({ method: 'GET', pathname: '/api/v1/server-log' }, (request) => { + const params = new URL(request.url).searchParams; + const offset = Number(params.get('offset') ?? 0); + const fileName = params.get('file') ?? longServerLogPath; + if (fileName === longServerLogPath && offset === 0) { + request.alias = 'serverLogInitialChunk'; + } else if (fileName === longServerLogPath && offset === 31) { + request.alias = 'serverLogNextChunk'; + } else if (fileName === 'results/router.log' && offset === searchJumpOffset) { + request.alias = 'serverLogJumpChunk'; + } else if (fileName === 'results/router.log' && offset === 0) { + request.alias = 'serverLogRouterStart'; + } + request.reply({ + body: + fileName === 'results/router.log' + ? { + id: 206885, + fileName, + serverLog: offset === searchJumpOffset ? routerJumpChunk : 'INFO router ready\n', + offset, + nextOffset: null, + } + : offset === 0 + ? { + id: 206885, + fileName, + serverLog: `\u001B[32m${firstChunk}\u001B[0m`, + offset: 0, + nextOffset: 31, + } + : { + id: 206885, + fileName, + serverLog: 'INFO benchmark complete\n', + offset, + nextOffset: null, + }, + }); + }).as('serverLogChunk'); + cy.intercept({ method: 'GET', pathname: '/api/v1/server-log-search' }, (request) => { + expect(new URL(request.url).searchParams.get('q')).to.equal('router ready'); + request.reply({ + body: { + id: 206885, + query: 'router ready', + matches: [ + { + fileName: 'results/router.log', + offset: searchMatchOffset, + before: 'INFO ', + match: 'router ready', + after: ' for requests', + }, + ], + truncated: false, + }, + }); + }).as('serverLogSearch'); + cy.intercept( + { + method: 'GET', + pathname: '/api/v1/server-log', + query: { + id: '206885', + file: longServerLogPath, + download: '1', + }, + }, + { + headers: { + 'content-disposition': 'attachment; filename="server.log"', + 'content-type': 'text/plain; charset=utf-8', + }, + body: firstChunk, + }, + ).as('serverLogDownload'); + cy.viewport(480, 900); + cy.get('[data-testid="detail-view-logs"]').click(); + cy.location('search').should('contain', 'view=logs'); + cy.wait('@serverLogFiles'); + cy.wait('@serverLogInitialChunk'); + + cy.get('[data-testid="agentic-server-log-viewer"]') + .should('contain.text', 'Log files') + .and('contain.text', longServerLogPath) + .then(($viewer) => { + cy.get('#agentic-log-file').then(($trigger) => { + const viewerBounds = $viewer[0].getBoundingClientRect(); + const triggerBounds = $trigger[0].getBoundingClientRect(); + expect(triggerBounds.left).to.be.at.least(viewerBounds.left); + expect(triggerBounds.right).to.be.at.most(viewerBounds.right); + }); + }); + cy.get('[data-testid="server-log-content"]') + .should('contain.text', 'INFO server ready') + .and('not.contain.text', '\u001B[32m'); + + cy.get('[data-testid="server-log-search"]').type('router ready'); + cy.wait('@serverLogSearch'); + cy.get('[data-testid="server-log-search-results"]') + .should('contain.text', '1 match') + .and('contain.text', 'results/router.log') + .and('contain.text', 'character 70,028') + .and('contain.text', 'INFO router ready for requests'); + + cy.get('[data-testid="go-to-server-log-match"]').click(); + cy.wait('@serverLogJumpChunk').then(({ request }) => { + const params = new URL(request.url).searchParams; + expect(params.get('file')).to.equal('results/router.log'); + expect(params.get('offset')).to.equal(String(searchJumpOffset)); + }); + cy.get('#agentic-log-file').should('contain.text', 'results/router.log'); + cy.get('[data-testid="server-log-jump-highlight"]') + .should('be.visible') + .and('have.text', 'router ready'); + + cy.get('#agentic-log-file').click(); + cy.contains('[role="option"]', longServerLogPath).click(); + cy.get('[data-testid="server-log-content"]').should('contain.text', 'INFO server ready'); + cy.get('[data-testid="download-selected-server-log"]') + .should('have.attr', 'href') + .and('include', 'download=1'); + cy.get('[data-testid="download-selected-server-log"]').click(); + cy.wait('@serverLogDownload'); + + cy.get('[data-testid="server-log-content"]').scrollTo('bottom'); + cy.wait('@serverLogNextChunk'); + cy.get('[data-testid="server-log-content"]') + .should('contain.text', 'INFO server ready') + .and('contain.text', 'INFO benchmark complete'); + cy.contains('End of stored log').should('be.visible'); + + cy.get('#agentic-log-file').click(); + cy.contains('[role="option"]', 'results/router.log').click(); + cy.wait('@serverLogRouterStart'); + cy.get('[data-testid="server-log-content"]').should('contain.text', 'INFO router ready'); + + cy.viewport(1280, 720); + cy.get('[data-testid="detail-view-point"]').click(); + }); + + it('keeps loaded log text visible when a later chunk fails', () => { + const retainedLog = `INFO retained after failure\n${'trace line\n'.repeat(2_000)}`; + cy.intercept( + { method: 'GET', pathname: '/api/v1/server-log-files' }, + { body: ['results/server.log', 'results/benchmark.log'] }, + ); + cy.intercept({ method: 'GET', pathname: '/api/v1/server-log' }, (request) => { + const params = new URL(request.url).searchParams; + const fileName = params.get('file'); + const offset = Number(params.get('offset') ?? 0); + if (fileName !== 'results/benchmark.log') { + request.reply({ + body: { + id: 206885, + fileName, + serverLog: 'INFO initial file\n', + offset, + nextOffset: null, + }, + }); + return; + } + request.alias = 'retainedLogChunk'; + if (offset === 0) { + request.reply({ + body: { + id: 206885, + fileName: 'results/benchmark.log', + serverLog: retainedLog, + offset: 0, + nextOffset: 28, + }, + }); + } else { + request.reply({ statusCode: 503, body: { error: 'temporary failure' } }); + } + }); + + cy.contains('button', 'TP8/DCP8 • c=8').should('be.visible'); + cy.get('[data-testid="detail-view-logs"]').click(); + cy.location('search').should('contain', 'view=logs'); + cy.get('#agentic-log-file').click(); + cy.contains('[role="option"]', 'results/benchmark.log').click(); + cy.wait('@retainedLogChunk'); + cy.get('[data-testid="server-log-content"]').should( + 'contain.text', + 'INFO retained after failure', + ); + cy.get('[data-testid="load-more-server-log"]').click(); + cy.contains('The next chunk could not be loaded', { timeout: 15_000 }).should('be.visible'); + cy.get('[data-testid="server-log-content"]').should( + 'contain.text', + 'INFO retained after failure', + ); + cy.get('[data-testid="detail-view-point"]').click(); + cy.location('search').should('not.contain', 'view=logs'); }); it('renders rolling P90 interactivity and TTFT by default using profiling requests only', () => { @@ -181,14 +477,22 @@ describe('Agentic point request metric time series', () => { // coverage is now continuous across [0s, 7s]: idle 0ms (0.0%). A 1.00s value // here would mean the Gantt had regressed to label-based filtering. cy.get('[data-testid="timeline-total-idle-time"]').should('have.text', 'idle 0ms (0.0%)'); + cy.get('[data-timeline-row-kind="parent"]') + .should('have.length', 2) + .then(($rows) => { + expect($rows.eq(0)).to.contain.text('conversation-1 · replay 1'); + expect($rows.eq(1)).to.contain.text('conversation-1 · replay 2'); + }); cy.get('[data-timeline-row-kind="aux"]') .should('have.css', 'padding-left', '24px') .and('contain.text', 'aux 011 · parallel'); }); it('restores the request timeline view after browser Back from a dataset route', () => { + cy.get('[data-testid="detail-view-timeline"]').click(); + cy.get('[data-testid="timeline-total-idle-time"]').should('be.visible'); cy.window().then((win) => { - win.history.pushState({}, '', '/datasets/test-dataset/conversations/conversation-1'); + win.history.pushState({}, '', '/agentx/test-dataset/conversations/conversation-1'); }); cy.go('back'); cy.location('pathname').should('eq', '/inference/agentic/206885'); @@ -204,6 +508,82 @@ describe('Agentic point request metric time series', () => { cy.get('svg path[stroke="#ef4444"]').should('have.length', 1); }); }); + + it('renders the log viewer in Simplified Chinese', () => { + cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: null }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { body: benchmarkSiblings }); + cy.intercept( + { method: 'GET', pathname: '/api/v1/server-log-files' }, + { + body: ['results/server.log'], + }, + ); + cy.intercept( + { method: 'GET', pathname: '/api/v1/server-log' }, + { + body: { + id: 206885, + fileName: 'results/server.log', + serverLog: 'INFO server ready\n', + offset: 0, + nextOffset: null, + }, + }, + ); + cy.visit('/zh/inference/agentic/206885?view=logs', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="detail-view-logs"]') + .should('have.attr', 'aria-selected', 'true') + .and('have.text', '日志'); + cy.get('[data-testid="agentic-server-log-viewer"]') + .should('contain.text', '日志文件') + .and('contain.text', 'results/server.log') + .and('contain.text', '搜索所有日志文件') + .and('contain.text', '已到达日志末尾'); + }); +}); + +describe('Agentic request timeline virtualization', () => { + const requests = Array.from({ length: 2_000 }, (_, index) => + timelineRequest(index, 10, 10, { + cid: `conversation-${String(index).padStart(4, '0')}`, + credit: index * 1_000_000_000, + start: index * 1_000_000_000, + end: (index + 1) * 1_000_000_000, + }), + ); + + beforeEach(() => { + cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: null }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-chart-data*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-timeline*', { + body: { + version: 5, + startNs: 0, + endNs: 2_000_000_000_000, + durationS: 2_000, + requests, + }, + }); + cy.visit('/inference/agentic/206885?view=timeline', { onBeforeLoad: unlockAgenticGate }); + }); + + it('keeps row labels and SVG request bars bounded to the viewport', () => { + cy.get('[data-testid="request-timeline-scroll"]').should('be.visible'); + cy.get('[data-timeline-row-kind]').should(($rows) => { + expect($rows.length).to.be.lessThan(40); + }); + cy.get('[data-testid="request-timeline-svg"] rect').should(($bars) => { + expect($bars.length).to.be.lessThan(100); + }); + + cy.get('[data-testid="request-timeline-scroll"]').scrollTo('bottom'); + cy.get('[data-timeline-row-kind]').should('contain.text', 'conversation-1999'); + cy.get('[data-timeline-row-kind]').should(($rows) => { + expect($rows.length).to.be.lessThan(40); + }); + }); }); const pointMeta = { @@ -279,7 +659,12 @@ describe('Agentic point orchestrator metric sources', () => { ); cy.intercept('GET', '/api/v1/trace-histograms*', { body: {} }); cy.intercept('GET', '/api/v1/benchmark-siblings*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-chart-data*', { statusCode: 404 }); cy.intercept('GET', '/api/v1/request-timeline*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/trace-server-metric-source*', (request) => { + const source = new URL(request.url).searchParams.get('source'); + request.reply({ body: source === decode.source.id ? decode : prefill }); + }); cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: { meta: pointMeta, @@ -296,7 +681,7 @@ describe('Agentic point orchestrator metric sources', () => { prefixCacheHitsTps: prefill.prefixCacheHitsTps, hostKvCacheUsage: [], kvCacheUsageByEngine: [], - metricSources: [prefill, decode], + metricSources: [{ source: prefill.source }, { source: decode.source }], }, }); cy.visit('/inference/agentic/206885', { onBeforeLoad: unlockAgenticGate }); @@ -353,3 +738,64 @@ describe('Agentic point orchestrator metric sources', () => { cy.get('[data-testid="throughput-series-decode"]').should('have.attr', 'aria-pressed', 'false'); }); }); + +const engineSeries = (engineLabel: string, value: number) => ({ + engineLabel, + points: [ + { t: 0, value }, + { t: 1, value: value + 0.05 }, + ], +}); + +describe('Agentic point per-engine KV overlay', () => { + beforeEach(() => { + cy.intercept('GET', '/api/v1/trace-histograms*', { body: {} }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-chart-data*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-timeline*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/trace-server-metrics*', { + body: { + meta: pointMeta, + startNs: 0, + endNs: 2_000_000_000, + durationS: 2, + timeslicesCount: 2, + kvCacheUsage: [ + { t: 0, value: 0.3 }, + { t: 1, value: 0.35 }, + ], + prefixCacheHitRate: [], + queueDepth: [], + promptTokensBySource: {}, + prefillTps: [], + decodeTps: [], + prefixCacheHitsTps: [], + hostKvCacheUsage: [], + // Bare DP ranks plus a role-qualified engine, as the ETL emits for a + // disaggregated run where the decode worker reports no rank label. + kvCacheUsageByEngine: [ + engineSeries('0', 0.2), + engineSeries('1', 0.4), + engineSeries('decode', 0.6), + ], + metricSources: [], + }, + }); + cy.visit('/inference/agentic/206885', { onBeforeLoad: unlockAgenticGate }); + }); + + it('draws one legend entry per engine and leaves named engines unprefixed', () => { + cy.contains('svg', 'KV cache (%)') + .first() + .within(() => { + cy.contains('text', 'DP 0').should('be.visible'); + cy.contains('text', 'DP 1').should('be.visible'); + // Already self-describing — must NOT come out as "DP decode". + cy.contains('text', 'decode').should('be.visible'); + cy.contains('text', 'DP decode').should('not.exist'); + cy.contains('text', 'Avg').should('be.visible'); + // One line per engine plus the average, and no duplicate chips. + cy.get('path[fill="none"]').should('have.length', 4); + }); + }); +}); diff --git a/packages/app/cypress/e2e/agentic-quick-filters.cy.ts b/packages/app/cypress/e2e/agentic-quick-filters.cy.ts new file mode 100644 index 000000000..9fa68cf49 --- /dev/null +++ b/packages/app/cypress/e2e/agentic-quick-filters.cy.ts @@ -0,0 +1,153 @@ +import { unlockAgenticGate } from '../support/e2e'; + +const MODEL = 'dsv4'; +const DATE = '2026-06-12'; + +const percentileLadder = (prefix: string, base: number): Record => ({ + [`median_${prefix}`]: base, + [`p75_${prefix}`]: base * 1.2, + [`p90_${prefix}`]: base * 1.5, + [`p95_${prefix}`]: base * 1.7, + [`p99_${prefix}`]: base * 2.2, + [`std_${prefix}`]: base * 0.3, +}); + +const metrics = (conc: number): Record => { + const scale = conc / 16; + const itl = 0.011 * scale; + return { + ...percentileLadder('ttft', 0.4 * scale), + ...percentileLadder('tpot', 0.012 * scale), + ...percentileLadder('itl', itl), + ...percentileLadder('e2el', 8 * scale), + median_intvty: 1 / itl, + p75_intvty: 1 / (itl * 1.2), + p90_intvty: 1 / (itl * 1.5), + p99_intvty: 1 / (itl * 2.2), + std_intvty: (1 / itl) * 0.1, + tput_per_gpu: 950 / Math.sqrt(scale), + output_tput_per_gpu: 210, + input_tput_per_gpu: 740, + total_tput_tps: 380 * conc, + }; +}; + +const configs = [ + { hardware: 'b200', framework: 'vllm', disagg: false }, + { hardware: 'mi300x', framework: 'sglang', disagg: true }, +]; + +const availability = [ + ...configs.map((config) => ({ + model: MODEL, + isl: null, + osl: null, + precision: 'fp4', + hardware: config.hardware, + framework: config.framework, + spec_method: 'mtp', + disagg: config.disagg, + benchmark_type: 'agentic_traces', + date: DATE, + })), + ...configs.map((config) => ({ + model: MODEL, + isl: 8192, + osl: 1024, + precision: 'fp4', + hardware: config.hardware, + framework: config.framework, + spec_method: 'none', + disagg: config.disagg, + benchmark_type: 'single_turn', + date: DATE, + })), +]; + +let benchmarkId = 980000; +const benchmarks = configs.flatMap((config) => + [16, 64, 128].map((conc) => ({ + id: benchmarkId++, + ...config, + model: MODEL, + precision: 'fp4', + spec_method: 'mtp', + is_multinode: false, + prefill_tp: 8, + decode_tp: 8, + num_prefill_gpu: 8, + num_decode_gpu: 8, + isl: null, + osl: null, + conc, + offload_mode: 'off', + benchmark_type: 'agentic_traces', + image: `${config.framework}/server:test`, + metrics: metrics(conc), + workers: null, + date: DATE, + run_url: null, + })), +); + +function visitAgenticQuickFilters() { + cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability'); + cy.intercept('GET', '/api/v1/benchmarks*', { body: benchmarks }).as('benchmarks'); + cy.visit('/inference?g_model=DeepSeek-V4-Pro&i_seq=agentic-traces&i_prec=fp4&i_spec=mtp', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + unlockAgenticGate(win); + }, + }); + cy.wait(['@availability', '@benchmarks']); + cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); +} + +describe('Agentic Quick Filters', () => { + it('opens from the legend, omits spec decoding, and filters the rendered agentic points', () => { + visitAgenticQuickFilters(); + + // A stale speculative-decoding URL selection is ignored for agentic data. + cy.get('.dot-group[data-hw-key^="b200"]').should('exist'); + cy.get('.dot-group[data-hw-key^="mi300x"]').should('exist'); + cy.get('[data-testid="quick-filters-dialog"]').should('not.exist'); + cy.get('[data-testid="scatter-quick-filters"]').should('contain.text', 'Quick Filters').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('be.visible'); + cy.get('[data-testid="quick-filters-selected-count"]').should('not.exist'); + + cy.get('[data-testid^="quick-filter-spec-"]').should('not.exist'); + cy.contains('Spec Decoding').should('not.exist'); + + cy.get('[data-testid="quick-filter-vendor-AMD"]').click(); + cy.get('.dot-group[data-hw-key^="b200"]').should('not.exist'); + cy.get('.dot-group[data-hw-key^="mi300x"]').should('exist'); + cy.get('[data-testid="quick-filters-selected-count"]').should('contain.text', '1 selected'); + + // An incompatible cross-group selection produces zero points. The dialog + // must stay mounted so the reader can recover without reloading the page. + cy.get('[data-testid="quick-filter-framework-vllm"]').click(); + cy.get('[data-testid="scatter-empty-quick-filters"]').should('exist'); + cy.get('[data-testid="quick-filters-dialog"]').should('be.visible'); + + cy.contains('button', 'Clear filters').click(); + cy.get('.dot-group[data-hw-key^="b200"]').should('exist'); + cy.get('.dot-group[data-hw-key^="mi300x"]').should('exist'); + cy.get('[data-testid="quick-filters-selected-count"]').should('not.exist'); + }); + + it('clears Quick Filters from the legend reset action', () => { + visitAgenticQuickFilters(); + + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filter-vendor-AMD"]').click(); + cy.contains('button', 'Done').click(); + + cy.get('.dot-group[data-hw-key^="b200"]').should('not.exist'); + cy.get('[data-testid="scatter-reset-filter"]').click(); + cy.get('.dot-group[data-hw-key^="b200"]').should('exist'); + cy.get('.dot-group[data-hw-key^="mi300x"]').should('exist'); + + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filter-vendor-AMD"]').should('have.attr', 'aria-pressed', 'false'); + }); +}); diff --git a/packages/app/cypress/e2e/agentx-optimizations.cy.ts b/packages/app/cypress/e2e/agentx-optimizations.cy.ts new file mode 100644 index 000000000..acf4e0616 --- /dev/null +++ b/packages/app/cypress/e2e/agentx-optimizations.cy.ts @@ -0,0 +1,148 @@ +import { unlockAgenticGate } from '../support/e2e'; + +const FRAMEWORK_SLUGS = [ + 'vllm', + 'sglang', + 'tensorrt-llm', + 'atom', + 'aiter', + 'dynamo', + 'lmcache', + 'mooncake', +]; + +describe('AgentX optimizations', () => { + beforeEach(() => { + cy.intercept('GET', '/api/v1/datasets', { statusCode: 200, body: [] }); + }); + + it('links from /agentx to the optimizations index and to each project', () => { + cy.visit('/agentx', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-optimizations-callout"]').within(() => { + cy.contains('h2', 'Optimizations for Agentic Workloads').should('be.visible'); + cy.contains('50+ upstream pull requests').should('be.visible'); + cy.get('[data-testid="agentx-optimizations-framework-link"]').should( + 'have.length', + FRAMEWORK_SLUGS.length, + ); + // Direct route into one project, for a reader who knows which engine they run. + cy.get('[data-testid="agentx-optimizations-framework-link"][data-framework="sglang"]').should( + 'have.attr', + 'href', + '/agentx/optimizations/sglang', + ); + cy.get('[data-testid="agentx-optimizations-cta"]') + .should('contain.text', 'Read the optimizations') + .click(); + }); + + cy.location('pathname').should('eq', '/agentx/optimizations'); + }); + + it('publishes the index with the ecosystem primer, every project, and sourced PRs', () => { + cy.visit('/agentx/optimizations', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-optimizations-index"]').within(() => { + cy.get('h1').should('have.text', 'Optimizations for Agentic Workloads'); + cy.get('[data-testid="agentx-optimizations-card"]').should( + 'have.length', + FRAMEWORK_SLUGS.length, + ); + cy.get('[data-testid="agentx-optimizations-section-ecosystem"]').should( + 'contain.text', + 'A brief introduction to the distributed inference ecosystem', + ); + cy.get('[data-testid="agentx-optimizations-figure-servingStack"] img').should('be.visible'); + // The day-zero section keeps its own upstream references. + cy.get('[data-testid="agentx-optimizations-section-other"]') + .find('[data-testid="agentx-optimizations-pr"]') + .first() + .should('have.attr', 'href') + .and('match', /^https:\/\/github\.com\/vllm-project\/vllm\/pull\/\d+$/u); + cy.get('[data-testid="agentx-optimizations-card"][data-framework="vllm"]').click(); + }); + + cy.location('pathname').should('eq', '/agentx/optimizations/vllm'); + }); + + it('renders a project page with its measurements, figures, and PR links', () => { + cy.visit('/agentx/optimizations/vllm', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-optimizations-article"]') + .should('have.attr', 'data-framework', 'vllm') + .within(() => { + cy.get('h1').should('have.text', 'vLLM'); + cy.contains('prefix-cache hit rate above 95%').should('be.visible'); + cy.contains('81.7% higher output throughput').should('be.visible'); + cy.get('[data-testid="agentx-optimizations-figure-vllmSelectiveRetention"] img').should( + 'be.visible', + ); + cy.get('[data-testid="agentx-optimizations-pr"]') + .should('have.length.greaterThan', 20) + .first() + .should('have.attr', 'href', 'https://github.com/vllm-project/vllm/pull/43447') + .and('contain.text', 'vLLM #43447'); + // Sibling projects stay one click away. + cy.get('[data-testid="agentx-optimizations-card"]').should( + 'have.length', + FRAMEWORK_SLUGS.length - 1, + ); + }); + }); + + it('serves every project page and pairs each with its Chinese sibling', () => { + for (const slug of FRAMEWORK_SLUGS) { + cy.request(`/agentx/optimizations/${slug}`).its('status').should('eq', 200); + cy.request(`/zh/agentx/optimizations/${slug}`).its('status').should('eq', 200); + } + }); + + it('returns 404 for a project that does not exist', () => { + cy.request({ url: '/agentx/optimizations/not-a-framework', failOnStatusCode: false }) + .its('status') + .should('eq', 404); + }); + + it('ships the Simplified Chinese index and project pages', () => { + cy.visit('/zh/agentx', { onBeforeLoad: unlockAgenticGate }); + cy.get('[data-testid="agentx-optimizations-callout"]').within(() => { + cy.contains('h2', '面向智能体负载的优化').should('be.visible'); + cy.get('[data-testid="agentx-optimizations-cta"]') + .should('contain.text', '查看优化详情') + .and('have.attr', 'href', '/zh/agentx/optimizations'); + }); + + cy.visit('/zh/agentx/optimizations', { onBeforeLoad: unlockAgenticGate }); + cy.get('[data-testid="agentx-optimizations-index"]').within(() => { + cy.get('h1').should('have.text', '面向智能体负载的优化'); + cy.contains('分布式推理生态简介').should('be.visible'); + cy.get('[data-testid="agentx-optimizations-card"]').should( + 'have.length', + FRAMEWORK_SLUGS.length, + ); + cy.get('[data-testid="agentx-optimizations-card"][data-framework="lmcache"]').should( + 'have.attr', + 'href', + '/zh/agentx/optimizations/lmcache', + ); + }); + cy.get('link[rel="alternate"][hreflang="en"]').should( + 'have.attr', + 'href', + 'https://inferencex.semianalysis.com/agentx/optimizations', + ); + + cy.visit('/zh/agentx/optimizations/sglang', { onBeforeLoad: unlockAgenticGate }); + cy.get('[data-testid="agentx-optimizations-article"]').within(() => { + // Product names stay English; the prose around them is translated. + cy.get('h1').should('have.text', 'SGLang'); + cy.contains('HiCache offload').should('be.visible'); + cy.contains('并发 384 下输出吞吐量提升 26.75%').should('be.visible'); + cy.get('[data-testid="agentx-optimizations-pr"]') + .first() + .should('have.attr', 'href') + .and('match', /^https:\/\/github\.com\/sgl-project\/sglang\/pull\/\d+$/u); + }); + }); +}); diff --git a/packages/app/cypress/e2e/agentx-telemetry.cy.ts b/packages/app/cypress/e2e/agentx-telemetry.cy.ts new file mode 100644 index 000000000..f3d5fc7eb --- /dev/null +++ b/packages/app/cypress/e2e/agentx-telemetry.cy.ts @@ -0,0 +1,157 @@ +import { keepTelemetryTutorial, unlockAgenticGate } from '../support/e2e'; + +// This spec owns the telemetry-tutorial card's storage, so it opts out of the +// global suppression seeded in support/e2e.ts (which also re-seeds on reload). +keepTelemetryTutorial(); + +const SECTION_IDS = [ + 'why-per-point-telemetry', + 'one-curve-per-stack', + 'point-detail-page', + 'kv-offload', + 'request-timeline', + 'flamegraph', +]; + +const FIGURE_KEYS = [ + 'perPointOverview', + 'chartPointTooltip', + 'pointDetailFull', + 'requestTimeline', + 'flamegraph', +]; + +describe('AgentX telemetry tutorial — English page', () => { + beforeEach(() => { + cy.visit('/agentx/telemetry'); + }); + + it('renders the article, every section, and every figure', () => { + cy.get('[data-testid="agentx-telemetry-article"]').should('be.visible'); + cy.contains('h1', 'Exploring Agentic Workloads: Detailed Telemetry').should('be.visible'); + + for (const id of SECTION_IDS) { + cy.get(`[data-testid="agentx-telemetry-section-${id}"]`).should('exist'); + // Every section id is also an anchor target for the on-this-page nav. + cy.get(`a[href="#${id}"]`).should('exist'); + } + + for (const key of FIGURE_KEYS) { + cy.get(`[data-testid="agentx-telemetry-figure-${key}"]`) + .should('exist') + // next/image lazy-loads below the fold, so nothing decodes until the + // figure is scrolled into view. + .scrollIntoView() + .find('img') + // naturalWidth is only non-zero once the file actually decodes, so this + // fails on a missing or broken asset rather than a missing tag. + .should(($img) => { + expect($img[0].naturalWidth, `decoded width of ${key}`).to.be.greaterThan(0); + }); + } + }); + + it('states the eleven-chart claim the detail page has to back', () => { + cy.get('[data-testid="agentx-telemetry-article"]') + .should('contain.text', '11') + .and('contain.text', 'per-point telemetry charts'); + }); + + it('links back to /agentx and out to the agentic results', () => { + cy.get('[data-testid="agentx-telemetry-results-cta"]') + .should('be.visible') + .and('have.attr', 'href', '/inference?i_seq=agentic-traces'); + + cy.contains('a', 'AgentX datasets').click(); + cy.location('pathname').should('eq', '/agentx'); + }); +}); + +describe('AgentX telemetry tutorial — Chinese page', () => { + it('renders translated copy and keeps the same section anchors', () => { + cy.visit('/zh/agentx/telemetry'); + cy.get('[data-testid="agentx-telemetry-article"]').should('be.visible'); + cy.contains('h1', '通过详细遥测数据解析智能体负载').should('be.visible'); + // Anchors are language-neutral so a deep link survives the locale switch. + for (const id of SECTION_IDS) { + cy.get(`[data-testid="agentx-telemetry-section-${id}"]`).should('exist'); + } + cy.get('[data-testid="agentx-telemetry-results-cta"]').should( + 'have.attr', + 'href', + '/zh/inference?i_seq=agentic-traces', + ); + }); +}); + +describe('AgentX telemetry tutorial — entry point on /agentx', () => { + it('shows the callout and navigates to the tutorial', () => { + cy.visit('/agentx'); + cy.get('[data-testid="agentx-telemetry-callout"]').should('be.visible'); + cy.get('[data-testid="agentx-telemetry-cta"]').click(); + cy.location('pathname').should('eq', '/agentx/telemetry'); + }); + + it('shows the Chinese callout on /zh/agentx', () => { + cy.visit('/zh/agentx'); + cy.get('[data-testid="agentx-telemetry-callout"]').should('be.visible'); + cy.get('[data-testid="agentx-telemetry-cta"]').click(); + cy.location('pathname').should('eq', '/zh/agentx/telemetry'); + }); +}); + +describe('AgentX telemetry tutorial — popup on the agentic point-detail page', () => { + beforeEach(() => { + // The detail page's own data is irrelevant here; stub it so the spec is + // about the nudge, not about whichever benchmark row happens to exist. + cy.intercept('GET', '/api/v1/trace-histograms*', { body: {} }); + cy.intercept('GET', '/api/v1/trace-server-metrics*', { body: null }); + cy.intercept('GET', '/api/v1/benchmark-siblings*', { statusCode: 404 }); + cy.intercept('GET', '/api/v1/request-timeline*', { statusCode: 404 }); + cy.visit('/inference/agentic/206885', { + onBeforeLoad: (win) => { + unlockAgenticGate(win); + win.localStorage.removeItem('inferencex-agentx-telemetry-tutorial-dismissed'); + }, + }); + }); + + it('appears after the charts settle, without a backdrop over them', () => { + cy.get('[data-testid="telemetry-tutorial-modal"]', { timeout: 10_000 }).should('be.visible'); + // A bottom-right card, not a centered dialog — the charts stay usable. + cy.get('[data-testid="telemetry-tutorial-modal"]').should('have.attr', 'aria-modal', 'false'); + cy.get('[data-testid="telemetry-tutorial-modal"]').should( + 'contain.text', + 'New to these charts?', + ); + }); + + it('is the only card in the corner — dashboard nudges stay off this route', () => { + // The detail page sits inside the (dashboard) route group, so the dashboard + // NudgeEngine would otherwise mount here too and stack its own bottom-right + // cards under the tutorial. `reproducibility` (1.5s timer, no conditions) is + // the one that always fires, so it is the canary for the carve-out in + // DashboardShell. + cy.get('[data-testid="telemetry-tutorial-modal"]', { timeout: 10_000 }).should('be.visible'); + cy.get('[data-testid="reproducibility-nudge"]').should('not.exist'); + cy.get('[data-testid="filter-hint-nudge"]').should('not.exist'); + }); + + it('opens the tutorial from its primary action', () => { + cy.get('[data-testid="telemetry-tutorial-modal-action"]', { timeout: 10_000 }).click(); + cy.location('pathname').should('eq', '/agentx/telemetry'); + }); + + it('stays dismissed across a reload', () => { + cy.get('[data-testid="telemetry-tutorial-modal-dismiss"]', { timeout: 10_000 }).click(); + cy.get('[data-testid="telemetry-tutorial-modal"]').should('not.exist'); + cy.reload(); + cy.get('[data-testid="detail-view-toggle"]').should('exist'); + // A `not.exist` assertion passes on its FIRST check, so a timeout would not + // outlast the nudge's 2.5s trigger delay — the card could still appear + // afterwards and the test would have gone green. Burn the delay first, then + // assert absence, so this only passes if dismissal actually persisted. + cy.wait(3_500); + cy.get('[data-testid="telemetry-tutorial-modal"]').should('not.exist'); + }); +}); diff --git a/packages/app/cypress/e2e/ai-chart.cy.ts b/packages/app/cypress/e2e/ai-chart.cy.ts new file mode 100644 index 000000000..f9ba41bc8 --- /dev/null +++ b/packages/app/cypress/e2e/ai-chart.cy.ts @@ -0,0 +1,171 @@ +type FixtureRow = Record & { metrics: Record }; + +function fixtureRow( + rows: FixtureRow[], + hardware: 'b200' | 'mi355x', + x: number, + throughput: number, + measuredPower?: number, +): FixtureRow { + const template = rows.find( + (row) => + row.hardware === hardware && + row.framework === 'sglang' && + row.spec_method === 'mtp' && + row.disagg === false && + row.isl === 8192 && + row.osl === 1024, + ); + if (!template) throw new Error(`Missing ${hardware} SGLang fixture row`); + + const metrics: Record = { + ...template.metrics, + median_intvty: x, + tput_per_gpu: throughput, + output_tput_per_gpu: throughput / 2, + input_tput_per_gpu: throughput / 2, + }; + if (measuredPower === undefined) delete metrics.avg_power_w; + else metrics.avg_power_w = measuredPower; + + return { + ...template, + conc: x, + metrics, + }; +} + +function interceptGeneration(spec: Record, rows: FixtureRow[]): void { + cy.intercept('GET', '**/api/v1/benchmarks?*', rows).as('benchmarks'); + cy.intercept('POST', 'https://api.openai.com/v1/chat/completions', (request) => { + const systemPrompt = request.body.messages?.[0]?.content ?? ''; + const content = systemPrompt.includes('chart generation assistant') + ? JSON.stringify(spec) + : 'Deterministic test summary.'; + request.reply({ choices: [{ message: { content } }] }); + }).as('openAi'); +} + +function generateChart(): void { + cy.get('input[placeholder="OpenAI API Key"]').type('test-api-key', { log: false }); + cy.get('textarea').type('Generate the requested chart'); + cy.contains('button', 'Generate Chart').click(); + cy.wait('@benchmarks'); +} + +function benchmarkSpec(overrides: Record): Record { + return { + chartType: 'bar', + dataSource: 'benchmarks', + model: 'DeepSeek-R1-0528', + sequence: '8k/1k', + hardwareKeys: ['b200', 'mi355x'], + precisions: [], + frameworks: ['sglang'], + disagg: false, + yAxisMetric: 'y_tpPerGpu', + yAxisLabel: 'Throughput/Chip', + targetInteractivity: 40, + sortOrder: 'desc', + radarMetrics: null, + topN: null, + topNDistinctGpus: true, + title: 'AI chart regression', + description: 'Deterministic fixture data.', + ...overrides, + }; +} + +describe('AI chart metric semantics', () => { + beforeEach(() => { + cy.visit('/ai-chart'); + }); + + it('excludes unavailable zero cost when selecting the top configuration', () => { + cy.fixture('api/benchmarks.json').then((fixtureRows) => { + const rows = [ + fixtureRow(fixtureRows, 'b200', 40, 0), + fixtureRow(fixtureRows, 'mi355x', 40, 10_000), + ]; + interceptGeneration( + benchmarkSpec({ + yAxisMetric: 'y_costh', + yAxisLabel: 'Cost per Million Total Tokens', + sortOrder: 'asc', + topN: 1, + topNDistinctGpus: false, + }), + rows, + ); + + generateChart(); + + cy.get('#ai-chart-bar-export') + .should('contain.text', 'MI355X') + .and('not.contain.text', 'B200'); + cy.get('#ai-chart-bar-export .bar').should('have.length', 1); + }); + }); + + it('renders lower measured power farther from the radar center', () => { + cy.fixture('api/benchmarks.json').then((fixtureRows) => { + const rows = [ + fixtureRow(fixtureRows, 'b200', 40, 100, 600), + fixtureRow(fixtureRows, 'mi355x', 40, 100, 300), + ]; + interceptGeneration( + benchmarkSpec({ + chartType: 'radar', + yAxisMetric: 'y_measuredAvgPower', + yAxisLabel: 'Measured Average Power per Chip', + radarMetrics: ['y_measuredAvgPower'], + }), + rows, + ); + + generateChart(); + + cy.get('#ai-chart-radar-export .radar-dot') + .should('have.length', 2) + .then(($dots) => { + const b200Radius = Math.hypot( + Number($dots.eq(0).attr('cx')), + Number($dots.eq(0).attr('cy')), + ); + const mi355xRadius = Math.hypot( + Number($dots.eq(1).attr('cx')), + Number($dots.eq(1).attr('cy')), + ); + expect(mi355xRadius).to.be.greaterThan(b200Radius); + }); + }); + }); + + it('keeps a missing middle metric as a line gap instead of a zero-valued dot', () => { + cy.fixture('api/benchmarks.json').then((fixtureRows) => { + const rows = [ + fixtureRow(fixtureRows, 'b200', 10, 100, 500), + fixtureRow(fixtureRows, 'b200', 20, 200), + fixtureRow(fixtureRows, 'b200', 30, 300, 400), + ]; + interceptGeneration( + benchmarkSpec({ + chartType: 'line', + hardwareKeys: ['b200'], + yAxisMetric: 'y_measuredAvgPower', + yAxisLabel: 'Measured Average Power per Chip', + }), + rows, + ); + + generateChart(); + + cy.get('#ai-chart-line-export .dot-group').should('have.length', 2); + cy.get('#ai-chart-line-export .line-path') + .invoke('attr', 'd') + .should((path) => { + expect(path?.match(/M/gu)).to.have.length(2); + }); + }); + }); +}); diff --git a/packages/app/cypress/e2e/api-documentation.cy.ts b/packages/app/cypress/e2e/api-documentation.cy.ts index f96d70f81..f22f9e9b5 100644 --- a/packages/app/cypress/e2e/api-documentation.cy.ts +++ b/packages/app/cypress/e2e/api-documentation.cy.ts @@ -76,6 +76,6 @@ describe('API documentation', () => { .and('have.attr', 'hreflang', 'en'); cy.get('[data-testid="footer-link-api"]') .should('have.attr', 'href', '/zh/api') - .and('have.text', 'API 参考文档'); + .and('have.text', 'API 文档'); }); }); diff --git a/packages/app/cypress/e2e/blog.cy.ts b/packages/app/cypress/e2e/blog.cy.ts index 0af6efc89..351f37fef 100644 --- a/packages/app/cypress/e2e/blog.cy.ts +++ b/packages/app/cypress/e2e/blog.cy.ts @@ -28,6 +28,9 @@ describe('Blog', () => { describe('Blog post page', () => { before(() => { + cy.intercept('GET', 'https://substack-post-media.s3.amazonaws.com/**', { + statusCode: 204, + }); cy.visit('/blog/inferencemax-open-source-inference-benchmarking'); }); diff --git a/packages/app/cypress/e2e/chart-overflow-continuation.cy.ts b/packages/app/cypress/e2e/chart-overflow-continuation.cy.ts index 5b78f53de..572ddfd1e 100644 --- a/packages/app/cypress/e2e/chart-overflow-continuation.cy.ts +++ b/packages/app/cypress/e2e/chart-overflow-continuation.cy.ts @@ -85,6 +85,9 @@ const availability = [ const visitOverflowChart = (withOverlay: boolean) => { cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability'); cy.intercept('GET', '/api/v1/benchmarks*', { body: rows(null) }).as('benchmarks'); + cy.intercept('GET', '/api/v1/workflow-info*', { + body: { runs: [], changelogs: [], configs: [] }, + }).as('workflowInfo'); if (withOverlay) { cy.intercept('GET', '/api/unofficial-run*', { body: { diff --git a/packages/app/cypress/e2e/collectivex.cy.ts b/packages/app/cypress/e2e/collectivex.cy.ts index a7cda6230..204e7a69d 100644 --- a/packages/app/cypress/e2e/collectivex.cy.ts +++ b/packages/app/cypress/e2e/collectivex.cy.ts @@ -43,6 +43,16 @@ const kvDataset = buildDataset({ ], meta: { run_id: '162', generated_at: '2026-08-07T12:20:00Z', source_sha: 'e'.repeat(40) }, }); +const kvComparisonDataset = buildDataset({ + shards: [makeRawShard()], + kv: [{}], + meta: { run_id: '164', generated_at: '2026-08-09T12:20:00Z', source_sha: 'a'.repeat(40) }, +}); +const kvOnlyDataset = buildDataset({ + shards: [], + kv: [{}], + meta: { run_id: '163', generated_at: '2026-08-08T12:20:00Z', source_sha: 'f'.repeat(40) }, +}); const ADMIN_TOKEN_KEY = 'collectivex-admin-token'; function installRuns(bodies: CollectiveXDataset[] = [dataset]) { @@ -174,7 +184,7 @@ describe('CollectiveX neutral run view', () => { .and('not.contain.text', 'Payload rate is derived'); }); - it('allows both measured kernel modes to be selected together', () => { + it('selects normal and low-latency kernel modes by default', () => { const withLowLatency = buildDataset({ shards: [makeRawShard(), makeRawShard({ mode: 'low-latency' }), makeRawShard({ ep: 16 })], }); @@ -185,13 +195,6 @@ describe('CollectiveX neutral run view', () => { cy.wait('@run'); cy.get('[data-testid="collectivex-mode-toggle"]').should('be.visible'); - cy.get('[data-testid="collectivex-mode-toggle"] button') - .contains('Normal') - .should('have.attr', 'aria-pressed', 'true'); - cy.get('[data-testid="collectivex-main-chart"]').should('contain.text', 'deepep-v2'); - cy.get('[data-testid="collectivex-explorer-chart"] .line-path').should('have.length', 1); - - cy.get('[data-testid="collectivex-mode-toggle"]').contains('Low-latency').click(); cy.get('[data-testid="collectivex-mode-toggle"] button[aria-pressed="true"]').should( 'have.length', 2, @@ -214,6 +217,42 @@ describe('CollectiveX neutral run view', () => { cy.get('[data-testid="collectivex-explorer-chart"] .line-path').should('have.length', 2); }); + it('normalizes runner-pool suffixes in SKU controls and chart labels', () => { + const withRunnerPoolSkus = buildDataset({ + shards: [ + makeRawShard({ sku: 'b200-nscale' }), + makeRawShard({ sku: 'h100-dgxc', mode: 'low-latency' }), + ], + }); + installRuns([withRunnerPoolSkus]); + installRun(withRunnerPoolSkus); + cy.reload(); + cy.wait('@runs'); + cy.wait('@run'); + + cy.get('[data-testid="chart-legend"]') + .should('contain.text', 'B200') + .and('contain.text', 'H100') + .and('not.contain.text', 'B200-NSCALE') + .and('not.contain.text', 'H100-DGXC'); + cy.get(`[data-testid="collectivex-run-row-${withRunnerPoolSkus.run.run_id}"]`) + .should('contain.text', 'B200') + .and('contain.text', 'H100') + .and('not.contain.text', 'B200-NSCALE') + .and('not.contain.text', 'H100-DGXC'); + cy.get('[data-testid="collectivex-sku-select"]').click(); + cy.get('[role="option"]').then(($options) => { + const labels = [...$options].map((option) => option.textContent?.trim()); + expect(labels).to.include.members(['B200', 'H100']); + expect(labels).not.to.include.members(['B200-NSCALE', 'H100-DGXC']); + }); + cy.contains('[role="option"]', 'B200').click(); + cy.get('[data-testid="chart-legend"]') + .should('contain.text', 'B200') + .and('not.contain.text', 'H100'); + cy.get('[data-testid="collectivex-explorer-chart"] .line-path').should('have.length', 1); + }); + it('selects the available phase when a partial run only measured prefill', () => { const prefill = buildDataset({ shards: [makeRawShard({ phase: 'prefill' })] }); installRuns([prefill]); @@ -333,14 +372,51 @@ describe('CollectiveX neutral run view', () => { cy.get('[data-testid="collectivex-explorer-chart"] .line-path').should('have.length', 1); }); - it('keeps the chart on top and presents the matrix inventory', () => { + it('filters the run table by EP and KV suite without hiding mixed-suite runs', () => { + installRuns([kvOnlyDataset, kvDataset, dataset]); + installRun(kvOnlyDataset, 'kvOnlyRun'); + cy.reload(); + cy.wait('@runs'); + cy.wait('@kvOnlyRun'); + + cy.get('[data-testid="collectivex-suite-filter"] [role="tab"][aria-selected="true"]').should( + 'contain.text', + 'All', + ); + cy.get(`[data-testid="collectivex-run-row-${kvOnlyDataset.run.run_id}"]`).should('be.visible'); + cy.get(`[data-testid="collectivex-run-row-${kvDataset.run.run_id}"]`).should('be.visible'); + cy.get(`[data-testid="collectivex-run-row-${dataset.run.run_id}"]`).should('be.visible'); + + cy.get('[data-testid="collectivex-suite-filter"]').contains('[role="tab"]', 'EP').click(); + cy.get(`[data-testid="collectivex-run-row-${kvOnlyDataset.run.run_id}"]`).should('not.exist'); + cy.get(`[data-testid="collectivex-run-row-${kvDataset.run.run_id}"]`).should('be.visible'); + cy.get(`[data-testid="collectivex-run-row-${dataset.run.run_id}"]`).should('be.visible'); + + cy.get('[data-testid="collectivex-suite-filter"]').contains('[role="tab"]', 'KV').click(); + cy.get(`[data-testid="collectivex-run-row-${kvOnlyDataset.run.run_id}"]`).should('be.visible'); + cy.get(`[data-testid="collectivex-run-row-${kvDataset.run.run_id}"]`).should('be.visible'); + cy.get(`[data-testid="collectivex-run-row-${dataset.run.run_id}"]`).should('not.exist'); + }); + + it('does not render the matrix case inventory', () => { cy.get('[data-testid="collectivex-main-chart"]').should('be.visible'); - cy.get('[data-testid="collectivex-inventory"]') - .should('contain.text', 'Matrix case inventory') - .and('contain.text', `${dataset.coverage.length} cases`); - cy.get('[data-testid="collectivex-inventory-table"]') - .should('contain.text', 'H200-DGXC') - .and('contain.text', 'B300'); + cy.get('[data-testid="collectivex-inventory"]').should('not.exist'); + cy.get('[data-testid="collectivex-display"]').should( + 'not.contain.text', + 'Matrix case inventory', + ); + }); + + it('localizes the suite filter on the Chinese route', () => { + cy.visit('/zh/collectivex'); + cy.wait('@runs'); + cy.wait('@run'); + + cy.get('[data-testid="collectivex-suite-filter"]') + .should('have.attr', 'aria-label', '按测试套件筛选 CollectiveX 运行') + .and('contain.text', '全部') + .and('contain.text', 'EP') + .and('contain.text', 'KV'); }); }); @@ -581,6 +657,44 @@ describe('CollectiveX kv-transfer card', () => { }); }); + it('plots the fixed frontier axes and keeps multi-run lines aligned with the legend', () => { + installRuns([kvComparisonDataset, kvDataset]); + installRun(kvComparisonDataset); + installRun(kvDataset, 'comparisonKvRun'); + openCollectiveX(); + + cy.get(`[data-testid="collectivex-run-visible-${kvDataset.run.run_id}"]`).check(); + cy.wait('@comparisonKvRun'); + cy.get('[data-testid="collectivex-kv-xaxis-toggle"]').contains('button', 'Frontier').click(); + + cy.get('[data-testid="collectivex-kv-metric-toggle"]').should('not.exist'); + cy.get('[data-testid="collectivex-kv-frontier-chart"]') + .should('contain.text', 'Aggregate pull bandwidth at p50 (GB/s, log)') + .and('contain.text', 'Burst p95 latency per in-flight request (ms, log)'); + cy.get('[data-testid="collectivex-kv-frontier-chart"] .line-path') + .should('have.length', 2) + .then(($lines) => { + expect([...$lines].map((line) => line.getAttribute('stroke-dasharray'))).to.have.members([ + 'none', + '9 4', + ]); + }); + }); + + it('localizes the frontier control and chart copy on the Chinese page', () => { + installRuns([kvDataset]); + installRun(kvDataset); + cy.visit('/zh/collectivex'); + cy.wait('@runs'); + cy.wait('@run'); + + cy.get('[data-testid="collectivex-kv-xaxis-toggle"]').contains('button', '帕累托前沿').click(); + cy.get('[data-testid="collectivex-kv-frontier-chart"]') + .should('contain.text', 'p50 聚合 pull 带宽(GB/s,对数)') + .and('contain.text', '每个在途请求的突发 p95 延迟(ms,对数)') + .and('contain.text', '越靠右下越优'); + }); + it('renders no kv card and no KV suite badge for an EP-only run', () => { installRuns(); installRun(); diff --git a/packages/app/cypress/e2e/compare-per-dollar-table.cy.ts b/packages/app/cypress/e2e/compare-per-dollar-table.cy.ts index e21a0baa5..f1134c682 100644 --- a/packages/app/cypress/e2e/compare-per-dollar-table.cy.ts +++ b/packages/app/cypress/e2e/compare-per-dollar-table.cy.ts @@ -1,3 +1,5 @@ +import { TCO_SOURCE_TITLE, TCO_SOURCE_URL } from '@semianalysisai/inferencex-constants'; + describe('Compare-per-dollar slug page — slimmed table + cross-link', () => { before(() => { cy.window().then((win) => { @@ -59,6 +61,12 @@ describe('Compare-per-dollar slug page — slimmed table + cross-link', () => { cy.contains('Performance per Dollar').should('be.visible'); }); + it('attributes pricing to the shared TCO source', () => { + cy.get('[data-testid="compare-per-dollar-pricing"]') + .contains('a', TCO_SOURCE_TITLE) + .should('have.attr', 'href', TCO_SOURCE_URL); + }); + it('renders an indexable comparison PNG with descriptive alt text', () => { cy.get('[data-testid="compare-per-dollar-indexed-image"] img') .should('be.visible') @@ -73,6 +81,15 @@ describe('Compare-per-dollar slug page — slimmed table + cross-link', () => { }); }); +describe('Chinese compare-per-dollar table', () => { + it('localizes the per-dollar metric label', () => { + cy.visit('/zh/compare-per-dollar/deepseek-r1-gb200-vs-h100'); + cy.get('[data-testid="compare-interpolated-table"] tbody') + .should('contain.text', '每百万 token 美元成本') + .and('not.contain.text', 'Dollar per Million Tokens'); + }); +}); + describe('Compare slug page — cross-link to per-dollar view', () => { before(() => { cy.window().then((win) => { diff --git a/packages/app/cypress/e2e/compare-precision.cy.ts b/packages/app/cypress/e2e/compare-precision.cy.ts index 9b07b388d..411c62ced 100644 --- a/packages/app/cypress/e2e/compare-precision.cy.ts +++ b/packages/app/cypress/e2e/compare-precision.cy.ts @@ -5,14 +5,74 @@ describe('Compare precision index page', () => { }); }); - it('renders the /compare index with precision and spec-decode CTA links', () => { + it('leads the /compare index with AgentX results and keeps fixed-sequence tools', () => { cy.visit('/compare'); + cy.get('[data-testid="compare-agentx-primary"]').within(() => { + cy.get('h1').should('have.text', 'Compare Realistic Agentic Inference Perf'); + cy.get('[data-testid^="compare-agentx-model-"]').should('have.length', 6); + cy.get('[data-testid="compare-agentx-model-kimi-k3"]').should( + 'have.attr', + 'href', + '/inference/kimi-k3', + ); + cy.get('[data-testid="compare-agentx-overview-link"]') + .should('contain.text', 'Overview') + .and('have.attr', 'href', '/overview'); + cy.get('[data-testid="compare-agentx-dashboard-link"]') + .should('contain.text', 'Full dashboard') + .and('have.attr', 'href', '/inference/kimi-k3'); + cy.get('[data-testid="compare-agentx-methodology-link"]').should('not.exist'); + }); + cy.get('[data-testid="compare-model-catalog"]') + .should('contain.text', 'AgentX and 8K→1K results') + .and('contain.text', 'Each card identifies its scenario'); cy.get('[data-testid="compare-index-precision-link"]') .should('have.attr', 'href', '/compare-precision') .and('contain.text', 'Compare precisions'); cy.get('[data-testid="compare-index-spec-decode-link"]') .should('have.attr', 'href', '/compare-spec-decode') .and('contain.text', 'Compare speculative decoding'); + cy.get('#deepseek-v4 a[data-scenario="AgentX"]') + .first() + .should('contain.text', 'AgentX') + .and('have.attr', 'href') + .and('match', /\/agentic$/u); + cy.get('#deepseek-r1 a[data-scenario="8K→1K"]') + .first() + .should('contain.text', '8K→1K') + .and('have.attr', 'href') + .and('match', /\/8k-1k$/u); + // Vendor logos render beside each hardware label in the pair cards: + // NVIDIA uses the full-color green mark, AMD its monochrome brand mark. + cy.get('a[data-scenario] img[src="/logos/nvidia-color.svg"]').should('exist'); + cy.get('a[data-scenario] img[src="/logos/amd.svg"]').should('exist'); + }); + + it('ships the same AgentX-first hierarchy on the Simplified Chinese page', () => { + cy.visit('/zh/compare'); + cy.get('[data-testid="compare-agentx-primary"]').within(() => { + cy.get('h1').should('have.text', '真实智能体工作负载下的推理性能对比'); + cy.get('[data-testid^="compare-agentx-model-"]').should('have.length', 6); + cy.get('[data-testid="compare-agentx-model-deepseek-v4"]').should( + 'have.attr', + 'href', + '/zh/inference/deepseek-v4', + ); + cy.get('[data-testid="compare-agentx-overview-link"]') + .should('contain.text', '总览') + .and('have.attr', 'href', '/zh/overview'); + cy.get('[data-testid="compare-agentx-dashboard-link"]') + .should('contain.text', '查看完整仪表板') + .and('have.attr', 'href', '/zh/inference/kimi-k3'); + cy.get('[data-testid="compare-agentx-methodology-link"]').should('not.exist'); + }); + cy.get('[data-testid="compare-model-catalog"]').should('contain.text', 'AgentX 与 8K→1K 结果'); + cy.get('#deepseek-v4 a[data-scenario="AgentX"]') + .first() + .should('have.attr', 'href') + .and('match', /^\/zh\/compare\/.+\/agentic$/u); + cy.get('a[data-scenario] img[src="/logos/nvidia-color.svg"]').should('exist'); + cy.get('a[data-scenario] img[src="/logos/amd.svg"]').should('exist'); }); it('renders the /compare-per-dollar index with precision and spec-decode CTA links', () => { diff --git a/packages/app/cypress/e2e/compare-scenario-routes.cy.ts b/packages/app/cypress/e2e/compare-scenario-routes.cy.ts new file mode 100644 index 000000000..994210bd4 --- /dev/null +++ b/packages/app/cypress/e2e/compare-scenario-routes.cy.ts @@ -0,0 +1,121 @@ +/** + * The workload is addressable as a path segment — `/compare//agentic`, + * `/compare//8k-1k` — not only as `?i_seq=`. The bare slug keeps + * rendering the pair's default workload. + */ +const AGENTX_SLUG = 'deepseek-v4-b200-vs-h200'; +const FIXED_SEQ_SLUG = 'deepseek-r1-h100-vs-h200'; + +describe('Compare scenario routes', () => { + beforeEach(() => { + cy.window().then((win) => { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }); + }); + + it('serves the agentic workload at its own path with the AgentX explainer', () => { + cy.visit(`/compare/${AGENTX_SLUG}/agentic`); + + cy.get('[data-testid="compare-agentic-intro"]') + .should('contain.text', 'AgentX replays real coding-agent sessions') + .find('[data-testid="compare-agentic-intro-link"]') + .should('contain.text', 'Learn more about AgentX') + .and('have.attr', 'href', '/agentx'); + }); + + it('serves the fixed-sequence workload at its own path, without the explainer', () => { + cy.visit(`/compare/${AGENTX_SLUG}/8k-1k`); + + cy.get('[data-testid="compare-narrative"]').should('exist'); + cy.get('[data-testid="compare-agentic-intro"]').should('not.exist'); + }); + + it('lets the path segment override the pair default', () => { + // Which workload a bare slug defaults to is `pickPairDefaults`, unit-tested + // in compare-pair-defaults.test.ts against rows this fixture set does not + // carry. What belongs here is the routing contract: whatever the default + // is, a scenario segment replaces it. + cy.visit(`/compare/${AGENTX_SLUG}`); + cy.get('[data-testid="compare-narrative"]') + .invoke('text') + .then((bare) => { + cy.visit(`/compare/${AGENTX_SLUG}/8k-1k`); + cy.get('[data-testid="compare-narrative"]').should('contain.text', '8k/1k'); + cy.get('[data-testid="compare-narrative"]') + .invoke('text') + .should((pinned) => { + expect(pinned, 'segment changes the rendered workload').to.not.equal(bare); + }); + }); + }); + + it('shows no AgentX explainer for a model with no AgentX data', () => { + cy.visit(`/compare/${FIXED_SEQ_SLUG}`); + cy.get('[data-testid="compare-agentic-intro"]').should('not.exist'); + }); + + it('canonicalizes every scenario view onto the bare slug URL', () => { + // The segments are views of one comparison; without a shared canonical the + // default view would appear twice in the index. + cy.visit(`/compare/${AGENTX_SLUG}/8k-1k`); + cy.get('link[rel="canonical"]') + .should('have.attr', 'href') + .and('match', new RegExp(`/compare/${AGENTX_SLUG}$`, 'u')); + }); + + it('404s an unknown scenario segment instead of silently defaulting', () => { + cy.request({ url: `/compare/${AGENTX_SLUG}/bogus`, failOnStatusCode: false }) + .its('status') + .should('eq', 404); + }); + + it('keeps the segment when redirecting a non-canonical slug', () => { + cy.request({ + url: `/compare/deepseek-v4-h200-vs-b200/8k-1k`, + followRedirect: false, + failOnStatusCode: false, + }).then((res) => { + expect(res.status).to.eq(308); + expect(res.headers.location).to.eq(`/compare/${AGENTX_SLUG}/8k-1k`); + }); + }); + + it('carries the scenario routes across the other compare families', () => { + for (const base of ['/compare-per-dollar', '/zh/compare']) { + cy.request({ url: `${base}/${AGENTX_SLUG}/agentic`, failOnStatusCode: false }) + .its('status') + .should('eq', 200); + } + }); + + it('links catalog cards to scenario paths rather than query strings', () => { + for (const base of ['/compare', '/compare-per-dollar']) { + cy.visit(base); + cy.get('#deepseek-v4 a[data-scenario="AgentX"]') + .first() + .should('have.attr', 'href') + .and('match', new RegExp(`^${base}/.+/agentic$`, 'u')); + cy.get('#deepseek-r1 a[data-scenario="8K→1K"]') + .first() + .should('have.attr', 'href') + .and('match', new RegExp(`^${base}/.+/8k-1k$`, 'u')); + } + }); + + it('splits the per-dollar catalog by scenario the way /compare does', () => { + cy.visit('/compare-per-dollar'); + cy.contains('Models with AgentX data open long-context').should('be.visible'); + cy.get('a[data-scenario="AgentX"]').should('have.length.greaterThan', 0); + cy.get('a[data-scenario="8K→1K"]').should('have.length.greaterThan', 0); + }); + + it('ships the Chinese explainer on the Chinese scenario route', () => { + cy.visit(`/zh/compare/${AGENTX_SLUG}/agentic`); + + cy.get('[data-testid="compare-agentic-intro"]') + .should('contain.text', 'AgentX 回放真实的编码 agent 会话') + .find('[data-testid="compare-agentic-intro-link"]') + .should('contain.text', '进一步了解 AgentX') + .and('have.attr', 'href', '/zh/agentx'); + }); +}); diff --git a/packages/app/cypress/e2e/csv-export-overlay.cy.ts b/packages/app/cypress/e2e/csv-export-overlay.cy.ts index b43295c38..0d0034c9e 100644 --- a/packages/app/cypress/e2e/csv-export-overlay.cy.ts +++ b/packages/app/cypress/e2e/csv-export-overlay.cy.ts @@ -35,9 +35,10 @@ function readCapturedCsv(): Cypress.Chainable { describe('Inference CSV export with an unofficial-run overlay', () => { before(() => { interceptOverlayRun(); - // The agentic default mode is E2E Normalized Interactivity (which suppresses overlays and - // fetches derived metrics) — stub the fetch, then switch to the - // Interactivity mode this suite's overlay assertions rely on. + // Agentic charts default to Interactivity, where the overlay renders. The + // derived-metrics fetch now happens only under E2E Normalized Interactivity, + // so the stub below is a guard against a stray request rather than a + // dependency of this suite. interceptDerivedAgenticMetrics(); cy.visit(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90`, { onBeforeLoad(win) { @@ -47,12 +48,11 @@ describe('Inference CSV export with an unofficial-run overlay', () => { }, }); cy.wait('@unofficialRun'); - // Interactivity is nested under the Advanced menu on agentic charts. - cy.get('[data-testid="x-axis-mode-advanced"]').click(); - cy.get('[data-testid="x-axis-mode-interactivity"]').click(); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('have.attr', 'data-state', 'active') - .and('contain.text', 'Interactivity'); + // Every x-axis metric is a top-level tab on agentic charts, and Interactivity + // is the default — clicked anyway so the suite does not depend on that. + cy.get('[data-testid="x-axis-mode-interactivity"]') + .click() + .should('have.attr', 'data-state', 'active'); cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should('exist'); }); diff --git a/packages/app/cypress/e2e/custom-user-values.cy.ts b/packages/app/cypress/e2e/custom-user-values.cy.ts index a45db8f9c..e5b7a6c34 100644 --- a/packages/app/cypress/e2e/custom-user-values.cy.ts +++ b/packages/app/cypress/e2e/custom-user-values.cy.ts @@ -24,7 +24,7 @@ describe('Custom User Values', () => { describe('Custom Chip Costs', () => { it('renders the custom costs input section when custom cost metric is selected', () => { selectCustomCostMetric(); - cy.get('[data-testid="custom-costs-section"]').should('be.visible'); + cy.get('[data-testid="custom-costs-section"]').scrollIntoView().should('be.visible'); cy.get('[data-testid="custom-costs-section"]').should('contain.text', 'Custom Chip Costs'); }); @@ -98,7 +98,7 @@ describe('Custom User Values', () => { describe('Custom Chip Powers', () => { it('renders the custom powers input section when custom power metric is selected', () => { selectCustomPowerMetric(); - cy.get('[data-testid="custom-powers-section"]').should('be.visible'); + cy.get('[data-testid="custom-powers-section"]').scrollIntoView().should('be.visible'); cy.get('[data-testid="custom-powers-section"]').should('contain.text', 'Custom Chip Powers'); }); diff --git a/packages/app/cypress/e2e/datasets-distributions.cy.ts b/packages/app/cypress/e2e/datasets-distributions.cy.ts index 0d2a7789a..a5759ce90 100644 --- a/packages/app/cypress/e2e/datasets-distributions.cy.ts +++ b/packages/app/cypress/e2e/datasets-distributions.cy.ts @@ -19,6 +19,20 @@ const distribution = (values: { }, }); +const LAYOUT_DATASET = { + id: 'layout-dataset', + slug: 'layout-dataset', + label: 'Layout dataset', + variant: 'full', + description: null, + hf_url: null, + license: 'apache-2.0', + conversation_count: 0, + summary: {}, + chart_data: {}, + ingested_at: '2026-06-23T00:00:00Z', +}; + describe('Dataset distribution percentiles', () => { before(() => { cy.intercept('GET', '/api/v1/datasets/test-dataset', { @@ -87,7 +101,7 @@ describe('Dataset distribution percentiles', () => { cy.intercept('GET', '/api/v1/datasets/test-dataset/conversations*', { body: { total: 0, items: [] }, }); - cy.visit('/datasets/test-dataset', { onBeforeLoad: unlockAgenticGate }); + cy.visit('/agentx/test-dataset', { onBeforeLoad: unlockAgenticGate }); }); it('shows P50/P75/P90/P95 for ISL, OSL, and uncached input', () => { @@ -133,3 +147,28 @@ describe('Dataset distribution percentiles', () => { } }); }); + +describe('Dataset detail loading stability', () => { + for (const path of ['/agentx/layout-dataset', '/zh/agentx/layout-dataset']) { + it(`keeps the footer below the viewport while ${path} loads`, () => { + cy.intercept('GET', '/api/v1/datasets/layout-dataset', { + delay: 1500, + body: LAYOUT_DATASET, + }).as('dataset'); + cy.intercept('GET', '/api/v1/datasets/layout-dataset/conversations*', { + body: { total: 0, items: [] }, + }); + + cy.visit(path, { onBeforeLoad: unlockAgenticGate }); + cy.get('[data-testid="dataset-detail-loading"]').should('be.visible'); + cy.get('[data-testid="footer"]').then(($footer) => { + cy.window().then((win) => { + expect($footer[0].getBoundingClientRect().top).to.be.at.least(win.innerHeight); + }); + }); + + cy.wait('@dataset'); + cy.contains('h1', 'Layout dataset').should('be.visible'); + }); + } +}); diff --git a/packages/app/cypress/e2e/datasets-flamegraph-time.cy.ts b/packages/app/cypress/e2e/datasets-flamegraph-time.cy.ts index bdb1adfc0..5a4e8df22 100644 --- a/packages/app/cypress/e2e/datasets-flamegraph-time.cy.ts +++ b/packages/app/cypress/e2e/datasets-flamegraph-time.cy.ts @@ -95,7 +95,7 @@ describe('Dataset conversation flamegraph timing', () => { }, }, }); - cy.visit('/datasets/test-dataset/conversations/conversation-1', { + cy.visit('/agentx/test-dataset/conversations/conversation-1', { onBeforeLoad: unlockAgenticGate, }); }); diff --git a/packages/app/cypress/e2e/datasets-methodology.cy.ts b/packages/app/cypress/e2e/datasets-methodology.cy.ts new file mode 100644 index 000000000..e9fccbb20 --- /dev/null +++ b/packages/app/cypress/e2e/datasets-methodology.cy.ts @@ -0,0 +1,165 @@ +import { unlockAgenticGate } from '../support/e2e'; + +const DATASET = { + id: 'agentx-v1-full', + slug: 'agentx-v1-full', + label: 'AgentX v1.0 (full)', + variant: 'full', + description: 'The complete AgentX v1.0 replay set.', + hf_url: null, + license: 'apache-2.0', + conversation_count: 393, + summary: { + medianRequestsPerConversation: 20, + meanRequestsPerConversation: 24, + mainTurns: 7800, + subagentGroups: 1300, + cachedPct: 0.84, + totalIn: 56_000_000, + totalOut: 175_000, + }, + ingested_at: '2026-06-21T00:00:00Z', +}; + +describe('AgentX dataset methodology', () => { + beforeEach(() => { + cy.intercept('GET', '/api/v1/datasets', { statusCode: 200, body: [DATASET] }); + }); + + it('explains the source, replay sequence, controls, and interpretation in English', () => { + cy.visit('/agentx', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-methodology"]').within(() => { + cy.get('h1').should('have.text', 'AgentX Benchmark Datasets'); + cy.get('[data-testid="agentx-methodology-step"]').should('have.length', 4); + cy.get('[data-testid="agentx-methodology-step"] h3').then(($headings) => { + expect([...$headings].map((heading) => heading.textContent)).to.deep.equal([ + 'Capture', + 'Transform', + 'Reconstruct', + 'Replay and measure', + ]); + }); + cy.contains('393 Claude Code sessions').should('be.visible'); + cy.contains('Claude Code 2.1.139 or newer').should('be.visible'); + cy.contains('SPEED-Bench').should('be.visible'); + cy.contains('Its synthetic payloads do not support model-quality evaluation.').should( + 'be.visible', + ); + cy.get('[data-testid="agentx-methodology-cta"]') + .should('contain.text', 'Read the full methodology') + .and('have.attr', 'href', '/agentx/methodology'); + cy.get('[data-testid="agentx-results-cta"]') + .should('contain.text', 'View AgentX Performance Results') + .and('have.attr', 'href', '/inference'); + }); + }); + + it('ships the same methodology on the Simplified Chinese page', () => { + cy.visit('/zh/agentx', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-methodology"]').within(() => { + cy.get('h1').should('have.text', 'AgentX 基准测试数据集'); + cy.get('[data-testid="agentx-methodology-step"]').should('have.length', 4); + cy.contains('AgentX 如何构建一次回放').should('be.visible'); + cy.contains('回放控制').should('be.visible'); + cy.contains('参与者自愿提供的 Claude Code 会话').should('be.visible'); + cy.contains('会话数').should('be.visible'); + cy.contains('单请求 input token 数中位数').should('be.visible'); + cy.contains('单请求 output token 数中位数').should('be.visible'); + cy.contains('对外报告的指标仅覆盖一小时 profiling 窗口').should('be.visible'); + cy.contains('合成 payload 不能用于模型质量评估').should('be.visible'); + cy.contains('对于没有标准 DRAM 配置的服务器,可用 DRAM 上限为 3 TB').should('be.visible'); + cy.get('[data-testid="agentx-methodology-cta"]') + .should('contain.text', '查看完整测试方法') + .and('have.attr', 'href', '/zh/agentx/methodology'); + cy.get('[data-testid="agentx-results-cta"]') + .should('contain.text', '查看 AgentX 性能结果') + .and('have.attr', 'href', '/zh/inference'); + }); + + cy.get('a[href="/zh/agentx/agentx-v1-full"]').within(() => { + cy.contains('会话数').should('be.visible'); + cy.contains('单会话请求数中位数').should('be.visible'); + cy.contains('单会话平均请求数').should('be.visible'); + cy.contains('main agent 轮次').should('be.visible'); + cy.contains('subagent 组').should('be.visible'); + cy.contains('cached input 占比').should('be.visible'); + cy.contains('input token 总数').should('be.visible'); + cy.contains('output token 总数').should('be.visible'); + }); + }); + + it('publishes the full English methodology with sourced figures and locale pairing', () => { + cy.visit('/agentx/methodology', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-methodology-article"]').within(() => { + cy.get('h1').should('have.text', 'AgentX Methodology'); + cy.contains('393 sessions built on June 21, 2026').should('be.visible'); + cy.contains('directed acyclic graph (DAG)').should('be.visible'); + cy.contains('following one-hour profiling window').should('be.visible'); + cy.contains('capped at 3 TB').should('be.visible'); + cy.get('figure[data-testid^="agentx-methodology-figure-"]').should('have.length', 21); + cy.get('[data-testid="agentx-methodology-figure-corpus"]') + .should('contain.text', 'View full-resolution image') + .parent('a') + .should('have.attr', 'href', '/images/agentx-methodology/corpus-scale.png') + .and('have.attr', 'target', '_blank'); + cy.get('[data-testid="agentx-methodology-figure-corpus"] img') + .invoke('attr', 'src') + .should('include', 'q=100'); + [ + 'requestDistributions256k', + 'subagentDistributions256k', + 'replayJoined', + 'replayFlatspawn', + 'replaySidecars', + ].forEach((figure) => { + cy.get(`[data-testid="agentx-methodology-figure-${figure}"] img`) + .should('be.visible') + .invoke('attr', 'alt') + .should('not.be.empty'); + }); + cy.get('a[href="https://arxiv.org/abs/2604.09557"]').should('exist'); + cy.contains('mostly vibe coded').should('not.exist'); + cy.contains('Distillation is bad').should('not.exist'); + }); + + cy.get('[data-testid="language-toggle"]').should('have.attr', 'href', '/zh/agentx/methodology'); + cy.get('link[rel="alternate"][hreflang="en"]').should('exist'); + cy.get('link[rel="alternate"][hreflang="zh-CN"]').should('exist'); + }); + + it('publishes the natural Simplified Chinese methodology sibling', () => { + cy.visit('/zh/agentx/methodology', { onBeforeLoad: unlockAgenticGate }); + + cy.get('[data-testid="agentx-methodology-article"]').within(() => { + cy.get('h1').should('have.text', 'AgentX 方法论'); + cy.contains('2026 年 6 月 21 日构建').should('be.visible'); + cy.contains('有向无环图(DAG)').should('be.visible'); + cy.contains('上限为 3 TB').should('be.visible'); + cy.get('figure[data-testid^="agentx-methodology-figure-"]').should('have.length', 21); + cy.get('[data-testid="agentx-methodology-figure-corpus"]') + .should('contain.text', '查看原始分辨率图片') + .parent('a') + .should('have.attr', 'href', '/images/agentx-methodology/corpus-scale.png'); + }); + + cy.get('[data-testid="language-toggle"]').should('have.attr', 'href', '/agentx/methodology'); + }); + + it('permanently redirects legacy dataset routes without dropping path or query', () => { + cy.request({ + url: '/datasets/test-set/conversations/abc?turn=3', + followRedirect: false, + }).then((response) => { + expect(response.status).to.eq(308); + expect(response.headers.location).to.eq('/agentx/test-set/conversations/abc?turn=3'); + }); + + cy.request({ url: '/zh/datasets', followRedirect: false }).then((response) => { + expect(response.status).to.eq(308); + expect(response.headers.location).to.eq('/zh/agentx'); + }); + }); +}); diff --git a/packages/app/cypress/e2e/dropdown-switching.cy.ts b/packages/app/cypress/e2e/dropdown-switching.cy.ts index 4f6f32058..1af072681 100644 --- a/packages/app/cypress/e2e/dropdown-switching.cy.ts +++ b/packages/app/cypress/e2e/dropdown-switching.cy.ts @@ -25,6 +25,11 @@ describe('Dropdown one-click switching', () => { }); it('only one MultiSelect content panel is open at a time when switching dropdowns', () => { + // The default model is FP4-only in the fixtures, which hides the Precision + // control — switch to a multi-precision model so both dropdowns exist. + cy.visit('/inference?g_model=DeepSeek-R1-0528'); + cy.get('[data-testid="inference-chart-display"]').should('exist'); + cy.get('[data-testid="model-selector"]').click(); cy.get('[data-slot="select-content"]').should('have.length', 1); @@ -44,6 +49,21 @@ describe('Dropdown one-click switching', () => { cy.get('[data-slot="select-content"]').should('not.exist'); }); + it('marks the featured AgentX models with a NEW pill in the dropdown', () => { + cy.get('[data-testid="model-selector"]').click(); + + // A featured AgentX model carries the pill… (MiniMax M3 rather than the + // Kimi K3 default because the availability fixtures don't ship kimik3 rows) + cy.contains('[role="option"]', 'MiniMax M3 428B') + .find('[data-new-badge="model-option"]') + .should('be.visible') + .and('have.text', 'NEW'); + // …while non-featured models render without one. + cy.contains('[role="option"]', 'DeepSeek R1 0528 671B') + .find('[data-new-badge="model-option"]') + .should('not.exist'); + }); + it('separates maintenance-mode models from deprecated models', () => { cy.get('[data-testid="model-selector"]').click(); diff --git a/packages/app/cypress/e2e/fixed-sequence-logs.cy.ts b/packages/app/cypress/e2e/fixed-sequence-logs.cy.ts new file mode 100644 index 000000000..590505536 --- /dev/null +++ b/packages/app/cypress/e2e/fixed-sequence-logs.cy.ts @@ -0,0 +1,55 @@ +import { unlockAgenticGate } from '../support/e2e'; + +describe('Fixed-sequence benchmark logs', () => { + beforeEach(() => { + cy.intercept('GET', '/api/v1/server-log-files*', { + body: ['watchtower-teal-cn03_decode_w0.out', 'results/benchmark.log'], + }).as('logFiles'); + cy.intercept({ method: 'GET', pathname: '/api/v1/server-log' }, (request) => { + const params = new URL(request.url).searchParams; + const fileName = params.get('file') ?? 'watchtower-teal-cn03_decode_w0.out'; + request.reply({ + body: { + id: 96255, + fileName, + serverLog: fileName.endsWith('.out') + ? 'INFO decode worker ready\n' + : 'benchmark complete\n', + offset: 0, + nextOffset: null, + }, + }); + }).as('logContent'); + }); + + it('opens all stored .out and .log files on the fixed-sequence detail route', () => { + cy.visit('/inference/logs/96255', { onBeforeLoad: unlockAgenticGate }); + cy.wait('@logFiles'); + cy.wait('@logContent'); + + cy.contains('h1', 'Benchmark logs').should('be.visible'); + cy.get('[data-testid="agentic-server-log-viewer"]') + .should('have.attr', 'data-log-context', 'fixed-sequence') + .and('contain.text', 'watchtower-teal-cn03_decode_w0.out'); + cy.get('[data-testid="server-log-content"]').should('contain.text', 'INFO decode worker ready'); + + cy.get('#agentic-log-file').click(); + cy.contains('[role="option"]', 'results/benchmark.log').click(); + cy.get('[data-testid="server-log-content"]').should('contain.text', 'benchmark complete'); + cy.get('[data-testid="download-selected-server-log"]') + .should('have.attr', 'href') + .and('include', 'id=96255') + .and('include', 'download=1'); + }); + + it('provides the same viewer on the Simplified Chinese route', () => { + cy.visit('/zh/inference/logs/96255', { onBeforeLoad: unlockAgenticGate }); + cy.wait('@logFiles'); + cy.wait('@logContent'); + + cy.contains('h1', '基准测试日志').should('be.visible'); + cy.get('[data-testid="agentic-server-log-viewer"]') + .should('contain.text', '日志文件') + .and('contain.text', '搜索所有日志文件'); + }); +}); diff --git a/packages/app/cypress/e2e/fleet-lifecycle.cy.ts b/packages/app/cypress/e2e/fleet-lifecycle.cy.ts new file mode 100644 index 000000000..30635edc2 --- /dev/null +++ b/packages/app/cypress/e2e/fleet-lifecycle.cy.ts @@ -0,0 +1,995 @@ +import { + availability as agenticAvailability, + b300Rows as agenticB300Rows, +} from '../support/overlay-fixtures'; +import { unlockAgenticGate } from '../support/e2e'; + +type CsvCaptureWindow = Cypress.AUTWindow & { + __capturedLifecycleCsvBlob?: Blob; +}; + +function captureLifecycleCsvDownloads(win: Cypress.AUTWindow): void { + const captureWindow = win as CsvCaptureWindow; + win.URL.createObjectURL = (object: Blob | MediaSource) => { + if (object instanceof win.Blob) captureWindow.__capturedLifecycleCsvBlob = object; + return 'blob:calculator-lifecycle-csv-test'; + }; + win.HTMLAnchorElement.prototype.click = () => {}; +} + +function readCapturedLifecycleCsv(): Cypress.Chainable { + return cy.window().then((win) => { + const blob = (win as CsvCaptureWindow).__capturedLifecycleCsvBlob; + expect(blob, 'captured lifecycle CSV Blob').to.be.instanceOf(win.Blob); + return blob!.text(); + }); +} + +// Fleet Lifecycle section on its own /fleet page. The distinguishing behaviours, all +// worth locking down: +// - it reads the FULL run history, so a row's operating point can come from an +// earlier date than the caption's run date — and must say which; +// - clamped (never-measured-at-target) reads are excluded rather than clamped, +// so chips can legitimately be absent and must be explained, not dropped; +// - the token price defaults to the cheapest visible chip's break-even; +// - agentic traces have no ISL/OSL, so history is keyed on benchmark_type +// instead, and their cached input tokens bill at a discount. + +/** + * This figure's export menu button. Scoped: the bar chart above has its own, and + * `export-button` is not unique across the page. + */ +const exportMenu = () => + cy.get('[data-testid="calculator-lifecycle-figure"]').find('[data-testid="export-button"]'); + +/** Switch the section to the table tab. Idempotent — clicking the active tab is a no-op. */ +const showTable = () => cy.get('[data-testid="calculator-lifecycle-table-view-btn"]').click(); + +/** Switch the section back to the chart tab. */ +const showChart = () => cy.get('[data-testid="calculator-lifecycle-chart-view-btn"]').click(); + +/** + * Put the section back on the chart tab if it has one rendered. + * + * `testIsolation` is false, so all of these tests share one page and one tab + * state. Without this, a test that reads the table leaves the next one looking + * for a chart that is not mounted — an ordering coupling that fails in whichever + * order the file happens to be written in. Tolerant of the tab not existing yet: + * when the power budget is cleared there is no figure at all. + */ +const resetToChart = () => + cy.get('body').then(($body) => { + if ($body.find('[data-testid="calculator-lifecycle-chart-view-btn"]').length > 0) showChart(); + }); + +/** + * A named cell of the first lifecycle table row. Switches to the table tab first: + * the section opens on the chart, and these assertions are about the numbers. + * + * Resolved by header text rather than by position on purpose — these were once + * bare indices, and inserting a column silently re-pointed nine assertions at + * their neighbours, each of which then failed on a value that was correct. + */ +const firstRowCell = (header: string) => { + showTable(); + return cy.get('[data-testid="calculator-lifecycle-table"] thead th').then(($ths) => { + const index = $ths.toArray().findIndex((th) => (th.textContent ?? '').includes(header)); + expect(index, `a column headed "${header}"`).to.be.greaterThan(-1); + return cy + .get('[data-testid="calculator-lifecycle-table"] tbody tr') + .first() + .find('td') + .eq(index); + }); +}; + +/** The time-axis tick labels, as one string — changes when the x domain moves. */ +const xAxisTicks = () => + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .x-axis .tick text').invoke('text'); + +/** The rect that owns hover for the whole plot area. */ +const plotOverlay = () => + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .proximity-overlay'); + +/** Move the cursor to a fraction of the way across the plot. */ +const hoverPlot = (fraction: number) => + plotOverlay().then(($rect) => { + const width = $rect[0]!.getBoundingClientRect().width; + plotOverlay().trigger('mousemove', width * fraction, 40); + }); + +/** Click at a fraction of the way across the plot. */ +const clickPlot = (fraction: number) => + plotOverlay().then(($rect) => { + const width = $rect[0]!.getBoundingClientRect().width; + plotOverlay().click(width * fraction, 40); + }); + +/** + * Plot x of the last step dot. Steps are matched to hover positions by proximity, + * so this is the x a reader points at to ask about a measured config. + */ +const stepX = () => + cy + .get('[data-testid="calculator-lifecycle-chart-svg"] .dot-group') + .last() + .invoke('attr', 'transform') + .then((transform) => { + const x = Number(/translate\((?[-\d.]+)/u.exec(String(transform))?.groups?.x); + expect(x, 'a numeric dot x').to.be.a('number'); + return x; + }); + +/** This chart's tooltip: a portal on , keyed by chart id. */ +const readout = () => cy.get('[data-chart-tooltip="fleet-lifecycle"]'); + +/** + * A table money cell as a number. Cells are compact ($1.2M, -$430k), so comparing + * their text lexically is meaningless — these have to be parsed to be compared. + */ +const money = (text: string): number => { + const match = /^(?-?)\$(?[\d.]+)(?[kMB])?/u.exec(text.trim()); + if (!match?.groups) throw new Error(`not a money cell: ${text}`); + const scale = { k: 1e3, M: 1e6, B: 1e9 }[match.groups.suffix ?? ''] ?? 1; + return Number(match.groups.value) * scale * (match.groups.sign ? -1 : 1); +}; + +/** All text in the chart SVG, including the axis labels. */ +const chartText = () => cy.get('[data-testid="calculator-lifecycle-chart-svg"]').invoke('text'); + +/** Vertex count of the first plotted line — a sampled rollout curve has many. */ +const lineVertices = () => + cy + .get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path') + .first() + .invoke('attr', 'd') + .then((d) => (d ?? '').split('L').length); + +describe('Fleet — Fleet Lifecycle', () => { + before(() => { + cy.window().then((win) => { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + // The reproducibility nudge is on a 1.5s timer, and showing a toast + // re-renders the chart tree — which rewrites the portal tooltip's inline + // styles and wipes an open hover readout. That is app-wide behaviour, not + // something this section can fix, so suppress the nudge rather than race it. + win.sessionStorage.setItem('inferencex-reproducibility-nudge-shown', '1'); + }); + cy.visit('/fleet'); + // Readiness: the lifecycle section only mounts once run data has loaded. + cy.get('[data-testid="calculator-lifecycle-section"]').should('exist'); + }); + + beforeEach(resetToChart); + + it('defaults the power budget to 10 MW and renders the lifecycle', () => { + cy.get('[data-testid="calculator-lifecycle-section"]') + .should('be.visible') + .and('contain.text', 'Fleet Lifecycle'); + cy.get('[data-testid="calc-fleet-mw-input"]').should('have.value', '10'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-empty"]').should('not.exist'); + }); + + it('the default MW budget drives the lifecycle table', () => { + // The figure opens on the chart; the table is the other tab. + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-table"]').should('not.exist'); + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-table"]').within(() => { + cy.contains('th', 'Config Now').should('exist'); + cy.contains('th', 'First Run').should('exist'); + cy.contains('th', 'Latest Best').should('exist'); + cy.contains('th', 'Improvements').should('exist'); + // Absorbed from the old Fleet Projection table: the physical sizing the + // economics rest on, on the same row as the economics. + cy.contains('th', 'Chips').should('exist'); + cy.contains('th', 'Concurrent Users now').should('exist'); + cy.contains('th', 'Margin $/day').should('exist'); + cy.contains('th', 'Payback').should('exist'); + cy.contains('th', 'Cumulative Margin').should('exist'); + cy.get('tbody tr').should('have.length.greaterThan', 0); + }); + cy.get('[data-testid="calculator-lifecycle-empty"]').should('not.exist'); + // No search box over one row per chip — every one of them is named in the + // legend and on the chart already. + cy.get('[data-testid="calculator-lifecycle-table"] input[type="text"]').should('not.exist'); + // No cached-input assumption on a fixed sequence: those runs record no cache + // hits at all, so the control would be a knob that moves nothing. + cy.get('[data-testid="calc-lifecycle-cache-input"]').should('not.exist'); + // Both prices are seeded as a pair, output at 4x input — roughly where the + // major vendors price. Asserted here, on a page nothing has edited yet: the + // spec shares one page across tests, so a later assertion would be reading + // whatever ratio an earlier test happened to leave behind. + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((input) => { + expect(Number(input)).to.be.greaterThan(0); + cy.get('[data-testid="calc-lifecycle-output-price-input"]') + .invoke('val') + .should((output) => { + expect(Number(output)).to.be.closeTo(Number(input) * 4, Number(input) * 0.01); + }); + }); + }); + + it('quotes both token prices in the caption, and tracks the fields', () => { + // Every margin plotted is linear in these two, and on the margin metrics they + // are seeded from break-even rather than typed — so without them in the + // caption two screenshots of different scenarios are indistinguishable. + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('12'); + cy.get('[data-testid="calc-lifecycle-output-price-input"]').clear().type('48'); + // A whole-dollar price reads as `$12`, not `$12.0000` — the fields pad for + // typing, the caption is prose. Cents survive; trailing zeros do not. + cy.get('[data-testid="calculator-lifecycle-figure"]').should('not.contain', '$12.0000'); + cy.get('[data-testid="calculator-lifecycle-figure"]').within(() => { + cy.contains('$12 in / $48 out per M tok').should('be.visible'); + // Beside the MW figure, not instead of it. + cy.contains('MW').should('be.visible'); + }); + + // It is the figure's caption, so it survives the tab switch with the rest. + showTable(); + cy.get('[data-testid="calculator-lifecycle-figure"]').should( + 'contain', + '$12 in / $48 out per M tok', + ); + showChart(); + + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('7.25'); + cy.get('[data-testid="calculator-lifecycle-figure"]').within(() => { + cy.contains('$7.25 in / $48 out per M tok').should('be.visible'); + }); + + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('7'); + cy.get('[data-testid="calculator-lifecycle-figure"]').within(() => { + cy.contains('$7 in / $48 out per M tok').should('be.visible'); + cy.contains('$12 in').should('not.exist'); + }); + + // `testIsolation: false`, so hand the prices back to break-even before + // leaving — later tests read the seeded values and a stale 7 makes them fail + // for a reason that has nothing to do with what they assert. The caption + // following the reset is the last thing worth checking anyway. + cy.get('[data-testid="calc-lifecycle-price-reset"]').click(); + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((seeded) => { + // The field pads to four decimals; the caption drops trailing zeros, so + // the expectation has to apply the same trim rather than echo the field. + const trimmed = String(seeded).includes('.') + ? String(seeded).replace(/\.?0+$/u, '') + : String(seeded); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('contain', `$${trimmed} in /`); + }); + }); + + it('puts the chart and the table on their own tabs, with a header on both', () => { + // The header is the figure's own caption, so it survives the switch — a reader + // on the table tab still sees what model, scenario and target they are reading. + cy.get('[data-testid="calculator-lifecycle-figure"]').within(() => { + cy.contains('h2', 'Fleet Lifecycle').should('be.visible'); + cy.contains('tok/s/user').should('be.visible'); + }); + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-table"]').should('not.exist'); + + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').should('not.exist'); + cy.get('[data-testid="calculator-lifecycle-figure"]').within(() => { + cy.contains('h2', 'Fleet Lifecycle').should('be.visible'); + }); + + showChart(); + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').should('be.visible'); + }); + + it('offers a download, and disables the PNG where a PNG is meaningless', () => { + exportMenu().should('be.visible').click(); + cy.get('[data-testid="export-csv-button"]').should('be.visible'); + // On the chart a PNG is a real artefact. + cy.get('[data-testid="export-png-button"]').should('not.have.attr', 'aria-disabled', 'true'); + cy.get('body').type('{esc}'); + + showTable(); + exportMenu().click(); + // A PNG of a paginated HTML table is not; the CSV is the export for that view. + cy.get('[data-testid="export-png-button"]').should('have.attr', 'aria-disabled', 'true'); + cy.get('[data-testid="export-csv-button"]').should('be.visible'); + cy.get('body').type('{esc}'); + }); + + it('every row is traceable to the dated run it came from', () => { + // The caption's run date no longer describes these numbers, so each row + // carries its own date and links its run. + firstRowCell('First Run') + .invoke('text') + .should('match', /\d{4}-\d{2}-\d{2}/u); + firstRowCell('Latest Best') + .invoke('text') + .should('match', /\d{4}-\d{2}-\d{2}/u); + }); + + it('defaults the token price to a break-even figure', () => { + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .should('match', /^\d+(?:\.\d+)?$/u); + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((val) => expect(Number(val)).to.be.greaterThan(0)); + // Break-even means the cheapest chip earns nothing: some row is at/near zero + // margin, and none of the visible fleet is comfortably profitable there. + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('contain.text', 'Never'); + }); + + it('raising the price above break-even turns margin positive and produces a payback', () => { + cy.get('[data-testid="calc-lifecycle-price-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-price-input"]').type('50'); + // Margin $/day column: positive, so no leading minus. + firstRowCell('Margin $/day').invoke('text').should('match', /^\$/u); + firstRowCell('Payback') + .invoke('text') + .should('match', /\d+(?:\.\d+)? mo/u); + }); + + it('editing the price offers a reset back to break-even', () => { + cy.get('[data-testid="calc-lifecycle-price-reset"]').should('be.visible').click(); + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((val) => expect(Number(val)).to.be.lessThan(50)); + }); + + it('bills the two streams apart — the output price alone moves margin', () => { + // On 8k/1k output is a ninth of the tokens, so a blended single price could + // not respond to this at all. + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('10'); + cy.get('[data-testid="calc-lifecycle-output-price-input"]').clear().type('10'); + firstRowCell('Margin $/day') + .invoke('text') + .then((before) => { + cy.get('[data-testid="calc-lifecycle-output-price-input"]').clear().type('500'); + firstRowCell('Margin $/day') + .invoke('text') + .should((after) => { + expect(money(after)).to.be.greaterThan(money(before)); + }); + }); + }); + + it('resets both prices together, keeping the ratio the user set', () => { + // Break-even with two prices is a line, so the reset has to pick a point on + // it: the one that preserves the ratio currently in the fields. + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('10'); + cy.get('[data-testid="calc-lifecycle-output-price-input"]').clear().type('30'); + cy.get('[data-testid="calc-lifecycle-price-reset"]').click(); + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((input) => { + expect(Number(input)).to.be.greaterThan(0); + cy.get('[data-testid="calc-lifecycle-output-price-input"]') + .invoke('val') + .should((output) => { + // 3x in, 3x out — not the 4x default, and not one field left behind. + expect(Number(output)).to.be.closeTo(Number(input) * 3, Number(input) * 0.01); + }); + }); + }); + + it('renders the lifecycle chart with one line per visible chip', () => { + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path').should( + 'have.length.greaterThan', + 0, + ); + // The break-even rule is what makes the sign of the margin readable. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should('exist'); + }); + + it('names each line with its chip at the right edge', () => { + // One label per line, so a series is identifiable without the legend. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path').then((lines) => { + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-series-label').should( + 'have.length', + lines.length, + ); + }); + // The label text is a chip name, matching the table's first column. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-series-label') + .first() + .invoke('text') + .should('match', /\S/u) + .then((label) => { + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('contain', String(label)); + }); + }); + + it('reads every plotted chip at the hovered date, not just one point', () => { + // Hovering anywhere in the plot has to answer "what is each chip doing here?". + // Per-point tooltips could not: comparing two chips at one date meant hovering + // twice and holding the first number in your head. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path').then((lines) => { + // Late in the window, where every chip has been measured — a chip first + // measured mid-window has no line, and so no row, before that. + hoverPlot(0.9); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .ruler-group').should( + 'have.css', + 'display', + 'block', + ); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .vertical-ruler').should('exist'); + // One row per line, each with a money figure, under the hovered date. + // Not `:visible` — Cypress treats a fixed-position element as hidden when + // something else answers elementFromPoint at its centre, and an unfrozen + // readout sets pointer-events: none, so the plot answers instead. + readout() + .should('have.css', 'display', 'block') + .and('contain.text', '$') + .invoke('text') + .should('match', /\d{2} \w{3} \d{4}/u); + readout().find('tbody tr').should('have.length', lines.length); + }); + // Earlier than the newest chip's first measurement, that chip has no number + // to give, and the readout says nothing rather than inventing one. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path').then((lines) => { + hoverPlot(0.05); + readout() + .find('tbody tr') + .should('have.length.greaterThan', 0) + .and('have.length.lessThan', lines.length); + }); + }); + + it('freezes the readout on click, and releases it on the next click', () => { + hoverPlot(0.4); + readout() + .invoke('text') + .then((hovered) => { + // The date is what identifies a readout; the rest of the text legitimately + // changes on freeze, since freezing adds any step's config detail. + const frozenDate = /\d{2} \w{3} \d{4}/u.exec(hovered)?.[0]; + expect(frozenDate, 'a date in the hover readout').to.be.a('string'); + clickPlot(0.4); + readout() + .should('have.css', 'pointer-events', 'auto') + .and('contain.text', String(frozenDate)); + // Frozen means frozen: moving the cursor elsewhere must not re-read. + hoverPlot(0.7); + readout().should('contain.text', String(frozenDate)); + // The same gesture releases it, so the reader is never stuck with a + // stale readout pinned over the lines. + clickPlot(0.7); + readout().should('have.css', 'display', 'none'); + }); + }); + + it('keeps config detail out of the hover readout, showing it only once frozen', () => { + // Hovering is for scanning: the popup keeps one shape wherever the cursor is. + // A step block appearing as the cursor crosses a dot reflows the rows under + // the reader and buries the comparison they came for. + stepX().then((x) => { + plotOverlay().trigger('mousemove', x, 40); + readout().should('have.css', 'display', 'block').and('not.contain.text', 'Config'); + plotOverlay().click(x, 40); + readout().should('contain.text', 'Config'); + // Leave nothing frozen for the next test. + plotOverlay().click(x, 40); + }); + }); + + it('links the run behind any step, not just the first and last', () => { + // The table links only the opening and closing sweeps. Intermediate rungs are + // exactly where an anomalous run that was never purged would sit, so freezing + // one has to expose its run — otherwise that rung is auditable nowhere. + // The hover grid never lands exactly on a step's instant, so steps match by + // proximity; this asserts that freezing works from a dot's own x. + stepX().then((x) => { + plotOverlay().click(x, 40); + }); + readout() + .find('a') + .should('have.length.greaterThan', 0) + .first() + .should('have.attr', 'href') + .and('match', /^https?:\/\//u); + // The pin is left in place: the next test's zoom dismisses it, and the + // tooltip is a portal outside the SVG so it does not disturb what follows. + }); + + it('zooms the time axis, as its own instructions promise', () => { + xAxisTicks().then((before) => { + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').trigger('wheel', { + deltaY: -400, + shiftKey: true, + clientX: 700, + clientY: 300, + bubbles: true, + }); + // The axis has to actually move: the chart advertises shift+scroll zoom in + // its caption, and for a while it advertised it without wiring it up. + xAxisTicks().should('not.equal', before); + // Double-click resets, as the same caption promises. Also leaves the chart + // unzoomed for the tests that follow — they share this page. + cy.get('[data-testid="calculator-lifecycle-chart-svg"]').dblclick(); + xAxisTicks().should('equal', before); + // d3's reset is a ~750ms transition whose trailing events repaint the axes + // from the scales captured when it started. Let it finish before the next + // test changes the metric, or that repaint lands on top of the new axis. + cy.wait(900); + }); + // The chip labels survive the zoom, one per line, still inside the SVG. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] path.line-path').then((lines) => { + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-series-label').should( + 'have.length', + lines.length, + ); + }); + }); + + it('switches the y axis between margin and revenue', () => { + chartText().should('contain', 'Margin ($/day)'); + // Break-even is meaningful on a margin axis, so the rule is drawn. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should('exist'); + + cy.get('[data-testid="calc-lifecycle-metric-revenue"]').click(); + chartText().should('contain', 'Revenue ($/day)'); + // Zero is not break-even on a revenue axis — each chip breaks even at its own + // cost line — so the rule must not be drawn claiming otherwise. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should( + 'not.exist', + ); + // Revenue drops the cost term, so nothing plotted can be negative. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .y-axis').should( + 'not.contain.text', + '-$', + ); + + // Cumulative revenue is a running total in $, not a rate in $/day. Zero is + // where every chip starts rather than a threshold anything crosses, so the + // break-even rule stays off here too. (The axis does carry one negative tick: + // the domain is padded 5% below its floor, which on a tens-of-millions span + // rounds to a visible one. That is scale padding, not plotted data.) + cy.get('[data-testid="calc-lifecycle-metric-cumulative-revenue"]').click(); + chartText().should('contain', 'Cumulative Revenue ($)'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should( + 'not.exist', + ); + + cy.get('[data-testid="calc-lifecycle-metric-margin"]').click(); + chartText().should('contain', 'Margin ($/day)'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should('exist'); + + // Margin per MW is the same quantity in the unit a power-constrained plan is + // written in, so it is a positive rescale: zero is still break-even and the + // rule stays. The axis ticks must differ from the $/day ones captured just + // above — the fleet is several MW, so a metric that was relabelled but never + // divided would reproduce them exactly. Comparing against any other metric's + // axis would pass without the division ever happening. + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .y-axis') + .invoke('text') + .then((perDay) => { + cy.get('[data-testid="calc-lifecycle-metric-margin-per-mw"]').click(); + chartText().should('contain', 'Margin ($/MW/day)'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should( + 'exist', + ); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .y-axis') + .invoke('text') + .should((perMw) => { + expect(perMw, 'the per-MW axis is not the $/day axis relabelled').to.not.equal(perDay); + }); + }); + + cy.get('[data-testid="calc-lifecycle-metric-margin"]').click(); + chartText().should('contain', 'Margin ($/day)'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should('exist'); + }); + + it('a longer horizon accumulates more of whatever the daily margin is', () => { + // Direction, not just difference: a regression that moved this the wrong way + // would satisfy a bare `not.equal` while inverting the finding. The direction + // is not "up" — the price defaults to the cheapest chip's break-even, so the + // first row is typically losing money, and more months of a negative rate is a + // deeper hole. Reading the sign off the $/day column keeps the assertion true + // of the physics rather than of one fixture's profitability. + cy.get('[data-testid="calc-lifecycle-horizon-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-horizon-input"]').type('24'); + firstRowCell('Margin $/day') + .invoke('text') + .then((perDay) => { + const sign = Math.sign(money(perDay)); + expect(sign, 'a non-zero daily margin to accumulate').to.not.equal(0); + firstRowCell('Cumulative Margin') + .invoke('text') + .then((short) => { + cy.get('[data-testid="calc-lifecycle-horizon-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-horizon-input"]').type('96'); + firstRowCell('Cumulative Margin') + .invoke('text') + .should((long) => { + const delta = (money(long) - money(short)) * sign; + expect(delta, 'cumulative margin moves with the daily rate').to.be.greaterThan(0); + }); + }); + }); + }); + + it('tracks measured config improvements as steps, not one flat plateau', () => { + showTable(); + // The point of the section: a chip's revenue follows its optimisation + // history, so at least one chip must show more than zero improvements. + cy.get('[data-testid="calculator-lifecycle-table"] tbody tr').then(($rows) => { + const improvements = [...$rows].map((row) => + Number.parseInt(row.querySelectorAll('td')[4]!.textContent ?? '0', 10), + ); + expect(Math.max(...improvements)).to.be.greaterThan(0); + }); + // Gain over the opening config is reported as a multiple. + firstRowCell('Gain') + .invoke('text') + .should('match', /^\d+(?:\.\d+)?×$/u); + }); + + it('rolls each config out over the ramp instead of stepping instantly', () => { + // Reads both views, so it switches deliberately: `firstRowCell` moves to the + // table, and the vertex counts are only meaningful on the chart. + cy.get('[data-testid="calc-lifecycle-ramp-input"]') + .should('have.value', '0.5') + .and('have.attr', 'step', '0.25'); + lineVertices().then((curved) => { + // Rollouts are sampled into curves, so the line cannot be a bare staircase. + expect(curved).to.be.greaterThan(20); + // A longer ramp reaches each config's numbers later, so it earns less. + firstRowCell('Cumulative Margin') + .invoke('text') + .then((short) => { + cy.get('[data-testid="calc-lifecycle-ramp-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-ramp-input"]').type('18'); + // "Earns less" is the claim, so assert less — not merely different. + firstRowCell('Cumulative Margin') + .invoke('text') + .should((long) => expect(money(long)).to.be.lessThan(money(short))); + // Ramp 0 means configs take effect instantly — a pure staircase, which + // needs far fewer vertices than the sampled curves. + cy.get('[data-testid="calc-lifecycle-ramp-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-ramp-input"]').type('0'); + showChart(); + lineVertices().should((stepped) => expect(stepped).to.be.lessThan(curved)); + }); + }); + }); + + it('a shorter MTBI lowers availability', () => { + cy.get('[data-testid="calc-lifecycle-mtbi-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-mtbi-input"]').type('60'); + firstRowCell('Availability') + .invoke('text') + .then((high) => { + cy.get('[data-testid="calc-lifecycle-mtbi-input"]').clear(); + cy.get('[data-testid="calc-lifecycle-mtbi-input"]').type('2'); + firstRowCell('Availability') + .invoke('text') + .should((low) => { + expect(Number.parseFloat(low)).to.be.lessThan(Number.parseFloat(high)); + }); + }); + }); + + it('states the hybrid basis and the unofficial-run exclusion', () => { + // Today's TCO rates on a dated operating point, and no overlay support. + cy.get('[data-testid="calculator-lifecycle-section"]') + .should('contain.text', 'Power and $/chip/hr are today') + .and('contain.text', 'Unofficial runs loaded via a run link are not shown here'); + cy.get('[data-testid="calculator-lifecycle-section"]').should( + 'contain.text', + 'SemiAnalysis Datacenter Industry Model', + ); + }); + + it('draws one row per chip, pooling its software configs', () => { + showTable(); + // The history holds many hwKeys per chip (b200_sglang, b200_trtllm, …) and + // they are one piece of silicon, so the chip must not repeat down the table. + cy.get('[data-testid="calculator-lifecycle-table"] tbody tr').then(($rows) => { + const chips = [...$rows].map((row) => row.querySelectorAll('td')[0]!.textContent?.trim()); + expect(chips.length).to.be.greaterThan(0); + expect(new Set(chips).size).to.equal(chips.length); + }); + // And each row names the config it ended up on, since that changes along the line. + firstRowCell('Config Now').invoke('text').should('not.be.empty'); + }); + + it('follows legend visibility', () => { + // The legend lives in the controls card, outside the figure, so it stays + // reachable from the table view and even when nothing is plottable. + showTable(); + // Legend entries are configs and clicking one isolates it. Configs sit one + // level below the lines now, so isolating a single config leaves at most one + // chip — and none at all when that config was never measured at the target, + // which is a legitimate state rather than a broken chart. The section-level + // assertion covers both: the table shows at most one row, or is absent + // entirely in favour of the explanatory message. + cy.get('[data-testid="calculator-lifecycle-table"] tbody tr').then(($rows) => { + const fullCount = $rows.length; + expect(fullCount).to.be.greaterThan(1); + cy.get('[data-testid="fleet-legend"] ul label').first().click(); + cy.get('[data-testid="calculator-lifecycle-section"]').should(($section) => { + expect( + $section.find('[data-testid="calculator-lifecycle-table"] tbody tr').length, + ).to.be.at.most(1); + }); + cy.get('[data-testid="fleet-legend"] ul label').first().click(); + cy.get('[data-testid="calculator-lifecycle-table"] tbody tr').should( + 'have.length', + fullCount, + ); + }); + }); + + it('explains chips that were never measured at an extreme target instead of dropping them', () => { + // Pushing the target to the top of the range leaves chips outside their + // measured interactivity; the honest answer is to say so, with the range. + cy.get('[data-testid="fleet-controls"] input[type="range"]') + .invoke('attr', 'max') + .then((max) => { + cy.get('[data-testid="fleet-controls"] input[type="number"]').clear(); + cy.get('[data-testid="fleet-controls"] input[type="number"]').type(`${max}{enter}`); + }); + cy.get('[data-testid="calculator-lifecycle-section"]').should( + 'contain.text', + 'Not measured at this interactivity', + ); + cy.get('[data-testid="calculator-lifecycle-unmeasured"]').should('contain.text', 'measured'); + }); + + it('names the budget, not the slider, when a budget is too small to size a fleet', () => { + // The two ways to have nothing to plot need different fixes, so they must not + // share a message: this one is the budget, and blaming the interactivity + // slider would send the reader to the wrong control. + // + // The previous test leaves the target at the top of the range, where nothing + // is measured — which is the *other* cause, and would make `noneMeasured` + // the correct answer. So put the target back in a measured range first, or + // this asserts nothing about the budget at all. + cy.get('[data-testid="fleet-controls"] input[type="number"]').clear(); + cy.get('[data-testid="fleet-controls"] input[type="number"]').type('35{enter}'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + + cy.get('[data-testid="calc-fleet-mw-input"]').clear(); + cy.get('[data-testid="calc-fleet-mw-input"]').type('0.0001'); + cy.get('[data-testid="calculator-lifecycle-none"]') + .should('be.visible') + .and('contain.text', 'too small to power a single chip'); + }); + + it('clearing the MW budget restores the empty state, and the input survives it', () => { + cy.get('[data-testid="calc-fleet-mw-input"]').clear(); + cy.get('[data-testid="calculator-lifecycle-empty"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-table"]').should('not.exist'); + // The one control that must never vanish with the body it gates. + cy.get('[data-testid="calc-fleet-mw-input"]').should('be.visible'); + }); +}); + +describe('Fleet — self-contained lifecycle regressions', () => { + before(() => { + const rows = agenticB300Rows(null); + cy.intercept('GET', '/api/v1/availability', { body: agenticAvailability }); + cy.intercept('GET', '/api/v1/benchmarks/history*', { body: rows }).as('legacyPriceHistory'); + cy.intercept('GET', '/api/v1/benchmarks*', { body: rows }).as('legacyPriceBenchmarks'); + cy.visit('/fleet?g_model=DeepSeek-V4-Pro&i_seq=agentic-traces&i_prec=fp4&c_mw=10&c_price=7', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + win.sessionStorage.setItem('inferencex-reproducibility-nudge-shown', '1'); + unlockAgenticGate(win); + }, + }); + cy.wait('@legacyPriceBenchmarks'); + cy.wait('@legacyPriceHistory'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + }); + + it('preserves c_price and derives a non-zero output price', () => { + cy.get('[data-testid="calc-lifecycle-price-input"]').should('have.value', '7'); + cy.get('[data-testid="calc-lifecycle-output-price-input"]') + .invoke('val') + .should((output) => { + expect(Number(output)).to.equal(28); + }); + }); + + it('plots revenue per MW as a real normalized y-axis metric', () => { + cy.get('[data-testid="calc-lifecycle-metric-revenue"]').click(); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .y-axis') + .invoke('text') + .then((revenuePerDay) => { + cy.get('[data-testid="calc-lifecycle-metric-revenue-per-mw"]').click(); + chartText().should('contain', 'Revenue ($/MW/day)'); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .lifecycle-zero-rule').should( + 'not.exist', + ); + cy.get('[data-testid="calculator-lifecycle-chart-svg"] .y-axis') + .invoke('text') + .should((revenuePerMw) => { + expect(revenuePerMw, 'Revenue/MW is not Revenue/day relabelled').to.not.equal( + revenuePerDay, + ); + }); + }); + cy.get('[data-testid="calc-lifecycle-metric-margin"]').click(); + }); + + it('exports the physical fleet sizing alongside its economics', () => { + showTable(); + cy.window().then(captureLifecycleCsvDownloads); + exportMenu().click(); + cy.get('[data-testid="export-csv-button"]').click(); + + readCapturedLifecycleCsv().then((csv) => { + const header = csv.split('\n').find((line) => line.startsWith('Chip,')); + expect(header, 'lifecycle CSV header').to.equal( + 'Chip,Config Now,First Run,Latest Best,Improvements,Gain,Chips,tok/s/MW now,Concurrent Users now,Revenue $/day,Cost $/day,Margin $/day,Payback,Cumulative Margin,Availability', + ); + }); + }); +}); + +describe('Fleet — Fleet Lifecycle with agentic traces', () => { + // Two run dates so the staircase has a step in it, and a measured cache hit + // rate on every row so the cached-input discount has something to apply to. + // The shared history fixture carries ZERO agentic rows, so this spec serves + // its own — the pattern gpu-compare-agentic-detail.cy.ts uses. + const LATER_DATE = '2026-07-26'; + const historyRows = () => { + const day1 = agenticB300Rows(null).map((row) => ({ + ...row, + metrics: { ...row.metrics, server_gpu_cache_hit_rate: 0.9 } as Record, + })); + // A later, faster sweep: the improvement the section exists to show. + const day2 = day1.map((row) => ({ + ...row, + id: row.id + 5000, + date: LATER_DATE, + metrics: { + ...row.metrics, + tput_per_gpu: row.metrics.tput_per_gpu * 1.4, + output_tput_per_gpu: row.metrics.output_tput_per_gpu * 1.4, + input_tput_per_gpu: row.metrics.input_tput_per_gpu * 1.4, + }, + })); + const b300 = [...day1, ...day2]; + return [...b300, ...b300.map((row) => ({ ...row, id: row.id + 20000, hardware: 'b200' }))]; + }; + + before(() => { + // Agentic rows have null ISL/OSL, which the default fixture model has none + // of — so serve them the same way the main calculator spec does. + const b300Rows = agenticB300Rows(null); + const b200Rows = b300Rows.map((row) => ({ ...row, hardware: 'b200' })); + cy.intercept('GET', '/api/v1/availability', { + body: [ + ...agenticAvailability, + ...agenticAvailability.map((row) => ({ ...row, hardware: 'b200' })), + ...agenticAvailability.map((row) => ({ ...row, date: LATER_DATE })), + ], + }); + cy.intercept('GET', '/api/v1/benchmarks/history*', { body: historyRows() }).as( + 'agenticHistory', + ); + cy.intercept('GET', '/api/v1/benchmarks*', { body: [...b300Rows, ...b200Rows] }).as( + 'agenticBenchmarks', + ); + cy.visit('/fleet?g_model=DeepSeek-V4-Pro&i_seq=agentic-traces&i_prec=fp4', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + unlockAgenticGate(win); + }, + }); + cy.wait('@agenticBenchmarks'); + // The agentic fixture exposes a single scenario, so the scenario + // control disappears entirely — no dropdown, no static readout. + cy.get('[data-testid="scenario-static-value"]').should('not.exist'); + cy.get('[data-testid="calc-fleet-mw-input"]').should('have.value', '10'); + cy.wait('@agenticHistory'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('be.visible'); + }); + + it('projects a lifecycle from history that has no ISL/OSL to key on', () => { + // The section used to refuse this outright. The endpoint keys agentic + // history on benchmark_type instead, so there is a real projection here. + cy.get('[data-testid="calculator-lifecycle-unsupported"]').should('not.exist'); + cy.get('[data-testid="calculator-lifecycle-table"] tbody tr').should( + 'have.length.greaterThan', + 0, + ); + }); + + it('names the cached tier in the caption, since most input bills at it', () => { + // On agentic the input price beside it is not what most tokens actually cost: + // the measured hit rate is ~0.9 here, so quoting the input price alone would + // overstate what a reader thinks the fleet charges. + cy.get('[data-testid="calculator-lifecycle-figure"]').should('contain', ', cached 10%'); + cy.get('[data-testid="calc-lifecycle-cache-input"]').clear().type('35'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('contain', ', cached 35%'); + cy.get('[data-testid="calc-lifecycle-cache-input"]').clear().type('10'); + }); + + it('raises the break-even price it seeds, because fewer tokens are billable', () => { + cy.get('[data-testid="calc-lifecycle-cache-input"]').should('have.value', '10'); + // While the price is still auto-seeded, the discount does NOT move margin — + // break-even re-solves to keep the cheapest fleet at zero either way. What it + // moves is the price you need to charge, which is the economically + // interesting statement: discounting 90% of a 133:1 input mix means asking + // materially more per token to stand still. + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .then((discountedPrice) => { + cy.get('[data-testid="calc-lifecycle-cache-input"]').clear().type('100'); + cy.get('[data-testid="calc-lifecycle-price-input"]') + .invoke('val') + .should((fullPrice) => { + expect(Number(discountedPrice)).to.be.greaterThan(Number(fullPrice)); + }); + }); + }); + + it('lowers margin at a fixed price, where the discount has nowhere to hide', () => { + // Take the price over so it stops re-seeding; now the cached-token discount + // lands where a reader would expect it — on the margin. + cy.get('[data-testid="calc-lifecycle-cache-input"]').clear().type('100'); + cy.get('[data-testid="calc-lifecycle-price-input"]').clear().type('500'); + firstRowCell('Margin $/day') + .invoke('text') + .then((full) => { + cy.get('[data-testid="calc-lifecycle-cache-input"]').clear().type('10'); + firstRowCell('Margin $/day') + .invoke('text') + .should((discounted) => { + expect(money(discounted)).to.be.lessThan(money(full)); + }); + }); + }); +}); + +describe('Fleet — Fleet Lifecycle in Chinese', () => { + before(() => { + cy.visit('/zh/fleet', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + // Readiness: the lifecycle section only mounts once run data has loaded. + cy.get('[data-testid="calculator-lifecycle-section"]').should('exist'); + }); + + it('translates the section, including the table headers and notes', () => { + cy.get('[data-testid="calculator-lifecycle-section"]') + .should('contain.text', '集群生命周期') + .and('contain.text', '设施功率 (MW)'); + cy.get('[data-testid="calc-fleet-mw-input"]').should('have.value', '10'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('be.visible'); + // The caption's price line is translated too, units and all. + cy.get('[data-testid="calculator-lifecycle-figure"]').should('contain', '每百万 token'); + cy.get('[data-testid="calculator-lifecycle-figure"]').should('contain', '输入 $'); + // The table headers this test checks are on the other tab now. + showTable(); + cy.get('[data-testid="calculator-lifecycle-table"]').should('be.visible'); + cy.get('[data-testid="calculator-lifecycle-section"]') + .should('contain.text', 'Token 价格') + .and('contain.text', '每 MW 收入') + .and('contain.text', '当前配置') + .and('contain.text', '首次运行') + .and('contain.text', '累计利润') + .and('contain.text', '来源:'); + // Nothing from the English table leaks through. + cy.get('[data-testid="calculator-lifecycle-section"]') + .should('not.contain.text', 'Enter a facility power budget') + .and('not.contain.text', 'Cumulative Margin') + .and('not.contain.text', 'First Run'); + }); +}); diff --git a/packages/app/cypress/e2e/gpu-compare-agentic-detail.cy.ts b/packages/app/cypress/e2e/gpu-compare-agentic-detail.cy.ts index d7ffe31fa..d643a2d44 100644 --- a/packages/app/cypress/e2e/gpu-compare-agentic-detail.cy.ts +++ b/packages/app/cypress/e2e/gpu-compare-agentic-detail.cy.ts @@ -1,4 +1,5 @@ import { interceptDerivedAgenticMetrics, unlockAgenticGate } from '../support/e2e'; +import { agenticMetrics } from '../support/agentic-fixtures'; // --------------------------------------------------------------------------- // Spec-scoped fixture helpers @@ -51,37 +52,6 @@ const agenticAvailability = [ })), ]; -// Minimal per-metric percentile ladder matching what the chart expects for -// agentic rows (median/p75/p90/p95/p99 + std for each family). -const percentileLadder = (prefix: string, base: number): Record => ({ - [`median_${prefix}`]: base, - [`p75_${prefix}`]: base * 1.2, - [`p90_${prefix}`]: base * 1.5, - [`p95_${prefix}`]: base * 1.7, - [`p99_${prefix}`]: base * 2.2, - [`std_${prefix}`]: base * 0.3, -}); - -const agenticMetrics = (conc: number): Record => { - const scale = conc / 16; - const itl = 0.011 * scale; - return { - ...percentileLadder('ttft', 0.4 * scale), - ...percentileLadder('tpot', 0.012 * scale), - ...percentileLadder('itl', itl), - ...percentileLadder('e2el', 8 * scale), - median_intvty: 1 / itl, - p75_intvty: 1 / (itl * 1.2), - p90_intvty: 1 / (itl * 1.5), - p99_intvty: 1 / (itl * 2.2), - std_intvty: (1 / itl) * 0.1, - tput_per_gpu: 950 / Math.sqrt(scale), - output_tput_per_gpu: 210, - input_tput_per_gpu: 740, - total_tput_tps: 7600 * conc * 0.05, - }; -}; - // IDs must be unique numbers — the GPU graph uses them as D3 data keys and // trace-availability is keyed on them. let benchIdCursor = 800100; @@ -147,6 +117,14 @@ describe('GPU comparison agentic point detail', () => { ); request.reply({ body: result }); }); + cy.intercept('GET', '/api/v1/log-availability*', (request) => { + const ids = new URL(request.url).searchParams.get('ids')?.split(',') ?? []; + request.reply({ + body: Object.fromEntries( + ids.filter((id) => agenticIds.has(Number(id))).map((id) => [id, true]), + ), + }); + }).as('gpuLogAvailability'); // The agentic default x-axis mode (E2E Normalized Interactivity) fetches derived metrics on // mount; without values every point drops out of the (remapped) data set. interceptDerivedAgenticMetrics(); @@ -160,6 +138,10 @@ describe('GPU comparison agentic point detail', () => { cy.get('[data-testid="gpu-multiselect"] [role="combobox"]').click({ force: true }); cy.get('[role="option"]').first().click(); + cy.contains('Comparison Date Range').should('be.visible'); + cy.contains('button', 'Select date range') + .should('not.have.class', 'animate-pulse') + .and('not.have.class', 'border-red-500'); cy.contains('button', 'Select date range').click(); cy.get('body').then(($body) => { if ($body.text().includes('View anyway')) { @@ -172,6 +154,7 @@ describe('GPU comparison agentic point detail', () => { cy.get('[data-testid="gpu-graph"]').first().should('be.visible'); cy.wait('@gpuTraceAvailability'); + cy.wait('@gpuLogAvailability'); cy.wait(100); cy.get('[data-testid="gpu-graph"]') .first() @@ -199,7 +182,26 @@ describe('GPU comparison agentic point detail', () => { .then(($link) => { expect($link).to.match('a'); expect($link).not.to.have.attr('target'); - expect($link.attr('href')).to.match(/^\/inference\/agentic\/\d+$/u); + const href = $link.attr('href') ?? ''; + expect(href).to.match(/^\/inference\/agentic\/\d+\?/u); + // The chart state rides along so the detail page can link back to the + // view the reader left — see agentic-detail-back-nav.cy.ts. Only + // non-default values are carried, so `g_model` (DeepSeek-V4-Pro IS the + // default) is absent while the scenario and GPU selection are not. + const params = new URLSearchParams(href.slice(href.indexOf('?'))); + expect(params.get('i_seq')).to.equal('agentic-traces'); + expect(params.get('i_gpus')).to.be.a('string').and.not.equal(''); + }); + cy.get('[data-chart-tooltip]:visible [data-action="view-logs"]') + .should('be.visible') + .then(($link) => { + expect($link).to.match('a'); + const href = $link.attr('href') ?? ''; + expect(href).to.match(/^\/inference\/agentic\/\d+\?/u); + const params = new URLSearchParams(href.slice(href.indexOf('?'))); + expect(params.get('view')).to.equal('logs'); + expect(params.get('i_seq')).to.equal('agentic-traces'); + expect(params.get('i_gpus')).to.be.a('string').and.not.equal(''); }); cy.location('pathname').should('eq', '/inference'); }); diff --git a/packages/app/cypress/e2e/gradient-labels.cy.ts b/packages/app/cypress/e2e/gradient-labels.cy.ts index 05df38a46..5fdce43df 100644 --- a/packages/app/cypress/e2e/gradient-labels.cy.ts +++ b/packages/app/cypress/e2e/gradient-labels.cy.ts @@ -1,3 +1,5 @@ +import { expandLegendAdvanced } from '../support/legend-advanced'; + describe('Gradient Labels Toggle', () => { before(() => { cy.visit('/inference', { @@ -8,6 +10,8 @@ describe('Gradient Labels Toggle', () => { // Wait for chart to load cy.get('[data-testid="scatter-graph"]').should('be.visible'); cy.get('.sidebar-legend').first().should('be.visible'); + // Parallelism Labels lives in the collapsed-by-default Advanced drawer. + expandLegendAdvanced(); }); it('Gradient Labels toggle exists in the legend', () => { @@ -94,6 +98,7 @@ describe('Gradient Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); }); }); diff --git a/packages/app/cypress/e2e/historical-trends.cy.ts b/packages/app/cypress/e2e/historical-trends.cy.ts index 55b0e2749..51c9d5b21 100644 --- a/packages/app/cypress/e2e/historical-trends.cy.ts +++ b/packages/app/cypress/e2e/historical-trends.cy.ts @@ -94,7 +94,12 @@ describe('Historical Trends — Content & Interactions', () => { cy.get('body').type('{esc}'); }); - it('precision multi-select is present', () => { + it('precision multi-select is present only for multi-precision models', () => { + // The default model is FP4-only in the fixtures, so the control is hidden; + // a multi-precision model brings it back. + cy.get('[data-testid="precision-multiselect"]').should('not.exist'); + cy.visit('/historical?g_model=DeepSeek-R1-0528'); + cy.get('[data-testid="historical-trends-display"]').should('be.visible'); cy.get('[data-testid="precision-multiselect"]').should('be.visible'); }); diff --git a/packages/app/cypress/e2e/inference-chart.cy.ts b/packages/app/cypress/e2e/inference-chart.cy.ts index 35ca6922e..33bcc96c9 100644 --- a/packages/app/cypress/e2e/inference-chart.cy.ts +++ b/packages/app/cypress/e2e/inference-chart.cy.ts @@ -1,3 +1,6 @@ +import { unlockAgenticGate } from '../support/e2e'; +import { interceptOverlayRun, OVERLAY_RUN_ID } from '../support/overlay-fixtures'; + describe('Inference Chart', () => { before(() => { cy.window().then((win) => { @@ -58,7 +61,9 @@ describe('Inference Chart', () => { }); it('renders quick filters and toggles a vendor pill', () => { - cy.get('[data-testid="quick-filters"]').should('exist'); + cy.get('[data-testid="quick-filters-dialog"]').should('not.exist'); + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('be.visible'); cy.get('[data-testid="quick-filter-deployment-single-node"]').should('contain', 'Single-node'); cy.get('[data-testid="quick-filter-deployment-multi-node"]').should('contain', 'Multi-node'); cy.get('[data-testid="quick-filter-deployment-disagg"]').should('contain', 'Disaggregated'); @@ -69,4 +74,132 @@ describe('Inference Chart', () => { .click() .should('have.attr', 'aria-pressed', 'false'); }); + + it('plots OpenRouter-priced token revenue for official and unofficial runs', () => { + cy.intercept('GET', 'https://openrouter.ai/api/v1/models', { + statusCode: 200, + body: { + data: [ + { + id: 'deepseek/deepseek-v4-pro-0813', + pricing: { + prompt: '0.000001122', + completion: '0.000003366', + }, + }, + ], + }, + }).as('openRouterPricing'); + interceptOverlayRun(); + cy.visit( + `/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90&i_metric=y_tokenRevenuePerGpuHour&i_revenue=openrouter`, + { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + unlockAgenticGate(win); + }, + }, + ); + cy.wait('@unofficialRun'); + cy.wait('@openRouterPricing'); + + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Token Revenue per GPU Hour', + ); + cy.get('[data-testid="token-revenue-price-source"]').should( + 'contain.text', + 'OpenRouter current pricing', + ); + cy.get('[data-testid="openrouter-price-summary"]') + .should('contain.text', 'Input $1.122/M tok') + .and('contain.text', 'Output $3.366/M tok'); + cy.get('[data-testid="chart-figure"]') + .first() + .find('[data-testid="token-revenue-subtitle-prices"]') + .should('have.text', 'Input $1.122/M tok · Output $3.366/M tok') + .then(($prices) => { + const subtitle = $prices.parent().text(); + expect(subtitle.indexOf($prices.text())).to.be.lessThan(subtitle.indexOf('Updated:')); + }); + cy.get('[data-testid="openrouter-pricing-link"]').should( + 'have.attr', + 'href', + 'https://openrouter.ai/deepseek/deepseek-v4-pro-0813', + ); + cy.get('[data-testid="chart-figure"]') + .first() + .find('h2') + .should('contain.text', 'Token Revenue per GPU Hour at OpenRouter Pricing'); + cy.get('[data-testid="inference-chart-display"] svg .dot-group').should( + 'have.length.greaterThan', + 0, + ); + cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should( + 'have.length.greaterThan', + 0, + ); + cy.get('[data-testid^="axis-metric-row-y-"]').first().click(); + cy.get('[data-testid^="axis-metric-body-y-"]') + .first() + .should('contain.text', 'OpenRouter') + .and('contain.text', '$/GPU/hr ='); + }); + + it('ships OpenRouter-priced token revenue in Chinese', () => { + cy.viewport(390, 844); + cy.intercept('GET', 'https://openrouter.ai/api/v1/models', { + statusCode: 200, + body: { + data: [ + { + id: 'deepseek/deepseek-v4-pro-0813', + pricing: { + prompt: '0.000001122', + completion: '0.000003366', + }, + }, + ], + }, + }).as('openRouterPricingZh'); + interceptOverlayRun(); + cy.visit( + `/zh/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90&i_metric=y_tokenRevenuePerGpuHour&i_revenue=openrouter`, + { onBeforeLoad: unlockAgenticGate }, + ); + cy.wait('@unofficialRun'); + cy.wait('@openRouterPricingZh'); + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + '每 GPU 小时 token 收入', + ); + cy.get('[data-testid="token-revenue-price-source"]').should( + 'contain.text', + 'OpenRouter 当前价格', + ); + cy.get('[data-testid="openrouter-price-summary"]') + .should('contain.text', '输入 $1.122/百万 token') + .and('contain.text', '输出 $3.366/百万 token'); + cy.get('[data-testid="chart-figure"]') + .first() + .find('[data-testid="token-revenue-subtitle-prices"]') + .should('have.text', '输入 $1.122/百万 token · 输出 $3.366/百万 token') + .then(($prices) => { + const subtitle = $prices.parent().text(); + expect(subtitle.indexOf($prices.text())).to.be.lessThan(subtitle.indexOf('更新时间:')); + }); + cy.get('[data-testid="chart-figure"]') + .first() + .find('h2') + .should('contain.text', '按 OpenRouter 价格计算的每 GPU 小时 token 收入'); + }); + + it('surfaces the error instead of an endless skeleton when availability fails', () => { + cy.intercept('GET', '/api/v1/availability*', { statusCode: 500, body: {} }).as( + 'availabilityFailure', + ); + cy.visit('/inference'); + cy.wait('@availabilityFailure'); + cy.contains('h2', 'Something went wrong!', { timeout: 10000 }).should('be.visible'); + }); }); diff --git a/packages/app/cypress/e2e/inference-replay.cy.ts b/packages/app/cypress/e2e/inference-replay.cy.ts index 8984b4e37..f44ce302f 100644 --- a/packages/app/cypress/e2e/inference-replay.cy.ts +++ b/packages/app/cypress/e2e/inference-replay.cy.ts @@ -1,3 +1,5 @@ +import { expandLegendAdvanced } from '../support/legend-advanced'; + const openReplayDialog = () => { cy.get('[data-testid="chart-figure"]') .first() @@ -120,23 +122,32 @@ describe('Inference Replay', () => { it('re-renders the replay frame when a parent-chart toggle changes', () => { cy.get('body').then(($body) => { if ($body.find('[data-testid="replay-panel-chart-0"]').length === 0) return; + // The log-scale switch lives in the collapsed-by-default Advanced drawer, + // so expand it on the parent chart's legend (not the replay panel's). + cy.get('[data-testid="legend-advanced-toggle"]').then(($toggles) => { + const parentAdvanced = [...$toggles].find( + (toggle) => !toggle.closest('[data-testid^="replay-panel-chart-"]'), + ); + if (!parentAdvanced) throw new Error('Parent chart Advanced toggle is missing'); + if (parentAdvanced.getAttribute('aria-expanded') !== 'true') { + cy.wrap(parentAdvanced).click({ force: true }); + } + }); // Capture the SVG path data for the first roofline as a stable signature. cy.get('[data-testid="replay-panel-chart-0"] svg path.roofline-path') .first() .invoke('attr', 'd') .then((beforeD) => { - // Toggle the log-scale setting in the underlying inference context — - // the replay panel shares state with the parent chart, so the chart - // re-renders without us touching the replay UI. - cy.window().then((win) => { - const url = new URL(win.location.href); - const cur = url.searchParams.get('i_log') === '1'; - url.searchParams.set('i_log', cur ? '0' : '1'); - win.history.replaceState(null, '', url.toString()); - // Dispatch a popstate so InferenceContext picks up the change. - win.dispatchEvent(new win.PopStateEvent('popstate')); + // Toggle the control on the parent chart rather than mutating the address + // bar. Dashboard URL state is intentionally snapshotted on load, so a + // synthetic popstate is not a supported control update. + cy.get('[data-testid="scatter-log-scale"]').then(($toggles) => { + const parentToggle = [...$toggles].find( + (toggle) => !toggle.closest('[data-testid^="replay-panel-chart-"]'), + ); + if (!parentToggle) throw new Error('Parent chart log-scale toggle is missing'); + cy.wrap(parentToggle).click({ force: true }); }); - cy.wait(400); cy.get('[data-testid="replay-panel-chart-0"] svg path.roofline-path') .first() .invoke('attr', 'd') @@ -153,6 +164,7 @@ describe('Inference Replay', () => { // Enable line labels inside the replay panel (scoped — the parent chart // renders the same control behind the dialog). cy.get('[data-testid="replay-panel-chart-0"]').within(() => { + expandLegendAdvanced(); cy.get('[data-testid="scatter-line-labels"]').then(($el) => { if ($el.attr('data-state') !== 'checked') cy.wrap($el).click(); }); diff --git a/packages/app/cypress/e2e/jalapeno-preview-notice.cy.ts b/packages/app/cypress/e2e/jalapeno-preview-notice.cy.ts new file mode 100644 index 000000000..73f99eac3 --- /dev/null +++ b/packages/app/cypress/e2e/jalapeno-preview-notice.cy.ts @@ -0,0 +1,131 @@ +const JALAPENO_QUERY = + 'g_model=DeepSeek-R1-0528&i_seq=8k%2F1k&i_prec=fp4&i_metric=y_outputTputPerGpu'; +const JALAPENO_NOTICE = '[data-testid="jalapeno-official-preview-notice"]'; +const VERA_RUBIN_NOTICE = '[data-testid="vera-rubin-official-preview-notice"]'; + +describe('official preview notices', () => { + beforeEach(() => { + // Keep this spec independent of Neon response time. The client appends the + // supplemental Jalapeño and July Rubin snapshots to these successful empty + // responses, which is exactly the path under test here. + cy.intercept('GET', '**/api/v1/availability*', { body: [] }); + cy.intercept('GET', '**/api/v1/benchmarks/history*', { body: [] }); + cy.intercept('GET', '**/api/v1/benchmarks?*', { body: [] }); + }); + + it('renders both graph notices and tracks Jalapeño legend visibility', () => { + cy.visit(`/inference?${JALAPENO_QUERY}`); + + cy.get(JALAPENO_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'InferenceX Official Preview') + .and('contain.text', 'Results may change as validation and publication continue.') + .and('not.contain.text', '\u2014'); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'Vera Rubin (July)') + .and('not.contain.text', '\u2014'); + + cy.get( + '[data-testid="chart-legend"] [role="button"][aria-label^="Hide"][aria-label*="Jalapeño"]', + ) + .first() + .click({ force: true }); + cy.get(JALAPENO_NOTICE).should('not.exist'); + cy.get(VERA_RUBIN_NOTICE).should('be.visible'); + }); + + it('keeps the July Vera Rubin notice off metrics where its curve is unsupported', () => { + cy.visit('/inference?g_model=DeepSeek-R1-0528&i_seq=8k%2F1k&i_prec=fp4&i_metric=y_tpPerGpu'); + + cy.get(JALAPENO_NOTICE, { timeout: 20000 }).should('be.visible'); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }).should('not.exist'); + }); + + it('appears with Jalapeño calculator results', () => { + cy.visit(`/calculator?${JALAPENO_QUERY}`); + + cy.get(JALAPENO_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'InferenceX Official Preview'); + + cy.get('[data-testid="calculator-controls"]').within(() => { + cy.get('#calc-cost-type').click(); + }); + cy.get('[role="option"]').contains('Output Tokens').click(); + cy.get('[data-testid="calculator-controls"] input[type="number"]') + .first() + .clear() + .type('100') + .blur(); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'Vera Rubin (July) results are an official preview'); + }); + + it('appears with Jalapeño historical trend lines', () => { + cy.visit(`/historical?${JALAPENO_QUERY}`); + + // The historical view defaults to 35 tok/s/user, below Jalapeño's measured + // range, and correctly does not extrapolate. Move into its measured range + // so a Jalapeño trend line, and therefore the preview notice, appears. + cy.get('[data-testid="historical-trends-display"] input[type="number"]', { + timeout: 20000, + }) + .clear() + .type('100') + .blur(); + cy.get(JALAPENO_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'InferenceX Official Preview'); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'Vera Rubin (July) results are an official preview'); + }); + + it('keeps the July Vera Rubin notice off unsupported historical metrics', () => { + cy.visit('/historical?g_model=DeepSeek-R1-0528&i_seq=8k%2F1k&i_prec=fp4&i_metric=y_tpPerGpu'); + + cy.get('[data-testid="historical-trends-display"] input[type="number"]', { + timeout: 20000, + }) + .clear() + .type('100') + .blur(); + cy.get(JALAPENO_NOTICE, { timeout: 20000 }).should('be.visible'); + cy.get(VERA_RUBIN_NOTICE).should('not.exist'); + }); + + it('shows the Vera Rubin preview when its output-token fleet curve is visible', () => { + cy.visit('/fleet?g_model=DeepSeek-R1-0528&i_seq=8k%2F1k&i_prec=fp4'); + + cy.get('[data-testid="fleet-cost-type-selector"]').click(); + cy.get('[role="option"]').contains('Output Tokens').click(); + cy.get('[data-testid="fleet-legend"]', { timeout: 20000 }).should('contain.text', 'Vera Rubin'); + cy.get('[data-testid="fleet-controls"] input[type="number"]').clear().type('100').blur(); + cy.get('[data-testid="calc-fleet-mw-input"]').should('have.value', '10'); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'Vera Rubin (July) results are an official preview') + .and('not.contain.text', '\u2014'); + }); + + it('uses the Chinese disclaimer on /zh and stays absent without Jalapeño results', () => { + cy.visit(`/zh/inference?${JALAPENO_QUERY}`); + cy.get(JALAPENO_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'InferenceX 官方预览') + .and('contain.text', '随着验证和发布工作的推进,结果可能会调整。'); + cy.get(VERA_RUBIN_NOTICE, { timeout: 20000 }) + .should('be.visible') + .and('contain.text', 'Vera Rubin (July)') + .and('contain.text', '随着验证和发布工作的推进,结果可能会调整。') + .and('not.contain.text', '\u2014'); + + cy.visit( + '/zh/inference?g_model=DeepSeek-V4-Pro&i_seq=8k%2F1k&i_prec=fp4&i_metric=y_outputTputPerGpu', + ); + cy.get(JALAPENO_NOTICE).should('not.exist'); + cy.get(VERA_RUBIN_NOTICE).should('not.exist'); + }); +}); diff --git a/packages/app/cypress/e2e/landing-performance.cy.ts b/packages/app/cypress/e2e/landing-performance.cy.ts index ed580a3d4..c114a7d32 100644 --- a/packages/app/cypress/e2e/landing-performance.cy.ts +++ b/packages/app/cypress/e2e/landing-performance.cy.ts @@ -47,7 +47,8 @@ describe('Landing page performance', () => { cy.viewport(412, 823); cy.request('/') .its('body') - .should('contain', 'See more supporters') + .should('contain', 'See full quotes') + .and('contain', 'href="/quotes#quote-minimax"') .and('contain', 'data-testid="launch-banner"'); cy.intercept('GET', '**/_next/static/**/*.js', (request) => { @@ -58,18 +59,20 @@ describe('Landing page performance', () => { cy.visit('/', { onBeforeLoad(win) { - win.localStorage.removeItem('inferencex-kimi-k3-modal-dismissed'); - win.localStorage.removeItem('inferencex-kimi-k3-banner-dismissed'); + win.localStorage.removeItem('inferencex-openai-rubin-banner-dismissed'); observeLayoutShifts(win); }, }); cy.get('[data-testid="launch-banner"]').should('be.visible'); - cy.get('[data-testid="intro-section"]').should('contain.text', 'See more supporters'); - cy.get('[data-testid="quote-carousel-more-row"]') + cy.get('[data-testid="intro-section"]').should( + 'contain.text', + 'See full quotes & more supporters', + ); + cy.get('[data-testid="supporters-more-row"]') .should('have.class', 'justify-end') .find('a') - .should('have.text', 'See more supporters →'); + .should('have.text', 'See full quotes & more supporters →'); expectLowCls(); }); @@ -77,14 +80,17 @@ describe('Landing page performance', () => { cy.viewport(412, 823); cy.visit('/', { onBeforeLoad(win) { - win.localStorage.setItem('inferencex-kimi-k3-banner-dismissed', '1'); + win.localStorage.setItem('inferencex-openai-rubin-banner-dismissed', '1'); observeLayoutShifts(win); }, }); cy.get('html').should('have.attr', 'data-landing-banner-dismissed'); cy.get('[data-testid="launch-banner"]').should('not.exist'); - cy.get('[data-testid="intro-section"]').should('contain.text', 'See more supporters'); + cy.get('[data-testid="intro-section"]').should( + 'contain.text', + 'See full quotes & more supporters', + ); expectLowCls(); }); @@ -100,11 +106,14 @@ describe('Landing page performance', () => { ); expect(resourceNames.some((name) => name.includes('/minecraft-click.mp3'))).to.eq(false); expect(resourceNames.some((name) => name.includes('/Monocraft-'))).to.eq(false); - // The carousel only renders the active quote's logo, and it always starts on - // the first (MiniMax) quote, so a mobile load fetches at most that one logo — - // never the full supporter set. - const carouselLogos = new Set(resourceNames.filter((name) => name.includes('/logos/'))); - expect(carouselLogos.size).to.be.lessThan(3); + // The landing AgentX ledger has five lazy model marks. A mobile viewport may + // fetch any visible subset, but the text-only supporter strip must not pull + // in its former logo set. + const landingLogos = new Set(resourceNames.filter((name) => name.includes('/logos/'))); + expect(landingLogos.size).to.be.lessThan(6); + expect( + [...landingLogos].filter((name) => !new URL(name).pathname.endsWith('-color.svg')), + ).to.deep.eq([]); expect( resourceNames.some( (name) => name.includes('/brand/logo-color.webp') && name.includes('w=128'), @@ -113,7 +122,7 @@ describe('Landing page performance', () => { }); }); - it('preloads only the default font and initially visible supporter logo', () => { + it('preloads only the default font and no supporter logos', () => { cy.request('/').then((response) => { // Next emits resource preloads as a `Link` response header (when `/` renders // dynamically) and/or as inlined tags in the document @@ -150,10 +159,9 @@ describe('Landing page performance', () => { } expect([...fonts]).to.have.length(1); - expect([...logos]).to.have.length(1); - // MiniMax is pinned first in the carousel order, so it's the initially - // visible supporter the server renders (index 0) and preloads. - expect([...logos][0]).to.eq('/logos/minimax.svg'); + // The supporters band no longer renders a quote block, so no supporter + // logo is visible on the landing page and none should be preloaded. + expect([...logos]).to.have.length(0); }); }); diff --git a/packages/app/cypress/e2e/line-labels.cy.ts b/packages/app/cypress/e2e/line-labels.cy.ts index 345ac17f1..d516fdb6f 100644 --- a/packages/app/cypress/e2e/line-labels.cy.ts +++ b/packages/app/cypress/e2e/line-labels.cy.ts @@ -1,3 +1,5 @@ +import { expandLegendAdvanced } from '../support/legend-advanced'; + describe('Line Labels Toggle', () => { before(() => { cy.visit('/inference', { @@ -8,6 +10,8 @@ describe('Line Labels Toggle', () => { // Wait for chart to load cy.get('[data-testid="scatter-graph"]').should('be.visible'); cy.get('.sidebar-legend').first().should('be.visible'); + // Label switches live in the collapsed-by-default Advanced drawer. + expandLegendAdvanced(); }); it('Line Labels toggle exists in the legend', () => { @@ -57,7 +61,7 @@ describe('Line Labels Toggle', () => { }); cy.get('[data-testid="scatter-graph"] svg g.line-label').should('have.length.greaterThan', 0); - cy.get('[data-testid="scatter-graph"] svg').then(($svg) => { + cy.get('[data-testid="scatter-graph"] [data-testid="d3-chart-svg"]').then(($svg) => { const svg = $svg[0]; const dots = svg.querySelectorAll('.dot-group'); const labels = svg.querySelectorAll('g.line-label'); @@ -88,7 +92,7 @@ describe('Line Labels Toggle', () => { // The chart requires Shift for wheel zoom (so bare scroll doesn't hijack // the page). Dispatch a few shift+wheel events over the plot to zoom in. - cy.get('[data-testid="scatter-graph"] svg').then(($svg) => { + cy.get('[data-testid="scatter-graph"] [data-testid="d3-chart-svg"]').then(($svg) => { const svg = $svg[0]; const r = svg.getBoundingClientRect(); for (let i = 0; i < 3; i++) { @@ -106,7 +110,7 @@ describe('Line Labels Toggle', () => { }); cy.wait(300); - cy.get('[data-testid="scatter-graph"] svg').then(($svg) => { + cy.get('[data-testid="scatter-graph"] [data-testid="d3-chart-svg"]').then(($svg) => { const svg = $svg[0]; const dots = svg.querySelectorAll('.dot-group'); const labels = svg.querySelectorAll('g.line-label'); @@ -156,6 +160,7 @@ describe('Line Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'checked'); // Labels should be rendered @@ -169,6 +174,7 @@ describe('Line Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'unchecked'); // Labels should not be rendered @@ -182,6 +188,7 @@ describe('Line Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'unchecked'); }); @@ -192,6 +199,7 @@ describe('Line Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); // Labels toggle is auto-enabled by the URL hydration so the advanced // (parallelism) point labels actually render. @@ -205,11 +213,12 @@ describe('Line Labels Toggle', () => { }, }); cy.get('[data-testid="scatter-graph"]').should('be.visible'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'unchecked'); }); - it('appends the precision to each line label when multiple precisions are selected', () => { + it('places the precision between the GPU and engine when multiple precisions are selected', () => { // Pair the FP4+FP8 selection with a model that has both precisions in the // fixtures. The default model (DeepSeek-V4-Pro) only has FP4, so // `effectivePrecisions` would drop FP8 and the test couldn't observe the @@ -222,8 +231,9 @@ describe('Line Labels Toggle', () => { cy.get('[data-testid="scatter-graph"]').should('be.visible'); // With both FP4 and FP8 shown, each curve is its own line and the label - // must carry the precision so the two curves of the same hardware are - // distinguishable (e.g. "B200 (vLLM) FP8" vs "B200 (vLLM) FP4"). + // must carry the precision between the GPU and engine so the two curves of + // the same hardware are distinguishable (e.g. "B200 FP8 (vLLM)" vs + // "B200 FP4 (vLLM)"). cy.get('[data-testid="scatter-graph"] svg g.line-label .ll-text') .should('have.length.greaterThan', 0) .then(($texts) => { @@ -237,6 +247,11 @@ describe('Line Labels Toggle', () => { labels.some((t) => /\bFP4\b/u.test(t)), 'an FP4 line label exists', ).to.equal(true); + for (const label of labels.filter((t) => /\bFP(?:4|8)\b/u.test(t) && /\(.+\)/u.test(t))) { + expect(label, 'precision precedes the engine suffix').to.match( + /^.+\sFP(?:4|8)\s\(.+\)$/u, + ); + } }); }); }); diff --git a/packages/app/cypress/e2e/model-architecture.cy.ts b/packages/app/cypress/e2e/model-architecture.cy.ts index a1dddb435..fb3b3c452 100644 --- a/packages/app/cypress/e2e/model-architecture.cy.ts +++ b/packages/app/cypress/e2e/model-architecture.cy.ts @@ -1,38 +1,42 @@ -describe('Model Architecture Diagram', () => { - before(() => { - // Use desktop viewport to ensure all UI elements are visible - cy.viewport(1280, 800); - // Pin to DeepSeek R1 — it has a rich architecture diagram (MoE + dense blocks) - // that the tests below exercise. The app's default model is DeepSeek V4 Pro, - // which has no architecture entry, so we select R1 explicitly via URL. - cy.visit('/inference?g_model=DeepSeek-R1-0528', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - // Wait for the page to load - cy.get('[data-testid="inference-chart-display"]').should('be.visible'); - }); - - it('architecture toggle renders for DeepSeek R1 with MoE badges', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('[data-testid="model-architecture-toggle"]').should( - 'contain.text', - 'Model Architecture', - ); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MLA'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '671B'); - }); - - it('clicking toggle expands and renders the SVG diagram', () => { - cy.get('[data-testid="model-architecture-toggle"]').click(); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); +/** + * The architecture diagram lives on the /model/[slug] deep-dive pages, always + * expanded (`variant="inline"`). The dashboard renders a link row to those + * pages where the collapsible drawer used to be — covered by the final + * describe block. + */ + +/** Visit a model deep-dive page and wait for the inline diagram to render. */ +function visitModelPage(slug: string) { + cy.viewport(1280, 800); + cy.visit(`/model/${slug}`, { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, }); + cy.get('[data-testid="model-architecture-inline"]').should('be.visible'); + cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); +} +describe('Model Architecture Diagram', () => { describe('Collapsible Transformer Blocks (MoE model - DeepSeek R1)', () => { before(() => { - // SVG is already visible from previous test (testIsolation: false) + // DeepSeek R1 has a rich architecture diagram (MoE + dense blocks) + // that the tests below exercise. + visitModelPage('deepseek-r1'); + }); + + it('inline architecture header renders with MoE badges', () => { + cy.get('[data-testid="model-architecture-inline"]').should( + 'contain.text', + 'Model Architecture', + ); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MLA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '671B'); + }); + + it('renders the SVG diagram without any toggle click', () => { + cy.get('[data-testid="model-architecture-toggle"]').should('not.exist'); cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); }); @@ -103,22 +107,7 @@ describe('Model Architecture Diagram', () => { describe('Collapsible Transformer Block (Dense model - Llama 3.3 70B)', () => { before(() => { - // Switch model and open architecture - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('Llama 3.3').click(); - - // Only click toggle if SVG is not already visible (previous describe may have left it open) - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('llama-3-3-70b'); }); it('shows single transformer block without dense sub-block', () => { @@ -140,34 +129,20 @@ describe('Model Architecture Diagram', () => { }); it('shows Dense badge and GQA badge', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'Dense'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'GQA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'Dense'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'GQA'); }); }); describe('Collapsible Transformer Blocks (MoE model - Kimi K2.5)', () => { before(() => { - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('Kimi K2.5/2.6').click(); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('kimi-k26'); }); it('shows MoE and MLA badges for Kimi K2.5', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MLA'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '1.0T'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MLA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '1.0T'); }); it('shows both dense and MoE transformer blocks', () => { @@ -195,27 +170,13 @@ describe('Model Architecture Diagram', () => { describe('Collapsible Transformer Blocks (MoE model - MiniMax M2.5)', () => { before(() => { - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('MiniMax M2.5/2.7').click(); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('minimax-m27'); }); it('shows MoE and GQA badges for MiniMax M2.5', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'GQA'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '230B'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'GQA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '230B'); }); it('shows single MoE transformer block without dense sub-block', () => { @@ -243,27 +204,13 @@ describe('Model Architecture Diagram', () => { describe('Collapsible Transformer Blocks (MoE model - MiniMax M3)', () => { before(() => { - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('MiniMax M3').click(); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('minimax-m3'); }); it('shows MoE and GQA badges for MiniMax M3', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'GQA'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '428B'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'GQA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '428B'); }); it('GQA attention is NOT expandable (sparse attention rendered as a static block)', () => { @@ -280,27 +227,13 @@ describe('Model Architecture Diagram', () => { describe('Alternating Attention Blocks (MoE model - gpt-oss 120B)', () => { before(() => { - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('gpt-oss').click(); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('gptoss-120b'); }); it('shows MoE and Sink/Full GQA badges for gpt-oss', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'Sink/Full GQA'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '120B'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'Sink/Full GQA'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '120B'); }); it('shows two separate transformer blocks (no single expand-transformer)', () => { @@ -356,27 +289,13 @@ describe('Model Architecture Diagram', () => { describe('Hybrid Attention Blocks (MoE model - DeepSeek V4 Pro)', () => { before(() => { - // Clear any stale Radix scroll lock from prior Select interactions - cy.document().then((doc) => { - delete doc.body.dataset.scrollLocked; - doc.body.style.removeProperty('pointer-events'); - }); - cy.get('[role="combobox"]').filter(':visible').first().click(); - cy.get('[role="option"]').contains('DeepSeek V4 Pro').click(); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + visitModelPage('deepseek-v4'); }); it('shows MoE and Hybrid badges for DeepSeek V4 Pro', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'Hybrid'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '1.6T'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'Hybrid'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '1.6T'); }); it('shows two separate hybrid (CSA/HCA) blocks with an alternating indicator', () => { @@ -446,48 +365,16 @@ describe('Model Architecture Diagram', () => { describe('Hybrid Attention Blocks (MoE model - Kimi K3)', () => { before(() => { - // The model dropdown only lists models that have availability rows, and the - // shared fixtures carry none for kimik3 (nothing ingested yet). Append one - // so K3 is selectable, then re-visit with the intercept in place. This block - // runs last in the spec, so no earlier block sees the patched availability. - cy.fixture('api/availability.json').then((rows: unknown[]) => { - cy.intercept('GET', '/api/v1/availability', { - body: [ - ...rows, - { - model: 'kimik3', - isl: 8192, - osl: 1024, - precision: 'fp4', - hardware: 'mi355x', - framework: 'vllm', - spec_method: 'none', - disagg: false, - date: '2026-07-18', - }, - ], - }).as('availability'); - }); - cy.visit('/inference?g_model=Kimi-K3', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="inference-chart-display"]').should('be.visible'); - - cy.get('[data-testid="model-architecture-toggle"]').should('be.visible'); - cy.get('body').then(($body) => { - if ($body.find('[data-testid="model-architecture-svg"]:visible').length === 0) { - cy.get('[data-testid="model-architecture-toggle"]').click(); - } - }); - cy.get('[data-testid="model-architecture-svg"]').should('be.visible'); + // The /model page renders the diagram regardless of benchmark + // availability, so no fixture patching is needed (unlike the old + // dashboard drawer, which required the model to be selectable). + visitModelPage('kimi-k3'); }); it('shows MoE and Hybrid badges for Kimi K3', () => { - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'MoE'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', 'Hybrid'); - cy.get('[data-testid="model-architecture-toggle"]').should('contain.text', '2.8T'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'MoE'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', 'Hybrid'); + cy.get('[data-testid="model-architecture-inline"]').should('contain.text', '2.8T'); }); it('renders the KDA and gated-MLA layer categories as two alternating blocks', () => { @@ -557,4 +444,107 @@ describe('Model Architecture Diagram', () => { cy.contains('Released by Moonshot AI').should('be.visible'); }); }); + + describe('Model index page (/model)', () => { + before(() => { + cy.viewport(1280, 800); + cy.visit('/model', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + }); + + it('lists every model page with architecture badges', () => { + cy.get('h1').should('contain.text', 'Model Architectures'); + cy.get('[data-testid="model-index-list"] a').should('have.length.at.least', 11); + cy.get('[data-testid="model-index-link-kimi-k3"]') + .should('contain.text', 'Kimi K3') + .and('contain.text', 'MoE') + .and('contain.text', 'Hybrid') + .and('contain.text', '2.8T') + .and('have.attr', 'href', '/model/kimi-k3'); + // Models without a MODEL_ARCHITECTURES entry still get a card (no badges). + cy.get('[data-testid="model-index-link-glm-5-2"]').should('contain.text', 'GLM-5.2'); + }); + + it('navigates to a model deep-dive page', () => { + cy.get('[data-testid="model-index-link-deepseek-r1"]').click(); + cy.url().should('include', '/model/deepseek-r1'); + cy.get('[data-testid="model-architecture-inline"]').should('be.visible'); + }); + + it('is linked from the footer', () => { + cy.get('[data-testid="footer-link-model-architectures"]') + .should('have.attr', 'href', '/model') + .and('contain.text', 'Model Architectures'); + }); + }); + + describe('Embedded dashboard changelog starts collapsed', () => { + before(() => { + cy.viewport(1280, 800); + // deepseek-r1 has fixture benchmark data, so the changelog renders. + cy.visit('/model/deepseek-r1', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + }); + + it('renders the changelog header collapsed inside the embed', () => { + cy.contains('button', 'Config Changelog', { timeout: 20000 }) + .should('be.visible') + .and('have.attr', 'aria-expanded', 'false'); + }); + + it('can still be expanded manually', () => { + cy.contains('button', 'Config Changelog').click(); + cy.contains('button', 'Config Changelog').should('have.attr', 'aria-expanded', 'true'); + }); + }); + + describe('Dashboard architecture icon link (replaces the banner row)', () => { + before(() => { + cy.viewport(1280, 800); + cy.visit('/inference?g_model=DeepSeek-R1-0528', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + cy.get('[data-testid="inference-chart-display"]').should('be.visible'); + }); + + it('renders an icon link beside the model selector instead of a banner row', () => { + cy.get('[data-testid="model-architecture-toggle"]').should('not.exist'); + cy.get('[data-testid="model-architecture-link"]').should('be.visible'); + cy.get('[data-testid="model-architecture-link"]').should( + 'have.attr', + 'aria-label', + 'Learn more about the DeepSeek R1 0528 671B architecture', + ); + cy.get('[data-testid="model-architecture-link"]') + .should('have.attr', 'href') + .and('equal', '/model/deepseek-r1'); + // The former banner copy and badges now live in the tooltip; Radix + // tooltips open on keyboard focus, which is more reliable in Cypress + // than synthetic hover. + cy.get('[data-testid="model-architecture-link"]').focus(); + cy.get('[data-testid="model-architecture-tooltip"]') + .should('be.visible') + .and('contain.text', 'Learn more about the DeepSeek R1 0528 671B architecture') + .and('contain.text', 'MoE') + .and('contain.text', 'MLA') + .and('contain.text', '671B'); + // Dismiss the tooltip so it doesn't cover the link for the next test. + cy.get('body').type('{esc}'); + }); + + it('navigates to the model deep-dive page', () => { + cy.get('[data-testid="model-architecture-link"]').click(); + cy.url().should('include', '/model/deepseek-r1'); + cy.get('[data-testid="model-architecture-inline"]').should('be.visible'); + cy.get('[data-testid="model-page-dashboard"]').should('exist'); + }); + }); }); diff --git a/packages/app/cypress/e2e/model-route-slugs.cy.ts b/packages/app/cypress/e2e/model-route-slugs.cy.ts new file mode 100644 index 000000000..1c20c43c9 --- /dev/null +++ b/packages/app/cypress/e2e/model-route-slugs.cy.ts @@ -0,0 +1,82 @@ +/** + * Per-model SEO routes: /historical/ and /calculator/ (and + * their /zh siblings) render the tab seeded to the routed model. Switching + * models on the page rewrites the pathname in place via history.replaceState + * — no full page reload and no App Router remount. + */ +const visitOptions = { + onBeforeLoad(win: Cypress.AUTWindow) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, +}; + +type MarkerWindow = Cypress.AUTWindow & { __modelRouteNoReloadMarker?: boolean }; + +const unlockPointerEvents = () => { + cy.document().then((doc) => { + delete doc.body.dataset.scrollLocked; + doc.body.style.removeProperty('pointer-events'); + }); +}; + +describe('Per-model dashboard routes', () => { + it('seeds /historical/ to the routed model and rewrites the URL on switch without a reload', () => { + cy.visit('/historical/minimax-m3', visitOptions); + cy.get('[data-testid="historical-trends-display"]').should('be.visible'); + cy.get('[data-testid="model-selector"]').should('contain.text', 'MiniMax'); + + // A full page load (or App Router remount via hard navigation) would wipe + // this marker; the in-place history.replaceState rewrite must keep it. + cy.window().then((win) => { + (win as MarkerWindow).__modelRouteNoReloadMarker = true; + }); + unlockPointerEvents(); + cy.get('[data-testid="model-selector"]').click(); + cy.get('[role="option"]').not('[aria-selected="true"]').first().click(); + + cy.location('pathname').should('not.eq', '/historical/minimax-m3'); + cy.location('pathname').should('match', /^\/historical\/[a-z0-9-]+$/u); + cy.window().its('__modelRouteNoReloadMarker').should('eq', true); + }); + + it('moves from the bare tab path to a slugged path when a non-default model is chosen', () => { + cy.visit('/historical', visitOptions); + cy.get('[data-testid="historical-trends-display"]').should('be.visible'); + unlockPointerEvents(); + cy.get('[data-testid="model-selector"]').click(); + cy.get('[role="option"]').not('[aria-selected="true"]').first().click(); + cy.location('pathname').should('match', /^\/historical\/[a-z0-9-]+$/u); + }); + + it('redirects alias slugs to the canonical model slug, keeping share-link params', () => { + cy.visit('/calculator/kimi?i_seq=8k%2F1k', visitOptions); + cy.location('pathname').should('eq', '/calculator/kimi-k26'); + cy.location('search').should('contain', 'i_seq=8k%2F1k'); + }); + + it('language toggle follows the in-place rewritten per-model path', () => { + cy.visit('/historical/minimax-m3', visitOptions); + cy.get('[data-testid="historical-trends-display"]').should('be.visible'); + unlockPointerEvents(); + cy.get('[data-testid="model-selector"]').click(); + cy.get('[role="option"]').not('[aria-selected="true"]').first().click(); + cy.location('pathname') + .should('match', /^\/historical\/[a-z0-9-]+$/u) + .then((rewritten) => { + cy.get('[data-testid="language-toggle"]').click(); + cy.location('pathname').should('eq', `/zh${rewritten}`); + }); + }); + + it('returns 404 for unknown model slugs', () => { + cy.request({ url: '/historical/not-a-model', failOnStatusCode: false }) + .its('status') + .should('eq', 404); + }); + + it('renders the /zh sibling with a model-specific Chinese intro', () => { + cy.visit('/zh/historical/minimax-m3', visitOptions); + cy.get('[data-testid="zh-tab-intro"]').should('be.visible').and('contain.text', 'MiniMax M3'); + cy.get('[data-testid="historical-trends-display"]').should('be.visible'); + }); +}); diff --git a/packages/app/cypress/e2e/navigation.cy.ts b/packages/app/cypress/e2e/navigation.cy.ts index 12fc661da..9ca19924a 100644 --- a/packages/app/cypress/e2e/navigation.cy.ts +++ b/packages/app/cypress/e2e/navigation.cy.ts @@ -46,18 +46,18 @@ describe('First-load navigation', () => { cy.visit('/', { onBeforeLoad(win) { win.localStorage.removeItem('inferencex-starred'); - win.localStorage.removeItem('inferencex-star-modal-dismissed'); - win.localStorage.removeItem('inferencex-kimi-k3-modal-dismissed'); - win.localStorage.removeItem('inferencex-kimi-k3-banner-dismissed'); + // Snoozed, not cleared: the star modal is the eligible landing nudge + // on first load, and its corner card would sit over the footer links + // these specs click. + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + win.localStorage.removeItem('inferencex-openai-rubin-banner-dismissed'); }, }); - // Banner (inline) and overlay modal coexist in independent slots. - cy.get('[data-testid="launch-modal"]').should('be.visible'); cy.get('body').should('not.have.attr', 'data-scroll-locked'); }); - it('navigates to articles from the footer while the launch modal is visible', () => { + it('navigates to articles from the footer', () => { cy.get('[data-testid="footer-link-articles"]').scrollIntoView().click(); cy.location('pathname').should('eq', '/blog'); }); @@ -77,17 +77,50 @@ describe('First-load navigation', () => { cy.location('pathname').should('eq', '/compare'); }); - it('navigates to overview and the full dashboard from the landing CTAs', () => { - cy.get('[data-testid="landing-overview-link"]') - .should('have.attr', 'href', '/overview') - .click(); - cy.location('pathname').should('eq', '/overview'); + it('navigates to AgentX from the header with one click', () => { + cy.get('[data-testid="nav-link-agentx"]') + .should('have.attr', 'href', '/agentx') + .find('[data-nav-badge="agentx"]') + .should('be.visible') + .and('have.text', 'NEW'); + cy.get('[data-testid="nav-link-agentx"]').click(); + cy.location('pathname').should('eq', '/agentx'); + }); - cy.visit('/'); - cy.get('[data-testid="landing-full-dashboard-link"]') - .should('have.attr', 'href', '/inference') - .click(); - cy.location('pathname').should('eq', '/inference'); + it('leads the landing page with the AgentX hero and its two CTAs', () => { + cy.get('[data-testid="compare-agentx-primary"]').within(() => { + // The hero owns /compare's h1; on the landing page it is a section heading. + cy.get('h2').should('have.text', 'Compare Realistic Agentic Inference Perf'); + cy.get('h1').should('not.exist'); + cy.get('[data-testid="compare-agentx-overview-link"]') + .should('contain.text', 'Overview') + .and('have.attr', 'href', '/overview'); + cy.get('[data-testid="compare-agentx-dashboard-link"]') + .should('contain.text', 'Full dashboard') + .and('have.attr', 'href', '/inference/kimi-k3'); + cy.get('[data-testid="compare-agentx-methodology-link"]').should('not.exist'); + cy.get('[data-testid^="compare-agentx-model-"]').should('have.length', 6); + // Editorial order, not alphabetical — see FEATURED_AGENTX_MODEL_SLUGS. + cy.get('[data-testid^="compare-agentx-model-"]').then(($rows) => { + const slugs = [...$rows].map((row) => + (row.dataset.testid ?? '').replace('compare-agentx-model-', ''), + ); + expect(slugs).to.deep.equal([ + 'kimi-k3', + 'deepseek-v4', + 'glm-5-2', + 'minimax-m3', + 'qwen-3-5', + 'qwen-3-8-flash-next', + ]); + }); + // Every featured ledger row carries the NEW pill. + cy.get('[data-testid^="compare-agentx-model-"] [data-new-badge="agentx-ledger"]') + .should('have.length', 6) + .each(($badge) => expect($badge.text()).to.equal('NEW')); + }); + cy.get('[data-testid="compare-agentx-overview-link"]').click(); + cy.location('pathname').should('eq', '/overview'); }); it('navigates to submissions from the landing CTA', () => { diff --git a/packages/app/cypress/e2e/nudge-system.cy.ts b/packages/app/cypress/e2e/nudge-system.cy.ts index a6d4fe031..530d8a775 100644 --- a/packages/app/cypress/e2e/nudge-system.cy.ts +++ b/packages/app/cypress/e2e/nudge-system.cy.ts @@ -14,8 +14,7 @@ function clearAllNudgeStorage(win: Cypress.AUTWindow) { const keys = [ 'inferencex-starred', 'inferencex-star-modal-dismissed', - 'inferencex-kimi-k3-modal-dismissed', - 'inferencex-kimi-k3-banner-dismissed', + 'inferencex-openai-rubin-banner-dismissed', 'inferencex-reproducibility-nudge-shown', 'inferencex-star-nudge-shown', 'inferencex-export-nudge-shown', @@ -43,90 +42,86 @@ beforeEach(() => { // --------------------------------------------------------------------------- describe('Landing nudges — modals', () => { - it('shows launch modal and banner simultaneously on fresh first load', () => { + it('shows the launch banner on fresh first load', () => { cy.visit('/', { onBeforeLoad: clearAllNudgeStorage, }); - // Banner (inline) and modal (overlay) occupy independent slots - cy.get('[data-testid="launch-banner"]').should('be.visible'); - cy.get('[data-testid="launch-modal"]') + cy.get('[data-testid="launch-banner"]') .should('be.visible') - .and('match', 'div[role="dialog"][aria-modal="false"]'); - // Only one overlay at a time — star modal should not appear - cy.get('[data-testid="github-star-modal"]').should('not.exist'); + .and('contain.text', "OpenAI's Latest In House Chip verus Rubin NVL72") + .and( + 'contain.text', + 'Compare Jalapeño (Teacup) with Vera Rubin (July) NVL72 on DeepSeek R1 at 8K / 1K.', + ) + .and('contain.text', 'View results'); + // Banner + header-nav badges, plus the six AgentX hero ledger rows — the + // shared pill must render at the same fixed size everywhere it appears. + cy.get('[data-new-badge]') + .should('have.length', 8) + .then(($badges) => { + const sizes = [...$badges].map((badge) => { + const rect = badge.getBoundingClientRect(); + return { width: rect.width, height: rect.height }; + }); + for (const size of sizes) { + expect(size.width).to.eq(sizes[0].width); + expect(size.height).to.eq(sizes[0].height); + } + expect(sizes[0]).to.deep.eq({ width: 32, height: 16 }); + + for (const badge of $badges) { + const label = badge.querySelector('[data-new-badge-label]'); + expect(label, 'badge label').not.to.eq(null); + + const badgeRect = badge.getBoundingClientRect(); + const labelRect = label!.getBoundingClientRect(); + const horizontalOffset = + labelRect.left + labelRect.width / 2 - (badgeRect.left + badgeRect.width / 2); + const verticalOffset = + labelRect.top + labelRect.height / 2 - (badgeRect.top + badgeRect.height / 2); + + expect(horizontalOffset).to.be.closeTo(0, 0.1); + expect(verticalOffset).to.eq(0); + + // The label box can sit dead centre while the glyphs themselves spill + // out of it, so measure the rendered text and not just its container. + const range = badge.ownerDocument.createRange(); + range.selectNodeContents(label!); + const inkRect = range.getBoundingClientRect(); + const inkOffset = + inkRect.left + inkRect.width / 2 - (badgeRect.left + badgeRect.width / 2); + + expect(inkRect.left, 'label ink stays inside the pill').to.be.at.least(badgeRect.left); + expect(inkRect.right, 'label ink stays inside the pill').to.be.at.most(badgeRect.right); + expect(inkOffset, 'label ink is centred').to.be.closeTo(0, 0.5); + } + }); }); - it('dismissing launch modal persists — not shown on reload', () => { - cy.visit('/', { + it('localizes the Rubin comparison banner title in Chinese', () => { + cy.visit('/zh', { onBeforeLoad: clearAllNudgeStorage, }); - cy.get('[data-testid="launch-modal"]').should('be.visible'); - cy.get('[data-testid="launch-modal-dismiss"]').click(); - cy.get('[data-testid="launch-modal"]').should('not.exist'); - - cy.reload(); - cy.get('[data-testid="launch-modal"]').should('not.exist'); + cy.get('[data-testid="launch-banner"]') + .should('be.visible') + .and('contain.text', 'OpenAI 最新自研芯片对比 Rubin NVL72') + .and( + 'contain.text', + '对比 Jalapeño (Teacup) 与 Vera Rubin (July) NVL72 在 DeepSeek R1 8K / 1K 工作负载下的表现。', + ) + .and('contain.text', '查看结果'); }); - it('launch modal Explore action persists dismissal in localStorage', () => { + it('does not float a duplicate GitHub star card over the footer', () => { + // The persistent star CTA lives in the footer grid (footer-star-cta) and + // the header; the old immediate star modal duplicated it and covered the + // footer, so it must stay gone. cy.visit('/', { onBeforeLoad: clearAllNudgeStorage, }); - cy.get('[data-testid="launch-modal"]').should('be.visible'); - - // The action writes localStorage synchronously before navigation. Check - // the storage value before the navigation completes; combined with the - // "Maybe Later" persists-across-reload test, this covers the explore - // path without needing to stub window.location. - cy.get('[data-testid="launch-modal-action"]').click(); - cy.window().then((win) => { - expect(win.localStorage.getItem('inferencex-kimi-k3-modal-dismissed')).to.eq('1'); - }); - }); - - it('shows star modal when launch modal was previously dismissed', () => { - cy.visit('/', { - onBeforeLoad(win) { - clearAllNudgeStorage(win); - win.localStorage.setItem('inferencex-kimi-k3-modal-dismissed', '1'); - }, - }); - cy.get('[data-testid="launch-modal"]').should('not.exist'); - cy.get('[data-testid="github-star-modal"]').should('be.visible'); - }); - - it('star modal dismiss uses timed strategy — re-shows after expiry', () => { - cy.visit('/', { - onBeforeLoad(win) { - clearAllNudgeStorage(win); - win.localStorage.setItem('inferencex-kimi-k3-modal-dismissed', '1'); - }, - }); - cy.get('[data-testid="github-star-modal"]').should('be.visible'); - cy.get('[data-testid="github-star-modal-dismiss"]').click(); - cy.get('[data-testid="github-star-modal"]').should('not.exist'); - - cy.window().then((win) => { - const value = win.localStorage.getItem('inferencex-star-modal-dismissed'); - expect(value).to.not.equal(null); - expect(Number(value)).to.be.greaterThan(0); - }); - }); - - it('starring permanently suppresses both star modal and star nudge', () => { - cy.visit('/', { - onBeforeLoad(win) { - clearAllNudgeStorage(win); - win.localStorage.setItem('inferencex-kimi-k3-modal-dismissed', '1'); - }, - }); - cy.get('[data-testid="github-star-modal"]').should('be.visible'); - cy.get('[data-testid="github-star-modal-action"]').click(); + cy.get('[data-testid="launch-banner"]').should('be.visible'); cy.get('[data-testid="github-star-modal"]').should('not.exist'); - - cy.window().then((win) => { - expect(win.localStorage.getItem('inferencex-starred')).to.eq('1'); - }); + cy.get('[data-testid="footer-star-cta"]').should('exist'); }); }); @@ -137,16 +132,8 @@ describe('Landing nudges — modals', () => { describe('Landing nudges — banner', () => { it('shows launch banner on landing page', () => { cy.visit('/', { - onBeforeLoad(win) { - clearAllNudgeStorage(win); - // Dismiss modals so the banner (highest priority at 60) is the active nudge. - // Actually the banner has priority 60 > launch modal 50, so it should show first. - // But the engine only shows one nudge at a time; the banner wins because of priority. - }, + onBeforeLoad: clearAllNudgeStorage, }); - // The banner has the highest priority (60), so it should appear. - // However, NudgeEngine only shows one nudge at a time. - // With immediate triggers and priority 60 > 50 > 40, the banner wins. cy.get('[data-testid="launch-banner"]').should('be.visible'); }); @@ -178,7 +165,7 @@ describe('Landing nudges — banner', () => { cy.get('[data-testid="launch-banner"]').should('be.visible'); cy.window().then((win) => { // Only the X button should persist a dismissal — show alone must not. - expect(win.localStorage.getItem('inferencex-kimi-k3-banner-dismissed')).to.eq(null); + expect(win.localStorage.getItem('inferencex-openai-rubin-banner-dismissed')).to.eq(null); }); }); @@ -189,11 +176,16 @@ describe('Landing nudges — banner', () => { cy.get('[data-testid="launch-banner"]').should('be.visible'); cy.get('[data-testid="launch-banner"]').click(); cy.location('pathname', { timeout: 10000 }).should('eq', '/inference'); + cy.location('search') + .should('include', 'g_model=DeepSeek-R1-0528') + .and('include', 'i_seq=8k%2F1k') + .and('include', 'i_prec=fp4') + .and('include', 'i_metric=y_outputTputPerMw'); // Body click must not write the dismissal key — the banner should still // render on a fresh visit to landing. cy.window().then((win) => { - expect(win.localStorage.getItem('inferencex-kimi-k3-banner-dismissed')).to.eq(null); + expect(win.localStorage.getItem('inferencex-openai-rubin-banner-dismissed')).to.eq(null); }); cy.visit('/'); @@ -261,6 +253,24 @@ describe('Dashboard nudges — filter-hint toast', () => { cy.get('[data-testid="filter-hint-nudge"]').should('not.exist'); }); + it('arms the filter hint before a client transition into inference', () => { + cy.visit('/evaluation', { onBeforeLoad: suppressCompetingDashboardToasts }); + cy.get('[data-testid="filter-hint-nudge"]').should('not.exist'); + + cy.get('[data-testid="tab-trigger-inference"]').click(); + cy.location('pathname').should('eq', '/inference'); + cy.get('[data-testid="filter-hint-nudge"]', { timeout: 3000 }).should('be.visible'); + }); + + it('re-evaluates the localized filter hint after a Chinese tab transition', () => { + cy.visit('/zh/evaluation', { onBeforeLoad: suppressCompetingDashboardToasts }); + cy.get('[data-testid="tab-trigger-inference"]').click(); + cy.location('pathname').should('eq', '/zh/inference'); + cy.get('[data-testid="filter-hint-nudge"]', { timeout: 3000 }) + .should('be.visible') + .and('contain.text', '图表太拥挤?'); + }); + it('dismissal persists to localStorage and the nudge stays gone after reload', () => { cy.visit('/inference', { onBeforeLoad: suppressCompetingDashboardToasts }); cy.get('[data-testid="filter-hint-nudge"]', { timeout: 5000 }).should('be.visible'); @@ -344,7 +354,6 @@ describe('Nudge scope isolation', () => { cy.visit('/inference', { onBeforeLoad: clearAllNudgeStorage, }); - cy.get('[data-testid="launch-modal"]').should('not.exist'); cy.get('[data-testid="github-star-modal"]').should('not.exist'); cy.get('[data-testid="launch-banner"]').should('not.exist'); }); @@ -354,8 +363,7 @@ describe('Nudge scope isolation', () => { onBeforeLoad(win) { clearAllNudgeStorage(win); // Dismiss all landing nudges so nothing blocks visibility checks - win.localStorage.setItem('inferencex-kimi-k3-modal-dismissed', '1'); - win.localStorage.setItem('inferencex-kimi-k3-banner-dismissed', '1'); + win.localStorage.setItem('inferencex-openai-rubin-banner-dismissed', '1'); win.localStorage.setItem('inferencex-starred', '1'); }, }); diff --git a/packages/app/cypress/e2e/overlay-legend-remove.cy.ts b/packages/app/cypress/e2e/overlay-legend-remove.cy.ts index d52df3965..07c555f80 100644 --- a/packages/app/cypress/e2e/overlay-legend-remove.cy.ts +++ b/packages/app/cypress/e2e/overlay-legend-remove.cy.ts @@ -4,7 +4,7 @@ * * Regression: with an overlay active, the chart reads official visibility from * `localOfficialOverride` (the unified overlay-mode selection), but the legend - * X routed straight to InferenceContext's `removeHwType`, which mutates + * X routed straight to the inference action domain's `removeHwType`, which mutates * `activeHwTypes` — a set the chart ignores in overlay mode. The click * appeared to do nothing. The legend toggle already had the overlay-aware * split (`unifiedToggle`); the X now shares it (`handleRemoveHwType`). @@ -22,9 +22,10 @@ describe('Official legend X works while an unofficial overlay is loaded', () => // Use distinct hardware so the engine-comparison exclusion policy does // not resolve the official and unofficial rows as one competing family. interceptOverlayRun({ overlayHardware: 'h100' }); - // The agentic default mode is E2E Normalized Interactivity (which suppresses overlays and - // fetches derived metrics) — stub the fetch, then switch to Interactivity - // where the overlay renders. + // Agentic charts default to Interactivity, where the overlay renders. The + // derived-metrics fetch now happens only under E2E Normalized Interactivity, + // so the stub below is a guard against a stray request rather than a + // dependency of this suite. interceptDerivedAgenticMetrics(); cy.visit(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90`, { onBeforeLoad(win) { @@ -33,9 +34,11 @@ describe('Official legend X works while an unofficial overlay is loaded', () => }, }); cy.wait('@unofficialRun'); - // Interactivity is nested under the Advanced menu on agentic charts. - cy.get('[data-testid="x-axis-mode-advanced"]').click(); - cy.get('[data-testid="x-axis-mode-interactivity"]').click(); + // Every x-axis metric is a top-level tab on agentic charts, and Interactivity + // is the default — clicked anyway so the suite does not depend on that. + cy.get('[data-testid="x-axis-mode-interactivity"]') + .click() + .should('have.attr', 'data-state', 'active'); cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should( 'have.length', @@ -43,6 +46,16 @@ describe('Official legend X works while an unofficial overlay is loaded', () => ); }); + // Cypress clears intercepts between tests, so the derived-metrics stub is + // re-registered per test rather than once in `before`. Until #736 the + // agentic default was E2E Normalized Interactivity, so the fetch happened + // during `before` while the stub was still alive and React Query held the + // result for the rest of the spec. The default no longer fetches, so any + // later switch into that mode issues a fresh request. + beforeEach(() => { + interceptDerivedAgenticMetrics(); + }); + it('shows official points and an official legend entry initially', () => { cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => { expect(countVisible($dots), 'visible official points').to.be.greaterThan(0); @@ -69,19 +82,50 @@ describe('Official legend X works while an unofficial overlay is loaded', () => cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => { expect(countVisible($dots), 'visible official points after remove').to.eq(0); }); - // The overlay series is untouched. Optimal Only still hides its trace-less - // points because unofficial rows cannot join the canonical frontier. + // The overlay series is untouched — hiding an official SKU must not disturb + // it. All five overlay points stay visible under Optimal Only: they are + // non-dominated on the interactivity axes, and since #736 lacking persisted + // traces no longer excludes them from the frontier. cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => { - expect(countVisible($pts), 'visible overlay X markers').to.eq(0); + expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length); }); // Inactive row: the hover affordance flips to the "+" restore indicator // (explicit "clicking the name brings it back"), and the Hide X is gone. cy.get('[data-testid="chart-legend"] [title^="Show B300"]').should('exist'); + // Best per SKU lives in the Quick Filters dialog now. + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filter-best-per-sku"]').should( + 'have.attr', + 'data-state', + 'unchecked', + ); + cy.contains('button', 'Done').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('not.exist'); cy.get( '[data-testid="chart-legend"] [role="button"][aria-label^="Hide"][aria-label*="B300"]', ).should('not.exist'); }); + it('keeps the official SKU hidden when chart metrics change', () => { + cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); + cy.contains('[role="option"]', 'Cost per Million Total Tokens (Owning - Hyperscaler)').click({ + force: true, + }); + + cy.get('[data-testid="chart-legend"] [title^="Show B300"]').should('exist'); + cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => { + expect(countVisible($dots), 'visible official points after Y-axis change').to.eq(0); + }); + + cy.get('[data-testid="x-axis-mode-ttft"]').click().should('have.attr', 'data-state', 'active'); + cy.get('[data-testid="chart-legend"] [title^="Show B300"]').should('exist'); + cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => { + expect(countVisible($pts), 'visible overlay points after metric changes').to.be.greaterThan( + 0, + ); + }); + }); + it('re-activating the SKU from the legend restores the official points', () => { cy.get('[data-testid="chart-legend"]').contains('B300').click(); cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => { diff --git a/packages/app/cypress/e2e/overlay-optimal-only.cy.ts b/packages/app/cypress/e2e/overlay-optimal-only.cy.ts index 520567806..f9f388c4d 100644 --- a/packages/app/cypress/e2e/overlay-optimal-only.cy.ts +++ b/packages/app/cypress/e2e/overlay-optimal-only.cy.ts @@ -1,22 +1,31 @@ /** - * Unofficial runs do not have persisted request traces, so they cannot join - * the canonical E2E Normalized Interactivity frontier. Optimal Only must hide - * them on every AgentX axis; Show All remains the explicit way to inspect them. + * Optimal Only must filter overlay (unofficial-run) points by the same rule it + * applies to official ones: the Pareto frontier of the *selected* axes. + * + * Until #736 the rule was different — unofficial runs have no persisted request + * traces, so they could never join the canonical E2E Normalized Interactivity + * frontier and Optimal Only hid every one of them. That gating is gone, so an + * overlay point now survives exactly when it is non-dominated on the axes on + * screen, and Show All remains the way to inspect the rest. */ import { interceptDerivedAgenticMetrics, unlockAgenticGate } from '../support/e2e'; import { countVisible, + DOMINATED_CONFIG, interceptOverlayRun, OVERLAY_RUN_ID, REAL_CONFIGS, } from '../support/overlay-fixtures'; -describe('Overlay points follow canonical Optimal Only policy (agentic interactivity)', () => { +// The five real configs are all non-dominated on the interactivity axes, so a +// sixth, deliberately dominated point is what gives Optimal Only something to +// remove. Without it this suite would assert the same count either way and pass +// no matter what the filter did. +const OVERLAY_CONFIGS = [...REAL_CONFIGS, DOMINATED_CONFIG]; + +describe('Overlay points follow Optimal Only on the selected axes', () => { before(() => { - interceptOverlayRun(); - // The agentic default mode is E2E Normalized Interactivity (which suppresses overlays and - // fetches derived metrics) — stub the fetch, then switch to the - // Interactivity mode this suite is about. + interceptOverlayRun({ overlayConfigs: OVERLAY_CONFIGS }); interceptDerivedAgenticMetrics(); cy.visit(`/inference?unofficialrun=${OVERLAY_RUN_ID}&i_seq=agentic-traces&i_pctl=p90`, { onBeforeLoad(win) { @@ -25,30 +34,37 @@ describe('Overlay points follow canonical Optimal Only policy (agentic interacti }, }); cy.wait('@unofficialRun'); - // Interactivity is nested under the Advanced menu on agentic charts. - cy.get('[data-testid="x-axis-mode-advanced"]').click(); - cy.get('[data-testid="x-axis-mode-interactivity"]').click(); + // Every x-axis metric is a top-level tab on agentic charts, and Interactivity + // is the default — clicked anyway so the suite does not depend on that. + cy.get('[data-testid="x-axis-mode-interactivity"]') + .click() + .should('have.attr', 'data-state', 'active'); cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('have.attr', 'data-state', 'active') - .and('contain.text', 'Interactivity'); + // All six are rendered; visibility is what Optimal Only changes. cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should( 'have.length', - REAL_CONFIGS.length, + OVERLAY_CONFIGS.length, ); }); - it('hides trace-less overlay points in the default Optimal Only view', () => { + // Cypress clears intercepts between tests, so the derived-metrics stub is + // re-registered per test rather than once in `before`. Until #736 the agentic + // default was E2E Normalized Interactivity, so the fetch happened during + // `before` while the stub was still alive and React Query held the result for + // the rest of the spec. The default no longer fetches. + beforeEach(() => { + interceptDerivedAgenticMetrics(); + }); + + it('drops only the dominated overlay point in the default Optimal Only view', () => { cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'checked'); - // The deterministic derived-metric stub puts all five official rows on the - // canonical frontier. cy.get('[data-testid="inference-chart-display"] svg .dot-group').should(($dots) => { expect(countVisible($dots), 'visible official points').to.eq(REAL_CONFIGS.length); }); - // Overlay rows have no persisted trace ids and therefore no canonical - // frontier membership. + // Five of six survive: every real config is non-dominated on these axes, and + // the overlay's trace-less rows are no longer excluded for lacking traces. cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => { - expect(countVisible($pts), 'visible overlay X markers').to.eq(0); + expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length); }); }); @@ -56,15 +72,15 @@ describe('Overlay points follow canonical Optimal Only policy (agentic interacti cy.get('#scatter-hide-non-optimal').click(); cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'unchecked'); cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => { - expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length); + expect(countVisible($pts), 'visible overlay X markers').to.eq(OVERLAY_CONFIGS.length); }); }); - it('re-hides trace-less overlay points when Optimal Only is re-enabled', () => { + it('re-drops the dominated overlay point when Optimal Only is re-enabled', () => { cy.get('#scatter-hide-non-optimal').click(); cy.get('#scatter-hide-non-optimal').should('have.attr', 'data-state', 'checked'); cy.get('[data-testid="inference-chart-display"] svg .unofficial-overlay-pt').should(($pts) => { - expect(countVisible($pts), 'visible overlay X markers').to.eq(0); + expect(countVisible($pts), 'visible overlay X markers').to.eq(REAL_CONFIGS.length); }); }); }); diff --git a/packages/app/cypress/e2e/overview-a11y.cy.ts b/packages/app/cypress/e2e/overview-a11y.cy.ts new file mode 100644 index 000000000..64ff36457 --- /dev/null +++ b/packages/app/cypress/e2e/overview-a11y.cy.ts @@ -0,0 +1,47 @@ +import type { Result } from 'axe-core'; + +const OVERVIEW_SURFACE = '[data-testid="overview-page"]'; + +const OVERVIEW_ROUTES = [ + { name: 'English hardware comparison', path: '/overview' }, + { name: 'Chinese hardware comparison', path: '/zh/overview' }, + { name: 'historical comparison', path: '/overview?compare=30d' }, +] as const; + +const OVERVIEW_THEMES = ['light', 'dark'] as const; + +const WCAG_AA_RULES = { + runOnly: { + type: 'tag' as const, + values: ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'], + }, +}; + +function reportViolations(violations: Result[]): void { + const summary = violations + .map( + ({ help, id, nodes }) => + `${id}: ${help}\n${nodes.map(({ target }) => ` ${target.join(' ')}`).join('\n')}`, + ) + .join('\n'); + throw new Error(`Overview accessibility violations:\n${summary}`); +} + +describe('Overview accessibility', () => { + for (const { name, path } of OVERVIEW_ROUTES) { + for (const theme of OVERVIEW_THEMES) { + it(`has no WCAG 2.0/2.1 A or AA violations in the ${name} view using the ${theme} theme`, () => { + cy.visit(path, { + onBeforeLoad(win) { + win.localStorage.setItem('theme', theme); + }, + }); + cy.get('html').should(theme === 'dark' ? 'have.class' : 'not.have.class', 'dark'); + cy.get(OVERVIEW_SURFACE).should('be.visible'); + + cy.injectAxe(); + cy.checkA11y(OVERVIEW_SURFACE, WCAG_AA_RULES, reportViolations); + }); + } + } +}); diff --git a/packages/app/cypress/e2e/overview.cy.ts b/packages/app/cypress/e2e/overview.cy.ts index 6ff5c79eb..875e9dd08 100644 --- a/packages/app/cypress/e2e/overview.cy.ts +++ b/packages/app/cypress/e2e/overview.cy.ts @@ -1,10 +1,13 @@ +import { TCO_SOURCE_TITLE, TCO_SOURCE_URL } from '@semianalysisai/inferencex-constants'; + // Order mirrors DEFAULT_MODELS (MODEL_CONFIG insertion order), which fixes the -// matrix row order. +// row order within each scenario group — AgentX rows lead the matrix, 8K/1K +// rows follow. const MODEL_LABELS = [ - 'DeepSeek V4 Pro 1.6T', + 'DeepSeek V4 Pro 0813 1.6T', 'Kimi K3 2.8T', 'MiniMax M3 428B', - 'GLM5.2', + 'GLM5.2/GLM5.3', 'Qwen3.5 397B', ]; @@ -19,24 +22,52 @@ const PLATFORM_HEADERS = [ ]; const SINGLE_TURN = 'single_turn_8k1k'; -/** Five models, three of them with both a single-turn and an AgentX row. */ -const MATRIX_ROWS = 8; +/** Six models: three with both a single-turn and an AgentX row, and three + * curated AgentX-only (Kimi K3, GLM 5.2, Qwen3.8-Flash-Next). */ +const MATRIX_ROWS = 9; const AGENTX = 'agentx'; const AGENTX_LABEL = 'Long Context Multi-Turn Realistic Agentic Scenario (AgentX)'; -const AGENTX_LABEL_ZH = '长上下文多轮真实智能体场景(AgentX)'; - -const PAGE_TITLE = 'Inference Cost per Million Tokens'; -const PAGE_TITLE_ZH = '推理每百万 token 成本'; -const SOURCE_NOTE = 'Source: InferenceX & SemiAnalysis Market July 2026 AI Cloud TCO Model'; -const SOURCE_LINK_TEXT = 'SemiAnalysis Market July 2026 AI Cloud TCO Model'; -const SOURCE_NOTE_ZH = '来源:InferenceX 与 SemiAnalysis Market July 2026 AI Cloud TCO Model'; -const SOURCE_HREF = 'https://semianalysis.com/ai-cloud-tco-model/'; +const AGENTX_LABEL_ZH = '长上下文、多轮交互的真实智能体场景(AgentX)'; +/** Shared by both locales: the scenario is named after its acronym. */ +const AGENTX_SHORT = 'AgentX'; + +const PAGE_TITLE = 'Agentic Inference Costs'; +const PAGE_TITLE_ZH = '智能体推理成本'; +const SOURCE_NOTE = `Source: InferenceX & ${TCO_SOURCE_TITLE}`; +const SOURCE_NOTE_ZH = `来源:InferenceX 与 ${TCO_SOURCE_TITLE}`; const SCOPE_METRIC = 'Hyperscaler cost'; const SCOPE_DIRECTION = '↓ Lower is better'; const SCOPE_LINE = `${SCOPE_METRIC} · ${SCOPE_DIRECTION} · ${SOURCE_NOTE}`; -const SCOPE_METRIC_ZH = '超大规模云(hyperscaler)成本'; +const SCOPE_METRIC_ZH = '超大规模云厂商成本'; const SCOPE_DIRECTION_ZH = '↓ 越低越好'; const SCOPE_LINE_ZH = `${SCOPE_METRIC_ZH} · ${SCOPE_DIRECTION_ZH} · ${SOURCE_NOTE_ZH}`; +const OVERVIEW_TIERS = [30, 50, 75, 100, 150, 200] as const; + +function selectOverviewTier(tier: (typeof OVERVIEW_TIERS)[number]) { + const targetIndex = OVERVIEW_TIERS.indexOf(tier); + // Unlike clicks, synthetic input events are not replayed by React, so a + // dispatch that lands before hydration finishes is silently dropped. The + // `.should` callback retries the whole dispatch until React has committed + // the selection, which only happens once the slider is interactive. + return cy.get('[data-testid="overview-tier-slider"]').should(([slider]) => { + const view = slider.ownerDocument.defaultView; + if (view === null) throw new Error('Slider has no window'); + const tracker = (slider as { _valueTracker?: { setValue: (next: string) => void } }) + ._valueTracker; + // React installs the value tracker when hydration commits the input. + expect(tracker, 'tier slider is hydrated').to.not.eq(undefined); + const nativeValueSetter = Object.getOwnPropertyDescriptor( + view.HTMLInputElement.prototype, + 'value', + )?.set; + if (nativeValueSetter === undefined) throw new Error('Range input has no native value setter'); + nativeValueSetter.call(slider, String(targetIndex)); + // Desync the tracker so React always registers the dispatched input. + tracker?.setValue(''); + slider.dispatchEvent(new view.Event('input', { bubbles: true })); + expect(slider.dataset.tier, 'slider committed tier').to.eq(String(tier)); + }); +} function expectNoHorizontalOverflow() { cy.document().then((doc) => { @@ -61,6 +92,52 @@ function expectNoHorizontalScroller(testId: string) { }); } +function stubFullscreenApi(win: Window) { + let fullscreenElement: Element | null = null; + const document = win.document; + const EventConstructor = document.defaultView?.Event ?? Event; + const ElementConstructor = document.defaultView?.Element ?? Element; + Object.defineProperty(document, 'fullscreenEnabled', { + configurable: true, + value: true, + }); + Object.defineProperty(document, 'fullscreenElement', { + configurable: true, + get: () => fullscreenElement, + }); + Object.defineProperty(document, 'exitFullscreen', { + configurable: true, + value: () => { + fullscreenElement = null; + document.dispatchEvent(new EventConstructor('fullscreenchange')); + return Promise.resolve(); + }, + }); + Object.defineProperty(ElementConstructor.prototype, 'requestFullscreen', { + configurable: true, + value() { + fullscreenElement = document.querySelector('[data-testid="overview-presentation-surface"]'); + document.dispatchEvent(new EventConstructor('fullscreenchange')); + return Promise.resolve(); + }, + }); +} + +/** + * The row header shows the acronym so the 22%-wide model column stays one line, + * and keeps the full scenario name for assistive tech and the hover title. Both + * strings are in the cell, so assert the two layers rather than their + * concatenation. Scenarios already named by a short label render one node. + */ +function expectAgentxScenario(fullLabel: string) { + cy.get('[data-testid="overview-model-scenario"]') + .should('have.attr', 'title', fullLabel) + .within(() => { + cy.get('.sr-only').should('have.text', fullLabel); + cy.get('[aria-hidden="true"]').should('have.text', AGENTX_SHORT); + }); +} + /** Visible dates and snapshot framing must be gone; evidence stays in labels. */ function expectNoVisibleDatesOrSnapshot() { cy.get('[data-testid="overview-pair-evidence-date"]').should('not.exist'); @@ -140,14 +217,14 @@ describe('Overview page', () => { 'preserved'; }); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.wait('@overviewJson'); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75'); + cy.location('search').should('eq', '?tier=75'); cy.window().its('__overviewNavigationSentinel').should('eq', 'preserved'); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '50').click(); + selectOverviewTier(50); cy.location('search').should('eq', ''); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.location('search').should('eq', '?tier=75'); cy.then(() => { expect(jsonRequests, 'one request; both visited selections are cached').to.equal(1); @@ -158,17 +235,17 @@ describe('Overview page', () => { .find('[data-overview-engine-scope="all"]') .click(); cy.wait('@overviewJson'); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75&engine=all'); + cy.location('search').should('eq', '?tier=75&engine=all'); cy.window().its('__overviewNavigationSentinel').should('eq', 'preserved'); cy.get('[data-overview-comparison="30d"]').click(); cy.wait('@overviewJson'); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75&engine=all&compare=30d'); + cy.location('search').should('eq', '?tier=75&engine=all&compare=30d'); cy.window().its('__overviewNavigationSentinel').should('eq', 'preserved'); cy.go('back'); cy.get('[data-overview-comparison="hardware"]').should('have.attr', 'aria-current', 'true'); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75&engine=all'); + cy.location('search').should('eq', '?tier=75&engine=all'); cy.then(() => { expect(rscRequests, 'selector and popstate RSC requests').to.equal(0); }); @@ -178,7 +255,7 @@ describe('Overview page', () => { cy.viewport(1280, 900); cy.visit('/overview'); cy.intercept('GET', '**/api/v1/overview*').as('overviewJson'); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.wait('@overviewJson'); cy.location('search').should('eq', '?tier=75'); @@ -211,16 +288,19 @@ describe('Overview page', () => { }); }).as('overviewJson'); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.get('[data-testid="overview-engine-scope-switcher"]') .find('[data-overview-engine-scope="all"]') .click(); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75&engine=all'); + cy.location('search').should('eq', '?tier=75&engine=all'); // The rendered state, not just the URL: the losing response must not win. - cy.get('[data-testid="overview-tier-switcher"] [aria-current="page"]', { + // Keeps its explicit timeout because it waits on the real API through + // `request.continue` plus the delay above — unlike the URL assertions + // around it, which the click resolves synchronously. + cy.get('[data-testid="overview-tier-slider"]', { timeout: 15_000, - }).should('have.text', '75'); + }).should('have.attr', 'data-tier', '75'); cy.get('[data-overview-engine-scope="all"]').should('have.attr', 'aria-current', 'true'); cy.location('search').should('eq', '?tier=75&engine=all'); }); @@ -234,15 +314,79 @@ describe('Overview page', () => { }); }).as('overviewJson'); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.get('[data-testid="overview-page"] [aria-busy="true"]').should('exist'); cy.wait('@overviewJson'); - cy.get('[data-testid="overview-page"] [aria-busy="true"]', { timeout: 15_000 }).should( - 'not.exist', + cy.get('[data-testid="overview-page"] [aria-busy="true"]').should('not.exist'); + }); + + it('keeps the last matrix and history entry when a popstate request fails', () => { + cy.viewport(1280, 900); + cy.visit('/overview'); + // A client-derived reference switch proves hydration and the popstate + // listener are established without warming the tier payload under test. + cy.get('[data-testid="overview-reference-select"]').click(); + cy.get('[data-overview-reference="b300"]').click(); + cy.location('search').should('eq', '?ref=b300'); + cy.intercept('GET', '**/api/v1/overview?tier=75', { statusCode: 500 }).as( + 'overviewPopstateFailure', ); + cy.window().then((win) => { + (win as Window & { __overviewNavigationSentinel?: string }).__overviewNavigationSentinel = + 'preserved'; + win.History.prototype.pushState.call( + win.history, + win.history.state, + '', + '/overview?tier=75&ref=b300', + ); + cy.wrap(win.history.length).as('overviewHistoryLength'); + }); + + cy.go('back'); + cy.location('search').should('eq', '?ref=b300'); + cy.go('forward'); + cy.wait('@overviewPopstateFailure'); + cy.get('[data-testid="overview-navigation-error"]') + .should('have.attr', 'role', 'alert') + .and( + 'have.text', + 'Could not load the selected comparison. Showing the last successfully loaded data.', + ); + cy.get('[data-testid="overview-tier-slider"]').should('have.attr', 'data-tier', '50'); + cy.get('[data-testid="overview-page"] [aria-busy="true"]').should('not.exist'); + cy.location('search').should('eq', '?tier=75&ref=b300'); + cy.get('@overviewHistoryLength').then((expectedHistoryLength) => { + cy.window().then((after) => { + expect(after.history.length).to.equal(expectedHistoryLength); + expect( + (after as Window & { __overviewNavigationSentinel?: string }) + .__overviewNavigationSentinel, + ).to.equal('preserved'); + }); + }); + }); + + it('shows a localized empty state after a selector returns no models', () => { + cy.viewport(1280, 900); + cy.request('/api/v1/overview?tier=75').then(({ body }) => { + cy.intercept('GET', '**/api/v1/overview?tier=75', { + statusCode: 200, + body: { ...body, models: [] }, + }).as('emptyOverview'); + cy.visit('/zh/overview'); + + selectOverviewTier(75); + cy.wait('@emptyOverview'); + cy.get('[data-testid="overview-empty-state"]') + .should('have.attr', 'role', 'status') + .and('have.text', '没有符合当前筛选条件的总览结果。'); + cy.get('[data-testid="overview-desktop-matrix"]').should('not.exist'); + cy.get('[data-testid="overview-mobile-list"]').should('not.exist'); + }); }); - it('rewrites one history entry when the overview request fails', () => { + it('keeps one history entry when the overview request fails', () => { cy.viewport(1280, 900); cy.visit('/inference'); cy.visit('/overview'); @@ -250,19 +394,27 @@ describe('Overview page', () => { cy.window().then((win) => { const before = win.history.length; - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.wait('@overviewJsonFailure'); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=75'); + cy.location('search').should('eq', '?tier=75'); // A plain `.should` would be satisfied by the transient extra entry. cy.window().then((after) => { expect(after.history.length - before, 'one entry for one selection').to.equal(1); }); + // The unsuccessful stop stays retryable: activating it again re-requests + // without stacking another history entry for the byte-identical href. + selectOverviewTier(75); + cy.wait('@overviewJsonFailure'); + cy.location('search').should('eq', '?tier=75'); + cy.window().then((after) => { + expect(after.history.length - before, 'retry adds no history entry').to.equal(1); + }); }); cy.go('back'); - cy.location('search', { timeout: 15_000 }).should('eq', ''); + cy.location('search').should('eq', ''); cy.go('back'); - cy.location('pathname', { timeout: 15_000 }).should('eq', '/inference'); + cy.location('pathname').should('eq', '/inference'); }); it('warms a hovered option and derives the reference without a request', () => { @@ -273,22 +425,27 @@ describe('Overview page', () => { jsonRequests += 1; }).as('overviewJson'); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '100').trigger('pointerover'); + cy.get('[data-testid="overview-engine-scope-switcher"]') + .find('[data-overview-engine-scope="all"]') + .trigger('pointerover'); cy.wait('@overviewJson'); cy.location('search').should('eq', ''); cy.then(() => { expect(jsonRequests, 'hover warms exactly one response').to.equal(1); }); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '100').click(); - cy.location('search').should('eq', '?tier=100'); + cy.get('[data-testid="overview-engine-scope-switcher"]') + .find('[data-overview-engine-scope="all"]') + .click(); + cy.location('search').should('eq', '?engine=all'); + cy.get('[data-overview-engine-scope="all"]').should('have.attr', 'aria-current', 'true'); cy.then(() => { expect(jsonRequests, 'the click reuses the warmed response').to.equal(1); }); cy.get('[data-testid="overview-reference-select"]').click(); cy.get('[data-overview-reference="b300"]').click(); - cy.location('search', { timeout: 15_000 }).should('eq', '?tier=100&ref=b300'); + cy.location('search').should('eq', '?engine=all&ref=b300'); cy.get('[data-overview-comparison="hardware"]').should('contain.text', 'vs B300'); cy.then(() => { expect(jsonRequests, 'a reference change is derived, not fetched').to.equal(1); @@ -300,6 +457,8 @@ describe('Overview page', () => { cy.visit('/overview'); cy.get('[data-overview-comparison="30d"]').click(); + // Explicit timeout retained: this waits on a real /api/v1/overview response, + // not a client-side URL change. cy.get('[data-overview-comparison="30d"]', { timeout: 15_000 }).should( 'have.attr', 'aria-current', @@ -321,7 +480,8 @@ describe('Overview page', () => { desktopModel('gpt-oss-120b') .find('[data-testid="overview-model-category-badge"]') .should('have.attr', 'data-category', 'deprecated') - .and('contain.text', 'Deprecated'); + .and('contain.text', 'Deprecated') + .and('contain.text', 'Model is no longer actively benchmarked.'); desktopModel('DeepSeek-R1-0528') .find('[data-testid="overview-model-category-badge"]') .should('have.attr', 'data-category', 'maintenance'); @@ -335,7 +495,7 @@ describe('Overview page', () => { ); }); - cy.get('[data-testid="overview-tier-switcher"]').contains('a', '75').click(); + selectOverviewTier(75); cy.location('search').should('eq', '?tier=75&models=all'); cy.get('[data-testid="overview-desktop-model"][data-model="gpt-oss-120b"]').should('exist'); @@ -350,13 +510,16 @@ describe('Overview page', () => { cy.viewport(1280, 900); cy.visit('/zh/overview?models=all'); - cy.get('[data-testid="overview-model-scope-toggle"]').should( - 'contain.text', - '隐藏已弃用与维护模式模型', - ); + // The chip carries the short label; the full sentence stays on the + // accessible name and hover title. + cy.get('[data-testid="overview-model-scope-toggle"]') + .should('contain.text', '隐藏停用模型') + .find('[data-overview-model-scope="default"]') + .should('have.attr', 'aria-label', '隐藏已弃用与维护模式模型'); desktopModel('gpt-oss-120b') .find('[data-testid="overview-model-category-badge"]') - .should('contain.text', '已弃用'); + .should('contain.text', '已弃用') + .and('contain.text', '目前已不再对该模型进行持续基准测试。'); }); it('uses a selectable hardware reference and preserves it across overview controls', () => { @@ -379,25 +542,46 @@ describe('Overview page', () => { platform('b200').find('[data-testid="overview-cost-delta"]').should('exist'); }); - cy.get('[data-testid="overview-tier-switcher"]') - .contains('a', '100') - .should('have.attr', 'href', '/overview?tier=100&ref=b300'); + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100&ref=b300'); cy.get('[data-testid="overview-engine-scope-switcher"]') .find('[data-overview-engine-scope="all"]') - .should('have.attr', 'href', '/overview?engine=all&ref=b300'); + .should('have.attr', 'href', '/overview?tier=100&engine=all&ref=b300'); cy.get('[data-overview-comparison="30d"]').should( 'have.attr', 'href', - '/overview?ref=b300&compare=30d', + '/overview?tier=100&ref=b300&compare=30d', ); cy.get('[data-testid="language-toggle"]') - .should('have.attr', 'href', '/zh/overview?ref=b300') + .should('have.attr', 'href', '/zh/overview?tier=100&ref=b300') .click(); cy.location('pathname').should('eq', '/zh/overview'); - cy.location('search').should('eq', '?ref=b300'); + cy.location('search').should('eq', '?tier=100&ref=b300'); cy.get('[data-overview-comparison="hardware"]').should('contain.text', '对比 B300'); }); + it('sends one RSC request when a slow language switch preserves overview state', () => { + let zhOverviewRscRequests = 0; + + cy.intercept('GET', '/zh/overview*', (request) => { + if (request.query._rsc === undefined && request.headers.rsc !== '1') return; + + zhOverviewRscRequests += 1; + request.continue((response) => { + response.setDelay(600); + }); + }); + + cy.visit('/overview?tier=100&ref=b300'); + cy.get('[data-testid="language-toggle"]') + .should('have.attr', 'href', '/zh/overview?tier=100&ref=b300') + .click(); + + cy.location('pathname').should('eq', '/zh/overview'); + cy.location('search').should('eq', '?tier=100&ref=b300'); + cy.then(() => expect(zhOverviewRscRequests).to.equal(1)); + }); + it('uses rack SKU labels when GB200 or GB300 is the comparison reference', () => { cy.viewport(1280, 900); cy.visit('/overview?ref=gb200'); @@ -410,8 +594,11 @@ describe('Overview page', () => { cy.viewport(1280, 900); cy.visit('/overview'); - cy.get('[data-testid="overview-page"]') - .children('[data-testid="overview-comparison-switcher"]') + // The tabs sit inside the surface handed to the Fullscreen API so that + // presenting keeps them with the matrix, which costs them their old spot as + // a direct child of the page. + cy.get('[data-testid="overview-presentation-surface"]') + .find('[data-testid="overview-comparison-switcher"]') .should('have.length', 1) .and('have.class', 'justify-center'); cy.get('[data-testid="overview-comparison-switcher"]') @@ -440,17 +627,16 @@ describe('Overview page', () => { cy.get('[data-testid="overview-desktop-matrix"] thead').should('contain.text', 'B200'); cy.get('[data-testid="overview-desktop-matrix"] thead').should('not.contain.text', 'Reference'); - cy.get('[data-testid="overview-tier-switcher"]') - .contains('a', '100') - .should('have.attr', 'href', '/overview?tier=100&compare=30d'); + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100&compare=30d'); cy.get('[data-testid="overview-engine-scope-switcher"]') .find('[data-overview-engine-scope="all"]') - .should('have.attr', 'href', '/overview?engine=all&compare=30d'); + .should('have.attr', 'href', '/overview?tier=100&engine=all&compare=30d'); cy.get('[data-testid="language-toggle"]') - .should('have.attr', 'href', '/zh/overview?compare=30d') + .should('have.attr', 'href', '/zh/overview?tier=100&compare=30d') .click(); cy.location('pathname').should('eq', '/zh/overview'); - cy.location('search').should('eq', '?compare=30d'); + cy.location('search').should('eq', '?tier=100&compare=30d'); cy.get('[data-testid="overview-comparison-switcher"]') .should('have.attr', 'aria-label', '对比方式') .within(() => { @@ -459,8 +645,109 @@ describe('Overview page', () => { .should('have.attr', 'aria-current', 'true') .and('have.text', '对比 1 个月前'); }); - cy.contains('当前成本及其相对 30–60 天前最近一次有效平台结果的变化。').should('exist'); - cy.contains('缺少有效 30 天对比的平台仅显示当前成本。').should('exist'); + // The prose is editorial; the stable contract is that both history notes + // render the selected window while the caption also renders its upper bound. + cy.get('[data-testid="overview-methodology"] p') + .eq(0) + .should('be.visible') + .and('contain.text', '30') + .should('contain.text', '60'); + cy.get('[data-testid="overview-methodology"] p') + .eq(1) + .should('be.visible') + .and('contain.text', '30'); + }); + + it('keeps client-only presentation intent across selectors without forcing fullscreen on load', () => { + cy.viewport(1280, 900); + cy.visit('/overview?tier=75&present=1&utm_source=deck#matrix', { + onBeforeLoad: stubFullscreenApi, + }); + + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'false', + ); + cy.get('[data-testid="language-toggle"]').should( + 'have.attr', + 'href', + '/zh/overview?tier=75&present=1&utm_source=deck', + ); + + cy.get('[data-testid="overview-present-toggle"]').click(); + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'true', + ); + cy.location('search').should('eq', '?tier=75&present=1&utm_source=deck'); + cy.location('hash').should('eq', '#matrix'); + + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100&present=1&utm_source=deck'); + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'true', + ); + + cy.get('[data-overview-comparison="30d"]').click(); + cy.location('search').should('eq', '?tier=100&compare=30d&present=1&utm_source=deck'); + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'true', + ); + + cy.get('[data-testid="overview-present-toggle"]').click(); + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'false', + ); + cy.location('search').should('eq', '?tier=100&compare=30d&utm_source=deck'); + cy.location('hash').should('eq', '#matrix'); + cy.get('[data-testid="language-toggle"]').should( + 'have.attr', + 'href', + '/zh/overview?tier=100&compare=30d&utm_source=deck', + ); + }); + + it('reasserts presentation intent when browser history changes during fullscreen', () => { + cy.viewport(1280, 900); + cy.visit('/overview?tier=75', { onBeforeLoad: stubFullscreenApi }); + + // A selector push creates a real same-document history entry. + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100'); + cy.get('[data-testid="overview-present-toggle"]').click(); + cy.location('search').should('eq', '?tier=100&present=1'); + + cy.window().then((win) => win.history.back()); + cy.location('search').should('eq', '?tier=75&present=1'); + cy.get('[data-testid="overview-presentation-surface"]').should( + 'have.attr', + 'data-presenting', + 'true', + ); + }); + + it('leaves comparison paging to a focused presentation select option', () => { + cy.viewport(1280, 900); + cy.visit('/overview?compare=30d', { onBeforeLoad: stubFullscreenApi }); + cy.get('[data-testid="overview-present-toggle"]').click(); + + cy.get('[data-testid="overview-history-window-select"]').click(); + cy.get('[role="option"]').first().focus().trigger('keydown', { + key: 'ArrowRight', + code: 'ArrowRight', + bubbles: true, + }); + + cy.location('search').should('eq', '?compare=30d&present=1'); + cy.get('[data-overview-comparison="30d"]').should('have.attr', 'aria-current', 'true'); }); it('switches the history window through the embedded selector', () => { @@ -469,7 +756,7 @@ describe('Overview page', () => { cy.get('[data-testid="overview-history-window-select"]').click(); cy.get('[data-overview-window="7d"]').click(); - cy.location('search', { timeout: 15_000 }).should('eq', '?compare=7d'); + cy.location('search').should('eq', '?compare=7d'); cy.get('[data-overview-comparison="7d"]').should('have.attr', 'aria-current', 'true'); cy.contains( 'Current cost and change versus the latest validated platform result 7–14 days earlier.', @@ -477,7 +764,7 @@ describe('Overview page', () => { cy.get('[data-testid="overview-history-window-select"]').click(); cy.get('[data-overview-window="90d"]').click(); - cy.location('search', { timeout: 15_000 }).should('eq', '?compare=90d'); + cy.location('search').should('eq', '?compare=90d'); cy.contains( 'Current cost and change versus the latest validated platform result 90–180 days earlier.', ).should('exist'); @@ -573,8 +860,11 @@ describe('Overview page', () => { cy.get('[data-testid="inference-chart-display"]').should('exist'); cy.get('[data-testid="model-selector"]').click(); - cy.contains('[role="option"]', 'DeepSeek V4 Pro 1.6T').click(); - cy.get('[data-testid="model-selector"]').should('contain.text', 'DeepSeek V4 Pro 1.6T'); + cy.contains('[role="option"]', 'DeepSeek V4 Pro 0813 1.6T').click(); + cy.get('[data-testid="model-selector"]').should( + 'contain.text', + 'DeepSeek V4 Pro 0813 1.6T', + ); cy.get('[data-testid="inference-chart-display"]').should( 'not.contain.text', 'No data available', @@ -598,6 +888,47 @@ describe('Overview page', () => { }); }); + it('keeps a benchmark result usable without comparison dates', () => { + cy.viewport(1280, 900); + cy.visit('/overview'); + + desktopModel('Qwen-3.5-397B-A17B', SINGLE_TURN) + .find( + '[data-testid="overview-pair-value"][data-hardware="mi355x"] [data-testid="overview-cost-evidence-link"]', + ) + .then(($link) => { + const href = String($link.attr('href')); + expect(href).not.to.contain('i_dates='); + cy.visit(href); + + cy.contains('Comparison Date Range').should('be.visible'); + cy.contains('button', 'Select date range') + .should('not.have.class', 'animate-pulse') + .and('not.have.class', 'border-red-500'); + cy.get('[data-testid="scatter-graph"]') + .should('be.visible') + .find('svg .dot-group') + .should('have.length.greaterThan', 0); + cy.contains('Select a date range or add a run to view chip comparison').should('not.exist'); + }); + }); + + it('does not require comparison dates when opening AgentX details', () => { + cy.viewport(1280, 900); + cy.visit('/overview'); + + desktopModel('Qwen-3.5-397B-A17B', AGENTX) + .contains('a', 'View details') + .then(($link) => { + const href = String($link.attr('href')); + expect(href).to.contain('i_seq=agentic-traces'); + expect(href).not.to.contain('i_dates='); + cy.visit(href); + + cy.contains('Comparison Date Range').should('not.exist'); + }); + }); + it('keeps the historical comparison complete and non-scrolling across desktop, tablet and phone', () => { for (const width of [320, 390, 768, 1024, 1279, 1280, 1440]) { cy.viewport(width, 900); @@ -626,6 +957,48 @@ describe('Overview page', () => { } }); + // Clicking the control, not just building its href: the matrix reads from a + // client data cache, and a cache keyed without the row params moves the + // address bar while leaving every row on screen. + it('narrows the matrix when the row filter is clicked in either comparison mode', () => { + cy.viewport(1280, 900); + + for (const [href, attribute, key, sentence] of [ + ['/overview?compare=30d', 'data-overview-row-scope', 'rows=changed', 'no 30-day change'], + [ + '/overview', + 'data-overview-hardware-row-scope', + 'hwrows=priced', + 'no result on any platform', + ], + ] as const) { + cy.visit(href); + cy.get('[data-testid="overview-desktop-model"]').should('have.length', MATRIX_ROWS); + + cy.get(`a[${attribute}]`) + .should('contain.text', 'Hide ') + .invoke('text') + .then((label) => { + const hidden = Number(/\d+/.exec(label)?.[0]); + expect(hidden, `hidden row count in "${label}"`).to.be.greaterThan(0); + + cy.get(`a[${attribute}]`).click(); + cy.location('search').should('contain', key); + cy.get('[data-testid="overview-desktop-model"]').should( + 'have.length', + MATRIX_ROWS - hidden, + ); + // The action label names the click, so it flips once the scope + // lands; the chip keeps the short label and the counted sentence + // rides the accessible name. + cy.get(`a[${attribute}]`) + .should('contain.text', 'Show all rows') + .and('contain.text', String(hidden)) + .and('have.attr', 'aria-label', `Show ${hidden} rows with ${sentence}`); + }); + } + }); + it('defaults to community engine scope and switches with canonical links preserving tier and locale', () => { cy.viewport(1280, 900); cy.visit('/overview'); @@ -655,7 +1028,7 @@ describe('Overview page', () => { }); desktopModel('GLM-5.2').within(() => { - cy.get('[data-testid="overview-model-scenario"]').should('have.text', AGENTX_LABEL); + expectAgentxScenario(AGENTX_LABEL); cy.get('[data-testid="overview-pair-missing"]').should('have.length', 5); }); cy.get( @@ -663,10 +1036,7 @@ describe('Overview page', () => { ).click(); cy.location('search').should('eq', '?engine=all'); desktopModel('GLM-5.2').find('[data-testid="overview-pair-missing"]').should('have.length', 5); - cy.get('[data-testid="overview-tier-switcher"]') - .contains('a', '100') - .should('have.attr', 'href', '/overview?tier=100&engine=all') - .click(); + selectOverviewTier(100); cy.location('search').should('eq', '?tier=100&engine=all'); cy.get('[data-testid="overview-engine-scope-switcher"]') @@ -720,12 +1090,12 @@ describe('Overview page', () => { .should( 'have.attr', 'title', - 'Estimated from validated benchmark runs. Open raw source dashboard for Jul 18: DeepSeek V4 Pro 1.6T · B200 · SGLang · FP4 · MTP', + 'Estimated from validated benchmark runs. Open raw source dashboard for Jul 18: DeepSeek V4 Pro 0813 1.6T · B200 · SGLang · FP4 · MTP', ) .and( 'have.attr', 'aria-label', - 'Approximately $0.059. Estimated from validated benchmark runs. Open raw source dashboard for Jul 18: DeepSeek V4 Pro 1.6T · B200 · SGLang · FP4 · MTP', + 'Approximately $0.059. Estimated from validated benchmark runs. Open raw source dashboard for Jul 18: DeepSeek V4 Pro 0813 1.6T · B200 · SGLang · FP4 · MTP', ); cy.get('[data-testid="overview-pair-missing"]').should('not.exist'); }); @@ -837,8 +1207,8 @@ describe('Overview page', () => { .and('not.contain.text', '8K→1K'); // The TCO model behind every $/GPU/hr is cited under the metric. cy.get('[data-testid="overview-source-link"]') - .should('have.text', SOURCE_LINK_TEXT) - .and('have.attr', 'href', SOURCE_HREF) + .should('have.text', TCO_SOURCE_TITLE) + .and('have.attr', 'href', TCO_SOURCE_URL) .and('have.attr', 'target', '_blank') .and('have.attr', 'rel', 'noopener noreferrer'); // Opening off-site is signalled by the shared external-link glyph. @@ -895,9 +1265,9 @@ describe('Overview page', () => { cy.get('[data-testid="overview-desktop-matrix"]').should('contain.text', label); } for (const model of ['Kimi-K3', 'GLM-5.2']) { - desktopModel(model) - .find('[data-testid="overview-model-scenario"]') - .should('have.text', AGENTX_LABEL); + desktopModel(model).within(() => { + expectAgentxScenario(AGENTX_LABEL); + }); } for (const model of ['DeepSeek-V4-Pro', 'MiniMax-M3', 'Qwen-3.5-397B-A17B']) { desktopModel(model, SINGLE_TURN) @@ -914,14 +1284,15 @@ describe('Overview page', () => { 'have.length', 2, ); - // Single-turn first, AgentX directly below it, both under the same label. + // The AgentX row sits in the leading AgentX group; the single-turn row + // follows in the 8K/1K group below it, both under the same label. cy.get('[data-testid="overview-desktop-model"][data-model="DeepSeek-V4-Pro"]').then(($rows) => { - expect([...$rows].map((row) => row.dataset.scenario)).to.deep.equal([SINGLE_TURN, AGENTX]); + expect([...$rows].map((row) => row.dataset.scenario)).to.deep.equal([AGENTX, SINGLE_TURN]); }); desktopModel('DeepSeek-V4-Pro', AGENTX).within(() => { - cy.get('[data-testid="overview-model-scenario"]').should('have.text', AGENTX_LABEL); - cy.contains('DeepSeek V4 Pro 1.6T').should('exist'); + expectAgentxScenario(AGENTX_LABEL); + cy.contains('DeepSeek V4 Pro 0813 1.6T').should('exist'); // Priced from the AgentX rows alone — the single-turn sweep never leaks in. cy.get( '[data-testid="overview-pair-value"][data-hardware="b200"] [data-testid="overview-cost-evidence-link"]', @@ -939,14 +1310,14 @@ describe('Overview page', () => { cy.contains('a', 'View details').should( 'have.attr', 'aria-label', - `View details: DeepSeek V4 Pro 1.6T · ${AGENTX_LABEL}`, + `View details: DeepSeek V4 Pro 0813 1.6T · ${AGENTX_LABEL}`, ); }); desktopModel('DeepSeek-V4-Pro', SINGLE_TURN).within(() => { cy.contains('a', 'View details').should( 'have.attr', 'aria-label', - 'View details: DeepSeek V4 Pro 1.6T · 8K/1K', + 'View details: DeepSeek V4 Pro 0813 1.6T · 8K/1K', ); }); }); @@ -1085,7 +1456,7 @@ describe('Overview page', () => { .should('not.match', /∞\s*%/); }); - it('re-renders the whole matrix at the service level the URL names, via plain links', () => { + it('re-renders the whole matrix from the six-stop SLO slider', () => { cy.viewport(1280, 900); cy.visit('/overview'); @@ -1095,22 +1466,42 @@ describe('Overview page', () => { .and('contain.text', 'SLO'); cy.get('body').should('not.contain.text', 'Service level'); cy.get('[data-testid="overview-tier-switcher"]').within(() => { - cy.get('[aria-current="page"]').should('have.text', '50'); - // 30 / 75 / 100 / 150 / 200 link out; the active 50 is inert text. - cy.get('a').should('have.length', 5); - cy.contains('a', '30').should('have.attr', 'href', '/overview?tier=30'); - cy.contains('a', '150').should('have.attr', 'href', '/overview?tier=150'); - cy.contains('a', '200').should('have.attr', 'href', '/overview?tier=200'); - cy.contains('a', '100').should('have.attr', 'href', '/overview?tier=100').click(); + cy.get('a, button').should('not.exist'); + cy.get('[data-tier-option]').should('have.length', 6); + cy.get('[data-tier-option="50"]').should('have.attr', 'data-selected', 'true'); + cy.get('[data-testid="overview-tier-ridge"]').should('have.length', 6); + cy.get('[data-tier-ridge="30"]').should('have.attr', 'data-state', 'filled'); + cy.get('[data-tier-ridge="50"]').should('have.attr', 'data-state', 'selected'); + cy.get('[data-tier-ridge="75"]').should('have.attr', 'data-state', 'unfilled'); + cy.get('[data-testid="overview-tier-slider"]') + .should('have.attr', 'min', '0') + .and('have.attr', 'max', '5') + .and('have.attr', 'step', '1') + .and('have.value', '1') + .and('have.attr', 'data-tier', '50') + .and('have.attr', 'aria-valuetext', '50 tok/s/user'); }); + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100'); // The metric line never repeats the tier — the switcher states it. cy.get('[data-testid="overview-scope"]').should('have.text', SCOPE_LINE); cy.get('[data-testid="overview-tier-switcher"]').within(() => { - cy.get('[aria-current="page"]').should('have.text', '100'); - cy.contains('a', '50').should('have.attr', 'href', '/overview'); + cy.get('[data-tier-option="100"]').should('have.attr', 'data-selected', 'true'); + cy.get('[data-tier-ridge="75"]').should('have.attr', 'data-state', 'filled'); + cy.get('[data-tier-ridge="100"]').should('have.attr', 'data-state', 'selected'); + cy.get('[data-tier-ridge="150"]').should('have.attr', 'data-state', 'unfilled'); + cy.get('[data-testid="overview-tier-slider"]') + .should('have.attr', 'type', 'range') + .should('have.value', '3') + .and('have.attr', 'data-tier', '100') + .and('have.attr', 'aria-valuetext', '100 tok/s/user'); }); + selectOverviewTier(75); + cy.location('search').should('eq', '?tier=75'); + selectOverviewTier(100); + cy.location('search').should('eq', '?tier=100'); desktopModel('Qwen-3.5-397B-A17B', SINGLE_TURN).within(() => { platform('b200').should('contain.text', '$0.124').and('contain.text', 'FP8'); @@ -1374,13 +1765,13 @@ describe('Overview page', () => { .should('have.attr', 'href', '/zh/overview') .and('contain.text', '总览'); cy.contains('h1', PAGE_TITLE_ZH).should('exist'); - cy.contains('按各模型标注的场景,基于各平台最佳观测服务包络线计算每百万总 token 成本').should( - 'exist', - ); + cy.contains( + '根据各模型标注的测试场景,采用各平台实测的最优服务包络线,计算每百万总 token 成本', + ).should('exist'); cy.get('[data-testid="overview-scope"]').should('have.text', SCOPE_LINE_ZH); cy.get('[data-testid="overview-source-link"]') - .should('have.text', SOURCE_LINK_TEXT) - .and('have.attr', 'href', SOURCE_HREF); + .should('have.text', TCO_SOURCE_TITLE) + .and('have.attr', 'href', TCO_SOURCE_URL); cy.get('body').should('not.contain.text', '一眼对比'); cy.contains('— = 无结果。∞ = 缺少 B200 基线。').should('exist'); cy.get('[data-testid="overview-methodology"]').children('p').should('have.length', 2); @@ -1398,11 +1789,11 @@ describe('Overview page', () => { ) .as('estimatedB200') .invoke('attr', 'title') - .should('include', '根据已验证的基准运行结果估算。') - .and('include', '原始数据仪表板:DeepSeek V4 Pro 1.6T · B200 · SGLang · FP4 · MTP'); + .should('include', '根据已验证的基准测试结果估算。') + .and('include', '原始数据仪表板:DeepSeek V4 Pro 0813 1.6T · B200 · SGLang · FP4 · MTP'); cy.get('@estimatedB200') .invoke('attr', 'aria-label') - .should('include', '约 $0.059。根据已验证的基准运行结果估算。'); + .should('include', '约 $0.059。根据已验证的基准测试结果估算。'); cy.get('@estimatedB200').should('have.text', '$0.059'); cy.get('@estimatedB200') .should('have.attr', 'href') @@ -1435,19 +1826,19 @@ describe('Overview page', () => { .and('have.attr', 'title', '缺少可比较的 B200 基线'); }); desktopModel('GLM-5.2').within(() => { - cy.get('[data-testid="overview-model-scenario"]').should('have.text', AGENTX_LABEL_ZH); + expectAgentxScenario(AGENTX_LABEL_ZH); cy.get('[data-testid="overview-pair-missing"]').should('have.length', 5); platform('b300') .find('[data-testid="overview-pair-missing"]') .should('contain.text', '该场景暂无数据'); }); desktopModel('DeepSeek-V4-Pro', AGENTX).within(() => { - cy.get('[data-testid="overview-model-scenario"]').should('have.text', AGENTX_LABEL_ZH); + expectAgentxScenario(AGENTX_LABEL_ZH); cy.get( '[data-testid="overview-pair-value"][data-hardware="b200"] [data-testid="overview-cost-evidence-link"]', ).should('have.text', '$0.064'); }); - cy.contains('若某款芯片不支持 FP4 推测解码,则采用次优的可用配置。').should('exist'); + cy.contains('如果某款芯片没有可用的 FP4 投机解码配置,则改用次优配置。').should('exist'); cy.visit('/zh/overview?tier=100'); cy.get('[data-testid="overview-scope"]').should('have.text', SCOPE_LINE_ZH); @@ -1456,7 +1847,11 @@ describe('Overview page', () => { .and('contain.text', 'SLO'); cy.get('body').should('not.contain.text', '服务档位'); cy.get('[data-testid="overview-tier-switcher"]').within(() => { - cy.contains('a', '50').should('have.attr', 'href', '/zh/overview'); + cy.get('[data-testid="overview-tier-slider"]') + .should('have.attr', 'data-tier', '100') + // tok/s/user is a protected unit that stays untranslated on the zh route. + .and('have.attr', 'aria-valuetext', '100 tok/s/user'); + cy.get('[data-tier-option]').should('have.length', 6); }); }); }); diff --git a/packages/app/cypress/e2e/resident-sequence-lengths.cy.ts b/packages/app/cypress/e2e/resident-sequence-lengths.cy.ts new file mode 100644 index 000000000..7f6e6b93c --- /dev/null +++ b/packages/app/cypress/e2e/resident-sequence-lengths.cy.ts @@ -0,0 +1,63 @@ +import { buildTokenLengthSketch } from '@semianalysisai/inferencex-constants'; + +import { interceptDerivedAgenticMetrics, unlockAgenticGate } from '../support/e2e'; +import { interceptOverlayRun, OVERLAY_RUN_ID } from '../support/overlay-fixtures'; + +function visitAgenticChart(extraQuery = '') { + cy.visit(`/inference?g_model=DeepSeek-V4-Pro&i_seq=agentic-traces&i_prec=fp4${extraQuery}`, { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + unlockAgenticGate(win); + }, + }); +} + +describe('resident sequence-length subtitle', () => { + it('summarizes every resident official point without following legend selection', () => { + interceptOverlayRun(); + interceptDerivedAgenticMetrics(); + cy.intercept('GET', '/api/v1/resident-sequence-lengths*', (request) => { + const ids = new URL(request.url).searchParams.get('ids')?.split(',').filter(Boolean) ?? []; + expect(ids.length).to.be.greaterThan(1); + request.reply({ + body: { + isl: buildTokenLengthSketch(Array.from({ length: 12_345 }, () => 8_192)), + osl: buildTokenLengthSketch(Array.from({ length: 12_345 }, () => 1_024)), + coveredPoints: ids.length, + requestedPoints: ids.length, + }, + }); + }).as('residentSequenceLengths'); + + visitAgenticChart(); + cy.wait('@residentSequenceLengths'); + cy.get('[data-testid="resident-sequence-lengths"]') + .should('be.visible') + .and('contain.text', 'Completed requests across all resident points (n=12,345)') + .and('contain.text', 'ISL p50 8.2k') + .and('contain.text', 'p95 8.2k') + .and('contain.text', 'OSL p50 1k'); + + // Legend visibility is downstream of the resident set. Hiding a series + // must not remove the subtitle or issue a narrower stats request. + cy.get('.sidebar-legend button').first().click({ force: true }); + cy.get('[data-testid="resident-sequence-lengths"]').should('be.visible'); + cy.get('@residentSequenceLengths.all').should('have.length', 1); + }); + + it('does not label official-only stats as covering an unofficial overlay', () => { + let requestCount = 0; + interceptOverlayRun(); + interceptDerivedAgenticMetrics(); + cy.intercept('GET', '/api/v1/resident-sequence-lengths*', (request) => { + requestCount += 1; + request.reply({ body: {} }); + }); + + visitAgenticChart(`&unofficialrun=${OVERLAY_RUN_ID}`); + cy.wait('@unofficialRun'); + cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); + cy.get('[data-testid="resident-sequence-lengths"]').should('not.exist'); + cy.then(() => expect(requestCount).to.equal(0)); + }); +}); diff --git a/packages/app/cypress/e2e/sanity.cy.ts b/packages/app/cypress/e2e/sanity.cy.ts index a18e11cc3..89366e088 100644 --- a/packages/app/cypress/e2e/sanity.cy.ts +++ b/packages/app/cypress/e2e/sanity.cy.ts @@ -58,11 +58,78 @@ describe('Page Load & Navigation', () => { ); }); }); + + it('opens and preserves a direct link to an FAQ answer', () => { + cy.visit('/about#faq-normalized-interactivity', { + onBeforeLoad(win) { + cy.stub(win.navigator.clipboard, 'writeText').as('writeFaqLink').resolves(); + }, + }); + + cy.get('#faq-normalized-interactivity') + .should('be.visible') + .within(() => { + cy.contains( + 'a[href="#faq-normalized-interactivity"]', + 'What is the difference between E2E Normalized Interactivity and Interactivity?', + ).should('be.visible'); + cy.contains('The normalized value penalizes slow TTFT').should('be.visible'); + cy.get('[data-testid="faq-copy-link-faq-normalized-interactivity"]') + .should('be.visible') + .and('have.text', '') + .and('have.attr', 'title', 'Copy link') + .find('svg.lucide-link') + .should('be.visible'); + cy.get('[data-testid="faq-copy-link-faq-normalized-interactivity"]') + .click() + .should('have.attr', 'title', 'Copied') + .find('svg.lucide-check') + .should('be.visible'); + }); + cy.get('@writeFaqLink').should( + 'have.been.calledOnceWith', + `${Cypress.config('baseUrl')}/about#faq-normalized-interactivity`, + ); + cy.location('hash').should('eq', '#faq-normalized-interactivity'); + }); + + it('shows a copy-link button for every FAQ question', () => { + cy.visit('/about'); + + cy.get('[data-testid^="faq-copy-link-"]') + .should('have.length', 15) + .each(($button) => { + cy.wrap($button) + .should('be.visible') + .and('have.text', '') + .and('have.attr', 'title', 'Copy link') + .find('svg.lucide-link') + .should('be.visible'); + }); + }); }); // Toggle visibility, click behavior, and aria-label are covered by // cypress/component/mode-toggle.cy.tsx. Only the reload-persistence test // requires a full page load (true e2e concern). +describe('Splash text', () => { + it('announces AgentX on the landing page in both light and dark mode', () => { + cy.visit('/', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + win.localStorage.setItem('theme', 'light'); + }, + }); + cy.get('html').should('not.have.class', 'dark'); + cy.get('[data-testid="splash-text"]').should('be.visible').and('have.text', 'AgentX is here!!'); + + // Same splash after switching themes — it is no longer minecraft-only. + cy.get('[data-testid="theme-toggle"]').click(); + cy.get('html').should('have.class', 'dark'); + cy.get('[data-testid="splash-text"]').should('be.visible').and('have.text', 'AgentX is here!!'); + }); +}); + describe('Theme Toggle', () => { it('theme persists across page reload (localStorage)', () => { cy.window().then((win) => { diff --git a/packages/app/cypress/e2e/speed-overlay.cy.ts b/packages/app/cypress/e2e/speed-overlay.cy.ts deleted file mode 100644 index 1b5c8ef1f..000000000 --- a/packages/app/cypress/e2e/speed-overlay.cy.ts +++ /dev/null @@ -1,204 +0,0 @@ -const advancedToggle = () => - cy.contains('figure', 'vs. Interactivity').find('[data-testid="legend-advanced-toggle"]'); - -describe('Bus / Race Car Speed Overlay', () => { - before(() => { - cy.visit('/inference', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="scatter-graph"]').should('be.visible'); - cy.get('.sidebar-legend').first().should('be.visible'); - }); - - it('toggle is hidden in the collapsed Advanced drawer and is off by default', () => { - advancedToggle().should('be.visible').and('have.attr', 'aria-expanded', 'false'); - cy.get('#scatter-speed-overlay').should('not.exist'); - - advancedToggle().click(); - advancedToggle().should('have.attr', 'aria-expanded', 'true'); - cy.get('#scatter-speed-overlay').should('exist'); - cy.get('label[for="scatter-speed-overlay"]').should('contain.text', 'Bus / Race Car'); - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'unchecked'); - cy.get('[data-testid="speed-overlay-bus"]').should('not.exist'); - cy.get('[data-testid="speed-overlay-car"]').should('not.exist'); - }); - - it('toggling on renders bus and car images on the chart', () => { - cy.get('#scatter-speed-overlay').click(); - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'checked'); - cy.get('[data-testid="speed-overlay-bus"]').should('exist'); - cy.get('[data-testid="speed-overlay-car"]').should('exist'); - }); - - it('default tput-vs-interactivity chart places bus top-left and car bottom-right', () => { - cy.get('[data-testid="speed-overlay-bus"]').should('have.attr', 'data-corner', 'top-left'); - cy.get('[data-testid="speed-overlay-car"]').should('have.attr', 'data-corner', 'bottom-right'); - }); - - it('switching to a cost metric flips the bus to bottom-left, car to top-right', () => { - cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); - cy.get('[role="option"]') - .contains('Cost per Million Total Tokens (Owning - Hyperscaler)') - .click({ force: true }); - cy.get('[data-testid="speed-overlay-bus"]').should('have.attr', 'data-corner', 'bottom-left'); - cy.get('[data-testid="speed-overlay-car"]').should('have.attr', 'data-corner', 'top-right'); - }); - - it('switching to an input-token throughput (TTFT X) places bus top-right, car bottom-left', () => { - cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); - cy.get('[role="option"]').contains('Input Token Throughput per Chip').click({ force: true }); - cy.get('[data-testid="speed-overlay-bus"]').should('have.attr', 'data-corner', 'top-right'); - cy.get('[data-testid="speed-overlay-car"]').should('have.attr', 'data-corner', 'bottom-left'); - }); - - it('toggling off removes the overlay images cleanly', () => { - cy.get('#scatter-speed-overlay').click(); - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'unchecked'); - cy.get('[data-testid="speed-overlay-bus"]').should('not.exist'); - cy.get('[data-testid="speed-overlay-car"]').should('not.exist'); - }); - - it('URL param i_speed=1 enables the overlay on first load', () => { - cy.visit('/inference?i_speed=1', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="scatter-graph"]').should('be.visible'); - advancedToggle().click(); - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'checked'); - cy.get('[data-testid="speed-overlay-bus"]').should('exist'); - cy.get('[data-testid="speed-overlay-car"]').should('exist'); - }); -}); - -describe('Donkey / Elytra Minecraft Overlay', () => { - before(() => { - cy.visit('/inference', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="scatter-graph"]').should('be.visible'); - cy.get('.sidebar-legend').first().should('be.visible'); - }); - - it('toggle is hidden in the Advanced drawer and is independent of the bus/car toggle', () => { - advancedToggle().should('have.attr', 'aria-expanded', 'false'); - cy.get('#scatter-minecraft-overlay').should('not.exist'); - - advancedToggle().click(); - cy.get('#scatter-minecraft-overlay').should('exist'); - cy.get('label[for="scatter-minecraft-overlay"]').should('contain.text', 'Donkey / Elytra'); - cy.get('#scatter-minecraft-overlay').should('have.attr', 'data-state', 'unchecked'); - cy.get('[data-testid="speed-overlay-minecraft-slow"]').should('not.exist'); - cy.get('[data-testid="speed-overlay-minecraft-fast"]').should('not.exist'); - }); - - it('toggling on renders the donkey-with-chest and elytra images', () => { - cy.get('#scatter-minecraft-overlay').click(); - cy.get('#scatter-minecraft-overlay').should('have.attr', 'data-state', 'checked'); - cy.get('[data-testid="speed-overlay-minecraft-slow"]') - .should('exist') - .and('have.attr', 'href') - .and('include', 'donkey-chest.png'); - cy.get('[data-testid="speed-overlay-minecraft-fast"]') - .should('exist') - .and('have.attr', 'href') - .and('include', 'elytra.png'); - }); - - it('donkey sits in the same corner as the bus would (batch-friendly side)', () => { - cy.get('[data-testid="speed-overlay-minecraft-slow"]').should( - 'have.attr', - 'data-corner', - 'top-left', - ); - cy.get('[data-testid="speed-overlay-minecraft-fast"]').should( - 'have.attr', - 'data-corner', - 'bottom-right', - ); - }); - - it('enabling both toggles stacks the second pair inward from the first', () => { - cy.get('#scatter-speed-overlay').click(); - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'checked'); - cy.get('[data-testid="speed-overlay-bus"]').should('exist'); - cy.get('[data-testid="speed-overlay-minecraft-slow"]').should('exist'); - // Both pairs anchor to the same corner — the donkey sits inward (larger x) - // when the slow corner is on the left. - cy.get('[data-testid="speed-overlay-bus"]').then(($bus) => { - const busX = Number($bus.attr('x')); - cy.get('[data-testid="speed-overlay-minecraft-slow"]').then(($donkey) => { - const donkeyX = Number($donkey.attr('x')); - expect(donkeyX).to.be.greaterThan(busX); - }); - }); - }); - - it('URL param i_mc=1 enables the minecraft overlay on first load', () => { - cy.visit('/inference?i_mc=1', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="scatter-graph"]').should('be.visible'); - advancedToggle().click(); - cy.get('#scatter-minecraft-overlay').should('have.attr', 'data-state', 'checked'); - cy.get('[data-testid="speed-overlay-minecraft-slow"]').should('exist'); - cy.get('[data-testid="speed-overlay-minecraft-fast"]').should('exist'); - // Bus/car toggle is independent — should remain off and not render. - cy.get('#scatter-speed-overlay').should('have.attr', 'data-state', 'unchecked'); - cy.get('[data-testid="speed-overlay-bus"]').should('not.exist'); - }); -}); - -describe('Y-Axis Metric Search', () => { - before(() => { - cy.visit('/inference', { - onBeforeLoad(win) { - win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); - }, - }); - cy.get('[data-testid="scatter-graph"]').should('be.visible'); - }); - - it('renders a search input inside the Y-axis selector dropdown', () => { - cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); - // Other selectors (legend, MultiSelect) reuse the "Search..." placeholder, so - // scope every lookup to the open Y-axis dropdown's content. - cy.get('[data-slot="select-content"]') - .find('input[placeholder="Search..."]') - .should('be.visible'); - cy.get('body').click(0, 0); - }); - - it('typing in the search filters the option list across groups', () => { - cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); - cy.get('[data-slot="select-content"]') - .find('input[placeholder="Search..."]') - .type('input token throughput'); - cy.get('[data-slot="select-content"]').find('[role="option"]').should('have.length', 2); - cy.get('[data-slot="select-content"]') - .find('[role="option"]') - .first() - .should('contain.text', 'Input Token Throughput'); - cy.get('body').click(0, 0); - }); - - it('selecting a filtered option applies the metric and closes the dropdown', () => { - cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); - cy.get('[data-slot="select-content"]') - .find('input[placeholder="Search..."]') - .type('cost per million total'); - cy.get('[data-slot="select-content"]') - .find('[role="option"]') - .contains('3 Year Rental') - .click({ force: true }); - cy.get('[data-slot="select-content"]').should('not.exist'); - cy.get('[data-testid="yaxis-metric-selector"]').should('contain.text', '3 Year Rental'); - }); -}); diff --git a/packages/app/cypress/e2e/throughput-calculator.cy.ts b/packages/app/cypress/e2e/throughput-calculator.cy.ts index 70fc19cc7..55a85f9de 100644 --- a/packages/app/cypress/e2e/throughput-calculator.cy.ts +++ b/packages/app/cypress/e2e/throughput-calculator.cy.ts @@ -1,3 +1,5 @@ +import { TCO_SOURCE_TITLE, TCO_SOURCE_URL } from '@semianalysisai/inferencex-constants'; + import { availability as agenticAvailability, b300Rows as agenticB300Rows, @@ -78,9 +80,12 @@ describe('TCO Calculator', () => { }); }); - it('renders Precision multi-selector', () => { + it('hides the Precision selector for a single-precision model', () => { + // DeepSeek-V4-Pro is FP4-only in the fixtures — with one precision there + // is nothing to choose, so the control is hidden entirely. + cy.get('[data-testid="calc-cost-selector"]').should('exist'); cy.get('[data-testid="calculator-controls"]').within(() => { - cy.contains('Precision').should('exist'); + cy.contains('Precision').should('not.exist'); }); }); @@ -217,6 +222,9 @@ describe('TCO Calculator', () => { it('shows TCO badges when cost metric is selected', () => { cy.get('[data-testid="calculator-cost-badges"]').should('contain.text', 'TCO $/chip/hr'); cy.get('[data-testid="calculator-cost-badges"]').should('contain.text', '$'); + cy.get('[data-testid="calculator-chart-section"]') + .contains('a', TCO_SOURCE_TITLE) + .should('have.attr', 'href', TCO_SOURCE_URL); }); it('displays chart title that updates when metric changes', () => { @@ -517,10 +525,13 @@ describe('TCO Calculator', () => { 'contain.text', 'Disaggregated inference configurations', ); - cy.get('[data-testid="calculator-chart-section"]').should( - 'contain.text', - 'throughput per decode chip', - ); + cy.get('[data-testid="calculator-chart-section"]') + // The direction is the point of the warning, not the phrasing: on these + // token types a disaggregated config reads high. Asserting the old wording + // alone let the note be reworded into something that no longer said which + // way it was wrong. + .should('contain.text', 'per prefill chip and per decode chip') + .and('contain.text', 'reads faster per chip than it is'); }); // A disagg config's input/output cost is attributed to only its prefill or @@ -541,7 +552,10 @@ describe('TCO Calculator', () => { cy.get('body').type('{esc}'); cy.get('[data-testid="calculator-disagg-cost-note"]') .should('be.visible') - .and('contain.text', 'cost per decode chip'); + .and('contain.text', 'per prefill chip and per decode chip') + // Cheaper, specifically — the flattering direction is what a reader + // comparing costs needs told. + .and('contain.text', 'reads cheaper than it is'); } cy.get('[data-testid="calc-cost-type-selector"]').click(); @@ -552,10 +566,13 @@ describe('TCO Calculator', () => { it('shows disaggregated throughput disclaimer for power metric', () => { cy.get('[data-testid="calculator-metric-power"]').click(); - cy.get('[data-testid="calculator-chart-section"]').should( - 'contain.text', - 'throughput per decode chip', - ); + cy.get('[data-testid="calculator-chart-section"]') + // The direction is the point of the warning, not the phrasing: on these + // token types a disaggregated config reads high. Asserting the old wording + // alone let the note be reworded into something that no longer said which + // way it was wrong. + .should('contain.text', 'per prefill chip and per decode chip') + .and('contain.text', 'reads faster per chip than it is'); }); }); @@ -578,7 +595,7 @@ describe('TCO Calculator', () => { // open straight to the right model without a flash of the default. See #430. it('?g_model= seeds the model selector before client hydration', () => { cy.request('/calculator?g_model=DeepSeek-V4-Pro').then((response) => { - expect(response.body).to.contain('DeepSeek V4 Pro 1.6T'); + expect(response.body).to.contain('DeepSeek V4 Pro 0813 1.6T'); expect(response.body).not.to.contain('DeepSeek R1 0528 671B'); }); }); @@ -588,7 +605,10 @@ describe('TCO Calculator', () => { win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); }); cy.visit('/calculator?g_model=DeepSeek-V4-Pro'); - cy.get('[data-testid="calc-model-selector"]').should('contain.text', 'DeepSeek V4 Pro 1.6T'); + cy.get('[data-testid="calc-model-selector"]').should( + 'contain.text', + 'DeepSeek V4 Pro 0813 1.6T', + ); }); }); @@ -596,6 +616,21 @@ describe('TCO Calculator', () => { // DeepSeek V4 agentic trace calculations // --------------------------------------------------------------------------- + describe('availability failure degradation', () => { + it('keeps usable benchmark results visible when availability metadata fails', () => { + cy.intercept('GET', '/api/v1/availability*', { + statusCode: 503, + body: { error: 'temporarily unavailable' }, + }).as('availabilityFailure'); + + cy.visit('/calculator'); + cy.wait('@availabilityFailure'); + cy.get('[data-testid="calculator-bar-chart"] svg .bar').should('have.length.greaterThan', 0); + cy.get('[data-testid="calculator-controls"]').should('be.visible'); + cy.contains('Error loading calculator data').should('not.exist'); + }); + }); + describe('DeepSeek V4 agentic calculations', () => { beforeEach(() => { const b300Rows = agenticB300Rows(null); @@ -624,7 +659,10 @@ describe('TCO Calculator', () => { }); it('renders throughput and cost calculations from null-ISL/OSL agentic rows', () => { - cy.get('[data-testid="calc-sequence-selector"]').should('contain.text', 'Agentic Traces'); + // The agentic fixture exposes a single scenario, so the scenario + // control disappears entirely — no dropdown, no static readout. + cy.get('[data-testid="scenario-static-value"]').should('not.exist'); + cy.get('[data-testid="calc-sequence-selector"]').should('not.exist'); cy.get('[data-testid="calc-percentile-selector"]').should('contain.text', 'p90'); cy.get('[data-testid="calculator-no-data"]').should('not.exist'); cy.get('[data-testid="calculator-bar-chart"] svg .bar').should('have.length', 2); @@ -681,7 +719,10 @@ describe('TCO Calculator', () => { }); cy.wait('@agenticBenchmarks'); - cy.get('[data-testid="calc-sequence-selector"]').should('contain.text', 'Agentic Traces'); + // Single-scenario fixture → the scenario control disappears entirely; + // the gate is locked, so no percentile selector either. + cy.get('[data-testid="scenario-static-value"]').should('not.exist'); + cy.get('[data-testid="calc-sequence-selector"]').should('not.exist'); cy.get('[data-testid="calc-percentile-selector"]').should('not.exist'); cy.get('[data-testid="calculator-chart-section"] h2') .first() @@ -694,10 +735,12 @@ describe('TCO Calculator', () => { }); // --------------------------------------------------------------------------- - // Fleet planner (MW projection + cost-target inverse lookup) + // Cost target: the inverse of the interactivity slider. The MW-budget input + // lives here now that the Fleet Lifecycle section moved to its own /fleet + // page — see fleet-lifecycle.cy.ts. Both pages share the `c_mw` URL param. // --------------------------------------------------------------------------- - describe('fleet planner', () => { + describe('cost target', () => { before(() => { cy.window().then((win) => { win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); @@ -706,42 +749,37 @@ describe('TCO Calculator', () => { cy.get('[data-testid="calculator-bar-chart"] svg .bar').should('have.length.greaterThan', 0); }); - it('renders both fleet planner cards with empty states', () => { - cy.get('[data-testid="calculator-fleet-section"]').should('be.visible'); - cy.get('[data-testid="calculator-fleet-section"]').should('contain.text', 'Fleet Projection'); - cy.get('[data-testid="calculator-fleet-empty"]').should('be.visible'); + it('sits directly under the interactivity slider, since it inverts it', () => { + // Reading order is the point: the slider fixes a speed and the cost falls + // out; this fixes the cost and the speed falls out. Anything between them + // makes the pair read as unrelated. cy.get('[data-testid="calculator-costcap-section"]').should('be.visible'); cy.get('[data-testid="calculator-costcap-empty"]').should('be.visible'); - }); - - it('entering a MW budget renders the fleet projection table', () => { - cy.get('[data-testid="calc-fleet-mw-input"]').type('10'); - cy.get('[data-testid="calculator-fleet-table"]').should('be.visible'); - cy.get('[data-testid="calculator-fleet-table"]').within(() => { - cy.contains('th', 'Chips').should('exist'); - cy.contains('th', 'Concurrent Users').should('exist'); - cy.contains('th', 'Fleet $/mo').should('exist'); - cy.get('tbody tr').should('have.length.greaterThan', 0); - // GPU counts are whole, comma-grouped numbers - cy.get('tbody tr') - .first() - .find('td') - .eq(1) - .invoke('text') - .should('match', /^[\d,]+$/u); + cy.get('[data-testid="calculator-controls"]').then(($controls) => { + cy.get('[data-testid="calculator-costcap-section"]').then(($costcap) => { + const order = $controls[0]!.compareDocumentPosition($costcap[0]!); + // eslint-disable-next-line no-bitwise + expect(order & Node.DOCUMENT_POSITION_FOLLOWING).to.be.greaterThan(0); + }); + }); + cy.get('[data-testid="calculator-chart-section"]').then(($chart) => { + cy.get('[data-testid="calculator-costcap-section"]').then(($costcap) => { + const order = $chart[0]!.compareDocumentPosition($costcap[0]!); + // eslint-disable-next-line no-bitwise + expect(order & Node.DOCUMENT_POSITION_PRECEDING).to.be.greaterThan(0); + }); }); - cy.get('[data-testid="calculator-fleet-empty"]').should('not.exist'); }); - it('fleet table shows the utilization and facility-power assumptions', () => { - cy.get('[data-testid="calculator-fleet-section"]').should( - 'contain.text', - 'Assumes 100% utilization', - ); - cy.get('[data-testid="calculator-fleet-section"]').should( - 'contain.text', - 'SemiAnalysis Datacenter Industry Model', - ); + it('owns the MW budget input now that the lifecycle section moved to /fleet', () => { + cy.get('[data-testid="calculator-costcap-section"]') + .find('[data-testid="calc-costcap-mw-input"]') + .should('have.value', '10'); + cy.get('[data-testid="calculator-lifecycle-section"]').should('not.exist'); + // A pointer to the new page replaces the section. + cy.get('[data-testid="calculator-fleet-pointer-link"]') + .should('have.attr', 'href') + .and('match', /\/fleet$/); }); it('entering a generous cost target renders reachable interactivity per GPU', () => { @@ -766,23 +804,6 @@ describe('TCO Calculator', () => { cy.get('[data-testid="calculator-costcap-table"]').should('contain.text', 'Not reachable'); }); - it('a budget too small for one GPU shows a dedicated message, not the enter-a-value prompt', () => { - cy.get('[data-testid="calc-fleet-mw-input"]').clear(); - cy.get('[data-testid="calc-fleet-mw-input"]').type('0.0001'); - cy.get('[data-testid="calculator-fleet-empty"]') - .should('be.visible') - .and('contain.text', 'too small to power a single chip'); - cy.get('[data-testid="calculator-fleet-table"]').should('not.exist'); - }); - - it('clearing the MW input restores the empty state', () => { - cy.get('[data-testid="calc-fleet-mw-input"]').clear(); - cy.get('[data-testid="calculator-fleet-empty"]') - .should('be.visible') - .and('contain.text', 'Enter a facility power budget'); - cy.get('[data-testid="calculator-fleet-table"]').should('not.exist'); - }); - it('cost-cap table follows legend visibility (soloing a GPU filters its rows)', () => { cy.get('[data-testid="calc-costcap-input"]').clear(); cy.get('[data-testid="calc-costcap-input"]').type('100'); @@ -802,4 +823,43 @@ describe('TCO Calculator', () => { }); }); }); + + describe('folding sections away', () => { + before(() => { + cy.window().then((win) => { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }); + cy.visit('/calculator'); + cy.get('[data-testid="calculator-bar-chart"] svg .bar').should('have.length.greaterThan', 0); + }); + + it('folds the chart away, keeping its title as the handle', () => { + // Folded means unmounted, not hidden: these bodies hold D3 charts, and a + // reader who folded one should not still be paying to render it. + cy.get('[data-testid="calculator-chart-collapse"]').click(); + cy.get('[data-testid="calculator-figure"]').should('not.exist'); + // The title only appears in the header once folded — while open the chart's + // own caption carries it, and two copies would be worse than none. + cy.get('[data-testid="calculator-chart-section"] h2') + .should('have.length', 1) + .and('contain.text', 'Total Token Throughput per Chip'); + + cy.get('[data-testid="calculator-chart-collapse"]').click(); + cy.get('[data-testid="calculator-figure"]').should('be.visible'); + cy.get('[data-testid="calculator-bar-chart"] svg .bar').should('have.length.greaterThan', 0); + }); + + it('folds the cost-target section away, keeping its heading as the handle', () => { + cy.get('[data-testid="calc-costcap-input"]').should('be.visible'); + cy.get('[data-testid="calculator-costcap-collapse"]').click(); + cy.get('[data-testid="calc-costcap-input"]').should('not.exist'); + // The heading stays put, so the folded section still says what it is. + cy.get('[data-testid="calculator-costcap-section"]').should( + 'contain.text', + 'Interactivity Within a Cost Target', + ); + cy.get('[data-testid="calculator-costcap-collapse"]').click(); + cy.get('[data-testid="calc-costcap-input"]').should('be.visible'); + }); + }); }); diff --git a/packages/app/cypress/e2e/tokens-per-currency.cy.ts b/packages/app/cypress/e2e/tokens-per-currency.cy.ts new file mode 100644 index 000000000..b7a4e74d8 --- /dev/null +++ b/packages/app/cypress/e2e/tokens-per-currency.cy.ts @@ -0,0 +1,51 @@ +/** + * The dashboard now opens on tokens purchasable per $1 USD, and the same + * quantities are also available priced in yuan. Fixed-sequence Quick Filters + * remain available from the chart legend. + */ +describe('Tokens per currency and agentic controls', () => { + beforeEach(() => { + cy.window().then((win) => { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }); + }); + + it('defaults the y-axis to total tokens per $1 USD', () => { + cy.visit('/inference'); + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Total Tokens per $1 USD (Owning - Neocloud Giant)', + ); + cy.get('[data-testid="scatter-graph"]') + .first() + .find('svg .dot-group') + .should('have.length.greaterThan', 0); + }); + + it('still honors an explicit ?i_metric=, so shared links are unaffected', () => { + cy.visit('/inference?i_metric=y_tpPerGpu'); + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Token Throughput per Chip', + ); + }); + + it('offers the same quantities priced in yuan', () => { + cy.visit('/inference'); + cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); + cy.get('[role="option"]') + .contains('Total Tokens per ¥1 RMB (Owning - Hyperscaler)') + .click({ force: true }); + cy.get('[data-testid="scatter-graph"]') + .first() + .find('svg .dot-group') + .should('have.length.greaterThan', 0); + }); + + it('keeps Quick Filters for a fixed-sequence scenario', () => { + cy.visit('/inference?i_seq=8k%2F1k'); + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('be.visible'); + cy.get('[data-testid="quick-filter-spec-mtp"]').should('exist'); + }); +}); diff --git a/packages/app/cypress/e2e/ttft-x-axis-toggle.cy.ts b/packages/app/cypress/e2e/ttft-x-axis-toggle.cy.ts index f62aed384..3dd584e8d 100644 --- a/packages/app/cypress/e2e/ttft-x-axis-toggle.cy.ts +++ b/packages/app/cypress/e2e/ttft-x-axis-toggle.cy.ts @@ -1,53 +1,26 @@ import { interceptDerivedAgenticMetrics, unlockAgenticGate } from '../support/e2e'; +import { agenticMetrics } from '../support/agentic-fixtures'; +import { expandLegendAdvanced } from '../support/legend-advanced'; // This spec exercises the agentic x-axis modes, which only exist when the -// selected model resolves to the Agentic Traces scenario. The default e2e -// fixtures (cypress/fixtures/api/*.json) have NO agentic rows for any model, and -// the app default scenario is 8K/1K, so bare /inference always resolves to a -// fixed-seq scenario. We therefore inject agentic availability + benchmark rows -// for the default model VIA SPEC-SCOPED INTERCEPTS (not the shared fixtures) and -// visit with an explicit ?i_seq=agentic-traces so this test — and only this -// test — sees the agentic view. +// selected model resolves to the Agentic scenario. The default e2e +// fixtures (cypress/fixtures/api/*.json) have NO agentic rows for any model, so +// bare /inference always resolves to a fixed-seq scenario there. We therefore +// inject agentic availability + benchmark rows for the default model VIA +// SPEC-SCOPED INTERCEPTS (not the shared fixtures) so this spec — and only this +// spec — sees the agentic view. Most cases still pass ?i_seq=agentic-traces +// explicitly; the "Default scenario" block below covers the implicit path, +// where availability opens the agentic scenario on its own. const DEFAULT_MODEL_DB_KEY = 'dsv4'; // DeepSeek-V4-Pro is the default model const AGENTIC_DATE = '2026-06-12'; -// Percentile ladder for one metric family (median/p75/p90/p95/p99/std). -const percentileLadder = (prefix: string, base: number): Record => ({ - [`median_${prefix}`]: base, - [`p75_${prefix}`]: base * 1.2, - [`p90_${prefix}`]: base * 1.5, - [`p95_${prefix}`]: base * 1.7, - [`p99_${prefix}`]: base * 2.2, - [`std_${prefix}`]: base * 0.3, -}); - -const agenticMetrics = (conc: number): Record => { - const scale = conc / 16; - const itl = 0.011 * scale; - return { - ...percentileLadder('ttft', 0.4 * scale), - ...percentileLadder('tpot', 0.012 * scale), - ...percentileLadder('itl', itl), - ...percentileLadder('e2el', 8 * scale), - median_intvty: 1 / itl, - p75_intvty: 1 / (itl * 1.2), - p90_intvty: 1 / (itl * 1.5), - p99_intvty: 1 / (itl * 2.2), - std_intvty: (1 / itl) * 0.1, - tput_per_gpu: 950 / Math.sqrt(scale), - output_tput_per_gpu: 210, - input_tput_per_gpu: 740, - total_tput_tps: 7600 * conc * 0.05, - }; -}; - const agenticGpus = [ { hardware: 'b200', framework: 'vllm', disagg: false }, { hardware: 'b300', framework: 'vllm', disagg: false }, ]; -// Availability: default model has BOTH agentic and fixed-seq. The agentic view -// is selected explicitly via ?i_seq=agentic-traces (the app default is 8K/1K). +// Availability: default model has BOTH agentic and fixed-seq, so the scenario +// the chart lands on is a real choice rather than the only option. const agenticAvailability = [ ...agenticGpus.map((g) => ({ model: DEFAULT_MODEL_DB_KEY, @@ -118,31 +91,42 @@ const interceptAgenticData = () => { cy.intercept('GET', '/api/v1/benchmarks*', { body: agenticBenchmarks }).as('benchmarks'); }; +// Same rows re-keyed to another model to prove the default follows availability +// rather than a hard-coded model registry. +const OTHER_MODEL_DB_KEY = 'dsr1'; +const otherModelAvailability = agenticAvailability.map((row) => ({ + ...row, + model: OTHER_MODEL_DB_KEY, +})); +const otherModelBenchmarks = agenticBenchmarks.map((row, index) => ({ + ...row, + id: 920000 + index, + model: OTHER_MODEL_DB_KEY, +})); + const interceptFixedSequenceData = () => { cy.intercept('GET', '/api/v1/availability', { body: agenticAvailability }).as('availability'); cy.intercept('GET', '/api/v1/benchmarks*', { body: fixedSequenceBenchmarks }).as('benchmarks'); }; /** - * On agentic charts, Interactivity / E2E Latency / TTFT live inside the - * "Advanced" menu rather than as top-level tabs, so they must be revealed - * before they can be clicked. The menu closes on select, so the post-select - * assertion targets the trigger, which carries the active state and the - * selected metric's label. + * Every x-axis metric is a top-level tab, on agentic charts as well as fixed + * sequences — #736 removed the "Advanced" popover that used to hide + * Interactivity / E2E Latency / TTFT behind a trigger. The tab itself now + * carries both the selected state and the metric's label. */ -function selectAdvancedXAxisMode(mode: 'interactivity' | 'e2e' | 'ttft', label: string) { - cy.get('[data-testid="x-axis-mode-advanced"]').click(); +function selectXAxisMode(mode: 'interactivity' | 'e2e' | 'ttft', label: string) { cy.get(`[data-testid="x-axis-mode-${mode}"]`).click(); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('have.attr', 'data-state', 'active') + cy.get(`[data-testid="x-axis-mode-${mode}"]`) + .should('have.attr', 'aria-selected', 'true') .and('contain.text', label); } describe('X-Axis Mode Toggle (inference chart)', () => { before(() => { interceptAgenticData(); - // The agentic default mode is E2E Normalized Interactivity, which fetches derived metrics - // on first render — stub them before the visit. + // Agentic defaults to Interactivity, which needs no derived metrics. The stub + // covers the first switch into E2E Normalized Interactivity further down. interceptDerivedAgenticMetrics(); cy.visit('/inference?i_seq=agentic-traces', { onBeforeLoad(win) { @@ -154,39 +138,69 @@ describe('X-Axis Mode Toggle (inference chart)', () => { cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); }); - it('shows E2E Normalized Interactivity by default for the agentic view, as the leftmost option', () => { - cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic Traces'); - // The three per-request latency modes are nested under Advanced on agentic. - cy.get('[data-testid="x-axis-mode-ttft"]').should('not.exist'); - cy.get('[data-testid="x-axis-mode-e2e"]').should('not.exist'); - cy.get('[data-testid="x-axis-mode-interactivity"]').should('not.exist'); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('be.visible') - .and('have.attr', 'data-state', 'inactive'); - cy.get('[data-testid="x-axis-mode-e2e-normalized-interactivity"]') - .should('be.visible') - .and('have.attr', 'aria-selected', 'true'); - // E2E Normalized Interactivity leads the mode list for agentic. + // Cypress clears intercepts between tests, so the derived-metrics stub is + // re-registered per test rather than once in `before`. Until #736 the + // agentic default was E2E Normalized Interactivity, so the fetch happened + // during `before` while the stub was still alive and React Query held the + // result for the rest of the spec. The default no longer fetches, so any + // later switch into that mode issues a fresh request. + beforeEach(() => { + interceptDerivedAgenticMetrics(); + }); + + it('defaults the agentic view to Interactivity, with all four modes as flat tabs', () => { + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); + // #736 made every latency mode a top-level tab and moved the default off E2E + // Normalized Interactivity, which still leads the strip without being selected. + cy.get('[data-testid="x-axis-mode-advanced"]').should('not.exist'); + for (const mode of ['e2e-normalized-interactivity', 'interactivity', 'e2e', 'ttft']) { + cy.get(`[data-testid="x-axis-mode-${mode}"]`).should('be.visible'); + } + cy.get('[data-testid="x-axis-mode-buttons"] [role="tab"]').should('have.length', 4); cy.get('[data-testid="x-axis-mode-buttons"] [role="tab"]') .first() - .should('have.attr', 'data-testid', 'x-axis-mode-e2e-normalized-interactivity'); - cy.get('[data-testid="chart-figure"] h2').should( - 'contain.text', - 'P90 E2E Normalized Interactivity', + .should('have.attr', 'data-testid', 'x-axis-mode-e2e-normalized-interactivity') + .and('have.attr', 'aria-selected', 'false'); + cy.get('[data-testid="x-axis-mode-interactivity"]').should( + 'have.attr', + 'aria-selected', + 'true', ); + cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Interactivity'); cy.get('[data-testid="chart-figure"] svg').should( 'contain.text', - 'P90 E2E Normalized Interactivity (tok/s/user)', + 'P90 Interactivity (tok/s/user)', ); }); - it('switches to Interactivity and updates the heading', () => { - selectAdvancedXAxisMode('interactivity', 'Interactivity'); + it('offers unobtrusive FAQ help beside E2E Normalized Interactivity', () => { + cy.get('[data-testid="normalized-interactivity-faq-link"]') + .should('be.visible') + .and('have.attr', 'href', '/about#faq-normalized-interactivity') + .and('have.attr', 'title', 'What does E2E Normalized Interactivity mean?') + .and('have.class', 'no-export') + .find('svg[aria-hidden="true"]') + .should('be.visible'); + + // The metric tabs sit outside the chart capture root, and no-export is a + // second guard if this control moves into that tree in the future. + cy.get('#chart-0 [data-testid="normalized-interactivity-faq-link"]').should('not.exist'); + }); + + it('switches to E2E Normalized Interactivity and updates the heading', () => { + // The first entry into this mode fetches the trace-derived metrics, which the + // suite's intercept stubs; the default no longer fetches them on load. + cy.get('[data-testid="x-axis-mode-e2e-normalized-interactivity"]').click(); + cy.get('[data-testid="chart-figure"] h2').should( + 'contain.text', + 'P90 E2E Normalized Interactivity', + ); + selectXAxisMode('interactivity', 'Interactivity'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Interactivity'); }); - it('explains the offload halo in the legend and distinguishes it from plain points', () => { - cy.get('#chart-0 [data-testid="offload-halo-key"]') + it('explains the offload halo in the info footer and distinguishes it from plain points', () => { + cy.get('[data-testid="axis-metric-footer-chart-0"] [data-testid="offload-halo-key"]') .should('be.visible') .and('contain.text', 'KV offload ON'); cy.get('#chart-0 .offload-halo').should('have.length.at.least', 1); @@ -197,30 +211,19 @@ describe('X-Axis Mode Toggle (inference chart)', () => { }); }); - it('keeps the offload halo explanation in the PNG export clone', () => { - cy.window().then((win) => { - const exportContainer = win.document.querySelector('#chart-0-export'); - expect(exportContainer).not.to.equal(null); - const state = { seen: false }; - const observer = new win.MutationObserver(() => { - if (exportContainer?.querySelector('[data-testid="offload-halo-key"]')) { - state.seen = true; - observer.disconnect(); - } - }); - observer.observe(exportContainer!, { childList: true, subtree: true }); - (win as typeof win & { __offloadHaloExportState: typeof state }).__offloadHaloExportState = - state; - }); - - cy.get('[data-testid="chart-figure"]').first().find('[data-testid="export-button"]').click(); - cy.get('[data-testid="export-png-button"]').click(); - cy.window().its('__offloadHaloExportState.seen').should('eq', true); + it('keeps the offload halo explanation out of the chart capture tree', () => { + // The key lives in the axis-metric info footer (a `no-export` sibling of + // the chart), so the PNG export clone — built from #chart-0 — carries the + // halo decoration itself but not the textual key. + cy.get('[data-testid="axis-metric-footer-chart-0"] [data-testid="offload-halo-key"]').should( + 'be.visible', + ); + cy.get('#chart-0 [data-testid="offload-halo-key"]').should('not.exist'); }); it('shows the selected percentile in the Interactivity axis label', () => { // Explicitly select the mode — do not rely on the agentic default mode. - selectAdvancedXAxisMode('interactivity', 'Interactivity'); + selectXAxisMode('interactivity', 'Interactivity'); // Agentic plots percentile fields (p90_intvty), so the axis label carries it. cy.get('[data-testid="chart-figure"] svg').should( 'contain.text', @@ -228,10 +231,14 @@ describe('X-Axis Mode Toggle (inference chart)', () => { ); }); - it('defaults to parallelism labels without line labels for the agentic view', () => { + it('defaults to parallelism, point, and line labels for the agentic view', () => { + // Line labels name the curve and point labels name the point, so the + // agentic view turns on both — it differs from fixed-seq only in the + // parallelism and point labels. + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'checked'); - cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'unchecked'); + cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'checked'); }); it('honors explicit label URL overrides for the agentic view', () => { @@ -239,25 +246,26 @@ describe('X-Axis Mode Toggle (inference chart)', () => { // Fresh page load → fresh React Query cache → the default E2E Normalized Interactivity // mode refetches derived metrics. interceptDerivedAgenticMetrics(); - cy.visit('/inference?i_seq=agentic-traces&i_label=0&i_advlabel=0&i_linelabel=1', { + cy.visit('/inference?i_seq=agentic-traces&i_label=0&i_advlabel=0&i_linelabel=0', { onBeforeLoad(win) { win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); unlockAgenticGate(win); }, }); - cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic Traces'); + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'unchecked'); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'unchecked'); - cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'checked'); + cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'unchecked'); }); it('switches the x-axis to TTFT and updates the heading', () => { - selectAdvancedXAxisMode('ttft', 'TTFT'); + selectXAxisMode('ttft', 'TTFT'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Time To First Token'); }); it('switches the x-axis to E2E Latency and updates the heading', () => { - selectAdvancedXAxisMode('e2e', 'E2E Latency'); + selectXAxisMode('e2e', 'E2E Latency'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'End-to-end Latency'); cy.get('[data-testid="chart-figure"] svg').should('contain.text', 'P90 End-to-end Latency (s)'); }); @@ -298,19 +306,12 @@ describe('X-Axis Mode Toggle (inference chart)', () => { ); }); - // Documents the intended pairing of the Advanced menu with manual tab - // activation: focus may move to the lone remaining tab without the x-axis - // snapping back to it. The load-bearing guard for manual activation is the - // 8K/1K test below, where focus-activation is directly observable; this one - // states the agentic-side expectation. - it('keeps the Advanced selection when the remaining tab is focused', () => { - selectAdvancedXAxisMode('ttft', 'TTFT'); - - // Let the popover finish closing first: Radix restores focus to its own - // trigger on close, which would otherwise win the race against the focus - // below and mask the revert this test is guarding against. - cy.get('[data-testid="x-axis-mode-advanced-menu"]').should('not.exist'); - cy.get('[data-testid="x-axis-mode-advanced"]').should('have.focus'); + // Tabs are manually activated: moving focus along the strip must not change + // the x-axis. The load-bearing guard is the 8K/1K test below, where the same + // behaviour is asserted on the fixed-sequence strip; this one states the + // agentic-side expectation, across the agentic-only mode. + it('keeps the selected mode when another tab is focused', () => { + selectXAxisMode('ttft', 'TTFT'); cy.get('[data-testid="x-axis-mode-e2e-normalized-interactivity"]').focus(); cy.get('[data-testid="x-axis-mode-e2e-normalized-interactivity"]').should('have.focus'); @@ -320,14 +321,12 @@ describe('X-Axis Mode Toggle (inference chart)', () => { 'aria-selected', 'false', ); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('have.attr', 'data-state', 'active') - .and('contain.text', 'TTFT'); + cy.get('[data-testid="x-axis-mode-ttft"]').should('have.attr', 'aria-selected', 'true'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Time To First Token'); }); it('switches back to Interactivity', () => { - selectAdvancedXAxisMode('interactivity', 'Interactivity'); + selectXAxisMode('interactivity', 'Interactivity'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Interactivity'); cy.get('[data-testid="chart-figure"] svg').should( 'contain.text', @@ -338,7 +337,7 @@ describe('X-Axis Mode Toggle (inference chart)', () => { it('follows the percentile selector in the Interactivity axis label', () => { // Select p75 here rather than inheriting it from another test — the axis // label must track the selector on its own. - selectAdvancedXAxisMode('interactivity', 'Interactivity'); + selectXAxisMode('interactivity', 'Interactivity'); cy.get('[data-testid="percentile-selector"]').click(); cy.contains('[role="option"]', 'p75').click(); cy.get('[data-testid="chart-figure"] svg').should( @@ -352,24 +351,28 @@ describe('X-axis mode URL param', () => { // Regression: the reconcile effect used to run before availability resolved // the sequence. It recorded the fixed-seq placeholder kind, then treated the // switch to agentic as a user-driven kind change and clobbered the - // URL-restored mode with the agentic default (E2E Normalized Interactivity). + // URL-restored mode with the agentic default. + // + // The restored mode must differ from that default or the test cannot fail: + // #736 moved the default to Interactivity, so restoring Interactivity now + // proves nothing. TTFT is a mode the snap would visibly overwrite. it('keeps a URL-restored mode through the agentic sequence resolving', () => { interceptAgenticData(); interceptDerivedAgenticMetrics(); - cy.visit('/inference?i_seq=agentic-traces&i_xmode=interactivity', { + cy.visit('/inference?i_seq=agentic-traces&i_xmode=ttft', { onBeforeLoad(win) { win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); unlockAgenticGate(win); }, }); - cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic Traces'); - cy.get('[data-testid="x-axis-mode-advanced"]') - .should('have.attr', 'data-state', 'active') - .and('contain.text', 'Interactivity'); + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); + cy.get('[data-testid="x-axis-mode-ttft"]') + .should('have.attr', 'aria-selected', 'true') + .and('contain.text', 'TTFT'); // Assert on the rendered chart too: the clobber happened one tick after // the buttons first painted, so a button-only check could pass too early. - cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Interactivity'); - cy.get('[data-testid="x-axis-mode-e2e-normalized-interactivity"]').should( + cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Time To First Token'); + cy.get('[data-testid="x-axis-mode-interactivity"]').should( 'have.attr', 'aria-selected', 'false', @@ -388,28 +391,61 @@ describe('X-axis mode URL param', () => { }, }); - cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic Traces'); + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); cy.get('[data-testid="percentile-selector"]').should('not.exist'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'P90'); }); }); describe('Default scenario', () => { - it('bare /inference defaults to 8K / 1K even when the model has agentic data', () => { - // Availability contains BOTH agentic and fixed-seq rows for the default - // model; the default selection must still resolve to 8K/1K, not agentic. - interceptFixedSequenceData(); + it('bare /inference opens on the Agentic scenario when the model has corresponding data', () => { + // Availability contains BOTH agentic and fixed-seq rows for DeepSeek-V4-Pro, + // so the untouched 8K/1K selection must not win. + interceptAgenticData(); + interceptDerivedAgenticMetrics(); cy.visit('/inference', { onBeforeLoad(win) { win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); }, }); + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); + // The explainer sits beside the trigger, linking out to the dataset page. + cy.get('[data-testid="scenario-agentic-info"]').should('exist'); + cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); + cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'P90'); + }); + + it('keeps 8K / 1K when the link asks for it explicitly', () => { + // An explicit selection always beats the availability-driven default. + interceptFixedSequenceData(); + cy.visit('/inference?i_seq=8k%2F1k', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); cy.get('[data-testid="scenario-selector"]').should('contain.text', '8K / 1K'); + cy.get('[data-testid="scenario-agentic-info"]').should('not.exist'); cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); // Fixed-seq plots the mean field — no percentile prefix on the axis label. cy.get('[data-testid="chart-figure"] svg').should('contain.text', 'Interactivity (tok/s/user)'); cy.get('[data-testid="chart-figure"] svg').should('not.contain.text', 'P90 Interactivity'); }); + + it('opens the Agentic scenario for another model with corresponding data', () => { + // DeepSeek-R1 is intentionally outside the original hard-coded model list; + // its agentic availability must still make the Agentic scenario the default. + cy.intercept('GET', '/api/v1/availability', { body: otherModelAvailability }).as( + 'availability', + ); + cy.intercept('GET', '/api/v1/benchmarks*', { body: otherModelBenchmarks }).as('benchmarks'); + cy.visit('/inference?g_model=DeepSeek-R1-0528', { + onBeforeLoad(win) { + win.localStorage.setItem('inferencex-star-modal-dismissed', String(Date.now())); + }, + }); + cy.get('[data-testid="scenario-selector"]').should('contain.text', 'Agentic'); + cy.get('[data-testid="scenario-agentic-info"]').should('exist'); + }); }); describe('Label defaults for fixed-sequence scenarios', () => { @@ -421,6 +457,7 @@ describe('Label defaults for fixed-sequence scenarios', () => { }, }); cy.get('[data-testid="scenario-selector"]').should('contain.text', '8K / 1K'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'unchecked'); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'unchecked'); cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'checked'); @@ -454,8 +491,9 @@ describe('Label defaults for fixed-sequence scenarios', () => { cy.get('[data-testid="x-axis-mode-ttft"]').should('have.attr', 'aria-selected', 'true'); }); - // Only agentic nests the latency modes: E2E Normalized Interactivity is - // agentic-only, so collapsing them here would leave an empty tab strip. + // The strip is flat everywhere since #736; what is still specific to a fixed + // sequence is that E2E Normalized Interactivity is agentic-only, so this strip + // carries three tabs rather than four. it('keeps the flat x-axis strip with no Advanced menu', () => { interceptFixedSequenceData(); cy.visit('/inference?i_seq=8k%2F1k', { @@ -481,6 +519,7 @@ describe('Label defaults for fixed-sequence scenarios', () => { }, }); cy.get('[data-testid="scenario-selector"]').should('contain.text', '8K / 1K'); + expandLegendAdvanced(); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); cy.get('#scatter-point-labels').should('have.attr', 'data-state', 'checked'); cy.get('#scatter-line-labels').should('have.attr', 'data-state', 'unchecked'); @@ -568,9 +607,19 @@ describe('X-Axis Mode Toggle — overlay path (finding #8 regression guard)', () cy.get('[data-testid="chart-figure"]').should('have.length.at.least', 1); }); + // Cypress clears intercepts between tests, so the derived-metrics stub is + // re-registered per test rather than once in `before`. Until #736 the + // agentic default was E2E Normalized Interactivity, so the fetch happened + // during `before` while the stub was still alive and React Query held the + // result for the rest of the spec. The default no longer fetches, so any + // later switch into that mode issues a fresh request. + beforeEach(() => { + interceptDerivedAgenticMetrics(); + }); + it('shows overlay (unofficial-run) watermark SVG when an overlay is loaded', () => { // Explicitly select Interactivity — do not rely on the agentic default mode. - selectAdvancedXAxisMode('interactivity', 'Interactivity'); + selectXAxisMode('interactivity', 'Interactivity'); // The unofficial-run pattern watermark appears when isUnofficialRun is true. cy.get('[data-testid="inference-chart-display"] svg pattern[id^="unofficial-pattern-"]').should( 'exist', @@ -580,13 +629,13 @@ describe('X-Axis Mode Toggle — overlay path (finding #8 regression guard)', () 'contain.text', 'P90 Interactivity (tok/s/user)', ); - cy.get('#chart-0 [data-testid="offload-halo-key"]') + cy.get('[data-testid="axis-metric-footer-chart-0"] [data-testid="offload-halo-key"]') .should('be.visible') .and('contain.text', 'KV offload ON'); }); it('switches to ttft x-axis mode and renders SVG with overlay points', () => { - selectAdvancedXAxisMode('ttft', 'TTFT'); + selectXAxisMode('ttft', 'TTFT'); cy.get('[data-testid="chart-figure"] h2').should('contain.text', 'Time To First Token'); // Overlay points render as triangles or circles inside the chart SVG. cy.get('[data-testid="inference-chart-display"] svg').should('exist'); diff --git a/packages/app/cypress/e2e/unofficial-watermark.cy.ts b/packages/app/cypress/e2e/unofficial-watermark.cy.ts index 043dc516f..944d25024 100644 --- a/packages/app/cypress/e2e/unofficial-watermark.cy.ts +++ b/packages/app/cypress/e2e/unofficial-watermark.cy.ts @@ -1,3 +1,5 @@ +import { expandLegendAdvanced } from '../support/legend-advanced'; + describe('Unofficial-run watermark', () => { before(() => { cy.fixture('api/benchmarks.json').then((benchmarks) => { @@ -20,6 +22,10 @@ describe('Unofficial-run watermark', () => { benchmarks: benchmarks.map((row: Record) => ({ ...row, is_multinode: true, + prefill_tp: 8, + prefill_ep: 1, + decode_tp: 8, + decode_ep: 1, metrics: { ...(row.metrics as Record | undefined), kv_offloading: 'dram', @@ -27,6 +33,8 @@ describe('Unofficial-run watermark', () => { kv_p2p_transfer: 'nixl', router_name: 'sglang-router', router_version: '0.3.2', + dcp_size: 8, + pcp_size: 4, server_gpu_cache_hit_rate: 0.875, }, run_url: runUrl, @@ -79,9 +87,18 @@ describe('Unofficial-run watermark', () => { expect(tooltip).to.contain.text('KV Offload Engine: HiCache'); expect(tooltip).to.contain.text('KV Transfer Engine: NIXL'); expect(tooltip).to.contain.text('Router: SGLang Router 0.3.2'); + expect(tooltip).to.contain.text('Decode Context Parallelism (DCP): 8'); + expect(tooltip).to.contain.text('Prefill Context Parallelism (PCP): 4'); expect(tooltip).to.contain.text('Chip Cache Hit Rate: 87.5%'); }); + expandLegendAdvanced(); + cy.get('#scatter-parallelism-labels').click(); + cy.get('[data-testid="scatter-graph"] .unofficial-overlay-pt') + .first() + .find('.overlay-label') + .should('contain.text', 'TP8/DCP8/PCP4'); + cy.get('[data-testid="scatter-graph"]').first().scrollIntoView(); cy.screenshot('unofficial-watermark', { capture: 'viewport' }); diff --git a/packages/app/cypress/e2e/url-params.cy.ts b/packages/app/cypress/e2e/url-params.cy.ts index 6c8272181..6f160bcec 100644 --- a/packages/app/cypress/e2e/url-params.cy.ts +++ b/packages/app/cypress/e2e/url-params.cy.ts @@ -3,6 +3,7 @@ * update the visible output (selector text, SVG axis labels). * Merged from url-params.cy.ts + chart-filter-effects.cy.ts + high-contrast.cy.ts. */ +import { expandLegendAdvanced } from '../support/legend-advanced'; const visitWithDismissedModal = (path: string) => { cy.visit(path, { onBeforeLoad(win) { @@ -38,10 +39,134 @@ describe('URL Parameter Persistence', () => { }); describe('Inference legend', () => { - it('i_legend=0 collapses the sidebar legend on load', () => { + it('i_legend=0 hides the sidebar legend on load and the reopen button restores it', () => { visitWithDismissedModal('/inference?i_legend=0'); + cy.get('[data-testid="legend-open-button"]').first().should('be.visible'); + cy.get('.sidebar-legend').should('not.exist'); + + cy.get('[data-testid="legend-open-button"]').first().click(); + cy.get('.sidebar-legend').first().should('be.visible'); + cy.get('[data-testid="legend-open-button"]').should('not.exist'); + }); + + // The address bar is deliberately left clean after load (see url-state.ts) + // — filter changes like closing the legend only reach the in-memory share + // state, so this asserts the UI transition rather than location.search. + it('legend close button hides the panel and the reopen button restores it', () => { + visitWithDismissedModal('/inference'); + cy.get('.sidebar-legend').first().should('be.visible'); + + cy.get('[data-testid="legend-close-button"]').first().click(); + cy.get('.sidebar-legend').should('not.exist'); + cy.get('[data-testid="legend-open-button"]').first().should('be.visible'); + + cy.get('[data-testid="legend-open-button"]').first().click(); cy.get('.sidebar-legend').first().should('be.visible'); - cy.get('.sidebar-legend').first().should('not.have.class', 'bg-accent'); + cy.get('[data-testid="legend-open-button"]').should('not.exist'); + }); + + it('preserves a legend subset when chart metrics change', () => { + visitWithDismissedModal('/inference'); + + cy.get('[data-testid="chart-legend"] input[type="checkbox"]:checked').should( + 'have.length.greaterThan', + 1, + ); + cy.get('[data-testid="chart-legend"] [role="button"][aria-label^="Hide "]') + .first() + .closest('li') + .find('input[type="checkbox"]') + .invoke('attr', 'id') + .then((hiddenInputId) => { + expect(hiddenInputId).to.be.a('string'); + expect(hiddenInputId).to.have.length.greaterThan(0); + const selector = `#${CSS.escape(hiddenInputId!)}`; + + cy.get(selector).parent().find('[role="button"][aria-label^="Hide "]').click(); + cy.get(selector).should('not.be.checked'); + + cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); + cy.contains('[role="option"]', 'All-in Provisioned Joules per Total Token').click({ + force: true, + }); + cy.get(selector).should('not.be.checked'); + + cy.get('[data-testid="x-axis-mode-ttft"]').click(); + cy.get('[data-testid="x-axis-mode-ttft"]').should('have.attr', 'aria-selected', 'true'); + cy.get(selector).should('not.be.checked'); + }); + }); + + it('refreshes the automatic Best per SKU selection when the metric changes', () => { + visitWithDismissedModal('/inference'); + + // Best per SKU lives in the Quick Filters dialog now. + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filter-best-per-sku"]').should( + 'have.attr', + 'data-state', + 'checked', + ); + cy.contains('button', 'Done').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('not.exist'); + + cy.get('[data-testid="chart-legend"] ul input[type="checkbox"]:checked') + .then(($inputs) => [...$inputs].map((input) => input.id).toSorted()) + .then((before) => { + cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); + cy.contains( + '[role="option"]', + 'Cost per Million Total Tokens (Owning - Hyperscaler)', + ).click({ force: true }); + + cy.get('[data-testid="scatter-quick-filters"]').click(); + cy.get('[data-testid="quick-filter-best-per-sku"]').should( + 'have.attr', + 'data-state', + 'checked', + ); + cy.contains('button', 'Done').click(); + cy.get('[data-testid="quick-filters-dialog"]').should('not.exist'); + cy.get('[data-testid="x-axis-mode-ttft"]').click(); + cy.get('[data-testid="x-axis-mode-ttft"]').should('have.attr', 'aria-selected', 'true'); + cy.get('[data-testid="chart-legend"] ul input[type="checkbox"]:checked').then( + ($inputs) => { + const after = [...$inputs].map((input) => input.id).toSorted(); + expect(after, 'metric-specific Best per SKU winners').not.to.deep.equal(before); + }, + ); + }); + }); + }); + + describe('Provider remount state', () => { + it('preserves filters across a standalone dashboard route', () => { + visitWithDismissedModal('/inference'); + cy.get('[data-testid="model-selector"]').click(); + cy.contains('[role="option"]', 'Qwen3.5 397B').click(); + cy.get('[data-testid="model-selector"]').should('contain.text', 'Qwen3.5 397B'); + // Navigate immediately to cover pending writes inside the debounce window. + cy.get('[data-testid="tab-trigger-gpu-specs"]').click(); + cy.url().should('include', '/gpu-specs'); + cy.get('[data-testid="tab-trigger-inference"]').click(); + + cy.get('[data-testid="model-selector"]').should('contain.text', 'Qwen3.5 397B'); + }); + + it('preserves the newest pending model across a retained-provider tab switch', () => { + visitWithDismissedModal('/inference'); + cy.get('[data-testid="model-selector"]').click(); + cy.contains('[role="option"]', 'MiniMax M3 428B').click(); + cy.get('[data-testid="model-selector"]').should('contain.text', 'MiniMax M3 428B'); + cy.wait(200); + + cy.get('[data-testid="model-selector"]').click(); + cy.contains('[role="option"]', 'Qwen3.5 397B').click(); + cy.get('[data-testid="tab-trigger-evaluation"]').click(); + cy.location('pathname').should('eq', '/evaluation'); + cy.get('[data-testid="tab-trigger-inference"]').click(); + cy.location('pathname').should('eq', '/inference'); + cy.get('[data-testid="model-selector"]').should('contain.text', 'Qwen3.5 397B'); }); }); @@ -63,10 +188,12 @@ describe('URL Parameter Persistence', () => { it('changing Y-axis metric via dropdown updates SVG axis label', () => { visitWithDismissedModal('/inference'); + // The dashboard opens on the tokens-per-dollar default; this asserts the + // starting label before switching, not that throughput is the default. cy.get('[data-testid="scatter-graph"]') .first() .find('svg text[transform="rotate(-90)"]') - .should('contain.text', 'Throughput'); + .should('contain.text', 'Total Tokens per $1 USD'); cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); cy.contains('[role="option"]', 'Cost per Million Total Tokens (Owning - Hyperscaler)').click({ @@ -79,6 +206,28 @@ describe('URL Parameter Persistence', () => { .should('have.text', 'Cost per Million Total Tokens ($)'); }); + it('tokens-per-dollar URL metric is independent from cost per million', () => { + visitWithDismissedModal('/inference?i_metric=y_tokensPerDollarH'); + + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Total Tokens per $1 USD (Owning - Hyperscaler)', + ); + cy.get('[data-testid="scatter-graph"]') + .first() + .find('svg text[transform="rotate(-90)"]') + .should('have.text', 'Total Tokens per $1 USD (tok/$)'); + }); + + it('keeps the legacy i_metric=y alias on raw throughput', () => { + visitWithDismissedModal('/inference?i_metric=y'); + + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Token Throughput per Chip', + ); + }); + it('selecting a Y-axis metric updates the displayed value', () => { visitWithDismissedModal('/inference'); cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); @@ -121,6 +270,62 @@ describe('URL Parameter Persistence', () => { .find('svg text[transform="rotate(-90)"]') .should('contain.text', 'Token Throughput per All in Utility MW'); }); + + it('keeps tooltip rulers aligned after a zoomed metric switch', () => { + visitWithDismissedModal('/inference?i_metric=y_tpPerGpu'); + expandLegendAdvanced(); + cy.get('#scatter-log-scale').first().click(); + + cy.get('[data-testid="scatter-graph"] [data-testid="d3-chart-svg"]') + .first() + .then(($svg) => { + const svg = $svg[0] as unknown as SVGSVGElement & { __zoom?: { k: number } }; + const bounds = svg.getBoundingClientRect(); + for (let i = 0; i < 2; i += 1) { + svg.dispatchEvent( + new WheelEvent('wheel', { + deltaY: -240, + clientX: bounds.x + bounds.width / 2, + clientY: bounds.y + bounds.height / 2, + shiftKey: true, + bubbles: true, + cancelable: true, + }), + ); + } + expect(svg.__zoom?.k, 'active zoom scale').to.be.greaterThan(1); + }); + + cy.get('[data-testid="yaxis-metric-selector"]').click({ force: true }); + cy.contains('[role="option"]', 'Input Token Throughput per Chip').click({ force: true }); + cy.get('[data-testid="yaxis-metric-selector"]').should( + 'contain.text', + 'Input Token Throughput per Chip', + ); + + cy.get('[data-testid="scatter-graph"]') + .first() + .within(() => { + cy.get('.dot-group') + .first() + .then(($point) => { + const match = $point + .attr('transform') + ?.match(/translate\((?[^,]+),(?[^)]+)\)/u); + expect(match, 'point transform').not.to.equal(null); + const pointX = Number(match?.groups?.x); + const pointY = Number(match?.groups?.y); + + cy.wrap($point).trigger('mouseenter', { force: true }); + cy.get('.vertical-ruler') + .invoke('attr', 'x1') + .then((x) => expect(Number(x)).to.be.closeTo(pointX, 1)); + cy.get('.horizontal-ruler') + .invoke('attr', 'y1') + .then((y) => expect(Number(y)).to.be.closeTo(pointY, 1)); + }); + }); + }); }); describe('Reliability date range', () => { @@ -196,7 +401,9 @@ describe('URL Parameter Persistence', () => { }); it('/inference with invalid ?i_prec=junk falls back to the default', () => { - visitWithErrorSpy('/inference?i_prec=junk'); + // Pair with a multi-precision model — the FP4-only default model hides + // the precision selector entirely, leaving nothing to assert against. + visitWithErrorSpy('/inference?g_model=DeepSeek-R1-0528&i_prec=junk'); cy.get('[data-testid="precision-multiselect"]').invoke('text').should('not.contain', 'junk'); assertNoHydrationMismatch(); }); @@ -239,24 +446,28 @@ describe('URL Parameter Persistence', () => { it('inference loads with high contrast off by default', () => { visitWithDismissedModal('/inference'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'unchecked'); }); it('i_hc=0 disables high contrast on load', () => { visitWithDismissedModal('/inference?i_hc=0'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'unchecked'); }); it('i_hc=1 applies high contrast on load', () => { visitWithDismissedModal('/inference?i_hc=1'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'checked'); }); it('multiple high contrast params can coexist in URL', () => { visitWithDismissedModal('/inference?i_hc=1&r_hc=1&e_hc=1'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'checked'); }); @@ -274,8 +485,8 @@ describe('URL Parameter Persistence', () => { }); it('historical trends tab shares the inference high-contrast default (off)', () => { - // Historical reads highContrast from the same InferenceContext as the - // scatter chart, so it inherits the default-off behavior. + // Historical reads highContrast from the shared inference display domain, + // so it inherits the default-off behavior. visitWithDismissedModal('/historical'); cy.get('[data-testid="historical-trends-display"]').should('exist'); cy.get('#historical-high-contrast').first().should('have.attr', 'data-state', 'unchecked'); @@ -292,6 +503,7 @@ describe('URL Parameter Persistence', () => { it('a bare /inference link with neither param renders high contrast AND parallelism labels off', () => { visitWithDismissedModal('/inference'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'unchecked'); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'unchecked'); }); @@ -299,6 +511,7 @@ describe('URL Parameter Persistence', () => { it('i_hc=1&i_advlabel=1 enables both high contrast and parallelism labels on load', () => { visitWithDismissedModal('/inference?i_hc=1&i_advlabel=1'); cy.get('[data-testid="scatter-graph"]').should('exist'); + expandLegendAdvanced(); cy.get('#scatter-high-contrast').first().should('have.attr', 'data-state', 'checked'); cy.get('#scatter-parallelism-labels').should('have.attr', 'data-state', 'checked'); }); diff --git a/packages/app/cypress/e2e/yaxis-metrics-render.cy.ts b/packages/app/cypress/e2e/yaxis-metrics-render.cy.ts index c2a9a7bec..98a66f46a 100644 --- a/packages/app/cypress/e2e/yaxis-metrics-render.cy.ts +++ b/packages/app/cypress/e2e/yaxis-metrics-render.cy.ts @@ -21,7 +21,26 @@ describe('Y-Axis Metrics All Render Data', () => { 'Cost per Million Input Tokens (Owning - Hyperscaler)', 'Cost per Million Input Tokens (Owning - Neocloud Giant)', 'Cost per Million Input Tokens (3 Year Rental)', + 'Total Tokens per $1 USD (Owning - Hyperscaler)', + 'Total Tokens per $1 USD (Owning - Neocloud Giant)', + 'Total Tokens per $1 USD (3 Year Rental)', + 'Output Tokens per $1 USD (Owning - Hyperscaler)', + 'Output Tokens per $1 USD (Owning - Neocloud Giant)', + 'Output Tokens per $1 USD (3 Year Rental)', + 'Input Tokens per $1 USD (Owning - Hyperscaler)', + 'Input Tokens per $1 USD (Owning - Neocloud Giant)', + 'Input Tokens per $1 USD (3 Year Rental)', + 'Total Tokens per ¥1 RMB (Owning - Hyperscaler)', + 'Total Tokens per ¥1 RMB (Owning - Neocloud Giant)', + 'Total Tokens per ¥1 RMB (3 Year Rental)', + 'Output Tokens per ¥1 RMB (Owning - Hyperscaler)', + 'Output Tokens per ¥1 RMB (Owning - Neocloud Giant)', + 'Output Tokens per ¥1 RMB (3 Year Rental)', + 'Input Tokens per ¥1 RMB (Owning - Hyperscaler)', + 'Input Tokens per ¥1 RMB (Owning - Neocloud Giant)', + 'Input Tokens per ¥1 RMB (3 Year Rental)', 'Cost per Million Total Tokens (Custom User Values)', + 'Total Tokens per $1 USD (Custom User Values)', 'Token Throughput per All in Utility MW (Custom User Values)', 'All-in Provisioned Joules per Total Token', 'All-in Provisioned Joules per Output Token', diff --git a/packages/app/cypress/e2e/zh-pages.cy.ts b/packages/app/cypress/e2e/zh-pages.cy.ts index 2ae1441db..7e114d37c 100644 --- a/packages/app/cypress/e2e/zh-pages.cy.ts +++ b/packages/app/cypress/e2e/zh-pages.cy.ts @@ -5,17 +5,13 @@ describe('Chinese (/zh) pages', () => { }); it('renders the Chinese landing content', () => { - cy.contains('h2', '探索 InferenceX').should('exist'); - cy.contains('快速对比').should('exist'); - }); - - it('links to the Chinese overview and full dashboard', () => { - cy.get('[data-testid="landing-overview-link"]') - .should('have.attr', 'href', '/zh/overview') - .and('have.text', '总览'); - cy.get('[data-testid="landing-full-dashboard-link"]') - .should('have.attr', 'href', '/zh/inference') - .and('have.text', '完整仪表板'); + cy.get('[data-testid="intro-section"]').should('contain.text', '智能体推理基准测试'); + cy.get('[data-testid="splash-text"]').should('have.text', 'AgentX 来了!!'); + // Quick Comparisons is hidden behind SHOW_QUICK_COMPARISONS in + // landing-page.tsx; the card and its Chinese strings still exist in the + // source, so assert it is not rendered rather than dropping the check. + cy.get('[data-testid="landing-quick-comparisons"]').should('not.exist'); + cy.contains('快速对比').should('not.exist'); }); it('sets hreflang alternates to the English homepage', () => { @@ -27,14 +23,40 @@ describe('Chinese (/zh) pages', () => { cy.get('[data-testid="language-toggle"]').should('have.attr', 'href', '/'); }); + it('header links to the Chinese AgentX page', () => { + cy.get('[data-testid="nav-link-agentx"]') + .should('contain.text', 'AgentX') + .and('have.attr', 'href', '/zh/agentx'); + }); + + it('renders the AgentX hero on the Chinese landing page', () => { + cy.get('[data-testid="compare-agentx-primary"]').within(() => { + cy.get('h2').should('have.text', '真实智能体工作负载下的推理性能对比'); + cy.get('[data-testid="compare-agentx-overview-link"]') + .should('contain.text', '总览') + .and('have.attr', 'href', '/zh/overview'); + cy.get('[data-testid="compare-agentx-methodology-link"]').should('not.exist'); + // Ledger NEW pills localize to 新 on the Chinese landing page. + cy.get('[data-testid^="compare-agentx-model-"] [data-new-badge="agentx-ledger"]') + .should('have.length', 6) + .each(($badge) => expect($badge.text()).to.equal('新')); + }); + }); + it('footer renders in Chinese with zh-internal links', () => { - cy.get('[data-testid="footer-brand-description"]').should('contain.text', '开源推理基准测试'); + cy.get('[data-testid="footer-brand-description"]').should( + 'contain.text', + 'InferenceX 持续开展开源的 agentic 推理基准测试', + ); cy.get('[data-testid="footer-link-supporters"]') - .should('contain.text', '支持者') + .should('contain.text', '业界评价') .and('have.attr', 'href', '/zh/quotes'); - cy.get('[data-testid="footer-link-datasets"]') - .should('contain.text', '数据集') - .and('have.attr', 'href', '/zh/datasets'); + cy.get('[data-testid="footer-link-agentx"]') + .should('contain.text', 'AgentX') + .and('have.attr', 'href', '/zh/agentx'); + cy.get('[data-testid="footer-link-about"]') + .should('have.text', '关于 SemiAnalysis') + .and('have.attr', 'href', 'https://semianalysis.com/about/'); cy.get('[data-testid="footer-link-articles"]') .should('contain.text', '文章') .and('have.attr', 'href', '/zh/blog'); @@ -43,9 +65,7 @@ describe('Chinese (/zh) pages', () => { 'href', '/zh/land-acknowledgement', ); - cy.get('[data-testid="footer-link-zh"]') - .should('contain.text', 'English') - .and('have.attr', 'href', '/'); + cy.get('[data-testid="footer-link-zh"]').should('not.exist'); }); }); @@ -56,7 +76,8 @@ describe('Chinese (/zh) pages', () => { it('renders the Chinese SEO intro above the chart', () => { cy.get('[data-testid="zh-tab-intro"]').within(() => { - cy.contains('h1', 'AI 推理基准测试').should('exist'); + cy.contains('h1', '智能体推理基准测试').should('exist'); + cy.contains('长上下文、多轮').should('exist'); }); }); @@ -68,6 +89,28 @@ describe('Chinese (/zh) pages', () => { }); }); + describe('zh FAQ', () => { + it('uses the same direct-link anchor as the English FAQ', () => { + cy.visit('/zh/about#faq-normalized-interactivity'); + + cy.get('#faq-normalized-interactivity') + .should('be.visible') + .within(() => { + cy.contains( + 'a[href="#faq-normalized-interactivity"]', + '端到端归一化交互性与交互性有何区别?', + ).should('be.visible'); + cy.contains('TTFT 越长,归一化指标越低').should('be.visible'); + cy.get('[data-testid="faq-copy-link-faq-normalized-interactivity"]') + .should('be.visible') + .and('have.attr', 'title', '复制链接') + .find('svg.lucide-link') + .should('be.visible'); + }); + cy.location('hash').should('eq', '#faq-normalized-interactivity'); + }); + }); + describe('zh blog', () => { before(() => { cy.visit('/zh/blog'); diff --git a/packages/app/cypress/fixtures/api/_manifest.json b/packages/app/cypress/fixtures/api/_manifest.json new file mode 100644 index 000000000..089c83483 --- /dev/null +++ b/packages/app/cypress/fixtures/api/_manifest.json @@ -0,0 +1,97 @@ +{ + "schemaVersion": 1, + "generatedAt": "2026-08-20T08:25:30.857Z", + "fixtures": { + "agentic-catalog": { + "bytes": 10027, + "capturedAt": null, + "sha256": "b4d7da73dbb839a8c9444af4bf90cdcd2e25c53970214937ecc122c86c13ae58", + "source": null, + "topLevel": "array" + }, + "availability": { + "bytes": 147942, + "capturedAt": null, + "sha256": "ffcf2ed4d741a592c50a4720d288acff176202b737942edb048ac46f0a9409ba", + "source": null, + "topLevel": "array" + }, + "benchmarks-history": { + "bytes": 1859995, + "capturedAt": null, + "sha256": "9e7efc17d018d25acd13095ac0c33ed3c20aaf0d86c225792c0a73bde8e12d15", + "source": null, + "topLevel": "array" + }, + "benchmarks": { + "bytes": 775884, + "capturedAt": null, + "sha256": "f46823a76cb63248705329f327e1494df02b4dfa4df0898b7ee084c75c20bd52", + "source": null, + "topLevel": "array" + }, + "collectivex-latest": { + "bytes": 63278, + "capturedAt": "2026-08-20T08:25:30.857Z", + "sha256": "66dd9287f7b5c1d33fa2dc76f69622b25dc42d89fcd62644b01c8edd7d6290b3", + "source": "synthetic://collectivex", + "topLevel": "object" + }, + "collectivex-run-159": { + "bytes": 20374, + "capturedAt": "2026-08-20T08:25:30.857Z", + "sha256": "54e33df39b51158ed3fb0956dd70e10e2c3540845dd62ab7c64fabad175d2259", + "source": "synthetic://collectivex", + "topLevel": "object" + }, + "collectivex-runs": { + "bytes": 687, + "capturedAt": "2026-08-20T08:25:30.857Z", + "sha256": "9f3d47af2543fc56b0afec7b0c577ba08fcfc2d6c91b07d98163e721e50146c8", + "source": "synthetic://collectivex", + "topLevel": "object" + }, + "evaluations": { + "bytes": 474940, + "capturedAt": null, + "sha256": "edd2740ddbf1104c763114018fa081e96c39d5edfa3f2e2424a7ce4b724a06c9", + "source": null, + "topLevel": "array" + }, + "overview-history-rows": { + "bytes": 5002, + "capturedAt": null, + "sha256": "73caa715550d07044dcad6f14dd6661fb28a203883470ad47f272df1eb3c4c82", + "source": null, + "topLevel": "object" + }, + "overview-rows": { + "bytes": 22152, + "capturedAt": null, + "sha256": "ccdc4c23fc6b5ab6f664e19db2ec31efe2b34318976baec06e9219987e240922", + "source": null, + "topLevel": "object" + }, + "reliability": { + "bytes": 530238, + "capturedAt": null, + "sha256": "cf36cdc2446c75bd191a22f027823bce6ec1380937b5387f274a773dbc19ee66", + "source": null, + "topLevel": "array" + }, + "submissions": { + "bytes": 130122, + "capturedAt": null, + "sha256": "cd3d37a878b708eab504b0a6bca6d2af408101d88bee47503f0a656d6c60f1f9", + "source": null, + "topLevel": "object" + }, + "workflow-info": { + "bytes": 4941, + "capturedAt": null, + "sha256": "f28d2220e5246ad70113e38e49394bc7a0bfe9afba8cedf08b90e39042e53357", + "source": null, + "topLevel": "object" + } + } +} diff --git a/packages/app/cypress/fixtures/api/agentic-catalog.json b/packages/app/cypress/fixtures/api/agentic-catalog.json new file mode 100644 index 000000000..d0f1536f8 --- /dev/null +++ b/packages/app/cypress/fixtures/api/agentic-catalog.json @@ -0,0 +1,519 @@ +[ + { + "id": 440056, + "model": "dsv4", + "hardware": "b200", + "framework": "sglang", + "precision": "fp4", + "points": 41, + "minConc": 1, + "maxConc": 68, + "latestDate": "2026-08-19" + }, + { + "id": 439967, + "model": "dsv4", + "hardware": "b200", + "framework": "vllm", + "precision": "fp4", + "points": 38, + "minConc": 1, + "maxConc": 196, + "latestDate": "2026-08-17" + }, + { + "id": 439778, + "model": "dsv4", + "hardware": "b300", + "framework": "sglang", + "precision": "fp4", + "points": 41, + "minConc": 1, + "maxConc": 196, + "latestDate": "2026-08-13" + }, + { + "id": 439526, + "model": "dsv4", + "hardware": "b300", + "framework": "vllm", + "precision": "fp4", + "points": 43, + "minConc": 1, + "maxConc": 576, + "latestDate": "2026-08-11" + }, + { + "id": 439970, + "model": "dsv4", + "hardware": "gb200", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 39, + "minConc": 1, + "maxConc": 576, + "latestDate": "2026-08-18" + }, + { + "id": 439331, + "model": "dsv4", + "hardware": "gb300", + "framework": "dynamo-sglang", + "precision": "fp4", + "points": 17, + "minConc": 1, + "maxConc": 1536, + "latestDate": "2026-08-10" + }, + { + "id": 439874, + "model": "dsv4", + "hardware": "gb300", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 6, + "minConc": 1, + "maxConc": 1152, + "latestDate": "2026-08-14" + }, + { + "id": 439550, + "model": "dsv4", + "hardware": "h200", + "framework": "dynamo-sglang", + "precision": "fp8", + "points": 5, + "minConc": 1, + "maxConc": 16, + "latestDate": "2026-08-11" + }, + { + "id": 439512, + "model": "dsv4", + "hardware": "mi355x", + "framework": "mori-sglang", + "precision": "fp4", + "points": 8, + "minConc": 1, + "maxConc": 128, + "latestDate": "2026-08-11" + }, + { + "id": 439840, + "model": "dsv4", + "hardware": "mi355x", + "framework": "sglang", + "precision": "fp4", + "points": 8, + "minConc": 1, + "maxConc": 48, + "latestDate": "2026-08-14" + }, + { + "id": 440031, + "model": "dsv4", + "hardware": "mi355x", + "framework": "vllm", + "precision": "fp4", + "points": 18, + "minConc": 1, + "maxConc": 64, + "latestDate": "2026-08-19" + }, + { + "id": 439990, + "model": "glm5.2", + "hardware": "b200", + "framework": "sglang", + "precision": "fp4", + "points": 10, + "minConc": 1, + "maxConc": 16, + "latestDate": "2026-08-18" + }, + { + "id": 439987, + "model": "glm5.2", + "hardware": "b300", + "framework": "sglang", + "precision": "fp4", + "points": 10, + "minConc": 1, + "maxConc": 16, + "latestDate": "2026-08-18" + }, + { + "id": 440079, + "model": "glm5.2", + "hardware": "gb200", + "framework": "dynamo-sglang", + "precision": "fp4", + "points": 14, + "minConc": 1, + "maxConc": 128, + "latestDate": "2026-08-19" + }, + { + "id": 439925, + "model": "glm5.2", + "hardware": "gb300", + "framework": "dynamo-sglang", + "precision": "fp4", + "points": 7, + "minConc": 2, + "maxConc": 192, + "latestDate": "2026-08-16" + }, + { + "id": 439294, + "model": "glm5.2", + "hardware": "h200", + "framework": "dynamo-sglang", + "precision": "fp8", + "points": 3, + "minConc": 8, + "maxConc": 16, + "latestDate": "2026-08-08" + }, + { + "id": 439298, + "model": "glm5.2", + "hardware": "mi325x", + "framework": "sglang", + "precision": "fp8", + "points": 7, + "minConc": 1, + "maxConc": 8, + "latestDate": "2026-08-08" + }, + { + "id": 440067, + "model": "glm5.2", + "hardware": "mi355x", + "framework": "atom", + "precision": "fp4", + "points": 7, + "minConc": 1, + "maxConc": 10, + "latestDate": "2026-08-19" + }, + { + "id": 439530, + "model": "glm5.2", + "hardware": "mi355x", + "framework": "sglang", + "precision": "fp4", + "points": 10, + "minConc": 1, + "maxConc": 16, + "latestDate": "2026-08-11" + }, + { + "id": 440070, + "model": "kimik3", + "hardware": "b200", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 15, + "minConc": 1, + "maxConc": 96, + "latestDate": "2026-08-19" + }, + { + "id": 439937, + "model": "kimik3", + "hardware": "b300", + "framework": "vllm", + "precision": "fp4", + "points": 20, + "minConc": 1, + "maxConc": 70, + "latestDate": "2026-08-16" + }, + { + "id": 440103, + "model": "kimik3", + "hardware": "gb200", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 31, + "minConc": 1, + "maxConc": 384, + "latestDate": "2026-08-18" + }, + { + "id": 440000, + "model": "kimik3", + "hardware": "gb300", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 7, + "minConc": 1, + "maxConc": 74, + "latestDate": "2026-08-18" + }, + { + "id": 438883, + "model": "kimik3", + "hardware": "h200", + "framework": "vllm", + "precision": "fp4", + "points": 35, + "minConc": 1, + "maxConc": 32, + "latestDate": "2026-08-07" + }, + { + "id": 439660, + "model": "kimik3", + "hardware": "mi355x", + "framework": "vllm", + "precision": "fp4", + "points": 4, + "minConc": 1, + "maxConc": 10, + "latestDate": "2026-08-12" + }, + { + "id": 440146, + "model": "minimaxm3", + "hardware": "b200", + "framework": "trt", + "precision": "fp4", + "points": 22, + "minConc": 1, + "maxConc": 45, + "latestDate": "2026-08-20" + }, + { + "id": 439916, + "model": "minimaxm3", + "hardware": "b200", + "framework": "vllm", + "precision": "fp4", + "points": 26, + "minConc": 1, + "maxConc": 40, + "latestDate": "2026-08-16" + }, + { + "id": 440040, + "model": "minimaxm3", + "hardware": "b300", + "framework": "trt", + "precision": "fp4", + "points": 11, + "minConc": 1, + "maxConc": 30, + "latestDate": "2026-08-19" + }, + { + "id": 439907, + "model": "minimaxm3", + "hardware": "b300", + "framework": "vllm", + "precision": "fp4", + "points": 24, + "minConc": 1, + "maxConc": 75, + "latestDate": "2026-08-16" + }, + { + "id": 439888, + "model": "minimaxm3", + "hardware": "gb200", + "framework": "dynamo-vllm", + "precision": "fp4", + "points": 17, + "minConc": 1, + "maxConc": 48, + "latestDate": "2026-08-15" + }, + { + "id": 439584, + "model": "minimaxm3", + "hardware": "h100", + "framework": "vllm", + "precision": "fp8", + "points": 7, + "minConc": 1, + "maxConc": 8, + "latestDate": "2026-08-12" + }, + { + "id": 439641, + "model": "minimaxm3", + "hardware": "h200", + "framework": "vllm", + "precision": "fp8", + "points": 8, + "minConc": 1, + "maxConc": 14, + "latestDate": "2026-08-12" + }, + { + "id": 439832, + "model": "minimaxm3", + "hardware": "mi300x", + "framework": "vllm", + "precision": "fp8", + "points": 6, + "minConc": 2, + "maxConc": 16, + "latestDate": "2026-08-13" + }, + { + "id": 439824, + "model": "minimaxm3", + "hardware": "mi325x", + "framework": "vllm", + "precision": "fp8", + "points": 9, + "minConc": 1, + "maxConc": 18, + "latestDate": "2026-08-13" + }, + { + "id": 439556, + "model": "minimaxm3", + "hardware": "mi355x", + "framework": "vllm", + "precision": "fp4", + "points": 17, + "minConc": 1, + "maxConc": 40, + "latestDate": "2026-08-12" + }, + { + "id": 438230, + "model": "qwen3.5", + "hardware": "b200", + "framework": "sglang", + "precision": "fp4", + "points": 32, + "minConc": 1, + "maxConc": 72, + "latestDate": "2026-08-03" + }, + { + "id": 438930, + "model": "qwen3.5", + "hardware": "b200", + "framework": "sglang", + "precision": "fp8", + "points": 32, + "minConc": 1, + "maxConc": 76, + "latestDate": "2026-08-07" + }, + { + "id": 438219, + "model": "qwen3.5", + "hardware": "b300", + "framework": "sglang", + "precision": "fp4", + "points": 28, + "minConc": 1, + "maxConc": 72, + "latestDate": "2026-08-03" + }, + { + "id": 438967, + "model": "qwen3.5", + "hardware": "b300", + "framework": "sglang", + "precision": "fp8", + "points": 34, + "minConc": 1, + "maxConc": 72, + "latestDate": "2026-08-07" + }, + { + "id": 440086, + "model": "qwen3.5", + "hardware": "gb200", + "framework": "dynamo-sglang", + "precision": "fp4", + "points": 33, + "minConc": 1, + "maxConc": 72, + "latestDate": "2026-08-19" + }, + { + "id": 438856, + "model": "qwen3.5", + "hardware": "gb300", + "framework": "dynamo-sglang", + "precision": "fp4", + "points": 14, + "minConc": 1, + "maxConc": 128, + "latestDate": "2026-08-07" + }, + { + "id": 439945, + "model": "qwen3.5", + "hardware": "gb300", + "framework": "dynamo-trt", + "precision": "fp4", + "points": 6, + "minConc": 7, + "maxConc": 704, + "latestDate": "2026-08-16" + }, + { + "id": 438985, + "model": "qwen3.5", + "hardware": "h100", + "framework": "sglang", + "precision": "fp8", + "points": 9, + "minConc": 1, + "maxConc": 16, + "latestDate": "2026-08-07" + }, + { + "id": 439978, + "model": "qwen3.5", + "hardware": "h200", + "framework": "sglang", + "precision": "fp8", + "points": 16, + "minConc": 2, + "maxConc": 24, + "latestDate": "2026-08-18" + }, + { + "id": 439573, + "model": "qwen3.5", + "hardware": "mi300x", + "framework": "sglang", + "precision": "fp8", + "points": 10, + "minConc": 4, + "maxConc": 32, + "latestDate": "2026-08-12" + }, + { + "id": 439628, + "model": "qwen3.5", + "hardware": "mi325x", + "framework": "sglang", + "precision": "fp8", + "points": 32, + "minConc": 1, + "maxConc": 80, + "latestDate": "2026-08-12" + }, + { + "id": 440140, + "model": "qwen3.5", + "hardware": "mi355x", + "framework": "sglang", + "precision": "fp4", + "points": 31, + "minConc": 1, + "maxConc": 40, + "latestDate": "2026-08-20" + } +] diff --git a/packages/app/cypress/fixtures/api/collectivex-latest.json b/packages/app/cypress/fixtures/api/collectivex-latest.json index 42f0c5e76..c0ae5a471 100644 --- a/packages/app/cypress/fixtures/api/collectivex-latest.json +++ b/packages/app/cypress/fixtures/api/collectivex-latest.json @@ -1 +1 @@ -{"version":1,"run":{"run_id":"160","run_attempt":1,"generated_at":"2026-07-08T12:20:00Z","conclusion":"success","source_sha":"cccccccccccccccccccccccccccccccccccccccc","requested_cases":5,"terminal_cases":4,"measured_cases":3,"unsupported_cases":1,"failed_cases":0,"requested_points":50,"terminal_points":40,"measured_points":30,"covered_skus":["b200-dgxc","b300","h200-dgxc","mi355x"]},"coverage":[{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · bf16","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"mi355x-mori-deepseek-v3-normal-decode-ep16-uniform-bf16","label":"mi355x · mori · normal · decode · EP16 · bf16","disposition":"runnable","sku":"mi355x","backend":"mori","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"xgmi","scale_out_transport":"rdma","topology_class":"mi355x-xgmi-rdma"},"points":[{"tokens_per_rank":1,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":4096,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":8192,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · fp8","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"fp8","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"b300-deepep-v2-deepseek-v3-normal-decode-ep16-uniform-bf16","label":"b300 · deepep-v2 · normal · decode · EP16 · bf16","disposition":"unsupported","sku":"b300","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":"rdma","topology_class":"b300-nvlink-rdma"},"points":[{"tokens_per_rank":1,"global_tokens":16,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":2,"global_tokens":32,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":4,"global_tokens":64,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":8,"global_tokens":128,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":16,"global_tokens":256,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":32,"global_tokens":512,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":64,"global_tokens":1024,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":128,"global_tokens":2048,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":256,"global_tokens":4096,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":512,"global_tokens":8192,"terminal_status":"unsupported","reason":"backend-platform-unsupported"}],"outcome":"unsupported","reason":"backend-platform-unsupported","detail":"unsupported by the selected backend/platform"},{"case_id":"b200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","label":"b200-dgxc · deepep-v2 · normal · decode · EP8 · bf16","disposition":"runnable","sku":"b200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"b200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"pending","reason":"pending"}],"outcome":"pending","reason":"pending","detail":null}],"series":[{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","phase":"decode","mode":"normal","precision":"bf16","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]},{"series_id":"mi355x-mori-deepseek-v3-normal-decode-ep16-uniform-bf16","phase":"decode","mode":"normal","precision":"bf16","backend":"mori","system":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"xgmi","scale_out_transport":"rdma","topology_class":"mi355x-xgmi-rdma","sku":"mi355x","vendor":"amd"},"points":[{"tokens_per_rank":1,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.95203836930456,"p90":55.51114663824496,"p95":53.52860568687907,"p99":49.96003197442047},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":100.19893333333334,"p90":92.77679012345678,"p95":89.46333333333332,"p99":83.4991111111111},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.34628571428572,"p90":56.802116402116404,"p95":54.7734693877551,"p99":51.121904761904766},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.25487947882736,"p90":49.310073591506814,"p95":47.54899953466728,"p99":44.37906623235614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.80861244019138,"p90":55.37834485202906,"p95":53.40054682159945,"p99":49.84051036682616},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":99.37084297520661,"p90":92.01003979185798,"p95":88.72396694214875,"p99":82.80903581267219},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.1901882951654,"p90":56.65758175478277,"p95":54.63409669211195,"p99":50.9918235793045},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.19711930585683,"p90":49.256591949867435,"p95":47.49742795165788,"p99":44.3309327548807},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.665871121718375,"p90":55.246176964554046,"p95":53.273099215819975,"p99":49.72155926809865},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":98.55632786885246,"p90":91.25585913782633,"p95":87.9967213114754,"p99":82.13027322404372},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.034883248730964,"p90":56.513780785862004,"p95":54.49543147208122,"p99":50.86240270727581},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.13948429035753,"p90":49.20322619477549,"p95":47.44596811639065,"p99":44.28290357529795},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.523809523809526,"p90":55.114638447971785,"p95":53.146258503401356,"p99":49.6031746031746},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":97.75505691056911,"p90":90.51394158386029,"p95":87.28130081300812,"p99":81.46254742547426},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.88036455696203,"p90":56.37070792311298,"p95":54.35746835443038,"p99":50.73363713080169},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.08197402597403,"p90":49.149975949975946,"p95":47.39461966604823,"p99":44.23497835497835},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.38242280285036,"p90":54.98372481745404,"p95":53.02002035968781,"p99":49.48535233570863},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":96.96670967741935,"p90":89.78399044205494,"p95":86.5774193548387,"p99":80.80559139784947},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.72662626262626,"p90":56.22835765057987,"p95":54.22020202020202,"p99":50.60552188552189},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.02458810810811,"p90":49.096840840840834,"p95":47.343382239382244,"p99":44.18715675675676},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.24170616113744,"p90":54.85343163068281,"p95":52.89438050101557,"p99":49.368088467614534},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":96.19097599999999,"p90":89.06571851851852,"p95":85.8848,"p99":80.15914666666667},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.57366246851385,"p90":56.08672450788319,"p95":54.08362720403022,"p99":50.47805205709488},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.96732613390929,"p90":49.04382049436045,"p95":47.29225547670472,"p99":44.1394384449244},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.1016548463357,"p90":54.72375448734786,"p95":52.7693346842283,"p99":49.25137903861309},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":95.42755555555556,"p90":88.3588477366255,"p95":85.2031746031746,"p99":79.52296296296296},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.42146733668342,"p90":55.94580308952168,"p95":53.947738693467336,"p99":50.351222780569515},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.910187702265375,"p90":48.99091453913461,"p95":47.24123901987979,"p99":44.09182308522115},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.9622641509434,"p90":54.594689028651295,"p95":52.64487870619945,"p99":49.13522012578617},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":94.67615748031496,"p90":87.66310877806941,"p95":84.53228346456692,"p99":78.89679790026247},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.2700350877193,"p90":55.80558804418454,"p95":53.812531328320794,"p99":50.22502923976609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.853172413793104,"p90":48.938122605363986,"p95":47.19033251231527,"p99":44.044310344827586},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.82352941176471,"p90":54.466230936819166,"p95":52.52100840336134,"p99":49.01960784313725},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":93.9365,"p90":86.97824074074073,"p95":83.87187499999999,"p99":78.28041666666667},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.11936,"p90":55.666074074074075,"p95":53.678,"p99":50.09946666666667},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.79627987082885,"p90":48.88544432484153,"p95":47.13953559895433,"p99":43.996899892357376},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":8192,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.68544600938967,"p90":54.33837593462006,"p95":52.39771965124077,"p99":48.904538341158066},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":93.20831007751937,"p90":86.30399081251794,"p95":83.22170542635658,"p99":77.67359173126616},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.96943640897756,"p90":55.52725593423847,"p95":53.544139650872815,"p99":49.97453034081463},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.739509677419356,"p90":48.83287933094384,"p95":47.08884792626728,"p99":43.94959139784947},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]},{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","phase":"decode","mode":"normal","precision":"fp8","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]}]} +{"version":1,"run":{"run_id":"160","run_attempt":1,"generated_at":"2026-07-08T12:20:00Z","conclusion":"success","source_sha":"cccccccccccccccccccccccccccccccccccccccc","requested_cases":5,"terminal_cases":4,"measured_cases":3,"unsupported_cases":1,"failed_cases":0,"requested_points":50,"terminal_points":40,"measured_points":30,"covered_skus":["b200-dgxc","b300","h200-dgxc","mi355x"],"kv_requested_cases":0,"kv_measured_cases":0},"coverage":[{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · bf16","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"mi355x-mori-deepseek-v3-normal-decode-ep16-uniform-bf16","label":"mi355x · mori · normal · decode · EP16 · bf16","disposition":"runnable","sku":"mi355x","backend":"mori","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"xgmi","scale_out_transport":"rdma","topology_class":"mi355x-xgmi-rdma"},"points":[{"tokens_per_rank":1,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":4096,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":8192,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · fp8","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"fp8","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null},{"case_id":"b300-deepep-v2-deepseek-v3-normal-decode-ep16-uniform-bf16","label":"b300 · deepep-v2 · normal · decode · EP16 · bf16","disposition":"unsupported","sku":"b300","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":"rdma","topology_class":"b300-nvlink-rdma"},"points":[{"tokens_per_rank":1,"global_tokens":16,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":2,"global_tokens":32,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":4,"global_tokens":64,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":8,"global_tokens":128,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":16,"global_tokens":256,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":32,"global_tokens":512,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":64,"global_tokens":1024,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":128,"global_tokens":2048,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":256,"global_tokens":4096,"terminal_status":"unsupported","reason":"backend-platform-unsupported"},{"tokens_per_rank":512,"global_tokens":8192,"terminal_status":"unsupported","reason":"backend-platform-unsupported"}],"outcome":"unsupported","reason":"backend-platform-unsupported","detail":"unsupported by the selected backend/platform"},{"case_id":"b200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","label":"b200-dgxc · deepep-v2 · normal · decode · EP8 · bf16","disposition":"runnable","sku":"b200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"bf16","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"b200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"pending","reason":"pending"},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"pending","reason":"pending"}],"outcome":"pending","reason":"pending","detail":null}],"series":[{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-bf16","phase":"decode","mode":"normal","precision":"bf16","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]},{"series_id":"mi355x-mori-deepseek-v3-normal-decode-ep16-uniform-bf16","phase":"decode","mode":"normal","precision":"bf16","backend":"mori","system":{"ep_size":16,"nodes":2,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"xgmi","scale_out_transport":"rdma","topology_class":"mi355x-xgmi-rdma","sku":"mi355x","vendor":"amd"},"points":[{"tokens_per_rank":1,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.95203836930456,"p90":55.51114663824496,"p95":53.52860568687907,"p99":49.96003197442047},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":100.19893333333334,"p90":92.77679012345678,"p95":89.46333333333332,"p99":83.4991111111111},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.34628571428572,"p90":56.802116402116404,"p95":54.7734693877551,"p99":51.121904761904766},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.25487947882736,"p90":49.310073591506814,"p95":47.54899953466728,"p99":44.37906623235614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.80861244019138,"p90":55.37834485202906,"p95":53.40054682159945,"p99":49.84051036682616},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":99.37084297520661,"p90":92.01003979185798,"p95":88.72396694214875,"p99":82.80903581267219},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.1901882951654,"p90":56.65758175478277,"p95":54.63409669211195,"p99":50.9918235793045},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.19711930585683,"p90":49.256591949867435,"p95":47.49742795165788,"p99":44.3309327548807},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.665871121718375,"p90":55.246176964554046,"p95":53.273099215819975,"p99":49.72155926809865},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":98.55632786885246,"p90":91.25585913782633,"p95":87.9967213114754,"p99":82.13027322404372},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":61.034883248730964,"p90":56.513780785862004,"p95":54.49543147208122,"p99":50.86240270727581},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.13948429035753,"p90":49.20322619477549,"p95":47.44596811639065,"p99":44.28290357529795},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.523809523809526,"p90":55.114638447971785,"p95":53.146258503401356,"p99":49.6031746031746},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":97.75505691056911,"p90":90.51394158386029,"p95":87.28130081300812,"p99":81.46254742547426},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.88036455696203,"p90":56.37070792311298,"p95":54.35746835443038,"p99":50.73363713080169},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.08197402597403,"p90":49.149975949975946,"p95":47.39461966604823,"p99":44.23497835497835},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.38242280285036,"p90":54.98372481745404,"p95":53.02002035968781,"p99":49.48535233570863},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":96.96670967741935,"p90":89.78399044205494,"p95":86.5774193548387,"p99":80.80559139784947},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.72662626262626,"p90":56.22835765057987,"p95":54.22020202020202,"p99":50.60552188552189},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":53.02458810810811,"p90":49.096840840840834,"p95":47.343382239382244,"p99":44.18715675675676},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.24170616113744,"p90":54.85343163068281,"p95":52.89438050101557,"p99":49.368088467614534},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":96.19097599999999,"p90":89.06571851851852,"p95":85.8848,"p99":80.15914666666667},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.57366246851385,"p90":56.08672450788319,"p95":54.08362720403022,"p99":50.47805205709488},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.96732613390929,"p90":49.04382049436045,"p95":47.29225547670472,"p99":44.1394384449244},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.1016548463357,"p90":54.72375448734786,"p95":52.7693346842283,"p99":49.25137903861309},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":95.42755555555556,"p90":88.3588477366255,"p95":85.2031746031746,"p99":79.52296296296296},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.42146733668342,"p90":55.94580308952168,"p95":53.947738693467336,"p99":50.351222780569515},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.910187702265375,"p90":48.99091453913461,"p95":47.24123901987979,"p99":44.09182308522115},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.9622641509434,"p90":54.594689028651295,"p95":52.64487870619945,"p99":49.13522012578617},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":94.67615748031496,"p90":87.66310877806941,"p95":84.53228346456692,"p99":78.89679790026247},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.2700350877193,"p90":55.80558804418454,"p95":53.812531328320794,"p99":50.22502923976609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.853172413793104,"p90":48.938122605363986,"p95":47.19033251231527,"p99":44.044310344827586},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.82352941176471,"p90":54.466230936819166,"p95":52.52100840336134,"p99":49.01960784313725},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":93.9365,"p90":86.97824074074073,"p95":83.87187499999999,"p99":78.28041666666667},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":60.11936,"p90":55.666074074074075,"p95":53.678,"p99":50.09946666666667},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.79627987082885,"p90":48.88544432484153,"p95":47.13953559895433,"p99":43.996899892357376},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":8192,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":58.68544600938967,"p90":54.33837593462006,"p95":52.39771965124077,"p99":48.904538341158066},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":93.20831007751937,"p90":86.30399081251794,"p95":83.22170542635658,"p99":77.67359173126616},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":59.96943640897756,"p90":55.52725593423847,"p95":53.544139650872815,"p99":49.97453034081463},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":52.739509677419356,"p90":48.83287933094384,"p95":47.08884792626728,"p99":43.94959139784947},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]},{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","phase":"decode","mode":"normal","precision":"fp8","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]}],"kv":[]} diff --git a/packages/app/cypress/fixtures/api/collectivex-run-159.json b/packages/app/cypress/fixtures/api/collectivex-run-159.json index 6cc60f58e..bc6b2fb26 100644 --- a/packages/app/cypress/fixtures/api/collectivex-run-159.json +++ b/packages/app/cypress/fixtures/api/collectivex-run-159.json @@ -1 +1 @@ -{"version":1,"run":{"run_id":"159","run_attempt":1,"generated_at":"2026-07-07T12:20:00Z","conclusion":"success","source_sha":"dddddddddddddddddddddddddddddddddddddddd","requested_cases":1,"terminal_cases":1,"measured_cases":1,"unsupported_cases":0,"failed_cases":0,"requested_points":10,"terminal_points":10,"measured_points":10,"covered_skus":["h200-dgxc"]},"coverage":[{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · fp8","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"fp8","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null}],"series":[{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","phase":"decode","mode":"normal","precision":"fp8","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]}]} +{"version":1,"run":{"run_id":"159","run_attempt":1,"generated_at":"2026-07-07T12:20:00Z","conclusion":"success","source_sha":"dddddddddddddddddddddddddddddddddddddddd","requested_cases":1,"terminal_cases":1,"measured_cases":1,"unsupported_cases":0,"failed_cases":0,"requested_points":10,"terminal_points":10,"measured_points":10,"covered_skus":["h200-dgxc"],"kv_requested_cases":0,"kv_measured_cases":0},"coverage":[{"case_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","label":"h200-dgxc · deepep-v2 · normal · decode · EP8 · fp8","disposition":"runnable","sku":"h200-dgxc","backend":"deepep-v2","phase":"decode","mode":"normal","precision":"fp8","topology":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island"},"points":[{"tokens_per_rank":1,"global_tokens":8,"terminal_status":"measured","reason":null},{"tokens_per_rank":2,"global_tokens":16,"terminal_status":"measured","reason":null},{"tokens_per_rank":4,"global_tokens":32,"terminal_status":"measured","reason":null},{"tokens_per_rank":8,"global_tokens":64,"terminal_status":"measured","reason":null},{"tokens_per_rank":16,"global_tokens":128,"terminal_status":"measured","reason":null},{"tokens_per_rank":32,"global_tokens":256,"terminal_status":"measured","reason":null},{"tokens_per_rank":64,"global_tokens":512,"terminal_status":"measured","reason":null},{"tokens_per_rank":128,"global_tokens":1024,"terminal_status":"measured","reason":null},{"tokens_per_rank":256,"global_tokens":2048,"terminal_status":"measured","reason":null},{"tokens_per_rank":512,"global_tokens":4096,"terminal_status":"measured","reason":null}],"outcome":"success","reason":null,"detail":null}],"series":[{"series_id":"h200-dgxc-deepep-v2-deepseek-v3-normal-decode-ep8-uniform-fp8","phase":"decode","mode":"normal","precision":"fp8","backend":"deepep-v2","system":{"ep_size":8,"nodes":1,"gpus_per_node":8,"scale_up_domain":8,"scale_up_transport":"nvlink","scale_out_transport":null,"topology_class":"h200-nvlink-island","sku":"h200-dgxc","vendor":"nvidia"},"points":[{"tokens_per_rank":1,"global_tokens":8,"components":{"dispatch":{"latency_us":{"p50":417,"p90":450.36,"p95":467.04,"p99":500.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":922.6952134292566,"p90":854.3474198419042,"p95":823.8350119904076,"p99":768.9126778577139},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.90407673860912,"p90":111.02229327648992,"p95":107.05721137375814,"p99":99.92006394884093},"payload_bytes":400000000},"stage":{"latency_us":{"p50":120,"p90":129.60000000000002,"p95":134.4,"p99":144},"activation_data_rate_gbps_at_latency_percentile":{"p50":1603.1829333333335,"p90":1484.4286419753084,"p95":1431.4133333333332,"p99":1335.9857777777777},"payload_data_rate_gbps_at_latency_percentile":{"p50":200.3978666666667,"p90":185.55358024691355,"p95":178.92666666666665,"p99":166.9982222222222},"payload_bytes":192381952},"combine":{"latency_us":{"p50":392,"p90":423.36,"p95":439.04,"p99":470.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":981.5405714285715,"p90":908.8338624338625,"p95":876.3755102040816,"p99":817.9504761904763},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.69257142857144,"p90":113.60423280423281,"p95":109.5469387755102,"p99":102.24380952380953},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":921,"p90":994.6800000000001,"p95":1031.5200000000002,"p99":1105.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":835.5350792616721,"p90":773.6435919089556,"p95":746.0134636264928,"p99":696.27923271806},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.50975895765473,"p90":98.62014718301363,"p95":95.09799906933456,"p99":88.75813246471228},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":2,"global_tokens":16,"components":{"dispatch":{"latency_us":{"p50":418,"p90":451.44000000000005,"p95":468.16,"p99":501.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":920.4878086124403,"p90":852.3035264930002,"p95":821.8641148325358,"p99":767.0731738437003},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.61722488038276,"p90":110.75668970405812,"p95":106.8010936431989,"p99":99.68102073365232},"payload_bytes":400000000},"stage":{"latency_us":{"p50":121,"p90":130.68,"p95":135.52,"p99":145.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1589.9334876033058,"p90":1472.1606366697276,"p95":1419.58347107438,"p99":1324.944573002755},"payload_data_rate_gbps_at_latency_percentile":{"p50":198.74168595041323,"p90":184.02007958371595,"p95":177.4479338842975,"p99":165.61807162534438},"payload_bytes":192381952},"combine":{"latency_us":{"p50":393,"p90":424.44000000000005,"p95":440.16,"p99":471.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":979.0430127226464,"p90":906.5213080765243,"p95":874.1455470737912,"p99":815.869177268872},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.3803765903308,"p90":113.31516350956554,"p95":109.2681933842239,"p99":101.983647158609},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":922,"p90":995.7600000000001,"p95":1032.64,"p99":1106.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":834.6288590021693,"p90":772.8044990760825,"p95":745.2043383947938,"p99":695.5240491684744},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.39423861171366,"p90":98.51318389973487,"p95":94.99485590331577,"p99":88.6618655097614},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":4,"global_tokens":32,"components":{"dispatch":{"latency_us":{"p50":419,"p90":452.52000000000004,"p95":469.28000000000003,"p99":502.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":918.2909403341289,"p90":850.2693891982674,"p95":819.9026252983293,"p99":765.2424502784409},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.33174224343675,"p90":110.49235392910809,"p95":106.54619843163995,"p99":99.4431185361973},"payload_bytes":400000000},"stage":{"latency_us":{"p50":122,"p90":131.76000000000002,"p95":136.64000000000001,"p99":146.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1576.9012459016394,"p90":1460.0937462052214,"p95":1407.9475409836064,"p99":1314.0843715846995},"payload_data_rate_gbps_at_latency_percentile":{"p50":197.11265573770493,"p90":182.51171827565267,"p95":175.9934426229508,"p99":164.26054644808744},"payload_bytes":192381952},"combine":{"latency_us":{"p50":394,"p90":425.52000000000004,"p95":441.28000000000003,"p99":472.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":976.5581319796954,"p90":904.2204925737921,"p95":871.9269035532996,"p99":813.798443316413},"payload_data_rate_gbps_at_latency_percentile":{"p50":122.06976649746193,"p90":113.02756157172401,"p95":108.99086294416244,"p99":101.72480541455162},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":923,"p90":996.84,"p95":1033.76,"p99":1107.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":833.724602383532,"p90":771.9672244291962,"p95":744.3969664138677,"p99":694.7705019862767},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.27896858071506,"p90":98.40645238955098,"p95":94.8919362327813,"p99":88.5658071505959},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":8,"global_tokens":64,"components":{"dispatch":{"latency_us":{"p50":420,"p90":453.6,"p95":470.40000000000003,"p99":504},"activation_data_rate_gbps_at_latency_percentile":{"p50":916.1045333333334,"p90":848.244938271605,"p95":817.9504761904761,"p99":763.4204444444445},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.04761904761905,"p90":110.22927689594357,"p95":106.29251700680271,"p99":99.2063492063492},"payload_bytes":400000000},"stage":{"latency_us":{"p50":123,"p90":132.84,"p95":137.76000000000002,"p99":147.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1564.0809105691058,"p90":1448.2230653417646,"p95":1396.5008130081299,"p99":1303.400758807588},"payload_data_rate_gbps_at_latency_percentile":{"p50":195.51011382113822,"p90":181.02788316772057,"p95":174.56260162601623,"p99":162.9250948509485},"payload_bytes":192381952},"combine":{"latency_us":{"p50":395,"p90":426.6,"p95":442.40000000000003,"p99":474},"activation_data_rate_gbps_at_latency_percentile":{"p50":974.0858329113925,"p90":901.9313267698077,"p95":869.7194936708861,"p99":811.738194092827},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.76072911392406,"p90":112.74141584622596,"p95":108.71493670886076,"p99":101.46727426160338},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":924,"p90":997.9200000000001,"p95":1034.88,"p99":1108.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":832.822303030303,"p90":771.1317620650954,"p95":743.5913419913419,"p99":694.0185858585859},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.16394805194805,"p90":98.29995189995189,"p95":94.78923933209646,"p99":88.4699567099567},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":16,"global_tokens":128,"components":{"dispatch":{"latency_us":{"p50":421,"p90":454.68,"p95":471.52000000000004,"p99":505.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":913.928513064133,"p90":846.230104689012,"p95":816.0076009501188,"p99":761.607094220111},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.76484560570071,"p90":109.96744963490808,"p95":106.04004071937563,"p99":98.97070467141727},"payload_bytes":400000000},"stage":{"latency_us":{"p50":124,"p90":133.92000000000002,"p95":138.88000000000002,"p99":148.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1551.4673548387095,"p90":1436.543847072879,"p95":1385.2387096774191,"p99":1292.8894623655915},"payload_data_rate_gbps_at_latency_percentile":{"p50":193.9334193548387,"p90":179.5679808841099,"p95":173.1548387096774,"p99":161.61118279569894},"payload_bytes":192381952},"combine":{"latency_us":{"p50":396,"p90":427.68,"p95":443.52000000000004,"p99":475.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":971.6260202020202,"p90":899.653722409278,"p95":867.5232323232323,"p99":809.6883501683502},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.45325252525252,"p90":112.45671530115975,"p95":108.44040404040403,"p99":101.21104377104378},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":925,"p90":999.0000000000001,"p95":1036,"p99":1110},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.9219545945946,"p90":770.298106106106,"p95":742.7874594594595,"p99":693.2682954954955},"payload_data_rate_gbps_at_latency_percentile":{"p50":106.04917621621622,"p90":98.19368168168167,"p95":94.68676447876449,"p99":88.37431351351351},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":32,"global_tokens":256,"components":{"dispatch":{"latency_us":{"p50":422,"p90":455.76000000000005,"p95":472.64000000000004,"p99":506.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":911.7628056872038,"p90":844.2248200807443,"p95":814.0739336492891,"p99":759.8023380726698},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.48341232227487,"p90":109.70686326136563,"p95":105.78876100203114,"p99":98.73617693522907},"payload_bytes":400000000},"stage":{"latency_us":{"p50":125,"p90":135,"p95":140,"p99":150},"activation_data_rate_gbps_at_latency_percentile":{"p50":1539.0556159999999,"p90":1425.0514962962964,"p95":1374.1568,"p99":1282.5463466666668},"payload_data_rate_gbps_at_latency_percentile":{"p50":192.38195199999998,"p90":178.13143703703705,"p95":171.7696,"p99":160.31829333333334},"payload_bytes":192381952},"combine":{"latency_us":{"p50":397,"p90":428.76000000000005,"p95":444.64000000000004,"p99":476.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":969.1785994962216,"p90":897.387592126131,"p95":865.3380352644836,"p99":807.648832913518},"payload_data_rate_gbps_at_latency_percentile":{"p50":121.1473249370277,"p90":112.17344901576638,"p95":108.16725440806044,"p99":100.95610411418976},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":926,"p90":1000.08,"p95":1037.1200000000001,"p99":1111.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":831.0235507559396,"p90":769.466250699944,"p95":741.9853131749459,"p99":692.5196256299496},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.93465226781858,"p90":98.0876409887209,"p95":94.58451095340943,"p99":88.2788768898488},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":64,"global_tokens":512,"components":{"dispatch":{"latency_us":{"p50":423,"p90":456.84000000000003,"p95":473.76000000000005,"p99":507.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":909.6073380614658,"p90":842.2290167235793,"p95":812.1494089834514,"p99":758.0061150512215},"payload_data_rate_gbps_at_latency_percentile":{"p50":118.2033096926714,"p90":109.44750897469572,"p95":105.5386693684566,"p99":98.50275807722618},"payload_bytes":400000000},"stage":{"latency_us":{"p50":126,"p90":136.08,"p95":141.12,"p99":151.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":1526.840888888889,"p90":1413.741563786008,"p95":1363.2507936507936,"p99":1272.3674074074074},"payload_data_rate_gbps_at_latency_percentile":{"p50":190.8551111111111,"p90":176.717695473251,"p95":170.4063492063492,"p99":159.04592592592593},"payload_bytes":192381952},"combine":{"latency_us":{"p50":398,"p90":429.84000000000003,"p95":445.76000000000005,"p99":477.59999999999997},"activation_data_rate_gbps_at_latency_percentile":{"p50":966.7434773869347,"p90":895.1328494323469,"p95":863.1638190954774,"p99":805.6195644891122},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.84293467336684,"p90":111.89160617904336,"p95":107.89547738693467,"p99":100.70244556113903},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":927,"p90":1001.1600000000001,"p95":1038.24,"p99":1112.3999999999999},"activation_data_rate_gbps_at_latency_percentile":{"p50":830.1270852211435,"p90":768.6361900195773,"p95":741.1848975188781,"p99":691.7725710176197},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.82037540453075,"p90":97.98182907826921,"p95":94.48247803975958,"p99":88.1836461704423},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":128,"global_tokens":1024,"components":{"dispatch":{"latency_us":{"p50":424,"p90":457.92,"p95":474.88000000000005,"p99":508.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":907.4620377358491,"p90":840.2426275331936,"p95":810.2339622641508,"p99":756.2183647798744},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.9245283018868,"p90":109.18937805730259,"p95":105.2897574123989,"p99":98.27044025157234},"payload_bytes":400000000},"stage":{"latency_us":{"p50":127,"p90":137.16,"p95":142.24,"p99":152.4},"activation_data_rate_gbps_at_latency_percentile":{"p50":1514.8185196850393,"p90":1402.6097404491106,"p95":1352.5165354330707,"p99":1262.3487664041995},"payload_data_rate_gbps_at_latency_percentile":{"p50":189.3523149606299,"p90":175.32621755613883,"p95":169.06456692913383,"p99":157.79359580052494},"payload_bytes":192381952},"combine":{"latency_us":{"p50":399,"p90":430.92,"p95":446.88000000000005,"p99":478.79999999999995},"activation_data_rate_gbps_at_latency_percentile":{"p50":964.3205614035088,"p90":892.8894087069526,"p95":861.0005012531327,"p99":803.6004678362574},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.5400701754386,"p90":111.61117608836908,"p95":107.62506265664159,"p99":100.45005847953217},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":928,"p90":1002.24,"p95":1039.3600000000001,"p99":1113.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":829.232551724138,"p90":767.8079182630906,"p95":740.3862068965517,"p99":691.0271264367817},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.70634482758621,"p90":97.87624521072797,"p95":94.38066502463055,"p99":88.08862068965517},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":256,"global_tokens":2048,"components":{"dispatch":{"latency_us":{"p50":425,"p90":459.00000000000006,"p95":476.00000000000006,"p99":510},"activation_data_rate_gbps_at_latency_percentile":{"p50":905.3268329411765,"p90":838.2655860566448,"p95":808.3275294117645,"p99":754.4390274509803},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.64705882352942,"p90":108.93246187363833,"p95":105.04201680672269,"p99":98.0392156862745},"payload_bytes":400000000},"stage":{"latency_us":{"p50":128,"p90":138.24,"p95":143.36,"p99":153.6},"activation_data_rate_gbps_at_latency_percentile":{"p50":1502.984,"p90":1391.6518518518517,"p95":1341.9499999999998,"p99":1252.4866666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":187.873,"p90":173.95648148148146,"p95":167.74374999999998,"p99":156.56083333333333},"payload_bytes":192381952},"combine":{"latency_us":{"p50":400,"p90":432,"p95":448.00000000000006,"p99":480},"activation_data_rate_gbps_at_latency_percentile":{"p50":961.90976,"p90":890.6571851851852,"p95":858.848,"p99":801.5914666666667},"payload_data_rate_gbps_at_latency_percentile":{"p50":120.23872,"p90":111.33214814814815,"p95":107.356,"p99":100.19893333333334},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":929,"p90":1003.32,"p95":1040.48,"p99":1114.8},"activation_data_rate_gbps_at_latency_percentile":{"p50":828.3399440258343,"p90":766.9814296535502,"p95":739.5892357373519,"p99":690.2832866881952},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.5925597416577,"p90":97.77088864968306,"p95":94.27907119790866,"p99":87.99379978471475},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}},{"tokens_per_rank":512,"global_tokens":4096,"components":{"dispatch":{"latency_us":{"p50":426,"p90":460.08000000000004,"p95":477.12000000000006,"p99":511.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":903.2016525821597,"p90":836.2978264649626,"p95":806.4300469483567,"p99":752.6680438184663},"payload_data_rate_gbps_at_latency_percentile":{"p50":117.37089201877934,"p90":108.67675186924012,"p95":104.79543930248154,"p99":97.80907668231613},"payload_bytes":400000000},"stage":{"latency_us":{"p50":129,"p90":139.32000000000002,"p95":144.48000000000002,"p99":154.79999999999998},"activation_data_rate_gbps_at_latency_percentile":{"p50":1491.33296124031,"p90":1380.863853000287,"p95":1331.5472868217053,"p99":1242.7774677002585},"payload_data_rate_gbps_at_latency_percentile":{"p50":186.41662015503874,"p90":172.60798162503588,"p95":166.44341085271316,"p99":155.3471834625323},"payload_bytes":192381952},"combine":{"latency_us":{"p50":401,"p90":433.08000000000004,"p95":449.12000000000006,"p99":481.2},"activation_data_rate_gbps_at_latency_percentile":{"p50":959.5109825436409,"p90":888.4360949478155,"p95":856.706234413965,"p99":799.5924854530341},"payload_data_rate_gbps_at_latency_percentile":{"p50":119.93887281795512,"p90":111.05451186847694,"p95":107.08827930174563,"p99":99.94906068162926},"payload_bytes":384763904},"roundtrip":{"latency_us":{"p50":930,"p90":1004.4000000000001,"p95":1041.6000000000001,"p99":1116},"activation_data_rate_gbps_at_latency_percentile":{"p50":827.4492559139785,"p90":766.1567184388689,"p95":738.7939784946236,"p99":689.541046594982},"payload_data_rate_gbps_at_latency_percentile":{"p50":105.47901935483871,"p90":97.66575866188768,"p95":94.17769585253455,"p99":87.89918279569893},"payload_bytes":784763904}},"roundtrip_token_rate_at_latency_percentile":{"p50":8338218,"p90":9005275.440000001,"p95":9338804.16,"p99":10005861.6}}]}],"kv":[]} diff --git a/packages/app/cypress/fixtures/api/collectivex-runs.json b/packages/app/cypress/fixtures/api/collectivex-runs.json index efdc80210..6888fbfea 100644 --- a/packages/app/cypress/fixtures/api/collectivex-runs.json +++ b/packages/app/cypress/fixtures/api/collectivex-runs.json @@ -1 +1 @@ -{"version":1,"runs":[{"run_id":"160","run_attempt":1,"generated_at":"2026-07-08T12:20:00Z","conclusion":"success","covered_skus":["b200-dgxc","b300","h200-dgxc","mi355x"],"requested_cases":5,"measured_cases":3,"requested_points":50,"terminal_points":40,"terminal_counts":{"measured":3,"unsupported":1,"failed":0}},{"run_id":"159","run_attempt":1,"generated_at":"2026-07-07T12:20:00Z","conclusion":"success","covered_skus":["h200-dgxc"],"requested_cases":1,"measured_cases":1,"requested_points":10,"terminal_points":10,"terminal_counts":{"measured":1,"unsupported":0,"failed":0}}],"discovery_complete":true} +{"version":1,"runs":[{"run_id":"160","run_attempt":1,"generated_at":"2026-07-08T12:20:00Z","conclusion":"success","covered_skus":["b200-dgxc","b300","h200-dgxc","mi355x"],"requested_cases":5,"measured_cases":3,"requested_points":50,"terminal_points":40,"terminal_counts":{"measured":3,"unsupported":1,"failed":0},"kv_cases":{"requested":0,"measured":0}},{"run_id":"159","run_attempt":1,"generated_at":"2026-07-07T12:20:00Z","conclusion":"success","covered_skus":["h200-dgxc"],"requested_cases":1,"measured_cases":1,"requested_points":10,"terminal_points":10,"terminal_counts":{"measured":1,"unsupported":0,"failed":0},"kv_cases":{"requested":0,"measured":0}}],"discovery_complete":true} diff --git a/packages/app/cypress/support/agentic-fixtures.ts b/packages/app/cypress/support/agentic-fixtures.ts new file mode 100644 index 000000000..66adff223 --- /dev/null +++ b/packages/app/cypress/support/agentic-fixtures.ts @@ -0,0 +1,30 @@ +export function percentileLadder(prefix: string, base: number): Record { + return { + [`median_${prefix}`]: base, + [`p75_${prefix}`]: base * 1.2, + [`p90_${prefix}`]: base * 1.5, + [`p95_${prefix}`]: base * 1.7, + [`p99_${prefix}`]: base * 2.2, + [`std_${prefix}`]: base * 0.3, + }; +} + +export function agenticMetrics(concurrency: number): Record { + const scale = concurrency / 16; + const itl = 0.011 * scale; + return { + ...percentileLadder('ttft', 0.4 * scale), + ...percentileLadder('tpot', 0.012 * scale), + ...percentileLadder('itl', itl), + ...percentileLadder('e2el', 8 * scale), + median_intvty: 1 / itl, + p75_intvty: 1 / (itl * 1.2), + p90_intvty: 1 / (itl * 1.5), + p99_intvty: 1 / (itl * 2.2), + std_intvty: (1 / itl) * 0.1, + tput_per_gpu: 950 / Math.sqrt(scale), + output_tput_per_gpu: 210, + input_tput_per_gpu: 740, + total_tput_tps: 7600 * concurrency * 0.05, + }; +} diff --git a/packages/app/cypress/support/e2e.ts b/packages/app/cypress/support/e2e.ts index b3c15b58b..a6678efa4 100644 --- a/packages/app/cypress/support/e2e.ts +++ b/packages/app/cypress/support/e2e.ts @@ -2,14 +2,54 @@ * Global e2e setup. Loaded before every `cy.visit` via `supportFile` in * `cypress.config.ts`. * - * Snoozes the feedback-modal nudge so it doesn't render its centered modal - * + backdrop on top of the UI under test. Specs that want to exercise the + * Suppresses the centered feedback-modal on the dashboard so its backdrop + * doesn't sit on top of the UI under test. Specs that want to exercise the * feedback-modal flow can clear `inferencex-feedback-modal-snoozed` in their - * own `onBeforeLoad`. + * own `onBeforeLoad`, which runs after this hook. */ +import 'cypress-axe'; + +let suppressTelemetryTutorial = true; +let suppressAgenticCoachMark = true; + +/** + * Opt the whole spec out of the telemetry-tutorial suppression. + * + * Clearing the key in a visit's `onBeforeLoad` is not enough: this hook also + * fires on `cy.reload()`, which takes no `onBeforeLoad`, so the key would be + * re-seeded behind the test and a "still dismissed after reload" assertion + * would pass even if dismissal never persisted anything. Call this at the top + * of any spec that owns telemetry-tutorial state. + * + * The card is a bottom-right modal on /inference/agentic/[id]. It has no + * backdrop, but it sits over the last chart in the grid, so agentic specs + * suppress it by default. + */ +export function keepTelemetryTutorial(): void { + suppressTelemetryTutorial = false; +} + +/** + * Opt the whole spec out of the agentic point coach-mark suppression, for the + * same reason as `keepTelemetryTutorial` — the key is re-seeded on + * `cy.reload()`, so a per-visit `onBeforeLoad` clear cannot own its state. + * + * The callout is anchored to a point inside `[data-testid="scatter-graph"]`, + * so on the agentic view it sits over the plot area that other specs click. + */ +export function keepAgenticCoachMark(): void { + suppressAgenticCoachMark = false; +} + Cypress.on('window:before:load', (win) => { try { win.localStorage.setItem('inferencex-feedback-modal-snoozed', String(Date.now())); + if (suppressTelemetryTutorial) { + win.localStorage.setItem('inferencex-agentx-telemetry-tutorial-dismissed', '1'); + } + if (suppressAgenticCoachMark) { + win.localStorage.setItem('inferencex-agentic-point-coach-mark-dismissed', '1'); + } } catch { // localStorage unavailable — fine, the test will just see the modal. } @@ -19,14 +59,14 @@ Cypress.on('window:before:load', (win) => { * Seed the shared feature-gate flag (the same localStorage key the ↑↑↓↓ konami * unlock writes — see use-feature-gate.ts). * - * The agentic surfaces (the "Agentic Traces" scenario, /datasets, - * /inference/agentic/[id], and the Datasets nav link) are now PUBLIC by default + * The agentic surfaces (the "Agentic" scenario, /agentx, + * /inference/agentic/[id], and the AgentX nav link) are now PUBLIC by default * — they no longer sit behind this gate — so agentic specs no longer need it. * The helper is retained as a harmless no-op for those specs (and still unlocks * the remaining hidden features: the "Hidden" tab dropdown and Measured Energy). * * Call from a spec's `cy.visit(..., { onBeforeLoad })`: - * cy.visit('/datasets/x', { onBeforeLoad: unlockAgenticGate }); + * cy.visit('/agentx/x', { onBeforeLoad: unlockAgenticGate }); * or compose inside an existing hook: `unlockAgenticGate(win)`. */ export function unlockAgenticGate(win: Window): void { diff --git a/packages/app/cypress/support/legend-advanced.ts b/packages/app/cypress/support/legend-advanced.ts new file mode 100644 index 000000000..ff0f48a48 --- /dev/null +++ b/packages/app/cypress/support/legend-advanced.ts @@ -0,0 +1,20 @@ +/** + * Expand the chart legend's collapsed-by-default Advanced drawer. + * + * Log Scale, Labels, High Contrast, Parallelism Labels, and Line Labels live + * inside the Advanced section, so tests must open the drawer before reading + * or clicking those switches. Idempotent — safe to call when the drawer is + * already expanded. + * + * `cy.get` respects `cy.within` scoping, so call this inside a `.within()` + * block to target a specific chart's legend (e.g. a replay panel). + */ +export const expandLegendAdvanced = (): void => { + cy.get('[data-testid="legend-advanced-toggle"]') + .first() + .then(($btn) => { + if ($btn.attr('aria-expanded') !== 'true') { + cy.wrap($btn).click(); + } + }); +}; diff --git a/packages/app/cypress/support/mock-data.ts b/packages/app/cypress/support/mock-data.ts index c66bf850c..b75ef5342 100644 --- a/packages/app/cypress/support/mock-data.ts +++ b/packages/app/cypress/support/mock-data.ts @@ -2,8 +2,11 @@ import type { AggDataEntry, ChartDefinition, HardwareConfig, - InferenceChartContextType, + InferenceActionsContextType, InferenceData, + InferenceDataContextType, + InferenceDisplayContextType, + InferenceFiltersContextType, } from '@/components/inference/types'; import type { EvaluationChartContextType, @@ -13,7 +16,13 @@ import type { ModelSuccessRateData, ReliabilityChartContextType, } from '@/components/reliability/types'; -import type { GlobalFilterContextType } from '@/components/GlobalFilterContext'; +import type { + GlobalFilterActionsContextType, + GlobalFilterAvailabilityContextType, + GlobalFilterRunContextType, + GlobalFilterSelectionContextType, + GlobalFilterWorkflowContextType, +} from '@/components/GlobalFilterContext'; import type { UnofficialRunContextType } from '@/components/unofficial-run-provider'; import { computeToggle } from '@/hooks/useTogglableSet'; import { Model, Sequence, Precision } from '@/lib/data-mappings'; @@ -153,12 +162,17 @@ export function createMockInferenceData(overrides?: Partial): Inf } // --------------------------------------------------------------------------- -// Inference context +// Inference domain contexts // --------------------------------------------------------------------------- -export function createMockInferenceContext( - overrides?: Partial, -): InferenceChartContextType { +export type MockInferenceContextValues = InferenceDataContextType & + InferenceFiltersContextType & + InferenceDisplayContextType & + InferenceActionsContextType; + +export function createMockInferenceContextValues( + overrides?: Partial, +): MockInferenceContextValues { const hwConfig = createMockHardwareConfig(); return { activeHwTypes: new Set(['h100', 'b200', 'b200_trt', 'mi300x', 'h200']), @@ -195,15 +209,23 @@ export function createMockInferenceContext( setSelectedPrecisions: namedStub('setSelectedPrecisions'), loading: false, error: null, - workflowInfo: null, selectedYAxisMetric: 'y_tpPerGpu', setSelectedYAxisMetric: namedStub('setSelectedYAxisMetric'), + tokenRevenuePriceSource: 'normalized', + tokenRevenuePricing: { + source: 'normalized', + inputPerMillion: 1, + outputPerMillion: 1, + }, + openRouterModelId: 'deepseek/deepseek-r1-0528', + openRouterPricingLoading: false, + openRouterPricingError: null, + setTokenRevenuePriceSource: namedStub('setTokenRevenuePriceSource'), selectedPercentile: 'p90', setSelectedPercentile: namedStub('setSelectedPercentile'), selectedXAxisMetric: null, setSelectedXAxisMetric: namedStub('setSelectedXAxisMetric'), selectedE2eXAxisMetric: null, - setSelectedE2eXAxisMetric: namedStub('setSelectedE2eXAxisMetric'), selectedXAxisMode: 'interactivity' as const, setSelectedXAxisMode: namedStub('setSelectedXAxisMode'), scaleType: 'auto', @@ -226,14 +248,12 @@ export function createMockInferenceContext( setLogScale: namedStub('setLogScale'), useAdvancedLabels: false, setUseAdvancedLabels: namedStub('setUseAdvancedLabels'), + showConcurrencyLabels: false, + setShowConcurrencyLabels: namedStub('setShowConcurrencyLabels'), showGradientLabels: false, setShowGradientLabels: namedStub('setShowGradientLabels'), showLineLabels: false, setShowLineLabels: namedStub('setShowLineLabels'), - showSpeedOverlay: false, - setShowSpeedOverlay: namedStub('setShowSpeedOverlay'), - showMinecraftOverlay: false, - setShowMinecraftOverlay: namedStub('setShowMinecraftOverlay'), selectedGPUs: [], setSelectedGPUs: namedStub('setSelectedGPUs'), availableGPUs: [ @@ -254,7 +274,7 @@ export function createMockInferenceContext( availableDates: ['2025-02-28', '2025-03-01'], dateRangeAvailableDates: ['2025-02-28', '2025-03-01'], isCheckingAvailableDates: false, - availableRuns: null, + availableRuns: {}, selectedRunId: '12345678', setSelectedRunId: namedStub('setSelectedRunId'), availablePrecisions: [Precision.FP4, Precision.FP8], @@ -414,14 +434,29 @@ export function createMockReliabilityContext( }; } -// --------------------------------------------------------------------------- -// Global filter context +// Global filter contexts // --------------------------------------------------------------------------- -export function createMockGlobalFilterContext( - overrides?: Partial, -): GlobalFilterContextType { - return { +export type GlobalFilterContextOverrides = Partial< + GlobalFilterSelectionContextType & + GlobalFilterActionsContextType & + GlobalFilterRunContextType & + GlobalFilterAvailabilityContextType & + GlobalFilterWorkflowContextType +>; + +export interface MockGlobalFilterContexts { + selection: GlobalFilterSelectionContextType; + actions: GlobalFilterActionsContextType; + run: GlobalFilterRunContextType; + availability: GlobalFilterAvailabilityContextType; + workflow: GlobalFilterWorkflowContextType; +} + +export function createMockGlobalFilterContexts( + overrides: GlobalFilterContextOverrides = {}, +): MockGlobalFilterContexts { + const values = { selectedModel: Model.DeepSeek_R1, setSelectedModel: namedStub('setSelectedModel_global'), selectedSequence: Sequence.EightK_OneK, @@ -429,8 +464,6 @@ export function createMockGlobalFilterContext( selectedPrecisions: [Precision.FP4], setSelectedPrecisions: namedStub('setSelectedPrecisions_global'), effectiveSequence: Sequence.EightK_OneK, - // Mocks represent a settled state: availability is known and the sequence is - // resolved. Tests exercising the pre-availability window override this. sequenceResolved: true, effectivePrecisions: [Precision.FP4], selectedRunDate: '2025-03-01', @@ -444,12 +477,51 @@ export function createMockGlobalFilterContext( availableDates: ['2025-02-28', '2025-03-01'], effectiveRunDate: '2025-03-01', availabilityRows: undefined, - workflowInfo: null, + availabilitySettled: true, + availabilityError: null, availableRuns: {}, workflowLoading: false, workflowError: null, ...overrides, }; + + return { + selection: { + selectedModel: values.selectedModel, + selectedSequence: values.selectedSequence, + selectedPrecisions: values.selectedPrecisions, + effectiveSequence: values.effectiveSequence, + sequenceResolved: values.sequenceResolved, + effectivePrecisions: values.effectivePrecisions, + }, + actions: { + setSelectedModel: values.setSelectedModel, + setSelectedSequence: values.setSelectedSequence, + setSelectedPrecisions: values.setSelectedPrecisions, + setSelectedRunDate: values.setSelectedRunDate, + setSelectedRunId: values.setSelectedRunId, + }, + run: { + selectedRunDate: values.selectedRunDate, + selectedRunDateRev: values.selectedRunDateRev, + selectedRunId: values.selectedRunId, + effectiveRunDate: values.effectiveRunDate, + }, + availability: { + availableModels: values.availableModels, + availableSequences: values.availableSequences, + availablePrecisions: values.availablePrecisions, + availableDates: values.availableDates, + availabilityRows: values.availabilityRows, + availabilitySettled: values.availabilitySettled, + availabilityError: values.availabilityError, + }, + workflow: { + availableRuns: values.availableRuns, + workflowLoading: values.workflowLoading, + workflowError: values.workflowError, + }, + }; } // --------------------------------------------------------------------------- @@ -476,12 +548,11 @@ export function createMockUnofficialRunContext( .stub() .returns(null) as unknown as UnofficialRunContextType['getOverlayData'], activeOverlayHwTypes: new Set(), - setActiveOverlayHwTypes: namedStub('setActiveOverlayHwTypes'), allOverlayHwTypes: new Set(), - toggleOverlayHwType: namedStub('toggleOverlayHwType'), - resetOverlayHwTypes: namedStub('resetOverlayHwTypes'), localOfficialOverride: null, - setLocalOfficialOverride: namedStub('setLocalOfficialOverride'), + reconcileOverlayScope: namedStub('reconcileOverlayScope'), + setUnifiedOverlaySelection: namedStub('setUnifiedOverlaySelection'), + resetOverlaySelection: namedStub('resetOverlaySelection'), ...overrides, }; } diff --git a/packages/app/cypress/support/mock-router.ts b/packages/app/cypress/support/mock-router.ts new file mode 100644 index 000000000..c7130ccef --- /dev/null +++ b/packages/app/cypress/support/mock-router.ts @@ -0,0 +1,16 @@ +import type { ContextType } from 'react'; +import type { AppRouterContext } from 'next/dist/shared/lib/app-router-context.shared-runtime'; + +export type MockAppRouter = NonNullable>; + +export function createMockRouter(): MockAppRouter { + return { + push: cy.stub(), + replace: cy.stub(), + refresh: cy.stub(), + back: cy.stub(), + forward: cy.stub(), + prefetch: cy.stub().resolves(), + bfcacheId: '', + }; +} diff --git a/packages/app/cypress/support/overlay-fixtures.ts b/packages/app/cypress/support/overlay-fixtures.ts index 2f3268500..ffd006585 100644 --- a/packages/app/cypress/support/overlay-fixtures.ts +++ b/packages/app/cypress/support/overlay-fixtures.ts @@ -4,8 +4,11 @@ * The benchmark values are the real numbers from GitHub run 29682242847 * (GLM5.2 B300 agentic hicache, offload=on rows): * conc, p90_intvty (tok/s/user), tput_per_gpu, p90_e2el (s) - * C=4 is dominated on e2e by C=8 (12874 tok/s @ 33.1s vs 9415 @ 48.0s), which - * makes the set a ready-made probe for the e2e-restricted frontier behaviors. + * C=4 is dominated on e2e by C=8 (12874 tok/s @ 33.1s vs 9415 @ 48.0s). That + * used to make the set a probe for the e2e-restricted frontier; #736 removed + * that gating, so on the interactivity axes all five are Pareto-optimal — + * throughput falls monotonically as interactivity rises. Use + * `DOMINATED_CONFIG` when a test needs a point Optimal Only will actually drop. */ export const DEFAULT_MODEL_DB_KEY = 'dsv4'; export const AGENTIC_DATE = '2026-07-19'; @@ -39,9 +42,21 @@ export const metricsFor = (intvty: number, tput: number, e2el: number): Record 60) and throughput (12874 > 5000). Since #736 the + * Pareto frontier is computed from the selected axes alone, so this is the point + * Optimal Only must drop — the e2e-dominated C=4 above no longer is one. + */ +export const DOMINATED_CONFIG: [number, number, number, number] = [6, 60, 5000, 40]; + let idCursor = 900000; -export const b300Rows = (runUrl: string | null, hardware = 'b300') => - REAL_CONFIGS.map(([conc, intvty, tput, e2el]) => ({ +export const b300Rows = ( + runUrl: string | null, + hardware = 'b300', + configs: [number, number, number, number][] = REAL_CONFIGS, +) => + configs.map(([conc, intvty, tput, e2el]) => ({ id: runUrl ? 0 : idCursor++, hardware, framework: 'sglang', @@ -81,8 +96,18 @@ export const availability = [ }, ]; -/** Intercept availability + benchmarks + unofficial-run with the B300 fixture. */ -export const interceptOverlayRun = ({ overlayHardware = 'b300' } = {}) => { +/** + * Intercept availability + benchmarks + unofficial-run with the B300 fixture. + * `overlayConfigs` overrides only the overlay run's rows, leaving the official + * rows as the standard five. + */ +export const interceptOverlayRun = ({ + overlayHardware = 'b300', + overlayConfigs = REAL_CONFIGS, +}: { + overlayHardware?: string; + overlayConfigs?: [number, number, number, number][]; +} = {}) => { cy.intercept('GET', '/api/v1/availability', { body: availability }).as('availability'); cy.intercept('GET', '/api/v1/benchmarks*', { body: b300Rows(null) }).as('benchmarks'); cy.intercept('GET', '/api/unofficial-run*', { @@ -100,7 +125,7 @@ export const interceptOverlayRun = ({ overlayHardware = 'b300' } = {}) => { isNonMainBranch: true, }, ], - benchmarks: b300Rows(OVERLAY_RUN_URL, overlayHardware), + benchmarks: b300Rows(OVERLAY_RUN_URL, overlayHardware, overlayConfigs), evaluations: [], }, }).as('unofficialRun'); diff --git a/packages/app/cypress/support/test-utils.tsx b/packages/app/cypress/support/test-utils.tsx index f7de968f1..ff7fe2784 100644 --- a/packages/app/cypress/support/test-utils.tsx +++ b/packages/app/cypress/support/test-utils.tsx @@ -1,35 +1,39 @@ import React from 'react'; import { QueryClient, QueryClientProvider } from '@tanstack/react-query'; -import { InferenceContext } from '@/components/inference/InferenceContext'; +import { InferenceContextsProvider } from '@/components/inference/InferenceContext'; import { EvaluationContext } from '@/components/evaluation/EvaluationContext'; import { ReliabilityContext } from '@/components/reliability/ReliabilityContext'; import { - GlobalFilterContext, - type GlobalFilterContextType, + GlobalFilterActionsContext, + GlobalFilterAvailabilityContext, + GlobalFilterRunContext, + GlobalFilterSelectionContext, + GlobalFilterWorkflowContext, } from '@/components/GlobalFilterContext'; import { UnofficialRunContext, type UnofficialRunContextType, } from '@/components/unofficial-run-provider'; -import type { InferenceChartContextType } from '@/components/inference/types'; import type { EvaluationChartContextType } from '@/components/evaluation/types'; import type { ReliabilityChartContextType } from '@/components/reliability/types'; import { - createMockInferenceContext, + createMockInferenceContextValues, createMockEvaluationContext, createMockReliabilityContext, - createMockGlobalFilterContext, + createMockGlobalFilterContexts, createMockUnofficialRunContext, + type GlobalFilterContextOverrides, + type MockInferenceContextValues, } from './mock-data'; export interface ProviderOverrides { - inference?: Partial; + inference?: Partial; evaluation?: Partial; reliability?: Partial; - globalFilters?: Partial; + globalFilters?: GlobalFilterContextOverrides; unofficial?: Partial; } @@ -53,8 +57,12 @@ export function mountWithProviders( let tree = component; if (overrides.inference !== undefined) { - const value = createMockInferenceContext(overrides.inference); - tree = {tree}; + const value = createMockInferenceContextValues(overrides.inference); + tree = ( + + {tree} + + ); } if (overrides.evaluation !== undefined) { @@ -68,8 +76,20 @@ export function mountWithProviders( } if (overrides.globalFilters !== undefined) { - const value = createMockGlobalFilterContext(overrides.globalFilters); - tree = {tree}; + const values = createMockGlobalFilterContexts(overrides.globalFilters); + tree = ( + + + + + + {tree} + + + + + + ); } if (overrides.unofficial !== undefined) { diff --git a/packages/app/next.config.ts b/packages/app/next.config.ts index b8445156c..bd96a0437 100644 --- a/packages/app/next.config.ts +++ b/packages/app/next.config.ts @@ -12,6 +12,20 @@ const nextConfig: NextConfig = { allowedDevOrigins: allowedDevOriginsFromEnv(), transpilePackages: ['@semianalysisai/inferencex-constants'], serverExternalPackages: ['shiki'], + redirects() { + return Promise.resolve([ + { + source: '/datasets/:path*', + destination: '/agentx/:path*', + permanent: true, + }, + { + source: '/zh/datasets/:path*', + destination: '/zh/agentx/:path*', + permanent: true, + }, + ]); + }, experimental: { optimizePackageImports: ['lucide-react', 'd3', '@tanstack/react-query'], // NOTE: experimental.inlineCss was evaluated (2026-07) for the PageSpeed @@ -26,6 +40,7 @@ const nextConfig: NextConfig = { ...(process.env.GITHUB_ACTIONS === 'true' && { turbopackFileSystemCacheForBuild: true }), }, images: { + qualities: [75, 100], remotePatterns: [ { hostname: 'placehold.co' }, { hostname: 'substack-post-media.s3.amazonaws.com' }, diff --git a/packages/app/package.json b/packages/app/package.json index 7ed615df2..579ec2852 100644 --- a/packages/app/package.json +++ b/packages/app/package.json @@ -5,7 +5,7 @@ "description": "InferenceX - Inference performance benchmarking and visualization", "repository": { "type": "git", - "url": "https://github.com/SemiAnalysisAI/inferencemax-app" + "url": "https://github.com/SemiAnalysisAI/InferenceX-app" }, "sideEffects": [ "**/*.css", @@ -20,7 +20,7 @@ "test:e2e": "bun run test:e2e:quick", "test:e2e:quick": "bun run test:e2e:quick:component && bun run test:e2e:quick:integration", "test:e2e:quick:component": "cypress run --component --spec cypress/component/chart-buttons.cy.tsx,cypress/component/scatter-graph.cy.tsx", - "test:e2e:quick:integration": "cypress run --spec cypress/e2e/sanity.cy.ts,cypress/e2e/inference-chart.cy.ts,cypress/e2e/evaluation-chart.cy.ts,cypress/e2e/reliability-chart.cy.ts,cypress/e2e/zh-pages.cy.ts,cypress/e2e/csv-export-overlay.cy.ts,cypress/e2e/overlay-optimal-only.cy.ts", + "test:e2e:quick:integration": "cypress run --spec cypress/e2e/sanity.cy.ts,cypress/e2e/inference-chart.cy.ts,cypress/e2e/evaluation-chart.cy.ts,cypress/e2e/reliability-chart.cy.ts,cypress/e2e/zh-pages.cy.ts,cypress/e2e/csv-export-overlay.cy.ts,cypress/e2e/overlay-optimal-only.cy.ts,cypress/e2e/resident-sequence-lengths.cy.ts,cypress/e2e/overview-a11y.cy.ts", "test:e2e:full": "cypress run --component && cypress run", "test:e2e:component": "cypress run --component", "test:e2e:integration": "cypress run", @@ -35,9 +35,8 @@ "dependencies": { "@chenglou/pretext": "^0.0.8", "@jpinsonneau/html-to-image": "^1.11.13", - "@noble/ciphers": "^2.2.0", - "@posthog/nextjs-config": "^1.9.68", - "@radix-ui/react-accordion": "^1.2.20", + "@noble/ciphers": "^2.3.0", + "@posthog/nextjs-config": "^1.9.69", "@radix-ui/react-dialog": "^1.1.23", "@radix-ui/react-label": "^2.1.15", "@radix-ui/react-popover": "^1.1.23", @@ -46,26 +45,26 @@ "@radix-ui/react-switch": "^1.3.7", "@radix-ui/react-tabs": "^1.1.21", "@radix-ui/react-tooltip": "^1.2.16", - "@react-three/fiber": "^9.6.1", + "@react-three/fiber": "^9.7.0", "@semianalysisai/inferencex-constants": "workspace:*", "@semianalysisai/inferencex-db": "workspace:*", - "@shikijs/rehype": "^4.3.1", + "@shikijs/rehype": "^4.4.3", "@tailwindcss/typography": "^0.5.20", "@tanstack/react-query": "^5.101.4", "@vercel/analytics": "^2.0.1", - "@vercel/blob": "^2.6.1", + "@vercel/blob": "^2.8.0", "@vercel/speed-insights": "^2.0.0", "adm-zip": "^0.6.0", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "d3": "^7.9.0", - "dompurify": "^3.4.12", + "dompurify": "^3.4.13", "gray-matter": "^4.0.3", "iwanthue": "^2.1.0", "katex": "^0.16.47", - "lucide-react": "^1.27.0", + "lucide-react": "^1.31.0", "mp4-muxer": "^5.2.2", - "next": "^16.2.12", + "next": "^16.3.1", "next-mdx-remote": "^6.0.0", "next-themes": "^0.4.6", "posthog-js": "^1.407.2", @@ -74,7 +73,7 @@ "rehype-katex": "^7.0.1", "remark-gfm": "^4.0.1", "remark-math": "^6.0.0", - "shiki": "^4.3.1", + "shiki": "^4.4.3", "tailwind-merge": "^3.6.0", "three": "^0.185.1" }, @@ -84,16 +83,17 @@ "@types/adm-zip": "^0.5.8", "@types/d3": "^7.4.3", "@types/mdast": "^4.0.4", - "@types/node": "^26.1.1", - "@types/react": "^19.2.17", - "@types/react-dom": "^19.2.3", - "@types/three": "^0.185.1", + "@types/node": "^26.2.0", + "@types/react": "19.2.17", + "@types/react-dom": "19.2.3", + "@types/three": "^0.185.4", "@vitest/coverage-v8": "^4.1.10", - "cypress": "^15.19.0", + "axe-core": "^4.13.0", + "cypress": "15.19.0", + "cypress-axe": "^1.7.0", "cypress-split": "^1.25.0", - "esbuild": "^0.28.1", - "fake-indexeddb": "^6.2.5", - "jsdom": "^29.1.1", + "esbuild": "^0.28.2", + "jsdom": "^30.0.1", "rimraf": "^6.1.3", "tailwindcss": "^4.3.3", "tw-animate-css": "^1.4.0", diff --git a/packages/app/public/decorative/bus.png b/packages/app/public/decorative/bus.png deleted file mode 100644 index 7ec6619bf..000000000 Binary files a/packages/app/public/decorative/bus.png and /dev/null differ diff --git a/packages/app/public/decorative/donkey-chest.png b/packages/app/public/decorative/donkey-chest.png deleted file mode 100644 index 7697e9b91..000000000 Binary files a/packages/app/public/decorative/donkey-chest.png and /dev/null differ diff --git a/packages/app/public/decorative/elytra.png b/packages/app/public/decorative/elytra.png deleted file mode 100644 index 79d83fc3e..000000000 Binary files a/packages/app/public/decorative/elytra.png and /dev/null differ diff --git a/packages/app/public/decorative/racing-car.png b/packages/app/public/decorative/racing-car.png deleted file mode 100644 index b463385a5..000000000 Binary files a/packages/app/public/decorative/racing-car.png and /dev/null differ diff --git a/packages/app/public/images/agentx-methodology/corpus-scale.png b/packages/app/public/images/agentx-methodology/corpus-scale.png new file mode 100644 index 000000000..a526a6df4 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/corpus-scale.png differ diff --git a/packages/app/public/images/agentx-methodology/engine-acceptance-controls.png b/packages/app/public/images/agentx-methodology/engine-acceptance-controls.png new file mode 100644 index 000000000..0606f78b0 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/engine-acceptance-controls.png differ diff --git a/packages/app/public/images/agentx-methodology/golden-acceptance-length.png b/packages/app/public/images/agentx-methodology/golden-acceptance-length.png new file mode 100644 index 000000000..5e2893d73 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/golden-acceptance-length.png differ diff --git a/packages/app/public/images/agentx-methodology/hash-token-ratio.png b/packages/app/public/images/agentx-methodology/hash-token-ratio.png new file mode 100644 index 000000000..1730ca439 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/hash-token-ratio.png differ diff --git a/packages/app/public/images/agentx-methodology/pareto-b200-minimax-m3.png b/packages/app/public/images/agentx-methodology/pareto-b200-minimax-m3.png new file mode 100644 index 000000000..dd8158824 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/pareto-b200-minimax-m3.png differ diff --git a/packages/app/public/images/agentx-methodology/profiling-cache-sources.png b/packages/app/public/images/agentx-methodology/profiling-cache-sources.png new file mode 100644 index 000000000..d3a3c49e1 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/profiling-cache-sources.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-flatspawn.png b/packages/app/public/images/agentx-methodology/replay-flatspawn.png new file mode 100644 index 000000000..19e99a110 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-flatspawn.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-linear.png b/packages/app/public/images/agentx-methodology/replay-linear.png new file mode 100644 index 000000000..75ab6d5aa Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-linear.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-multi-subagent.png b/packages/app/public/images/agentx-methodology/replay-multi-subagent.png new file mode 100644 index 000000000..5e8072111 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-multi-subagent.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-single-subagent.png b/packages/app/public/images/agentx-methodology/replay-single-subagent.png new file mode 100644 index 000000000..185068040 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-single-subagent.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-subagent-sidecars.png b/packages/app/public/images/agentx-methodology/replay-subagent-sidecars.png new file mode 100644 index 000000000..a60af7fed Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-subagent-sidecars.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-two-subagents-joined.png b/packages/app/public/images/agentx-methodology/replay-two-subagents-joined.png new file mode 100644 index 000000000..3e39e3de2 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-two-subagents-joined.png differ diff --git a/packages/app/public/images/agentx-methodology/replay-two-subagents.png b/packages/app/public/images/agentx-methodology/replay-two-subagents.png new file mode 100644 index 000000000..54d03ed10 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/replay-two-subagents.png differ diff --git a/packages/app/public/images/agentx-methodology/request-distributions-256k.png b/packages/app/public/images/agentx-methodology/request-distributions-256k.png new file mode 100644 index 000000000..8f6e03660 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/request-distributions-256k.png differ diff --git a/packages/app/public/images/agentx-methodology/request-distributions.png b/packages/app/public/images/agentx-methodology/request-distributions.png new file mode 100644 index 000000000..640c7b68f Binary files /dev/null and b/packages/app/public/images/agentx-methodology/request-distributions.png differ diff --git a/packages/app/public/images/agentx-methodology/request-queue-depth.png b/packages/app/public/images/agentx-methodology/request-queue-depth.png new file mode 100644 index 000000000..724e4dba2 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/request-queue-depth.png differ diff --git a/packages/app/public/images/agentx-methodology/subagent-distributions-256k.png b/packages/app/public/images/agentx-methodology/subagent-distributions-256k.png new file mode 100644 index 000000000..e695991ca Binary files /dev/null and b/packages/app/public/images/agentx-methodology/subagent-distributions-256k.png differ diff --git a/packages/app/public/images/agentx-methodology/subagent-distributions.png b/packages/app/public/images/agentx-methodology/subagent-distributions.png new file mode 100644 index 000000000..01b72a3e3 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/subagent-distributions.png differ diff --git a/packages/app/public/images/agentx-methodology/warmup-cache-sources.png b/packages/app/public/images/agentx-methodology/warmup-cache-sources.png new file mode 100644 index 000000000..f1bdfd761 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/warmup-cache-sources.png differ diff --git a/packages/app/public/images/agentx-methodology/warmup-snapshot.png b/packages/app/public/images/agentx-methodology/warmup-snapshot.png new file mode 100644 index 000000000..d806e911f Binary files /dev/null and b/packages/app/public/images/agentx-methodology/warmup-snapshot.png differ diff --git a/packages/app/public/images/agentx-methodology/weka-trace-record.png b/packages/app/public/images/agentx-methodology/weka-trace-record.png new file mode 100644 index 000000000..43c8263c3 Binary files /dev/null and b/packages/app/public/images/agentx-methodology/weka-trace-record.png differ diff --git a/packages/app/public/images/agentx-optimizations/aiter-64bit-offsets.png b/packages/app/public/images/agentx-optimizations/aiter-64bit-offsets.png new file mode 100644 index 000000000..76923c99d Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/aiter-64bit-offsets.png differ diff --git a/packages/app/public/images/agentx-optimizations/atom-prefix-promotion.png b/packages/app/public/images/agentx-optimizations/atom-prefix-promotion.png new file mode 100644 index 000000000..371389de3 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/atom-prefix-promotion.png differ diff --git a/packages/app/public/images/agentx-optimizations/dynamo-ownership-table.png b/packages/app/public/images/agentx-optimizations/dynamo-ownership-table.png new file mode 100644 index 000000000..b3b39ad11 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/dynamo-ownership-table.png differ diff --git a/packages/app/public/images/agentx-optimizations/lmcache-tiers.png b/packages/app/public/images/agentx-optimizations/lmcache-tiers.png new file mode 100644 index 000000000..7bdb5d247 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/lmcache-tiers.png differ diff --git a/packages/app/public/images/agentx-optimizations/serving-stack.png b/packages/app/public/images/agentx-optimizations/serving-stack.png new file mode 100644 index 000000000..afc647a1b Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/serving-stack.png differ diff --git a/packages/app/public/images/agentx-optimizations/sglang-hicache-asymmetry.png b/packages/app/public/images/agentx-optimizations/sglang-hicache-asymmetry.png new file mode 100644 index 000000000..86a1159f9 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/sglang-hicache-asymmetry.png differ diff --git a/packages/app/public/images/agentx-optimizations/sglang-radix-staging.png b/packages/app/public/images/agentx-optimizations/sglang-radix-staging.png new file mode 100644 index 000000000..5e6674142 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/sglang-radix-staging.png differ diff --git a/packages/app/public/images/agentx-optimizations/trtllm-pipelined-transfer.png b/packages/app/public/images/agentx-optimizations/trtllm-pipelined-transfer.png new file mode 100644 index 000000000..67c531479 Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/trtllm-pipelined-transfer.png differ diff --git a/packages/app/public/images/agentx-optimizations/vllm-connector-layer.png b/packages/app/public/images/agentx-optimizations/vllm-connector-layer.png new file mode 100644 index 000000000..a5d79fccf Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/vllm-connector-layer.png differ diff --git a/packages/app/public/images/agentx-optimizations/vllm-selective-retention.png b/packages/app/public/images/agentx-optimizations/vllm-selective-retention.png new file mode 100644 index 000000000..da734273f Binary files /dev/null and b/packages/app/public/images/agentx-optimizations/vllm-selective-retention.png differ diff --git a/packages/app/public/images/agentx-telemetry/chart-point-tooltip.png b/packages/app/public/images/agentx-telemetry/chart-point-tooltip.png new file mode 100644 index 000000000..bac3efc25 Binary files /dev/null and b/packages/app/public/images/agentx-telemetry/chart-point-tooltip.png differ diff --git a/packages/app/public/images/agentx-telemetry/flamegraph.png b/packages/app/public/images/agentx-telemetry/flamegraph.png new file mode 100644 index 000000000..4864123aa Binary files /dev/null and b/packages/app/public/images/agentx-telemetry/flamegraph.png differ diff --git a/packages/app/public/images/agentx-telemetry/per-point-overview.png b/packages/app/public/images/agentx-telemetry/per-point-overview.png new file mode 100644 index 000000000..49b205c8c Binary files /dev/null and b/packages/app/public/images/agentx-telemetry/per-point-overview.png differ diff --git a/packages/app/public/images/agentx-telemetry/point-detail-full.png b/packages/app/public/images/agentx-telemetry/point-detail-full.png new file mode 100644 index 000000000..e4577b359 Binary files /dev/null and b/packages/app/public/images/agentx-telemetry/point-detail-full.png differ diff --git a/packages/app/public/images/agentx-telemetry/request-timeline.png b/packages/app/public/images/agentx-telemetry/request-timeline.png new file mode 100644 index 000000000..38c4b7e46 Binary files /dev/null and b/packages/app/public/images/agentx-telemetry/request-timeline.png differ diff --git a/packages/app/public/images/brief-overview-of-agentic-workloads/context-accumulation.png b/packages/app/public/images/brief-overview-of-agentic-workloads/context-accumulation.png new file mode 100644 index 000000000..ac2a8db38 Binary files /dev/null and b/packages/app/public/images/brief-overview-of-agentic-workloads/context-accumulation.png differ diff --git a/packages/app/public/logos/deepseek-color.svg b/packages/app/public/logos/deepseek-color.svg new file mode 100644 index 000000000..3fc230240 --- /dev/null +++ b/packages/app/public/logos/deepseek-color.svg @@ -0,0 +1 @@ +DeepSeek \ No newline at end of file diff --git a/packages/app/public/logos/kimi-color.svg b/packages/app/public/logos/kimi-color.svg new file mode 100644 index 000000000..214eb84f3 --- /dev/null +++ b/packages/app/public/logos/kimi-color.svg @@ -0,0 +1 @@ +Kimi diff --git a/packages/app/public/logos/kvcache-ai.svg b/packages/app/public/logos/kvcache-ai.svg new file mode 100644 index 000000000..e8e3bb33b --- /dev/null +++ b/packages/app/public/logos/kvcache-ai.svg @@ -0,0 +1,5 @@ + + + + + diff --git a/packages/app/public/logos/meta-color.svg b/packages/app/public/logos/meta-color.svg new file mode 100644 index 000000000..a8a8d9485 --- /dev/null +++ b/packages/app/public/logos/meta-color.svg @@ -0,0 +1 @@ +Meta \ No newline at end of file diff --git a/packages/app/public/logos/minimax-color.svg b/packages/app/public/logos/minimax-color.svg new file mode 100644 index 000000000..beb7adb86 --- /dev/null +++ b/packages/app/public/logos/minimax-color.svg @@ -0,0 +1 @@ +Minimax \ No newline at end of file diff --git a/packages/app/public/logos/nvidia-color.svg b/packages/app/public/logos/nvidia-color.svg new file mode 100644 index 000000000..f711e8830 --- /dev/null +++ b/packages/app/public/logos/nvidia-color.svg @@ -0,0 +1 @@ +NVIDIA diff --git a/packages/app/public/logos/qwen-color.svg b/packages/app/public/logos/qwen-color.svg new file mode 100644 index 000000000..f2d0ada16 --- /dev/null +++ b/packages/app/public/logos/qwen-color.svg @@ -0,0 +1 @@ +Qwen \ No newline at end of file diff --git a/packages/app/public/logos/zai-color.svg b/packages/app/public/logos/zai-color.svg new file mode 100644 index 000000000..d46c329da --- /dev/null +++ b/packages/app/public/logos/zai-color.svg @@ -0,0 +1 @@ +Z.ai diff --git a/packages/app/public/promo-poster.webp b/packages/app/public/promo-poster.webp new file mode 100644 index 000000000..108e035c7 Binary files /dev/null and b/packages/app/public/promo-poster.webp differ diff --git a/packages/app/scripts/capture-cypress-fixtures.ts b/packages/app/scripts/capture-cypress-fixtures.ts index e30c042d3..0b1e61595 100644 --- a/packages/app/scripts/capture-cypress-fixtures.ts +++ b/packages/app/scripts/capture-cypress-fixtures.ts @@ -11,7 +11,7 @@ * bun run --cwd packages/app capture:fixtures -- --collectivex-only (synthetic multi-run data) */ -import { mkdir, writeFile } from 'node:fs/promises'; +import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'; import { resolve } from 'node:path'; import { buildRunSummary } from '@semianalysisai/inferencex-db/collectivex/reader'; @@ -20,6 +20,16 @@ import { makeCollectiveXDataset, makeRawShard, } from '@semianalysisai/inferencex-db/collectivex/test-fixture'; +import type { BenchmarkRow } from '@semianalysisai/inferencex-db/queries/benchmarks'; +import { + FIXTURE_MANIFEST_FILENAME, + FIXTURE_MANIFEST_SCHEMA_VERSION, + type FixtureManifest, + type FixtureManifestEntry, + assertFixtureContent, + fixtureSha256, + fixtureTopLevel, +} from '../src/lib/test-fixture-manifest'; const cliArgs = process.argv.filter((argument) => argument !== '--').slice(2); const collectiveXOnly = cliArgs.includes('--collectivex-only'); @@ -139,6 +149,44 @@ async function writeFixture(name: string, data: unknown): Promise { await writeFile(resolve(fixturesDir, `${name}.json`), body); return body.length; } +async function writeFixtureManifest(updatedNames: Set, source: string): Promise { + const manifestPath = resolve(fixturesDir, FIXTURE_MANIFEST_FILENAME); + let previous: FixtureManifest | null = null; + try { + previous = JSON.parse(await readFile(manifestPath, 'utf8')) as FixtureManifest; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error; + } + + const capturedAt = new Date().toISOString(); + const directoryEntries = await readdir(fixturesDir); + const fixtureFiles = directoryEntries + .filter((name) => name.endsWith('.json') && name !== FIXTURE_MANIFEST_FILENAME) + .toSorted(); + const fixtures: Record = {}; + for (const filename of fixtureFiles) { + const name = filename.slice(0, -'.json'.length); + const body = await readFile(resolve(fixturesDir, filename), 'utf8'); + const value = JSON.parse(body) as unknown; + assertFixtureContent(name, value); + const oldEntry = previous?.fixtures[name]; + const wasUpdated = updatedNames.has(name); + fixtures[name] = { + bytes: Buffer.byteLength(body), + capturedAt: wasUpdated ? capturedAt : (oldEntry?.capturedAt ?? null), + sha256: fixtureSha256(body), + source: wasUpdated ? source : (oldEntry?.source ?? null), + topLevel: fixtureTopLevel(value), + }; + } + + const manifest: FixtureManifest = { + schemaVersion: FIXTURE_MANIFEST_SCHEMA_VERSION, + generatedAt: capturedAt, + fixtures, + }; + await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +} async function writeCollectiveXFixtures(): Promise<[string, number][]> { const latest = makeCollectiveXDataset(); @@ -182,7 +230,9 @@ async function main() { await mkdir(fixturesDir, { recursive: true }); if (collectiveXOnly) { console.log('Generating synthetic CollectiveX fixtures'); - printSizes(await writeCollectiveXFixtures()); + const sizes = await writeCollectiveXFixtures(); + await writeFixtureManifest(new Set(sizes.map(([name]) => name)), 'synthetic://collectivex'); + printSizes(sizes); return; } @@ -200,14 +250,6 @@ async function main() { // Latest-snapshot: already deduped to one row per config, no date filter. // ~20 conc levels per (hw, fw, prec, isl, osl) — sample down to keep the // scatter visually populated without writing every concurrency point. - interface BenchmarkRow { - conc: number; - hardware: string; - framework: string; - precision: string; - isl: number; - osl: number; - } const benchmarks = await fetchJson( `/api/v1/benchmarks?model=${encodeURIComponent(BENCHMARK_MODEL)}`, ); @@ -309,6 +351,12 @@ async function main() { ...collectiveXSizes, ]; + const collectiveXNames = new Set(collectiveXSizes.map(([name]) => name)); + await writeFixtureManifest( + new Set(sizes.map(([name]) => name).filter((name) => !collectiveXNames.has(name))), + baseUrl, + ); + await writeFixtureManifest(collectiveXNames, 'synthetic://collectivex'); printSizes(sizes); } diff --git a/packages/app/scripts/check-typography.ts b/packages/app/scripts/check-typography.ts new file mode 100644 index 000000000..3722881f7 --- /dev/null +++ b/packages/app/scripts/check-typography.ts @@ -0,0 +1,96 @@ +#!/usr/bin/env bun +/** + * Typography ratchet gate — CLI. + * + * bun run check:typography # verify (CI + lefthook) + * bun run check:typography --update # shrink the allowlist after fixes + * + * Scans packages/app/src for arbitrary font sizes (text-[11px]), arbitrary + * letter-spacing (tracking-[0.16em]), and quoted font-size literals in the + * shared D3 chart library, then compares per-file counts against + * scripts/typography-allowlist.json. New offenders fail immediately; the + * allowlisted ones are the migration burndown and may only ever decrease. + * Rules and exemptions live in src/lib/typography-gate.ts; docs/typography.md + * explains the token system these rules point at. + */ +import { readdirSync, readFileSync, writeFileSync } from 'node:fs'; +import path from 'node:path'; + +import { + compareToAllowlist, + countViolations, + scanSource, + type ViolationCounts, +} from '../src/lib/typography-gate'; + +const APP_ROOT = path.resolve(import.meta.dirname, '..'); +const ALLOWLIST_PATH = path.join(import.meta.dirname, 'typography-allowlist.json'); +const UPDATE = process.argv.includes('--update'); + +function collectSourceFiles(dir: string, out: string[] = []): string[] { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) { + collectSourceFiles(full, out); + } else if (/\.tsx?$/u.test(entry.name)) { + out.push(full); + } + } + return out; +} + +const files = collectSourceFiles(path.join(APP_ROOT, 'src')) + .map((absPath) => ({ + relPath: path.relative(APP_ROOT, absPath).replaceAll(path.sep, '/'), + source: readFileSync(absPath, 'utf8'), + })) + .sort((a, b) => a.relPath.localeCompare(b.relPath)); + +const counts = countViolations(files); + +if (UPDATE) { + const sorted: ViolationCounts = Object.fromEntries( + Object.keys(counts) + .sort() + .map((file) => [file, counts[file]]), + ); + writeFileSync(ALLOWLIST_PATH, `${JSON.stringify(sorted, null, 2)}\n`); + const total = Object.values(sorted) + .flatMap((rules) => Object.values(rules)) + .reduce((sum, n) => sum + n, 0); + console.log( + `check-typography: allowlist rewritten with ${Object.keys(sorted).length} file(s), ${total} violation(s) remaining.`, + ); + process.exit(0); +} + +const allowlist: ViolationCounts = JSON.parse(readFileSync(ALLOWLIST_PATH, 'utf8')); +const { errors, stale } = compareToAllowlist(counts, allowlist); + +if (errors.length > 0) { + console.error('check-typography: new typography violations (see docs/typography.md):\n'); + for (const error of errors) { + console.error(` ${error}`); + const [file] = error.split(':', 1); + const entry = files.find((candidate) => candidate.relPath === file); + if (!entry) continue; + for (const violation of scanSource(entry.relPath, entry.source)) { + console.error( + ` ${file}:${violation.line} ${violation.match} → ${violation.suggestion}`, + ); + } + } +} + +if (stale.length > 0) { + console.error( + '\ncheck-typography: violations were fixed — ratchet down by running:\n bun run check:typography --update\n', + ); + for (const line of stale) console.error(` ${line}`); +} + +if (errors.length > 0 || stale.length > 0) { + process.exit(1); +} + +console.log('check-typography: OK'); diff --git a/packages/app/scripts/typography-allowlist.json b/packages/app/scripts/typography-allowlist.json new file mode 100644 index 000000000..bd3b12185 --- /dev/null +++ b/packages/app/scripts/typography-allowlist.json @@ -0,0 +1,26 @@ +{ + "src/app/land-acknowledgement/page.tsx": { + "arbitrary-tracking": 2 + }, + "src/app/zh/land-acknowledgement/page.tsx": { + "arbitrary-tracking": 2 + }, + "src/components/compare/agentx-compare-hero.tsx": { + "arbitrary-text-size": 1 + }, + "src/components/evaluation/ui/EvaluationTable.tsx": { + "arbitrary-text-size": 1 + }, + "src/components/glossary/glossary-browser.tsx": { + "arbitrary-tracking": 1 + }, + "src/components/inference/agentic-point/request-timeline.tsx": { + "arbitrary-text-size": 2 + }, + "src/components/inference/agentic-point/timeline-tooltips.tsx": { + "arbitrary-text-size": 1 + }, + "src/lib/d3-chart/layers/perf-ruler.ts": { + "chart-font-literal": 1 + } +} diff --git a/packages/app/src/app/(dashboard)/calculator/[model]/page.tsx b/packages/app/src/app/(dashboard)/calculator/[model]/page.tsx new file mode 100644 index 000000000..7675cafc9 --- /dev/null +++ b/packages/app/src/app/(dashboard)/calculator/[model]/page.tsx @@ -0,0 +1,44 @@ +import type { Metadata } from 'next'; +import { notFound, permanentRedirect } from 'next/navigation'; + +import ThroughputCalculatorDisplay from '@/components/calculator/ThroughputCalculatorDisplay'; +import { resolveCalculatorUrlSeed } from '@/components/calculator/url-seed'; +import { modelRoutePath, pathWithSearchParams, resolveModelRouteSlug } from '@/lib/model-routes'; +import { modelTabMetadata } from '@/lib/tab-meta'; + +/** + * `/calculator/` — the Throughput & TCO calculator seeded to a specific + * model, giving every model an indexable URL. Switching models in the UI + * rewrites the pathname in place via `history.replaceState` (no reload, no + * remount). No `generateStaticParams`: like the bare /calculator route this + * page reads searchParams, so it renders dynamically anyway. + */ + +interface Props { + params: Promise<{ model: string }>; + searchParams: Promise>; +} + +export async function generateMetadata({ params }: Props): Promise { + const { model } = await params; + const resolved = resolveModelRouteSlug(model); + if (!resolved) return {}; + return modelTabMetadata('calculator', resolved.route); +} + +export default async function CalculatorModelPage({ params, searchParams }: Props) { + const [{ model }, sp] = await Promise.all([params, searchParams]); + const resolved = resolveModelRouteSlug(model); + if (!resolved) notFound(); + // Aliases and non-canonical casing 308 to the canonical slug, keeping any + // share-link params, mirroring /compare/[slug]. + if (resolved.isAlias) { + permanentRedirect(pathWithSearchParams(modelRoutePath('calculator', resolved.route.slug), sp)); + } + const seed = resolveCalculatorUrlSeed(sp); + // A legacy explicit ?g_model= wins over the path — same precedence as + // GlobalFilterProvider, which then canonicalizes the pathname client-side. + return ( + + ); +} diff --git a/packages/app/src/app/(dashboard)/current-inferencex-image/page.tsx b/packages/app/src/app/(dashboard)/current-inferencex-image/page.tsx index 00c9f32df..d573d5fba 100644 --- a/packages/app/src/app/(dashboard)/current-inferencex-image/page.tsx +++ b/packages/app/src/app/(dashboard)/current-inferencex-image/page.tsx @@ -1,19 +1,7 @@ -import type { Metadata } from 'next'; - -import { SITE_URL } from '@semianalysisai/inferencex-constants'; import { CurrentImageContent } from '@/components/latest-image/latest-image-content'; +import { tabMetadata } from '@/lib/tab-meta'; -export const metadata: Metadata = { - title: 'Current InferenceX Image', - description: 'Current InferenceX Docker image tags for each model, chip SKU, and configuration.', - alternates: { canonical: `${SITE_URL}/current-inferencex-image` }, - openGraph: { - title: 'Current InferenceX Image | InferenceX by SemiAnalysis', - description: - 'Current InferenceX Docker image tags for each model, chip SKU, and configuration.', - url: `${SITE_URL}/current-inferencex-image`, - }, -}; +export const metadata = tabMetadata('current-inferencex-image'); export default function CurrentInferenceXImagePage() { return ; diff --git a/packages/app/src/app/(dashboard)/fleet/page.tsx b/packages/app/src/app/(dashboard)/fleet/page.tsx new file mode 100644 index 000000000..29ff67ef9 --- /dev/null +++ b/packages/app/src/app/(dashboard)/fleet/page.tsx @@ -0,0 +1,17 @@ +import type { Metadata } from 'next'; + +import FleetLifecycleDisplay from '@/components/calculator/FleetLifecycleDisplay'; +import { resolveCalculatorUrlSeed } from '@/components/calculator/url-seed'; +import { tabMetadata } from '@/lib/tab-meta'; + +export const metadata: Metadata = tabMetadata('fleet'); + +interface Props { + searchParams: Promise>; +} + +export default async function FleetPage({ searchParams }: Props) { + const sp = await searchParams; + const seed = resolveCalculatorUrlSeed(sp); + return ; +} diff --git a/packages/app/src/app/(dashboard)/historical/[model]/page.tsx b/packages/app/src/app/(dashboard)/historical/[model]/page.tsx new file mode 100644 index 000000000..0991b851b --- /dev/null +++ b/packages/app/src/app/(dashboard)/historical/[model]/page.tsx @@ -0,0 +1,55 @@ +import type { Metadata } from 'next'; +import { notFound, permanentRedirect } from 'next/navigation'; + +import { InferenceProvider } from '@/components/inference/InferenceContext'; +import HistoricalTrendsDisplay from '@/components/trends/HistoricalTrendsDisplay'; +import { + MODEL_ROUTES, + modelRoutePath, + pathWithSearchParams, + resolveModelRouteSlug, +} from '@/lib/model-routes'; +import { modelTabMetadata } from '@/lib/tab-meta'; + +/** + * `/historical/` — the Historical Trends tab seeded to a specific + * model, giving every model an indexable URL. `GlobalFilterProvider` (mounted + * by the dashboard shell) reads the model straight from the pathname, so the + * page body is identical to the bare tab; switching models in the UI rewrites + * the pathname in place via `history.replaceState` (no reload, no remount). + */ + +interface Props { + params: Promise<{ model: string }>; + searchParams: Promise>; +} + +export function generateStaticParams() { + return MODEL_ROUTES.map((route) => ({ model: route.slug })); +} + +export async function generateMetadata({ params }: Props): Promise { + const { model } = await params; + const resolved = resolveModelRouteSlug(model); + if (!resolved) return {}; + return modelTabMetadata('historical', resolved.route); +} + +export default async function HistoricalModelPage({ params, searchParams }: Props) { + const { model } = await params; + const resolved = resolveModelRouteSlug(model); + if (!resolved) notFound(); + // Aliases and non-canonical casing 308 to the canonical slug, keeping any + // share-link params, mirroring /compare/[slug]. `searchParams` is awaited + // only on this branch so the canonical slugs stay statically generated. + if (resolved.isAlias) { + permanentRedirect( + pathWithSearchParams(modelRoutePath('historical', resolved.route.slug), await searchParams), + ); + } + return ( + + + + ); +} diff --git a/packages/app/src/app/(dashboard)/inference/[model]/page.tsx b/packages/app/src/app/(dashboard)/inference/[model]/page.tsx new file mode 100644 index 000000000..02dce4651 --- /dev/null +++ b/packages/app/src/app/(dashboard)/inference/[model]/page.tsx @@ -0,0 +1,86 @@ +import type { Metadata } from 'next'; +import { notFound, permanentRedirect } from 'next/navigation'; + +import { AUTHOR_NAME, SITE_NAME, SITE_URL } from '@semianalysisai/inferencex-constants'; + +import { InferenceProvider } from '@/components/inference/InferenceContext'; +import InferenceChartDisplay from '@/components/inference/ui/ChartDisplay'; +import { enAlternates } from '@/lib/i18n'; +import { inferenceModelMeta } from '@/lib/inference-model-meta'; +import { + getInferenceModelBySlug, + INFERENCE_MODEL_SLUGS, + inferenceModelPath, +} from '@/lib/inference-model-slug'; + +/** + * `/inference/` — an indexable path form of `/inference?g_model=`. + * + * The query form stays supported for share links, but this page gives every + * model its own crawlable URL with model-specific metadata, a self-canonical, + * and a `/zh` hreflang sibling. The chart body is identical to `/inference`; + * the model is pinned from the pathname by the dashboard shell + * (`GlobalFilterProvider` seeding via `inferenceModelForPathname`), and an + * explicit `?g_model=` param still wins over the path pin. + */ + +interface Props { + params: Promise<{ model: string }>; + searchParams: Promise>; +} + +export function generateStaticParams(): { model: string }[] { + return INFERENCE_MODEL_SLUGS.map((entry) => ({ model: entry.slug })); +} + +export async function generateMetadata({ params }: Props): Promise { + const { model } = await params; + const entry = getInferenceModelBySlug(model); + if (!entry) return {}; + const { title, description } = inferenceModelMeta(entry); + const enPath = inferenceModelPath(entry.slug); + const url = `${SITE_URL}${enPath}`; + return { + title: { absolute: `${title} | ${SITE_NAME} by ${AUTHOR_NAME}` }, + description, + alternates: enAlternates(enPath), + openGraph: { + title: `${title} | ${SITE_NAME}`, + description, + url, + type: 'website', + }, + twitter: { + card: 'summary_large_image', + title: `${title} | ${SITE_NAME}`, + description, + }, + }; +} + +export default async function InferenceModelPage({ params, searchParams }: Props) { + const { model } = await params; + const entry = getInferenceModelBySlug(model); + if (!entry) notFound(); + // Aliases (family names, superseded versions, raw `g_model` display names, + // uppercase variants) collapse onto the one canonical URL per model. + // Preserves the query string so share-link params like `?i_seq=` and + // `?i_prec=` survive the redirect — same treatment as the compare pages. + if (model !== entry.slug) { + const sp = await searchParams; + const qs = Object.entries(sp) + .flatMap(([k, v]) => { + if (Array.isArray(v)) return v.map((vv) => [k, vv] as const); + if (v === undefined) return []; + return [[k, v] as const]; + }) + .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`) + .join('&'); + permanentRedirect(`${inferenceModelPath(entry.slug)}${qs ? `?${qs}` : ''}`); + } + return ( + + + + ); +} diff --git a/packages/app/src/app/(dashboard)/inference/agentic/page.tsx b/packages/app/src/app/(dashboard)/inference/agentic/page.tsx new file mode 100644 index 000000000..3c6a85d99 --- /dev/null +++ b/packages/app/src/app/(dashboard)/inference/agentic/page.tsx @@ -0,0 +1,49 @@ +import type { Metadata } from 'next'; + +import { AgenticCatalogHero } from '@/components/inference/agentic-catalog/agentic-catalog-hero'; +import { AgenticCatalogList } from '@/components/inference/agentic-catalog/agentic-catalog-list'; +import { JsonLd } from '@/components/json-ld'; +import { getAgenticCatalogGroups } from '@/lib/agentic-catalog'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +const TITLE = 'AgentX Telemetry'; +const DESCRIPTION = + 'Browse every AgentX benchmark run with stored per-request telemetry, grouped by model and serving stack. Each configuration opens its point-detail charts, request timeline, and cache behavior.'; + +export const metadata: Metadata = { + title: TITLE, + description: DESCRIPTION, + alternates: enAlternates('/inference/agentic'), + openGraph: { + title: `${TITLE} | InferenceX`, + description: DESCRIPTION, + url: `${SITE_URL}/inference/agentic`, + }, + twitter: { title: `${TITLE} | InferenceX`, description: DESCRIPTION }, +}; + +// The catalog reads the same live benchmark tables the dashboard does, so a +// build-time snapshot would go stale the moment a new run lands. +export const dynamic = 'force-dynamic'; + +const jsonLd = { + '@context': 'https://schema.org', + '@type': 'CollectionPage', + name: `${TITLE} | InferenceX`, + description: DESCRIPTION, + url: `${SITE_URL}/inference/agentic`, + inLanguage: 'en', + isPartOf: { '@type': 'WebSite', name: 'InferenceX', url: SITE_URL }, +}; + +export default async function AgenticTelemetryCatalogPage() { + const groups = await getAgenticCatalogGroups(); + return ( + <> + + + + + ); +} diff --git a/packages/app/src/app/(dashboard)/inference/logs/[id]/page.tsx b/packages/app/src/app/(dashboard)/inference/logs/[id]/page.tsx new file mode 100644 index 000000000..681af75ca --- /dev/null +++ b/packages/app/src/app/(dashboard)/inference/logs/[id]/page.tsx @@ -0,0 +1,17 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { BenchmarkLogDetail } from '@/components/inference/log-viewer/benchmark-log-detail'; +import { isPersistedBenchmarkId } from '@/lib/benchmark-id'; + +export const metadata: Metadata = { + title: 'Benchmark logs | InferenceX', + robots: { index: false }, +}; + +export default async function BenchmarkLogPage({ params }: { params: Promise<{ id: string }> }) { + const { id } = await params; + const numericId = Number(id); + if (!isPersistedBenchmarkId(numericId)) notFound(); + return ; +} diff --git a/packages/app/src/app/about/page.tsx b/packages/app/src/app/about/page.tsx index 5e9348afe..ae75c761e 100644 --- a/packages/app/src/app/about/page.tsx +++ b/packages/app/src/app/about/page.tsx @@ -4,38 +4,29 @@ import Link from 'next/link'; import { Card } from '@/components/ui/card'; import { FAQ_ITEMS } from '@/components/about/faq-data'; import { AgentXFaq } from '@/components/about/agentx-faq'; +import { buildFaqJsonLd, FaqList } from '@/components/about/faq'; +import { PromoVideo } from '@/components/about/promo-video'; import { JsonLd } from '@/components/json-ld'; import { enAlternates } from '@/lib/i18n'; import { GITHUB_OWNER, GITHUB_REPO, SITE_URL } from '@semianalysisai/inferencex-constants'; -const faqJsonLd = { - '@context': 'https://schema.org', - '@type': 'FAQPage', - mainEntity: FAQ_ITEMS.map((item) => ({ - '@type': 'Question', - name: item.question, - acceptedAnswer: { - '@type': 'Answer', - text: [item.answer, item.link?.text, ...(item.list ?? [])].filter(Boolean).join(' '), - }, - })), -}; +const faqJsonLd = buildFaqJsonLd(FAQ_ITEMS); + +const ABOUT_DESCRIPTION = + 'InferenceX benchmarks agentic and fixed-sequence AI inference across accelerators and serving stacks. AgentX is its long-context, multi-turn coding scenario.'; export const metadata: Metadata = { title: 'About', - description: - 'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.', + description: ABOUT_DESCRIPTION, alternates: enAlternates('/about'), openGraph: { title: 'About | InferenceX', - description: - 'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.', + description: ABOUT_DESCRIPTION, url: `${SITE_URL}/about`, }, twitter: { title: 'About | InferenceX', - description: - 'InferenceX is an independent, vendor neutral, reproducible benchmark which continuously benchmarks inference software across a wide range of AI accelerators.', + description: ABOUT_DESCRIPTION, }, }; @@ -44,11 +35,15 @@ export default function AboutPage() { ); diff --git a/packages/app/src/app/datasets/[slug]/conversations/[convId]/page.tsx b/packages/app/src/app/agentx/[slug]/conversations/[convId]/page.tsx similarity index 94% rename from packages/app/src/app/datasets/[slug]/conversations/[convId]/page.tsx rename to packages/app/src/app/agentx/[slug]/conversations/[convId]/page.tsx index 8957c8b4d..8a9d79226 100644 --- a/packages/app/src/app/datasets/[slug]/conversations/[convId]/page.tsx +++ b/packages/app/src/app/agentx/[slug]/conversations/[convId]/page.tsx @@ -19,7 +19,7 @@ export async function generateMetadata({ params }: Props): Promise { title, description, alternates: { - canonical: `${SITE_URL}/datasets/${slug}/conversations/${encodeURIComponent(convId)}`, + canonical: `${SITE_URL}/agentx/${slug}/conversations/${encodeURIComponent(convId)}`, }, robots: { index: false }, // per-conversation pages are too numerous to index }; diff --git a/packages/app/src/app/datasets/[slug]/page.tsx b/packages/app/src/app/agentx/[slug]/page.tsx similarity index 83% rename from packages/app/src/app/datasets/[slug]/page.tsx rename to packages/app/src/app/agentx/[slug]/page.tsx index 567f5588d..613ccde1a 100644 --- a/packages/app/src/app/datasets/[slug]/page.tsx +++ b/packages/app/src/app/agentx/[slug]/page.tsx @@ -10,16 +10,16 @@ interface Props { export async function generateMetadata({ params }: Props): Promise { const { slug } = await params; - const title = `${slug} | Agentic Datasets`; + const title = `${slug} | AgentX Datasets`; const description = `Distributions, token statistics, and per-conversation flamegraphs for the ${slug} agentic trace dataset.`; return { title, description, alternates: { - canonical: `${SITE_URL}/datasets/${slug}`, - languages: languageAlternates(`/datasets/${slug}`), + canonical: `${SITE_URL}/agentx/${slug}`, + languages: languageAlternates(`/agentx/${slug}`), }, - openGraph: { title: `${title} | InferenceX`, description, url: `${SITE_URL}/datasets/${slug}` }, + openGraph: { title: `${title} | InferenceX`, description, url: `${SITE_URL}/agentx/${slug}` }, twitter: { title: `${title} | InferenceX`, description }, }; } diff --git a/packages/app/src/app/agentx/methodology/page.tsx b/packages/app/src/app/agentx/methodology/page.tsx new file mode 100644 index 000000000..4aee25e6d --- /dev/null +++ b/packages/app/src/app/agentx/methodology/page.tsx @@ -0,0 +1,42 @@ +import type { Metadata } from 'next'; + +import { AgentXMethodologyArticle } from '@/components/datasets/agentx-methodology-article'; +import { JsonLd } from '@/components/json-ld'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +const DESCRIPTION = + 'How AgentX transforms opt-in coding-agent traces into replay graphs with controlled KV-cache warmup and benchmark settings.'; + +export const metadata: Metadata = { + title: 'AgentX Methodology', + description: DESCRIPTION, + alternates: enAlternates('/agentx/methodology'), + openGraph: { + title: 'AgentX Methodology | InferenceX', + description: DESCRIPTION, + url: `${SITE_URL}/agentx/methodology`, + }, + twitter: { title: 'AgentX Methodology | InferenceX', description: DESCRIPTION }, +}; + +const jsonLd = { + '@context': 'https://schema.org', + '@type': 'TechArticle', + headline: 'AgentX Methodology', + description: DESCRIPTION, + url: `${SITE_URL}/agentx/methodology`, + inLanguage: 'en', + isPartOf: { '@type': 'WebSite', name: 'InferenceX', url: SITE_URL }, +}; + +export default function AgentXMethodologyPage() { + return ( +
+ +
+ +
+
+ ); +} diff --git a/packages/app/src/app/agentx/optimizations/[framework]/page.tsx b/packages/app/src/app/agentx/optimizations/[framework]/page.tsx new file mode 100644 index 000000000..b6dff301e --- /dev/null +++ b/packages/app/src/app/agentx/optimizations/[framework]/page.tsx @@ -0,0 +1,64 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { AgentXOptimizationsArticle } from '@/components/datasets/agentx-optimizations-article'; +import { JsonLd } from '@/components/json-ld'; +import { + AGENTX_OPTIMIZATION_FRAMEWORKS, + getOptimizationFramework, +} from '@/lib/agentx-optimizations'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +interface Props { + params: Promise<{ framework: string }>; +} + +export function generateStaticParams() { + return AGENTX_OPTIMIZATION_FRAMEWORKS.map((entry) => ({ framework: entry.slug })); +} + +export const dynamicParams = false; + +export async function generateMetadata({ params }: Props): Promise { + const { framework: slug } = await params; + const framework = getOptimizationFramework(slug); + if (!framework) return {}; + const title = `${framework.name} Agentic Optimizations`; + return { + title, + description: framework.summary, + alternates: enAlternates(`/agentx/optimizations/${slug}`), + openGraph: { + title: `${title} | InferenceX`, + description: framework.summary, + url: `${SITE_URL}/agentx/optimizations/${slug}`, + }, + twitter: { title: `${title} | InferenceX`, description: framework.summary }, + }; +} + +export default async function AgentXOptimizationsFrameworkPage({ params }: Props) { + const { framework: slug } = await params; + const framework = getOptimizationFramework(slug); + if (!framework) notFound(); + + const jsonLd = { + '@context': 'https://schema.org', + '@type': 'TechArticle', + headline: `${framework.name} Agentic Optimizations`, + description: framework.summary, + url: `${SITE_URL}/agentx/optimizations/${slug}`, + inLanguage: 'en', + isPartOf: { '@type': 'WebSite', name: 'InferenceX', url: SITE_URL }, + }; + + return ( +
+ +
+ +
+
+ ); +} diff --git a/packages/app/src/app/agentx/optimizations/page.tsx b/packages/app/src/app/agentx/optimizations/page.tsx new file mode 100644 index 000000000..559df325a --- /dev/null +++ b/packages/app/src/app/agentx/optimizations/page.tsx @@ -0,0 +1,45 @@ +import type { Metadata } from 'next'; + +import { AgentXOptimizationsIndex } from '@/components/datasets/agentx-optimizations-article'; +import { JsonLd } from '@/components/json-ld'; +import { OPTIMIZATIONS_OVERVIEW } from '@/lib/agentx-optimizations'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +const TITLE = 'AgentX Industry Impact: Optimizations for Agentic Workloads'; +const DESCRIPTION = + 'The 50+ upstream pull requests AgentX drove across vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, LMCache, and Mooncake.'; + +export const metadata: Metadata = { + title: 'AgentX Industry Impact', + description: DESCRIPTION, + alternates: enAlternates('/agentx/optimizations'), + openGraph: { + title: `${TITLE} | InferenceX`, + description: DESCRIPTION, + url: `${SITE_URL}/agentx/optimizations`, + }, + twitter: { title: `${TITLE} | InferenceX`, description: DESCRIPTION }, +}; + +const jsonLd = { + '@context': 'https://schema.org', + '@type': 'TechArticle', + headline: TITLE, + description: DESCRIPTION, + url: `${SITE_URL}/agentx/optimizations`, + inLanguage: 'en', + about: OPTIMIZATIONS_OVERVIEW.title, + isPartOf: { '@type': 'WebSite', name: 'InferenceX', url: SITE_URL }, +}; + +export default function AgentXOptimizationsPage() { + return ( +
+ +
+ +
+
+ ); +} diff --git a/packages/app/src/app/agentx/page.tsx b/packages/app/src/app/agentx/page.tsx new file mode 100644 index 000000000..41308e6ec --- /dev/null +++ b/packages/app/src/app/agentx/page.tsx @@ -0,0 +1,48 @@ +import type { Metadata } from 'next'; + +import { AgentXMethodology } from '@/components/datasets/agentx-methodology'; +import { DatasetList } from '@/components/datasets/dataset-list'; +import { JsonLd } from '@/components/json-ld'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +const DESCRIPTION = + 'AgentX replays workload shapes derived from opt-in Claude Code sessions. Explore its methodology, distributions, and per-conversation flamegraphs.'; + +export const metadata: Metadata = { + title: 'AgentX Methodology and Datasets', + description: DESCRIPTION, + alternates: enAlternates('/agentx'), + openGraph: { + title: 'AgentX Methodology and Datasets | InferenceX', + description: DESCRIPTION, + url: `${SITE_URL}/agentx`, + }, + twitter: { title: 'AgentX Methodology and Datasets | InferenceX', description: DESCRIPTION }, +}; + +const jsonLd = { + '@context': 'https://schema.org', + '@type': 'CollectionPage', + name: 'InferenceX AgentX Datasets', + description: DESCRIPTION, + url: `${SITE_URL}/agentx`, +}; + +export default function AgentXPage() { + return ( +
+ +
+
+ +
+ +
+

Datasets

+ +
+
+
+ ); +} diff --git a/packages/app/src/app/agentx/telemetry/page.tsx b/packages/app/src/app/agentx/telemetry/page.tsx new file mode 100644 index 000000000..374f1fbcb --- /dev/null +++ b/packages/app/src/app/agentx/telemetry/page.tsx @@ -0,0 +1,45 @@ +import type { Metadata } from 'next'; + +import { AgentXTelemetryArticle } from '@/components/datasets/agentx-telemetry-article'; +import { JsonLd } from '@/components/json-ld'; +import { AGENTX_TELEMETRY_GUIDE } from '@/lib/agentx-telemetry'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_URL } from '@semianalysisai/inferencex-constants'; + +const TITLE = 'Exploring Agentic Workloads: Detailed Telemetry'; +const DESCRIPTION = + 'How to read the telemetry behind a single AgentX point: the per-point charts, the request timeline, KV offload markers, and the per-conversation flamegraph.'; + +export const metadata: Metadata = { + title: 'AgentX Telemetry Tutorial', + description: DESCRIPTION, + alternates: enAlternates('/agentx/telemetry'), + openGraph: { + title: `${TITLE} | InferenceX`, + description: DESCRIPTION, + url: `${SITE_URL}/agentx/telemetry`, + }, + twitter: { title: `${TITLE} | InferenceX`, description: DESCRIPTION }, +}; + +const jsonLd = { + '@context': 'https://schema.org', + '@type': 'TechArticle', + headline: TITLE, + description: DESCRIPTION, + url: `${SITE_URL}/agentx/telemetry`, + inLanguage: 'en', + about: AGENTX_TELEMETRY_GUIDE.title, + isPartOf: { '@type': 'WebSite', name: 'InferenceX', url: SITE_URL }, +}; + +export default function AgentXTelemetryPage() { + return ( +
+ +
+ +
+
+ ); +} diff --git a/packages/app/src/app/api/unofficial-run/route.test.ts b/packages/app/src/app/api/unofficial-run/route.test.ts index b9c45b801..d4a8e5f4d 100644 --- a/packages/app/src/app/api/unofficial-run/route.test.ts +++ b/packages/app/src/app/api/unofficial-run/route.test.ts @@ -144,6 +144,70 @@ describe('normalizeArtifactRows', () => { expect(m.mean_e2el).toBe(1.5); }); + it.each([ + { + name: 'boolean verdict and junk-suffixed schema', + input: { power_valid: true, power_metric_schema_version: '2garbage' }, + expectedVerdict: 0, + expectedSchema: undefined, + }, + { + name: 'garbage verdict and valid numeric schema', + input: { power_valid: 'garbage', power_metric_schema_version: 2 }, + expectedVerdict: 0, + expectedSchema: 2, + }, + { + name: 'canonical numeric strings', + input: { power_valid: '1', power_metric_schema_version: '2' }, + expectedVerdict: 1, + expectedSchema: 2, + }, + { + name: 'explicit invalid verdict', + input: { power_valid: 0, power_metric_schema_version: 2 }, + expectedVerdict: 0, + expectedSchema: 2, + }, + { + name: 'legacy row without power contract fields', + input: {}, + expectedVerdict: undefined, + expectedSchema: undefined, + }, + ])( + 'normalizes overlay power contract discriminators: $name', + ({ input, expectedVerdict, expectedSchema }) => { + const [row] = normalizeArtifactRows([rawRow(input)], '2026-03-01'); + + if (expectedVerdict === undefined) { + expect(row.metrics).not.toHaveProperty('power_valid'); + } else { + expect(row.metrics.power_valid).toBe(expectedVerdict); + } + if (expectedSchema === undefined) { + expect(row.metrics).not.toHaveProperty('power_metric_schema_version'); + } else { + expect(row.metrics.power_metric_schema_version).toBe(expectedSchema); + } + }, + ); + + it('preserves recipe identity for unofficial overlays', () => { + const rows = normalizeArtifactRows( + [ + rawRow({ + recipe_fingerprint: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + }), + ], + '2026-03-01', + ); + + expect(rows[0].recipe_fingerprint).toBe( + 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', + ); + }); + it('surfaces pipeline-parallelism fields in metrics (auto-capture)', () => { // pp has no configs-table column: the frontend reads it from the metrics // JSONB (rowToAggDataEntry), so the overlay route must keep passing it diff --git a/packages/app/src/app/api/unofficial-run/route.ts b/packages/app/src/app/api/unofficial-run/route.ts index 168d05d51..b827b6d75 100644 --- a/packages/app/src/app/api/unofficial-run/route.ts +++ b/packages/app/src/app/api/unofficial-run/route.ts @@ -68,6 +68,7 @@ export function normalizeArtifactRows( osl: params.osl, conc: params.conc, image: params.image, + recipe_fingerprint: params.recipeFingerprint, metrics: params.metrics, // Surface the same per-worker payload the DB path emits so unofficial // overlays carry the multinode measured-power breakdown too. diff --git a/packages/app/src/app/api/v1/agentic-aggregates/route.ts b/packages/app/src/app/api/v1/agentic-aggregates/route.ts index 9808cec6e..f0cd38408 100644 --- a/packages/app/src/app/api/v1/agentic-aggregates/route.ts +++ b/packages/app/src/app/api/v1/agentic-aggregates/route.ts @@ -32,7 +32,7 @@ const getCachedAgenticAggregates = cachedQuery( /** * GET /api/v1/agentic-aggregates?ids=1,2,3 * - * Returns per-id mean/p50/p75/p90/p99 for ISL, OSL, KV cache utilization, + * Returns per-id mean/p50/p75/p90/p95/p99 for ISL, OSL, KV cache utilization, * and prefix cache hit rate — computed live from the stored aiperf * profile_export.jsonl + server_metrics_json blobs. Ids without a * trace_replay blob (or with no usable samples) get nulls. diff --git a/packages/app/src/app/api/v1/agentic-cache-keys.test.ts b/packages/app/src/app/api/v1/agentic-cache-keys.test.ts index 7177f9960..57be6bbcc 100644 --- a/packages/app/src/app/api/v1/agentic-cache-keys.test.ts +++ b/packages/app/src/app/api/v1/agentic-cache-keys.test.ts @@ -26,12 +26,17 @@ vi.mock('@/lib/api-cache', () => ({ import { STATS_VERSION } from '@semianalysisai/inferencex-db/queries/agentic-aggregates'; import { REQUEST_TIMELINE_VERSION } from '@semianalysisai/inferencex-db/etl/compute-request-timeline'; +import { CHART_SERIES_VERSION } from '@semianalysisai/inferencex-db/etl/compute-chart-series'; +import { REQUEST_CHART_DATA_VERSION } from '@semianalysisai/inferencex-db/queries/request-chart-data'; import { TRACE_SERVER_METRICS_VERSION } from '@semianalysisai/inferencex-db/queries/trace-server-metrics'; import { CACHE_KEY_PREFIX as agenticAggregatesKey } from './agentic-aggregates/route'; import { CACHE_KEY_PREFIX as requestTimelineKey } from './request-timeline/route'; +import { CACHE_KEY_PREFIX as requestChartDataKey } from './request-chart-data/route'; +import { CACHE_KEY_PREFIX as traceServerMetricSourceKey } from './trace-server-metric-source/route'; import { CACHE_KEY_PREFIX as traceServerMetricsKey } from './trace-server-metrics/route'; import { CACHE_KEY_PREFIX as traceHistogramsKey } from './trace-histograms/route'; +import { CACHE_KEY_PREFIX as residentSequenceLengthsKey } from './resident-sequence-lengths/route'; describe('agentic blob-cache keys are version-derived', () => { it('agentic-aggregates key embeds STATS_VERSION', () => { @@ -42,20 +47,35 @@ describe('agentic blob-cache keys are version-derived', () => { expect(requestTimelineKey).toBe(`request-timeline-v${REQUEST_TIMELINE_VERSION}`); }); + it('request-chart-data key embeds REQUEST_CHART_DATA_VERSION', () => { + expect(requestChartDataKey).toBe(`request-chart-data-v${REQUEST_CHART_DATA_VERSION}`); + }); + it('trace-server-metrics key embeds its composite response version', () => { expect(traceServerMetricsKey).toBe(`trace-server-metrics-v${TRACE_SERVER_METRICS_VERSION}`); }); + it('trace-server-metric-source key embeds CHART_SERIES_VERSION', () => { + expect(traceServerMetricSourceKey).toBe(`trace-server-metric-source-v${CHART_SERIES_VERSION}`); + }); + it('trace-histograms key embeds REQUEST_TIMELINE_VERSION (its payload is read from request_timeline)', () => { expect(traceHistogramsKey).toBe(`trace-histograms-v${REQUEST_TIMELINE_VERSION}`); }); + it('resident-sequence-lengths key embeds STATS_VERSION', () => { + expect(residentSequenceLengthsKey).toBe(`resident-sequence-lengths-v${STATS_VERSION}`); + }); + it('every key actually contains a version segment (no unversioned literals)', () => { for (const key of [ agenticAggregatesKey, requestTimelineKey, + requestChartDataKey, traceServerMetricsKey, + traceServerMetricSourceKey, traceHistogramsKey, + residentSequenceLengthsKey, ]) { expect(key).toMatch(/-v\d+$/u); } diff --git a/packages/app/src/app/api/v1/availability/route.ts b/packages/app/src/app/api/v1/availability/route.ts index c3c4bab0a..b8db29c86 100644 --- a/packages/app/src/app/api/v1/availability/route.ts +++ b/packages/app/src/app/api/v1/availability/route.ts @@ -1,23 +1,15 @@ -import { NextResponse } from 'next/server'; - -import { FIXTURES_MODE, getDb } from '@semianalysisai/inferencex-db/connection'; +import { getDb } from '@semianalysisai/inferencex-db/connection'; import { getAvailabilityData } from '@semianalysisai/inferencex-db/queries/workflow-info'; -import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { cachedReadRoute } from '@/lib/cached-read-route'; import { loadFixture } from '@/lib/test-fixtures'; export const dynamic = 'force-dynamic'; -const getCachedAvailability = cachedQuery(() => getAvailabilityData(getDb()), 'availability'); - -export async function GET() { - if (FIXTURES_MODE) return cachedJson(loadFixture('availability')); - try { - const rows = await getCachedAvailability(); - return cachedJson(rows); - } catch (error) { - console.error('Error fetching availability:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); - } -} +export const GET = cachedReadRoute({ + cacheKey: 'availability', + fetch: () => getAvailabilityData(getDb()), + fixture: () => loadFixture('availability'), + logLabel: 'availability', +}); diff --git a/packages/app/src/app/api/v1/benchmark-siblings/route.ts b/packages/app/src/app/api/v1/benchmark-siblings/route.ts index 1e7482278..a5a5ce5ec 100644 --- a/packages/app/src/app/api/v1/benchmark-siblings/route.ts +++ b/packages/app/src/app/api/v1/benchmark-siblings/route.ts @@ -13,9 +13,9 @@ export const dynamic = 'force-dynamic'; const getCachedSiblings = cachedQuery( (id: number): Promise => getBenchmarkSiblings(getDb(), id), - // v2 adds PP and makes the aggregate topology explicit. Roll the blob - // namespace so old TP0 sibling payloads cannot survive the schema change. - 'benchmark-siblings-v2', + // v3 adds DCP/PCP. Roll the blob namespace so cached sibling payloads gain + // the complete parallelism mapping used by chart point labels. + 'benchmark-siblings-v3', ); /** diff --git a/packages/app/src/app/api/v1/benchmarks/history/route.ts b/packages/app/src/app/api/v1/benchmarks/history/route.ts index 113a6ed65..28ba786ba 100644 --- a/packages/app/src/app/api/v1/benchmarks/history/route.ts +++ b/packages/app/src/app/api/v1/benchmarks/history/route.ts @@ -1,11 +1,13 @@ -import { type NextRequest, NextResponse } from 'next/server'; +import type { NextRequest } from 'next/server'; -import { DISPLAY_MODEL_TO_DB } from '@semianalysisai/inferencex-constants'; +import { DISPLAY_MODEL_TO_DB, islOslToSequence } from '@semianalysisai/inferencex-constants'; import { FIXTURES_MODE, getDb } from '@semianalysisai/inferencex-db/connection'; import { getAllBenchmarksForHistory } from '@semianalysisai/inferencex-db/queries/benchmarks'; import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { toCalculatorBenchmarkRows } from '@/lib/benchmark-api-view'; +import { PUBLIC_API_ERRORS, publicApiError } from '@/lib/public-api-errors'; import { loadFixture } from '@/lib/test-fixtures'; import { agenticWorkflowMetadataOnly } from '@/lib/agentic-workflow-metadata'; @@ -24,6 +26,34 @@ const getCachedAgenticBenchmarkHistory = cachedQuery( { blobOnly: true }, ); +/** + * Trim a history response to the calculator's metric allowlist. + * + * Opt-in only. `CALCULATOR_METRIC_KEYS` excludes the measured-power metrics + * (`measured_avg_power_w`, `joules_per_output_token`, …) that Historical Trends + * plots, so applying this to every history response would silently blank those + * charts. The sequence is derived from the route's own isl/osl rather than + * taken as a parameter, so the two can never disagree. + * + * Agentic requests carry no isl/osl and so have no sequence to key on. Those + * responses pass through untrimmed rather than guessing a sequence — the trim is + * a payload optimisation, and the agentic payload is a fraction of a fixed one. + * The calculator's lifecycle section consumes them either way. + */ +function applyCalculatorView< + T extends { + benchmark_type: string; + isl: number | null; + osl: number | null; + metrics: Record; + }, +>(rows: T[], isl: number | null, osl: number | null): T[] { + if (isl === null || osl === null) return rows; + const sequence = islOslToSequence(isl, osl); + if (!sequence) return rows; + return toCalculatorBenchmarkRows(rows, sequence); +} + export async function GET(request: NextRequest) { const model = request.nextUrl.searchParams.get('model') ?? ''; const rawIsl = request.nextUrl.searchParams.get('isl'); @@ -32,26 +62,34 @@ export async function GET(request: NextRequest) { const isl = rawIsl === null ? null : Number(rawIsl); const osl = rawOsl === null ? null : Number(rawOsl); const isAgentic = benchmarkType === 'agentic_traces'; + const calculatorView = request.nextUrl.searchParams.get('view') === 'calculator'; if (!model) { - return NextResponse.json({ error: 'model, isl, and osl are required' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.benchmarkHistoryParameters, 400); } if (!isAgentic && (!isl || !osl)) { - return NextResponse.json({ error: 'model, isl, and osl are required' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.benchmarkHistoryParameters, 400); + } + if (FIXTURES_MODE) { + const fixture = loadFixture('benchmarks-history'); + return cachedJson( + calculatorView && Array.isArray(fixture) ? applyCalculatorView(fixture, isl, osl) : fixture, + ); } - if (FIXTURES_MODE) return cachedJson(loadFixture('benchmarks-history')); try { const modelKeys = DISPLAY_MODEL_TO_DB[model]; if (!modelKeys || modelKeys.length === 0) { - return NextResponse.json({ error: 'Unknown model' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.unknownModel, 400); } const rows = isAgentic ? await getCachedAgenticBenchmarkHistory(modelKeys) : await getCachedBenchmarkHistory(modelKeys, isl!, osl!); - return cachedJson(agenticWorkflowMetadataOnly(rows)); + // Both trims run after the cache, so every view shares one cached query. + const trimmed = agenticWorkflowMetadataOnly(rows); + return cachedJson(calculatorView ? applyCalculatorView(trimmed, isl, osl) : trimmed); } catch (error) { console.error('Error fetching benchmark history:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } } diff --git a/packages/app/src/app/api/v1/benchmarks/route.ts b/packages/app/src/app/api/v1/benchmarks/route.ts index 84ef529f8..a75260d7b 100644 --- a/packages/app/src/app/api/v1/benchmarks/route.ts +++ b/packages/app/src/app/api/v1/benchmarks/route.ts @@ -11,6 +11,7 @@ import { import { cachedJson, cachedQuery } from '@/lib/api-cache'; import { toCalculatorBenchmarkRows } from '@/lib/benchmark-api-view'; +import { PUBLIC_API_ERRORS, publicApiError } from '@/lib/public-api-errors'; import { agenticWorkflowMetadataOnly } from '@/lib/agentic-workflow-metadata'; import { loadFixture } from '@/lib/test-fixtures'; @@ -23,8 +24,9 @@ const getCachedBenchmarks = cachedQuery( { blobOnly: true }, ); -// Exactly one run's results (GPU comparison of individual same-day runs). Cached -// under a distinct key prefix so it never collides with the latest/as-of query. +// One logical run snapshot (GPU comparison of individual same-day runs). For an +// append-only run this includes its same-image predecessor chain. Cached under a +// distinct key prefix so it never collides with the latest/as-of query. const getCachedBenchmarksForRun = cachedQuery( (dbModelKeys: string[], runId: string) => getBenchmarksForRun(getDb(), dbModelKeys, runId), 'benchmarks-run-agentic-run-metadata', @@ -52,7 +54,7 @@ export async function GET(request: NextRequest) { const sequence = params.get('sequence') ?? ''; const dbModelKeys = DISPLAY_MODEL_TO_DB[model]; if (!dbModelKeys || dbModelKeys.length === 0) { - return NextResponse.json({ error: 'Unknown model' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.unknownModel, 400); } if (view === 'calculator' && !['1k/1k', '1k/8k', '8k/1k', 'agentic-traces'].includes(sequence)) { return NextResponse.json({ error: 'Unknown calculator sequence' }, { status: 400 }); @@ -74,6 +76,6 @@ export async function GET(request: NextRequest) { return cachedJson(agenticWorkflowMetadataOnly(rows)); } catch (error) { console.error('Error fetching benchmarks:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } } diff --git a/packages/app/src/app/api/v1/collectivex/latest/route.test.ts b/packages/app/src/app/api/v1/collectivex/latest/route.test.ts index da3ee6929..b806694b6 100644 --- a/packages/app/src/app/api/v1/collectivex/latest/route.test.ts +++ b/packages/app/src/app/api/v1/collectivex/latest/route.test.ts @@ -2,15 +2,15 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { makeCollectiveXDataset } from '@semianalysisai/inferencex-db/collectivex/test-fixture'; -const { mockGetLatest, mockFromRow, mockGetDb, mockEnsureLatest } = vi.hoisted(() => ({ +const { mockGetLatest, mockFromRow, mockGetWriteDb, mockEnsureLatest } = vi.hoisted(() => ({ mockGetLatest: vi.fn(), mockFromRow: vi.fn(), - mockGetDb: vi.fn(() => 'mock-sql'), + mockGetWriteDb: vi.fn(() => 'mock-write-sql'), mockEnsureLatest: vi.fn(), })); vi.mock('@semianalysisai/inferencex-db/connection', () => ({ - getCollectiveXDb: mockGetDb, + getCollectiveXWriteDb: mockGetWriteDb, FIXTURES_MODE: false, })); @@ -73,7 +73,7 @@ describe('GET /api/v1/collectivex/latest', () => { expect(res.status).toBe(200); expect(await res.json()).toEqual(dataset); expect(mockEnsureLatest).toHaveBeenCalledWith(1); - expect(mockGetLatest).toHaveBeenCalledWith('mock-sql', 1); + expect(mockGetLatest).toHaveBeenCalledWith('mock-write-sql', 1); }); it('serves the stored run when GitHub discovery fails', async () => { diff --git a/packages/app/src/app/api/v1/collectivex/latest/route.ts b/packages/app/src/app/api/v1/collectivex/latest/route.ts index 264b5a842..302063309 100644 --- a/packages/app/src/app/api/v1/collectivex/latest/route.ts +++ b/packages/app/src/app/api/v1/collectivex/latest/route.ts @@ -1,7 +1,7 @@ import { type NextRequest, NextResponse } from 'next/server'; import { parseCollectiveXVersion } from '@semianalysisai/inferencex-db/collectivex/types'; -import { FIXTURES_MODE, getCollectiveXDb } from '@semianalysisai/inferencex-db/connection'; +import { FIXTURES_MODE, getCollectiveXWriteDb } from '@semianalysisai/inferencex-db/connection'; import { collectiveXDatasetFromRow, getLatestCollectiveXRun, @@ -34,7 +34,8 @@ export async function GET(request: NextRequest) { } try { - const row = await getLatestCollectiveXRun(getCollectiveXDb(), version); + // Discovery may have persisted a run, so read from the same primary. + const row = await getLatestCollectiveXRun(getCollectiveXWriteDb(), version); if (row !== null) { if (ensureError) console.error('CollectiveX discovery failed; serving stored run:', ensureError); diff --git a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.fixtures.test.ts b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.fixtures.test.ts index 4c2b1b5ce..cd03846ed 100644 --- a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.fixtures.test.ts +++ b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.fixtures.test.ts @@ -12,7 +12,6 @@ const { mockLoadFixture } = vi.hoisted(() => ({ vi.mock('@semianalysisai/inferencex-db/connection', () => ({ FIXTURES_MODE: true, - getCollectiveXDb: vi.fn(), getCollectiveXWriteDb: vi.fn(), })); diff --git a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.test.ts b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.test.ts index c2f2c51ad..4ceed09db 100644 --- a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.test.ts +++ b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.test.ts @@ -2,19 +2,17 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; import { makeCollectiveXDataset } from '@semianalysisai/inferencex-db/collectivex/test-fixture'; -const { mockGetRun, mockDelete, mockFromRow, mockGetDb, mockGetWriteDb, mockPurge, mockEnsureRun } = +const { mockGetRun, mockDelete, mockFromRow, mockGetWriteDb, mockPurge, mockEnsureRun } = vi.hoisted(() => ({ mockGetRun: vi.fn(), mockDelete: vi.fn(), mockFromRow: vi.fn(), - mockGetDb: vi.fn(() => 'mock-sql'), mockGetWriteDb: vi.fn(() => 'mock-write-sql'), mockPurge: vi.fn(() => Promise.resolve(0)), mockEnsureRun: vi.fn(), })); vi.mock('@semianalysisai/inferencex-db/connection', () => ({ - getCollectiveXDb: mockGetDb, getCollectiveXWriteDb: mockGetWriteDb, FIXTURES_MODE: false, })); @@ -99,7 +97,7 @@ describe('GET /api/v1/collectivex/runs/[runId]', () => { expect(res.status).toBe(200); expect(await res.json()).toEqual(dataset); expect(mockEnsureRun).toHaveBeenCalledWith(1, runId); - expect(mockGetRun).toHaveBeenCalledWith('mock-sql', 1, runId); + expect(mockGetRun).toHaveBeenCalledWith('mock-write-sql', 1, runId); }); it.each([ diff --git a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.ts b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.ts index e8150c1fb..43883cbd5 100644 --- a/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.ts +++ b/packages/app/src/app/api/v1/collectivex/runs/[runId]/route.ts @@ -1,5 +1,3 @@ -import { timingSafeEqual } from 'crypto'; - import { type NextRequest, NextResponse } from 'next/server'; import { @@ -7,11 +5,7 @@ import { type CollectiveXRunSummary, parseCollectiveXVersion, } from '@semianalysisai/inferencex-db/collectivex/types'; -import { - FIXTURES_MODE, - getCollectiveXDb, - getCollectiveXWriteDb, -} from '@semianalysisai/inferencex-db/connection'; +import { FIXTURES_MODE, getCollectiveXWriteDb } from '@semianalysisai/inferencex-db/connection'; import { collectiveXDatasetFromRow, deleteCollectiveXRun, @@ -24,6 +18,7 @@ import { cachedJson, purgeCollectiveX, } from '@/lib/api-cache'; +import { bearerMatches } from '@/lib/bearer-auth'; import { collectiveXSweepErrorStatus, ensureCollectiveXRun } from '@/lib/collectivex-lazy-ingest'; import { loadFixture } from '@/lib/test-fixtures'; @@ -58,7 +53,8 @@ export async function GET(request: NextRequest, context: { params: Promise<{ run } try { - const row = await getCollectiveXRun(getCollectiveXDb(), version, runId); + // Ensure may have inserted or refreshed this run, so read from the same primary. + const row = await getCollectiveXRun(getCollectiveXWriteDb(), version, runId); if (row === null) { if (ensureError) { const status = collectiveXSweepErrorStatus(ensureError); @@ -87,15 +83,6 @@ export async function GET(request: NextRequest, context: { params: Promise<{ run } } -/** Constant-time Bearer check that tolerates multibyte header bytes. */ -function bearerMatches(header: string, secret: string): boolean { - const provided = Buffer.from(header); - const expected = Buffer.from(`Bearer ${secret}`); - // Compare BYTE lengths — a multibyte char can make the JS string lengths - // equal while the buffers differ, and timingSafeEqual throws on that. - return provided.length === expected.length && timingSafeEqual(provided, expected); -} - /** * Admin deletion of an ingested run. Authenticated with a dedicated Bearer * secret — the token is remembered in browser localStorage, so it must not diff --git a/packages/app/src/app/api/v1/collectivex/runs/route.test.ts b/packages/app/src/app/api/v1/collectivex/runs/route.test.ts index 658d7b178..b4230fcaf 100644 --- a/packages/app/src/app/api/v1/collectivex/runs/route.test.ts +++ b/packages/app/src/app/api/v1/collectivex/runs/route.test.ts @@ -3,15 +3,15 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { buildRunSummary } from '@semianalysisai/inferencex-db/collectivex/reader'; import { makeCollectiveXDataset } from '@semianalysisai/inferencex-db/collectivex/test-fixture'; -const { mockList, mockGetDb, mockEnsureList, mockCachedJson } = vi.hoisted(() => ({ +const { mockList, mockGetWriteDb, mockEnsureList, mockCachedJson } = vi.hoisted(() => ({ mockList: vi.fn(), - mockGetDb: vi.fn(() => 'mock-sql'), + mockGetWriteDb: vi.fn(() => 'mock-write-sql'), mockEnsureList: vi.fn(), mockCachedJson: vi.fn((data: unknown) => Response.json(data)), })); vi.mock('@semianalysisai/inferencex-db/connection', () => ({ - getCollectiveXDb: mockGetDb, + getCollectiveXWriteDb: mockGetWriteDb, FIXTURES_MODE: false, })); @@ -76,7 +76,7 @@ describe('GET /api/v1/collectivex/runs', () => { discovery_complete: true, }); expect(mockEnsureList).toHaveBeenCalledWith(1); - expect(mockList).toHaveBeenCalledWith('mock-sql', 1); + expect(mockList).toHaveBeenCalledWith('mock-write-sql', 1); }); it('marks the list incomplete when another bounded discovery pass is needed', async () => { diff --git a/packages/app/src/app/api/v1/collectivex/runs/route.ts b/packages/app/src/app/api/v1/collectivex/runs/route.ts index fce3f2a2d..62b2ca22d 100644 --- a/packages/app/src/app/api/v1/collectivex/runs/route.ts +++ b/packages/app/src/app/api/v1/collectivex/runs/route.ts @@ -1,7 +1,7 @@ import { type NextRequest, NextResponse } from 'next/server'; import { parseCollectiveXVersion } from '@semianalysisai/inferencex-db/collectivex/types'; -import { FIXTURES_MODE, getCollectiveXDb } from '@semianalysisai/inferencex-db/connection'; +import { FIXTURES_MODE, getCollectiveXWriteDb } from '@semianalysisai/inferencex-db/connection'; import { listCollectiveXRuns } from '@semianalysisai/inferencex-db/queries/collectivex'; import { COLLECTIVEX_CACHE_CONTROL, cachedJson, collectiveXCacheTag } from '@/lib/api-cache'; @@ -35,7 +35,8 @@ export async function GET(request: NextRequest) { } try { - const runs = await listCollectiveXRuns(getCollectiveXDb(), version); + // Backfill may have persisted runs, so read from the same primary. + const runs = await listCollectiveXRuns(getCollectiveXWriteDb(), version); if (runs.length === 0 && ensureError) { console.error('CollectiveX run backfill failed with no stored fallback:', ensureError); const status = collectiveXSweepErrorStatus(ensureError) ?? 502; diff --git a/packages/app/src/app/api/v1/datasets/route.ts b/packages/app/src/app/api/v1/datasets/route.ts index 242a2b086..ec0a3160e 100644 --- a/packages/app/src/app/api/v1/datasets/route.ts +++ b/packages/app/src/app/api/v1/datasets/route.ts @@ -1,25 +1,14 @@ -import { NextResponse } from 'next/server'; - import { getDb } from '@semianalysisai/inferencex-db/connection'; import { listDatasets, type DatasetRecord } from '@semianalysisai/inferencex-db/queries/datasets'; -import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { cachedReadRoute } from '@/lib/cached-read-route'; export const dynamic = 'force-dynamic'; -const getCachedDatasets = cachedQuery( - (): Promise => listDatasets(getDb()), - 'datasets', -); - /** GET /api/v1/datasets — all ingested cc-traces-weka datasets (registry cards). */ -export async function GET() { - try { - const data = await getCachedDatasets(); - return cachedJson(data); - } catch (error) { - console.error('Error fetching datasets:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); - } -} +export const GET = cachedReadRoute({ + cacheKey: 'datasets', + fetch: () => listDatasets(getDb()), + logLabel: 'datasets', +}); diff --git a/packages/app/src/app/api/v1/eval-samples-live/route.ts b/packages/app/src/app/api/v1/eval-samples-live/route.ts index a98775dbf..e985f553e 100644 --- a/packages/app/src/app/api/v1/eval-samples-live/route.ts +++ b/packages/app/src/app/api/v1/eval-samples-live/route.ts @@ -6,6 +6,7 @@ import { type NextRequest, NextResponse } from 'next/server'; import { extractDemonstrations } from '@/lib/eval-sample-utils'; +import { parseEvalSampleWindow } from '@/lib/eval-sample-params'; import { type EvalArtifactConfig, fetchAndParseSamples, @@ -15,10 +16,6 @@ import { fetchGithubRunArtifacts, getGithubToken } from '@/lib/github-artifacts' export const dynamic = 'force-dynamic'; -const ALLOWED_FILTERS = new Set(['all', 'passed', 'failed']); -const DEFAULT_LIMIT = 200; -const MAX_LIMIT = 500; - /** * GET /api/v1/eval-samples-live * ?run_id= @@ -42,10 +39,6 @@ export async function GET(request: NextRequest) { const params = request.nextUrl.searchParams; const runId = params.get('run_id') ?? ''; const task = params.get('task') ?? ''; - const filterParam = params.get('filter') ?? 'all'; - const offset = Math.max(0, Math.trunc(Number(params.get('offset') ?? '0'))); - const requestedLimit = Math.trunc(Number(params.get('limit') ?? String(DEFAULT_LIMIT))); - const limit = Math.min(MAX_LIMIT, Math.max(1, requestedLimit || DEFAULT_LIMIT)); if (!/^\d+$/u.test(runId)) { return NextResponse.json({ error: 'run_id must be a positive integer' }, { status: 400 }); @@ -53,13 +46,11 @@ export async function GET(request: NextRequest) { if (!task) { return NextResponse.json({ error: 'task is required' }, { status: 400 }); } - if (!ALLOWED_FILTERS.has(filterParam)) { - return NextResponse.json( - { error: `filter must be one of: ${[...ALLOWED_FILTERS].join(', ')}` }, - { status: 400 }, - ); + const window = parseEvalSampleWindow(params); + if ('error' in window) { + return NextResponse.json(window, { status: 400 }); } - const filter = filterParam as 'all' | 'passed' | 'failed'; + const { filter, offset, limit } = window; const config: EvalArtifactConfig = { model: params.get('model') ?? '', diff --git a/packages/app/src/app/api/v1/eval-samples/route.ts b/packages/app/src/app/api/v1/eval-samples/route.ts index 897eb5d2a..93d5f8670 100644 --- a/packages/app/src/app/api/v1/eval-samples/route.ts +++ b/packages/app/src/app/api/v1/eval-samples/route.ts @@ -8,13 +8,10 @@ import { import { cachedJson, cachedQuery } from '@/lib/api-cache'; import { extractDemonstrations } from '@/lib/eval-sample-utils'; +import { parseEvalSampleWindow } from '@/lib/eval-sample-params'; export const dynamic = 'force-dynamic'; -const ALLOWED_FILTERS = new Set(['all', 'passed', 'failed']); -const DEFAULT_LIMIT = 200; -const MAX_LIMIT = 500; - const getCachedEvalSamples = cachedQuery( (evalResultId: number, filter: 'all' | 'passed' | 'failed', offset: number, limit: number) => getEvalSamples(getDb(), evalResultId, filter, offset, limit), @@ -38,12 +35,8 @@ const getCachedEvalSampleOffset = cachedQuery( export async function GET(request: NextRequest) { const params = request.nextUrl.searchParams; const evalResultId = Number(params.get('eval_result_id')); - const filterParam = params.get('filter') ?? 'all'; - const offset = Math.max(0, Math.trunc(Number(params.get('offset') ?? '0'))); const docIdParam = params.get('doc_id'); const docId = docIdParam === null ? null : Math.trunc(Number(docIdParam)); - const requestedLimit = Math.trunc(Number(params.get('limit') ?? String(DEFAULT_LIMIT))); - const limit = Math.min(MAX_LIMIT, Math.max(1, requestedLimit || DEFAULT_LIMIT)); if (!evalResultId || !Number.isFinite(evalResultId) || evalResultId <= 0) { return NextResponse.json( @@ -54,13 +47,11 @@ export async function GET(request: NextRequest) { if (docId !== null && (!Number.isFinite(docId) || docId < 0)) { return NextResponse.json({ error: 'doc_id must be a non-negative integer' }, { status: 400 }); } - if (!ALLOWED_FILTERS.has(filterParam)) { - return NextResponse.json( - { error: `filter must be one of: ${[...ALLOWED_FILTERS].join(', ')}` }, - { status: 400 }, - ); + const window = parseEvalSampleWindow(params); + if ('error' in window) { + return NextResponse.json(window, { status: 400 }); } - const filter = filterParam as 'all' | 'passed' | 'failed'; + const { filter, offset, limit } = window; try { const sampleOffset = diff --git a/packages/app/src/app/api/v1/evaluations/route.ts b/packages/app/src/app/api/v1/evaluations/route.ts index 9fafff4b0..23477a4a4 100644 --- a/packages/app/src/app/api/v1/evaluations/route.ts +++ b/packages/app/src/app/api/v1/evaluations/route.ts @@ -1,23 +1,15 @@ -import { NextResponse } from 'next/server'; - -import { FIXTURES_MODE, getDb } from '@semianalysisai/inferencex-db/connection'; +import { getDb } from '@semianalysisai/inferencex-db/connection'; import { getAllEvalResults } from '@semianalysisai/inferencex-db/queries/evaluations'; -import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { cachedReadRoute } from '@/lib/cached-read-route'; import { loadFixture } from '@/lib/test-fixtures'; export const dynamic = 'force-dynamic'; -const getCachedEvaluations = cachedQuery(() => getAllEvalResults(getDb()), 'evaluations'); - -export async function GET() { - if (FIXTURES_MODE) return cachedJson(loadFixture('evaluations')); - try { - const rows = await getCachedEvaluations(); - return cachedJson(rows); - } catch (error) { - console.error('Error fetching evaluations:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); - } -} +export const GET = cachedReadRoute({ + cacheKey: 'evaluations', + fetch: () => getAllEvalResults(getDb()), + fixture: () => loadFixture('evaluations'), + logLabel: 'evaluations', +}); diff --git a/packages/app/src/app/api/v1/id-routes.ts b/packages/app/src/app/api/v1/id-routes.ts index fea9221bd..920e6ad4b 100644 --- a/packages/app/src/app/api/v1/id-routes.ts +++ b/packages/app/src/app/api/v1/id-routes.ts @@ -1,6 +1,7 @@ import { type NextRequest, NextResponse } from 'next/server'; import { cachedJson } from '@/lib/api-cache'; +import { PUBLIC_API_ERRORS, publicApiError, tooManyIdsError } from '@/lib/public-api-errors'; /** * Shared GET-handler factories for the agentic benchmark routes, which all @@ -22,7 +23,7 @@ import { cachedJson } from '@/lib/api-cache'; export function parseIdsParam(request: NextRequest, maxIds: number): number[] | NextResponse { const raw = request.nextUrl.searchParams.get('ids'); if (!raw) { - return NextResponse.json({ error: 'ids query param is required' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.idsRequired, 400); } const ids = [ @@ -34,10 +35,10 @@ export function parseIdsParam(request: NextRequest, maxIds: number): number[] | ), ]; if (ids.length === 0) { - return NextResponse.json({ error: 'no valid ids provided' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.noValidIds, 400); } if (ids.length > maxIds) { - return NextResponse.json({ error: `too many ids (max ${maxIds})` }, { status: 400 }); + return publicApiError(tooManyIdsError(maxIds), 400); } return ids.toSorted((a, b) => a - b); } @@ -57,7 +58,7 @@ export function idsQueryRoute(options: { return cachedJson(await fetch(ids)); } catch (error) { console.error(`Error fetching ${logLabel}:`, error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } }; } @@ -71,15 +72,15 @@ export function idQueryRoute(options: { return async (request: NextRequest) => { const id = Number(request.nextUrl.searchParams.get('id')); if (!id || !Number.isFinite(id)) { - return NextResponse.json({ error: 'id is required (benchmark_result_id)' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.idRequired, 400); } try { const data = await fetch(id); - if (!data) return NextResponse.json({ error: 'Not found' }, { status: 404 }); + if (!data) return publicApiError(PUBLIC_API_ERRORS.notFound, 404); return cachedJson(data); } catch (error) { console.error(`Error fetching ${logLabel}:`, error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } }; } diff --git a/packages/app/src/app/api/v1/invalidate/route.ts b/packages/app/src/app/api/v1/invalidate/route.ts index e648e75b1..129b3926e 100644 --- a/packages/app/src/app/api/v1/invalidate/route.ts +++ b/packages/app/src/app/api/v1/invalidate/route.ts @@ -1,14 +1,12 @@ -import { timingSafeEqual } from 'crypto'; - import { NextResponse } from 'next/server'; +import { bearerMatches } from '@/lib/bearer-auth'; + import { COLLECTIVEX_CACHE_SCOPE, purgeAll, purgeCollectiveX } from '@/lib/api-cache'; export async function POST(request: Request) { const secret = process.env.INVALIDATE_SECRET; const authHeader = request.headers.get('Authorization') ?? ''; - const provided = Buffer.from(authHeader); - const expected = Buffer.from(`Bearer ${secret}`); // The shared staging deployment is already protected by Vercel. Its CI // caller must present the project-scoped automation bypass before Vercel // forwards this header to the route, so a second app secret is redundant. @@ -19,12 +17,7 @@ export async function POST(request: Request) { process.env.VERCEL_GIT_COMMIT_REF === 'staging' && Boolean(request.headers.get('x-vercel-protection-bypass')); - // Compare BYTE lengths — a multibyte char can make the JS string lengths - // equal while the buffers differ, and timingSafeEqual throws on that. - if ( - !isProtectedStagingRequest && - (!secret || provided.length !== expected.length || !timingSafeEqual(provided, expected)) - ) { + if (!isProtectedStagingRequest && (!secret || !bearerMatches(authHeader, secret))) { return new NextResponse('Unauthorized', { status: 401 }); } diff --git a/packages/app/src/app/api/v1/latest-images/route.ts b/packages/app/src/app/api/v1/latest-images/route.ts index 001f01ea5..eaa15ff79 100644 --- a/packages/app/src/app/api/v1/latest-images/route.ts +++ b/packages/app/src/app/api/v1/latest-images/route.ts @@ -1,22 +1,13 @@ -import { NextResponse } from 'next/server'; - import { getDb } from '@semianalysisai/inferencex-db/connection'; import { getLatestImages } from '@semianalysisai/inferencex-db/queries/latest-images'; -import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { cachedReadRoute } from '@/lib/cached-read-route'; export const dynamic = 'force-dynamic'; -const getCachedLatestImages = cachedQuery(() => getLatestImages(getDb()), 'latest-images-v2', { +export const GET = cachedReadRoute({ + cacheKey: 'latest-images-v2', + fetch: () => getLatestImages(getDb()), + logLabel: 'latest images', blobOnly: true, }); - -export async function GET() { - try { - const rows = await getCachedLatestImages(); - return cachedJson(rows); - } catch (error) { - console.error('Error fetching latest images:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); - } -} diff --git a/packages/app/src/app/api/v1/log-availability/route.test.ts b/packages/app/src/app/api/v1/log-availability/route.test.ts new file mode 100644 index 000000000..ffc9e8ead --- /dev/null +++ b/packages/app/src/app/api/v1/log-availability/route.test.ts @@ -0,0 +1,38 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +const { mockGetServerLogAvailability, mockGetDb } = vi.hoisted(() => ({ + mockGetServerLogAvailability: vi.fn(), + mockGetDb: vi.fn(() => 'mock-sql'), +})); + +vi.mock('@semianalysisai/inferencex-db/connection', () => ({ getDb: mockGetDb })); +vi.mock('@semianalysisai/inferencex-db/queries/server-logs', () => ({ + getServerLogAvailability: mockGetServerLogAvailability, +})); +vi.mock('@/lib/api-cache', () => ({ + cachedQuery: (fn: (...args: any[]) => any) => fn, + cachedJson: (data: unknown) => Response.json(data), +})); + +import { NextRequest } from 'next/server'; +import { GET } from './route'; + +const req = (url: string) => new NextRequest(new URL(url, 'http://localhost')); + +beforeEach(() => vi.clearAllMocks()); + +describe('GET /api/v1/log-availability', () => { + it('returns a presence map for valid, deduplicated ids', async () => { + mockGetServerLogAvailability.mockResolvedValueOnce({ 4: true }); + const response = await GET(req('/api/v1/log-availability?ids=9,4,4')); + expect(response.status).toBe(200); + await expect(response.json()).resolves.toEqual({ 4: true }); + expect(mockGetServerLogAvailability).toHaveBeenCalledWith('mock-sql', [4, 9]); + }); + + it('returns 400 when no valid ids are provided', async () => { + const response = await GET(req('/api/v1/log-availability?ids=bad,0')); + expect(response.status).toBe(400); + expect(mockGetServerLogAvailability).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/app/src/app/api/v1/log-availability/route.ts b/packages/app/src/app/api/v1/log-availability/route.ts new file mode 100644 index 000000000..77e7c0a48 --- /dev/null +++ b/packages/app/src/app/api/v1/log-availability/route.ts @@ -0,0 +1,28 @@ +import { getDb } from '@semianalysisai/inferencex-db/connection'; +import { + getServerLogAvailability, + type ServerLogAvailabilityMap, +} from '@semianalysisai/inferencex-db/queries/server-logs'; + +import { cachedQuery } from '@/lib/api-cache'; + +import { idsQueryRoute } from '../id-routes'; + +export const dynamic = 'force-dynamic'; + +const getCachedServerLogAvailability = cachedQuery( + (ids: number[]): Promise => getServerLogAvailability(getDb(), ids), + 'server-log-availability', +); + +/** + * GET /api/v1/log-availability?ids=1,2,3 + * + * Returns `{[id]: true}` for benchmark points with a linked server log. The + * chart uses this lightweight check before offering its "View logs" action. + */ +export const GET = idsQueryRoute({ + maxIds: 500, + logLabel: 'server log availability', + fetch: getCachedServerLogAvailability, +}); diff --git a/packages/app/src/app/api/v1/overview/route.test.ts b/packages/app/src/app/api/v1/overview/route.test.ts index 2a0829d2b..e68107f73 100644 --- a/packages/app/src/app/api/v1/overview/route.test.ts +++ b/packages/app/src/app/api/v1/overview/route.test.ts @@ -37,12 +37,22 @@ describe('GET /api/v1/overview', () => { mockGetOverviewPageData.mockResolvedValueOnce(data); const response = await GET( - request('/api/v1/overview?tier=75&engine=all&compare=30d&ref=b300&models=all'), + request( + '/api/v1/overview?tier=75&engine=all&compare=30d&ref=b300&models=all&rows=changed&hwrows=priced', + ), ); expect(response.status).toBe(200); expect(await response.json()).toEqual(data); - expect(mockGetOverviewPageData).toHaveBeenCalledWith(75, 'all', '30d', 'b300', 'all'); + expect(mockGetOverviewPageData).toHaveBeenCalledWith( + 75, + 'all', + '30d', + 'b300', + 'all', + 'changed', + 'priced', + ); expect(mockCachedJson).toHaveBeenCalledWith(data); }); @@ -50,7 +60,9 @@ describe('GET /api/v1/overview', () => { mockGetOverviewPageData.mockResolvedValueOnce({ models: [] }); await GET( - request('/api/v1/overview?tier=999&engine=vendor&compare=weekly&ref=h100&models=inactive'), + request( + '/api/v1/overview?tier=999&engine=vendor&compare=weekly&ref=h100&models=inactive&rows=some&hwrows=blank', + ), ); expect(mockGetOverviewPageData).toHaveBeenCalledWith( @@ -59,6 +71,8 @@ describe('GET /api/v1/overview', () => { 'hardware', 'b200', 'default', + 'all', + 'all', ); }); diff --git a/packages/app/src/app/api/v1/overview/route.ts b/packages/app/src/app/api/v1/overview/route.ts index ba6819160..3842b87c7 100644 --- a/packages/app/src/app/api/v1/overview/route.ts +++ b/packages/app/src/app/api/v1/overview/route.ts @@ -4,8 +4,10 @@ import { cachedJson } from '@/lib/api-cache'; import { resolveOverviewComparisonMode, resolveOverviewEngineScope, + resolveOverviewHardwareRowScope, resolveOverviewModelScope, resolveOverviewReferenceHardware, + resolveOverviewRowScope, resolveOverviewTier, } from '@/lib/overview-data'; import { getOverviewPageData } from '@/lib/overview-data.server'; @@ -22,6 +24,8 @@ export async function GET(request: NextRequest) { resolveOverviewComparisonMode(params.get('compare') ?? undefined), resolveOverviewReferenceHardware(params.get('ref') ?? undefined), resolveOverviewModelScope(params.get('models') ?? undefined), + resolveOverviewRowScope(params.get('rows') ?? undefined), + resolveOverviewHardwareRowScope(params.get('hwrows') ?? undefined), ); return cachedJson(data); } catch (error) { diff --git a/packages/app/src/app/api/v1/reliability/route.ts b/packages/app/src/app/api/v1/reliability/route.ts index eb3641433..faa0ea6c1 100644 --- a/packages/app/src/app/api/v1/reliability/route.ts +++ b/packages/app/src/app/api/v1/reliability/route.ts @@ -1,23 +1,15 @@ -import { NextResponse } from 'next/server'; - -import { FIXTURES_MODE, getDb } from '@semianalysisai/inferencex-db/connection'; +import { getDb } from '@semianalysisai/inferencex-db/connection'; import { getReliabilityStats } from '@semianalysisai/inferencex-db/queries/reliability'; -import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { cachedReadRoute } from '@/lib/cached-read-route'; import { loadFixture } from '@/lib/test-fixtures'; export const dynamic = 'force-dynamic'; -const getCachedReliability = cachedQuery(() => getReliabilityStats(getDb()), 'reliability'); - -export async function GET() { - if (FIXTURES_MODE) return cachedJson(loadFixture('reliability')); - try { - const rows = await getCachedReliability(); - return cachedJson(rows); - } catch (error) { - console.error('Error fetching reliability stats:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); - } -} +export const GET = cachedReadRoute({ + cacheKey: 'reliability', + fetch: () => getReliabilityStats(getDb()), + fixture: () => loadFixture('reliability'), + logLabel: 'reliability stats', +}); diff --git a/packages/app/src/app/api/v1/request-chart-data/route.ts b/packages/app/src/app/api/v1/request-chart-data/route.ts new file mode 100644 index 000000000..ddba93496 --- /dev/null +++ b/packages/app/src/app/api/v1/request-chart-data/route.ts @@ -0,0 +1,33 @@ +import { getDb } from '@semianalysisai/inferencex-db/connection'; + +import { + getRequestChartData, + REQUEST_CHART_DATA_VERSION, + type RequestChartDataWire, +} from '@semianalysisai/inferencex-db/queries/request-chart-data'; + +import { cachedQuery } from '@/lib/api-cache'; + +import { idQueryRoute } from '../id-routes'; + +export const dynamic = 'force-dynamic'; + +/** Version-derived blob-cache key namespace (exported for the key-derivation test). */ +export const CACHE_KEY_PREFIX = `request-chart-data-v${REQUEST_CHART_DATA_VERSION}`; + +const getCachedRequestChartData = cachedQuery( + (id: number): Promise => getRequestChartData(getDb(), id), + CACHE_KEY_PREFIX, + { blobOnly: true }, +); + +/** + * GET /api/v1/request-chart-data?id=N + * + * Returns the compact request fields used by the default point charts. The + * full request timeline remains a separate, timeline-view-only download. + */ +export const GET = idQueryRoute({ + logLabel: 'request chart data', + fetch: getCachedRequestChartData, +}); diff --git a/packages/app/src/app/api/v1/resident-sequence-lengths/route.ts b/packages/app/src/app/api/v1/resident-sequence-lengths/route.ts new file mode 100644 index 000000000..794a53357 --- /dev/null +++ b/packages/app/src/app/api/v1/resident-sequence-lengths/route.ts @@ -0,0 +1,32 @@ +import { getDb } from '@semianalysisai/inferencex-db/connection'; +import { STATS_VERSION } from '@semianalysisai/inferencex-db/queries/agentic-shared'; +import { + getResidentSequenceLengthSketches, + type ResidentSequenceLengthSketches, +} from '@semianalysisai/inferencex-db/queries/resident-sequence-lengths'; + +import { cachedQuery } from '@/lib/api-cache'; + +import { idsQueryRoute } from '../id-routes'; + +export const dynamic = 'force-dynamic'; + +export const CACHE_KEY_PREFIX = `resident-sequence-lengths-v${STATS_VERSION}`; + +const getCachedResidentSequenceLengths = cachedQuery( + (ids: number[]): Promise => + getResidentSequenceLengthSketches(getDb(), ids), + CACHE_KEY_PREFIX, +); + +/** + * GET /api/v1/resident-sequence-lengths?ids=1,2,3 + * + * Returns one mergeable ISL/OSL sketch for the requested resident chart + * points. The client chunks large charts and merges these bounded payloads. + */ +export const GET = idsQueryRoute({ + maxIds: 200, + logLabel: 'resident sequence lengths', + fetch: getCachedResidentSequenceLengths, +}); diff --git a/packages/app/src/app/api/v1/server-log-files/route.test.ts b/packages/app/src/app/api/v1/server-log-files/route.test.ts new file mode 100644 index 000000000..36707c18f --- /dev/null +++ b/packages/app/src/app/api/v1/server-log-files/route.test.ts @@ -0,0 +1,50 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +const { mockGetServerLogFileNames, mockGetDb } = vi.hoisted(() => ({ + mockGetServerLogFileNames: vi.fn(), + mockGetDb: vi.fn(() => 'mock-sql'), +})); + +vi.mock('@semianalysisai/inferencex-db/connection', () => ({ + getDb: mockGetDb, + FIXTURES_MODE: false, +})); + +vi.mock('@semianalysisai/inferencex-db/queries/server-logs', () => ({ + getServerLogFileNames: mockGetServerLogFileNames, +})); + +vi.mock('@/lib/api-cache', () => ({ + cachedQuery: (fn: (...args: any[]) => any) => fn, + cachedJson: (data: unknown) => Response.json(data), +})); + +import { NextRequest } from 'next/server'; +import { GET } from './route'; + +const req = (url: string) => new NextRequest(new URL(url, 'http://localhost')); + +beforeEach(() => vi.clearAllMocks()); + +describe('GET /api/v1/server-log-files', () => { + it('returns artifact-relative filenames in database order', async () => { + mockGetServerLogFileNames.mockResolvedValueOnce([ + 'results/server.log', + 'results/benchmark.log', + 'results/router.log', + ]); + const response = await GET(req('/api/v1/server-log-files?id=439516')); + expect(response.status).toBe(200); + await expect(response.json()).resolves.toEqual([ + 'results/server.log', + 'results/benchmark.log', + 'results/router.log', + ]); + }); + + it('returns 404 when no log bundle is linked', async () => { + mockGetServerLogFileNames.mockResolvedValueOnce(null); + const response = await GET(req('/api/v1/server-log-files?id=9')); + expect(response.status).toBe(404); + }); +}); diff --git a/packages/app/src/app/api/v1/server-log-files/route.ts b/packages/app/src/app/api/v1/server-log-files/route.ts new file mode 100644 index 000000000..308ace8d9 --- /dev/null +++ b/packages/app/src/app/api/v1/server-log-files/route.ts @@ -0,0 +1,19 @@ +import { getDb } from '@semianalysisai/inferencex-db/connection'; +import { getServerLogFileNames } from '@semianalysisai/inferencex-db/queries/server-logs'; + +import { cachedQuery } from '@/lib/api-cache'; + +import { idQueryRoute } from '../id-routes'; + +export const dynamic = 'force-dynamic'; + +const getCachedServerLogFileNames = cachedQuery( + (id: number) => getServerLogFileNames(getDb(), id), + 'server-log-files', +); + +/** GET /api/v1/server-log-files?id=1 — artifact-relative .log/.out filenames. */ +export const GET = idQueryRoute({ + logLabel: 'server log filenames', + fetch: getCachedServerLogFileNames, +}); diff --git a/packages/app/src/app/api/v1/server-log-search/route.test.ts b/packages/app/src/app/api/v1/server-log-search/route.test.ts new file mode 100644 index 000000000..2f78ffc79 --- /dev/null +++ b/packages/app/src/app/api/v1/server-log-search/route.test.ts @@ -0,0 +1,81 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +const { mockSearchServerLogs, mockGetDb } = vi.hoisted(() => ({ + mockSearchServerLogs: vi.fn(), + mockGetDb: vi.fn(() => 'mock-sql'), +})); + +vi.mock('@semianalysisai/inferencex-db/connection', () => ({ + getDb: mockGetDb, + FIXTURES_MODE: false, +})); + +vi.mock('@semianalysisai/inferencex-db/queries/server-logs', () => ({ + searchServerLogs: mockSearchServerLogs, +})); + +vi.mock('@/lib/api-cache', () => ({ + cachedJson: (data: unknown) => Response.json(data), +})); + +import { NextRequest } from 'next/server'; +import { GET } from './route'; + +function req(url: string): NextRequest { + return new NextRequest(new URL(url, 'http://localhost')); +} + +beforeEach(() => vi.clearAllMocks()); + +describe('GET /api/v1/server-log-search', () => { + it('searches complete stored logs with a bounded result count', async () => { + mockSearchServerLogs.mockResolvedValueOnce({ + matches: [ + { + fileName: 'results/router.log', + offset: 12, + before: 'router ', + match: 'ready', + after: '\n', + }, + ], + truncated: false, + }); + + const response = await GET(req('/api/v1/server-log-search?id=42&q=ready&limit=20')); + expect(response.status).toBe(200); + await expect(response.json()).resolves.toEqual({ + id: 42, + query: 'ready', + matches: [ + { + fileName: 'results/router.log', + offset: 12, + before: 'router ', + match: 'ready', + after: '\n', + }, + ], + truncated: false, + }); + expect(mockSearchServerLogs).toHaveBeenCalledWith('mock-sql', 42, 'ready', 20); + }); + + it.each([ + '/api/v1/server-log-search?q=ready', + '/api/v1/server-log-search?id=42&q=', + `/api/v1/server-log-search?id=42&q=${'x'.repeat(257)}`, + '/api/v1/server-log-search?id=42&q=ready&limit=0', + '/api/v1/server-log-search?id=42&q=ready&limit=101', + ])('rejects invalid search request %s', async (url) => { + const response = await GET(req(url)); + expect(response.status).toBe(400); + expect(mockSearchServerLogs).not.toHaveBeenCalled(); + }); + + it('returns 500 when the database search fails', async () => { + mockSearchServerLogs.mockRejectedValueOnce(new Error('Connection reset')); + const response = await GET(req('/api/v1/server-log-search?id=42&q=ready')); + expect(response.status).toBe(500); + }); +}); diff --git a/packages/app/src/app/api/v1/server-log-search/route.ts b/packages/app/src/app/api/v1/server-log-search/route.ts new file mode 100644 index 000000000..6a03a3728 --- /dev/null +++ b/packages/app/src/app/api/v1/server-log-search/route.ts @@ -0,0 +1,46 @@ +import { type NextRequest, NextResponse } from 'next/server'; + +import { getDb } from '@semianalysisai/inferencex-db/connection'; +import { searchServerLogs } from '@semianalysisai/inferencex-db/queries/server-logs'; + +import { cachedJson } from '@/lib/api-cache'; + +export const dynamic = 'force-dynamic'; + +const DEFAULT_RESULT_LIMIT = 50; +const MAX_RESULT_LIMIT = 100; +const MAX_SEARCH_LENGTH = 256; + +function parseLimit(value: string | null): number | null { + if (value === null) return DEFAULT_RESULT_LIMIT; + if (!/^\d+$/u.test(value)) return null; + const parsed = Number(value); + return Number.isSafeInteger(parsed) && parsed >= 1 && parsed <= MAX_RESULT_LIMIT ? parsed : null; +} + +export async function GET(request: NextRequest) { + const id = Number(request.nextUrl.searchParams.get('id')); + if (!id || !Number.isFinite(id)) { + return NextResponse.json({ error: 'id is required (benchmark_result_id)' }, { status: 400 }); + } + + const query = request.nextUrl.searchParams.get('q')?.trim() ?? ''; + if (query.length === 0 || query.length > MAX_SEARCH_LENGTH || query.includes('\u0000')) { + return NextResponse.json( + { error: `q must contain 1-${MAX_SEARCH_LENGTH} characters` }, + { status: 400 }, + ); + } + + const limit = parseLimit(request.nextUrl.searchParams.get('limit')); + if (limit === null) { + return NextResponse.json({ error: `limit must be 1-${MAX_RESULT_LIMIT}` }, { status: 400 }); + } + + try { + return cachedJson({ id, query, ...(await searchServerLogs(getDb(), id, query, limit)) }); + } catch (error) { + console.error('Error searching server logs:', error); + return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + } +} diff --git a/packages/app/src/app/api/v1/server-log/route.test.ts b/packages/app/src/app/api/v1/server-log/route.test.ts index e8159c846..dbd4b3aa0 100644 --- a/packages/app/src/app/api/v1/server-log/route.test.ts +++ b/packages/app/src/app/api/v1/server-log/route.test.ts @@ -1,7 +1,8 @@ import { describe, expect, it, vi, beforeEach } from 'vitest'; -const { mockGetServerLog, mockGetDb } = vi.hoisted(() => ({ +const { mockGetServerLog, mockGetServerLogChunk, mockGetDb } = vi.hoisted(() => ({ mockGetServerLog: vi.fn(), + mockGetServerLogChunk: vi.fn(), mockGetDb: vi.fn(() => 'mock-sql'), })); @@ -12,6 +13,7 @@ vi.mock('@semianalysisai/inferencex-db/connection', () => ({ vi.mock('@semianalysisai/inferencex-db/queries/server-logs', () => ({ getServerLog: mockGetServerLog, + getServerLogChunk: mockGetServerLogChunk, })); vi.mock('@/lib/api-cache', () => ({ @@ -72,6 +74,137 @@ describe('GET /api/v1/server-log', () => { expect(mockGetServerLog).toHaveBeenCalledWith('mock-sql', 42); }); + it('returns a bounded chunk when offset or limit is provided', async () => { + mockGetServerLogChunk.mockResolvedValueOnce({ + fileName: 'results/router.log', + serverLog: 'INFO ready\n', + offset: 65536, + nextOffset: 65547, + }); + + const res = await GET( + req('/api/v1/server-log?id=42&file=results%2Frouter.log&offset=65536&limit=1024'), + ); + expect(res.status).toBe(200); + await expect(res.json()).resolves.toEqual({ + id: 42, + fileName: 'results/router.log', + serverLog: 'INFO ready\n', + offset: 65536, + nextOffset: 65547, + }); + expect(mockGetServerLogChunk).toHaveBeenCalledWith( + 'mock-sql', + 42, + 65536, + 1024, + 'results/router.log', + ); + expect(mockGetServerLog).not.toHaveBeenCalled(); + }); + + it('uses bounded defaults when only one chunk parameter is provided', async () => { + mockGetServerLogChunk.mockResolvedValueOnce({ + fileName: 'server.log', + serverLog: 'start', + offset: 0, + nextOffset: null, + }); + + const res = await GET(req('/api/v1/server-log?id=42&offset=0')); + expect(res.status).toBe(200); + expect(mockGetServerLogChunk).toHaveBeenCalledWith('mock-sql', 42, 0, 64 * 1024, undefined); + }); + + it('streams the complete selected file as a download attachment', async () => { + mockGetServerLogChunk + .mockResolvedValueOnce({ + fileName: 'nested/router log.out', + serverLog: 'INFO', + offset: 0, + nextOffset: 4, + }) + .mockResolvedValueOnce({ + fileName: 'nested/router log.out', + serverLog: ' ready\n', + offset: 4, + nextOffset: null, + }); + + const res = await GET( + req('/api/v1/server-log?id=42&file=nested%2Frouter%20log.out&download=1'), + ); + + expect(res.status).toBe(200); + expect(res.headers.get('content-type')).toBe('text/plain; charset=utf-8'); + expect(res.headers.get('content-disposition')).toContain( + 'attachment; filename="router log.out"', + ); + await expect(res.text()).resolves.toBe('INFO ready\n'); + expect(mockGetServerLogChunk).toHaveBeenNthCalledWith( + 1, + 'mock-sql', + 42, + 0, + 256 * 1024, + 'nested/router log.out', + ); + expect(mockGetServerLogChunk).toHaveBeenNthCalledWith( + 2, + 'mock-sql', + 42, + 4, + 256 * 1024, + 'nested/router log.out', + ); + expect(mockGetServerLog).not.toHaveBeenCalled(); + }); + + it.each(['download=0', 'download=yes', 'download=1&offset=0', 'download=1&limit=1'])( + 'returns 400 for invalid download parameters %s', + async (query) => { + const res = await GET(req(`/api/v1/server-log?id=42&${query}`)); + expect(res.status).toBe(400); + expect(mockGetServerLogChunk).not.toHaveBeenCalled(); + }, + ); + + it.each([ + 'offset=-1', + 'offset=1.5', + 'offset=2000000001', + 'limit=0', + 'limit=262145', + 'limit=nope', + ])('returns 400 for invalid chunk parameter %s', async (query) => { + const res = await GET(req(`/api/v1/server-log?id=42&${query}`)); + expect(res.status).toBe(400); + expect(mockGetServerLogChunk).not.toHaveBeenCalled(); + }); + + it('returns 400 for an empty filename', async () => { + const res = await GET(req('/api/v1/server-log?id=42&file=')); + expect(res.status).toBe(400); + expect(mockGetServerLog).not.toHaveBeenCalled(); + }); + + it('returns a complete named log for compatibility clients', async () => { + mockGetServerLog.mockResolvedValueOnce('router output'); + const res = await GET(req('/api/v1/server-log?id=42&file=router.log')); + await expect(res.json()).resolves.toEqual({ + id: 42, + fileName: 'router.log', + serverLog: 'router output', + }); + expect(mockGetServerLog).toHaveBeenCalledWith('mock-sql', 42, 'router.log'); + }); + + it('returns 404 when a requested chunk has no linked log', async () => { + mockGetServerLogChunk.mockResolvedValueOnce(null); + const res = await GET(req('/api/v1/server-log?id=999&offset=0')); + expect(res.status).toBe(404); + }); + it('returns 500 when query throws', async () => { mockGetServerLog.mockRejectedValueOnce(new Error('Connection reset')); diff --git a/packages/app/src/app/api/v1/server-log/route.ts b/packages/app/src/app/api/v1/server-log/route.ts index 55de025b4..dafaaff4f 100644 --- a/packages/app/src/app/api/v1/server-log/route.ts +++ b/packages/app/src/app/api/v1/server-log/route.ts @@ -2,33 +2,177 @@ import { type NextRequest, NextResponse } from 'next/server'; import { getDb } from '@semianalysisai/inferencex-db/connection'; -import { getServerLog } from '@semianalysisai/inferencex-db/queries/server-logs'; +import { getServerLog, getServerLogChunk } from '@semianalysisai/inferencex-db/queries/server-logs'; import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { PUBLIC_API_ERRORS, publicApiError } from '@/lib/public-api-errors'; export const dynamic = 'force-dynamic'; const getCachedServerLog = cachedQuery((id: number) => getServerLog(getDb(), id), 'server-log', { blobOnly: true, }); +const getCachedNamedServerLog = cachedQuery( + (id: number, fileName: string) => getServerLog(getDb(), id, fileName), + 'server-log-file', + { blobOnly: true }, +); +const getCachedServerLogChunk = cachedQuery( + (id: number, offset: number, limit: number, fileName: string | null) => + getServerLogChunk(getDb(), id, offset, limit, fileName ?? undefined), + 'server-log-chunk', +); + +export const DEFAULT_SERVER_LOG_CHUNK_SIZE = 64 * 1024; +export const MAX_SERVER_LOG_CHUNK_SIZE = 256 * 1024; +const MAX_SERVER_LOG_OFFSET = 2_000_000_000; +const MAX_SERVER_LOG_FILE_NAME_LENGTH = 1024; + +function downloadFileName(fileName: string): string { + const leafName = fileName.replaceAll('\\', '/').split('/').at(-1) || 'server.log'; + const asciiName = [...leafName] + .map((character) => { + const codePoint = character.codePointAt(0) ?? 0; + return codePoint >= 32 && codePoint <= 126 && character !== '"' && character !== '\\' + ? character + : '_'; + }) + .join('') + .slice(0, 180); + const encodedName = encodeURIComponent(leafName).replaceAll( + /[!'()*]/gu, + (character) => `%${character.codePointAt(0)?.toString(16).toUpperCase() ?? ''}`, + ); + return `attachment; filename="${asciiName || 'server.log'}"; filename*=UTF-8''${encodedName}`; +} + +function parseIntegerParam( + value: string | null, + fallback: number, + minimum: number, + maximum: number, +): number | null { + if (value === null) return fallback; + if (!/^\d+$/u.test(value)) return null; + const parsed = Number(value); + return Number.isSafeInteger(parsed) && parsed >= minimum && parsed <= maximum ? parsed : null; +} export async function GET(request: NextRequest) { const id = Number(request.nextUrl.searchParams.get('id')); if (!id || !Number.isFinite(id)) { - return NextResponse.json({ error: 'id is required (benchmark_result_id)' }, { status: 400 }); + return publicApiError(PUBLIC_API_ERRORS.idRequired, 400); } try { - const serverLog = await getCachedServerLog(id); + const fileName = request.nextUrl.searchParams.get('file'); + if ( + fileName !== null && + (fileName.length === 0 || + fileName.length > MAX_SERVER_LOG_FILE_NAME_LENGTH || + fileName.includes('\u0000')) + ) { + return NextResponse.json({ error: 'file is invalid' }, { status: 400 }); + } + const wantsChunk = + request.nextUrl.searchParams.has('offset') || request.nextUrl.searchParams.has('limit'); + const download = request.nextUrl.searchParams.get('download'); + if ((download !== null && download !== '1') || (download === '1' && wantsChunk)) { + return NextResponse.json( + { error: 'download must be 1 and cannot be combined with offset or limit' }, + { status: 400 }, + ); + } + + if (download === '1') { + const firstChunk = await getCachedServerLogChunk(id, 0, MAX_SERVER_LOG_CHUNK_SIZE, fileName); + if (firstChunk === null) { + return publicApiError(PUBLIC_API_ERRORS.notFound, 404); + } + const encoder = new TextEncoder(); + let nextOffset = firstChunk.nextOffset; + const stream = new ReadableStream({ + start(controller) { + if (firstChunk.serverLog.length > 0) { + controller.enqueue(encoder.encode(firstChunk.serverLog)); + } + if (nextOffset === null) controller.close(); + }, + async pull(controller) { + if (nextOffset === null) return; + try { + const chunk = await getCachedServerLogChunk( + id, + nextOffset, + MAX_SERVER_LOG_CHUNK_SIZE, + fileName, + ); + if (chunk === null) { + controller.error(new Error('Log file disappeared during download')); + return; + } + if (chunk.serverLog.length > 0) { + controller.enqueue(encoder.encode(chunk.serverLog)); + } + nextOffset = chunk.nextOffset; + if (nextOffset === null) controller.close(); + } catch (error) { + controller.error(error); + } + }, + }); + return new Response(stream, { + headers: { + 'Cache-Control': 'private, no-cache, no-store, max-age=0', + 'Content-Disposition': downloadFileName(fileName ?? 'server.log'), + 'Content-Type': 'text/plain; charset=utf-8', + 'X-Content-Type-Options': 'nosniff', + }, + }); + } + + if (wantsChunk) { + const offset = parseIntegerParam( + request.nextUrl.searchParams.get('offset'), + 0, + 0, + MAX_SERVER_LOG_OFFSET, + ); + const limit = parseIntegerParam( + request.nextUrl.searchParams.get('limit'), + DEFAULT_SERVER_LOG_CHUNK_SIZE, + 1, + MAX_SERVER_LOG_CHUNK_SIZE, + ); + if (offset === null || limit === null) { + return NextResponse.json( + { + error: `offset must be 0-${MAX_SERVER_LOG_OFFSET}; limit must be 1-${MAX_SERVER_LOG_CHUNK_SIZE}`, + }, + { status: 400 }, + ); + } + + const chunk = await getCachedServerLogChunk(id, offset, limit, fileName); + if (chunk === null) { + return publicApiError(PUBLIC_API_ERRORS.notFound, 404); + } + return cachedJson({ id, ...chunk }); + } + + const serverLog = + fileName === null + ? await getCachedServerLog(id) + : await getCachedNamedServerLog(id, fileName); if (serverLog === null) { - return NextResponse.json({ error: 'Not found' }, { status: 404 }); + return publicApiError(PUBLIC_API_ERRORS.notFound, 404); } - return cachedJson({ id, serverLog }); + return cachedJson(fileName === null ? { id, serverLog } : { id, fileName, serverLog }); } catch (error) { console.error('Error fetching server log:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } } diff --git a/packages/app/src/app/api/v1/trace-server-metric-source/route.ts b/packages/app/src/app/api/v1/trace-server-metric-source/route.ts new file mode 100644 index 000000000..ed4bf8b83 --- /dev/null +++ b/packages/app/src/app/api/v1/trace-server-metric-source/route.ts @@ -0,0 +1,43 @@ +import { type NextRequest, NextResponse } from 'next/server'; + +import { getDb } from '@semianalysisai/inferencex-db/connection'; +import { CHART_SERIES_VERSION } from '@semianalysisai/inferencex-db/etl/compute-chart-series'; +import { + getTraceServerMetricSource, + type MetricSourceSeries, +} from '@semianalysisai/inferencex-db/queries/trace-server-metrics'; + +import { cachedJson, cachedQuery } from '@/lib/api-cache'; + +export const dynamic = 'force-dynamic'; +export const maxDuration = 300; + +/** Version-derived blob-cache key namespace (exported for the key-derivation test). */ +export const CACHE_KEY_PREFIX = `trace-server-metric-source-v${CHART_SERIES_VERSION}`; + +const getCachedMetricSource = cachedQuery( + (id: number, source: string): Promise => + getTraceServerMetricSource(getDb(), id, source), + CACHE_KEY_PREFIX, + { blobOnly: true }, +); + +/** GET /api/v1/trace-server-metric-source?id=N&source=SOURCE_ID */ +export async function GET(request: NextRequest): Promise { + const id = Number(request.nextUrl.searchParams.get('id')); + const source = request.nextUrl.searchParams.get('source')?.trim(); + if (!id || !Number.isFinite(id)) { + return NextResponse.json({ error: 'id is required (benchmark_result_id)' }, { status: 400 }); + } + if (!source || source.length > 512) { + return NextResponse.json({ error: 'source is required' }, { status: 400 }); + } + try { + const data = await getCachedMetricSource(id, source); + if (!data) return NextResponse.json({ error: 'Not found' }, { status: 404 }); + return cachedJson(data); + } catch (error) { + console.error('Error fetching trace server metric source:', error); + return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + } +} diff --git a/packages/app/src/app/api/v1/trace-server-metrics/route.ts b/packages/app/src/app/api/v1/trace-server-metrics/route.ts index 1334b580e..a2adf3d54 100644 --- a/packages/app/src/app/api/v1/trace-server-metrics/route.ts +++ b/packages/app/src/app/api/v1/trace-server-metrics/route.ts @@ -12,6 +12,15 @@ import { idQueryRoute } from '../id-routes'; export const dynamic = 'force-dynamic'; +// The slow path re-parses `server_metrics_json_gz`, which reaches 448 MB +// compressed (several GB of JSON) on the largest agentic points and takes +// minutes to stream. That only happens for rows whose stored `chart_series` +// predates the current version — i.e. after a version bump, until +// `db:backfill-chart-series` drains them — but the platform default would +// cut those requests off mid-parse and the blob cache only populates on +// success, so every visitor would re-pay. Ask for the maximum instead. +export const maxDuration = 300; + // Key derived from TRACE_SERVER_METRICS_VERSION (governs chart_series plus // the separately queried point-metadata payload). // The blob cache is write-once with no post-backfill purge, so the diff --git a/packages/app/src/app/api/v1/workflow-info/route.ts b/packages/app/src/app/api/v1/workflow-info/route.ts index d8d646e97..075c472e0 100644 --- a/packages/app/src/app/api/v1/workflow-info/route.ts +++ b/packages/app/src/app/api/v1/workflow-info/route.ts @@ -1,4 +1,4 @@ -import { type NextRequest, NextResponse } from 'next/server'; +import type { NextRequest } from 'next/server'; import { FIXTURES_MODE, getDb } from '@semianalysisai/inferencex-db/connection'; @@ -10,6 +10,7 @@ import { } from '@semianalysisai/inferencex-db/queries/workflow-info'; import { cachedJson, cachedQuery } from '@/lib/api-cache'; +import { PUBLIC_API_ERRORS, publicApiError } from '@/lib/public-api-errors'; import { loadFixture } from '@/lib/test-fixtures'; export const dynamic = 'force-dynamic'; @@ -25,16 +26,9 @@ async function loadWorkflowInfo(date: string, benchmarkType?: 'agentic_traces') return { runs, changelogs, configs, runConfigs }; } -// Preserve the established public cache and response for calls without a -// scenario. Scenario-scoped calls use a separate cache namespace. -const getCachedWorkflowInfo = cachedQuery( - (date: string) => loadWorkflowInfo(date), - 'workflow-info', -); -const getCachedScenarioWorkflowInfo = cachedQuery( - (date: string) => loadWorkflowInfo(date, 'agentic_traces'), - 'workflow-info-scenario', -); +// Scenario is an explicit cache dimension: fixed-sequence and Agentic Traces +// requests for the same date must never share a cached runConfigs payload. +const getCachedWorkflowInfo = cachedQuery(loadWorkflowInfo, 'workflow-info-v2'); export async function GET(request: NextRequest) { const date = request.nextUrl.searchParams.get('date') ?? ''; @@ -43,20 +37,15 @@ export async function GET(request: NextRequest) { ? 'agentic_traces' : undefined; if (date && !/^\d{4}-\d{2}-\d{2}$/u.test(date)) { - return NextResponse.json( - { error: 'Invalid date format (YYYY-MM-DD required)' }, - { status: 400 }, - ); + return publicApiError(PUBLIC_API_ERRORS.invalidDate, 400); } if (FIXTURES_MODE) return cachedJson(loadFixture('workflow-info')); try { - const data = benchmarkType - ? await getCachedScenarioWorkflowInfo(date) - : await getCachedWorkflowInfo(date); + const data = await getCachedWorkflowInfo(date, benchmarkType); return cachedJson(data); } catch (error) { console.error('Error fetching workflow info:', error); - return NextResponse.json({ error: 'Internal server error' }, { status: 500 }); + return publicApiError(PUBLIC_API_ERRORS.internal, 500); } } diff --git a/packages/app/src/app/blog/[slug]/og-image-render.tsx b/packages/app/src/app/blog/[slug]/og-image-render.tsx index 4d1e0732e..f758a6063 100644 --- a/packages/app/src/app/blog/[slug]/og-image-render.tsx +++ b/packages/app/src/app/blog/[slug]/og-image-render.tsx @@ -43,7 +43,16 @@ async function loadTiles() { return TILE_GRID.map((t) => (t ? { src: cache[t.file], rotate: t.rotate } : null)); } -export async function renderOgImage(meta: BlogPostMeta) { +export async function renderOgImage( + meta: Pick, + opts?: { + /** + * Pre-formatted footer label. When set, `meta.date` is ignored — used by + * non-blog reusers (e.g. /model pages) whose dates are free-form text. + */ + dateLabel?: string; + }, +) { const [logoSrc, tiles] = await Promise.all([ readFile(join(process.cwd(), 'public/brand/logo-color.png')).then( (buf) => `data:image/png;base64,${buf.toString('base64')}`, @@ -174,12 +183,13 @@ export async function renderOgImage(meta: BlogPostMeta) {
- {new Date(`${meta.date}T00:00:00Z`).toLocaleDateString('en-US', { - year: 'numeric', - month: 'long', - day: 'numeric', - timeZone: 'UTC', - })} + {opts?.dateLabel ?? + new Date(`${meta.date}T00:00:00Z`).toLocaleDateString('en-US', { + year: 'numeric', + month: 'long', + day: 'numeric', + timeZone: 'UTC', + })} SemiAnalysis
diff --git a/packages/app/src/app/blog/[slug]/page.tsx b/packages/app/src/app/blog/[slug]/page.tsx index 92baa0d65..4fa7a7bc2 100644 --- a/packages/app/src/app/blog/[slug]/page.tsx +++ b/packages/app/src/app/blog/[slug]/page.tsx @@ -1,12 +1,5 @@ import type { Metadata } from 'next'; import { notFound } from 'next/navigation'; -import { compileMDX } from 'next-mdx-remote/rsc'; -import rehypeShikiFromHighlighter from '@shikijs/rehype/core'; -import rehypeKatex from 'rehype-katex'; -import remarkGfm from 'remark-gfm'; -import remarkMath from 'remark-math'; -import { createHighlighterCore } from 'shiki/core'; -import { createOnigurumaEngine } from 'shiki/engine/oniguruma'; import 'katex/dist/katex.min.css'; @@ -14,7 +7,6 @@ import { BlogBackLink } from '@/components/blog/blog-back-link'; import { BlogPostNav } from '@/components/blog/blog-post-nav'; import { BlogToc } from '@/components/blog/blog-toc'; import { HashScroll } from '@/components/blog/hash-scroll'; -import { createMdxComponents } from '@/components/blog/mdx-components'; import { ReadingProgressBar } from '@/components/blog/reading-progress-bar'; import { ShareTwitterButton, ShareLinkedInButton } from '@/components/share-buttons'; import { Card } from '@/components/ui/card'; @@ -29,6 +21,7 @@ import { getPostBySlug, hasZhTranslation, } from '@/lib/blog'; +import { compileBlogMdx } from '@/lib/blog-mdx'; import { languageAlternates } from '@/lib/i18n'; import { AUTHOR_HANDLE, @@ -85,33 +78,6 @@ export async function generateMetadata({ params }: Props): Promise { }; } -let highlighterPromise: ReturnType | null = null; - -function getHighlighter() { - if (!highlighterPromise) { - highlighterPromise = createHighlighterCore({ - themes: [import('shiki/themes/github-dark.mjs'), import('shiki/themes/github-light.mjs')], - langs: [ - import('shiki/langs/typescript.mjs'), - import('shiki/langs/javascript.mjs'), - import('shiki/langs/python.mjs'), - import('shiki/langs/bash.mjs'), - import('shiki/langs/json.mjs'), - import('shiki/langs/yaml.mjs'), - import('shiki/langs/css.mjs'), - import('shiki/langs/html.mjs'), - import('shiki/langs/tsx.mjs'), - import('shiki/langs/jsx.mjs'), - import('shiki/langs/sql.mjs'), - import('shiki/langs/go.mjs'), - import('shiki/langs/rust.mjs'), - ], - engine: createOnigurumaEngine(import('shiki/wasm')), - }); - } - return highlighterPromise; -} - export default async function BlogPostPage({ params }: Props) { const { slug } = await params; const result = getPostBySlug(slug); @@ -120,32 +86,7 @@ export default async function BlogPostPage({ params }: Props) { const { meta, raw } = result; const adjacent = getAdjacentPosts(slug); const headings = extractHeadings(raw); - const highlighter = await getHighlighter(); - - const { content } = await compileMDX({ - source: raw, - components: createMdxComponents(), - options: { - mdxOptions: { - // `singleDollarTextMath: false` keeps prices like $1.95/GPU/hr from being parsed - // as inline math; math must be delimited with `$$`. - remarkPlugins: [remarkGfm, [remarkMath, { singleDollarTextMath: false }]], - rehypePlugins: [ - // `strict: false` keeps a single malformed expression from failing the whole build; - // KaTeX renders it inline in red instead. - [rehypeKatex, { strict: false }], - [ - rehypeShikiFromHighlighter, - highlighter, - { - themes: { dark: 'github-dark', light: 'github-light' }, - defaultColor: false, - }, - ], - ], - }, - }, - }); + const { content } = await compileBlogMdx(raw); const jsonLd = buildBlogPostingJsonLd(meta, raw); const breadcrumbJsonLd = buildBlogBreadcrumbJsonLd(slug, meta.title); diff --git a/packages/app/src/app/blog/page.tsx b/packages/app/src/app/blog/page.tsx index bad1c01c1..b2a98ea2b 100644 --- a/packages/app/src/app/blog/page.tsx +++ b/packages/app/src/app/blog/page.tsx @@ -11,11 +11,11 @@ import { SITE_URL, SITE_NAME, AUTHOR_NAME } from '@semianalysisai/inferencex-con export const metadata: Metadata = { title: 'Articles', - description: `Technical articles from ${SITE_NAME} by ${AUTHOR_NAME} — AI inference benchmarking, chip performance analysis, and ML infrastructure insights.`, + description: `Technical articles from ${SITE_NAME} by ${AUTHOR_NAME} on agentic inference benchmarks, AgentX results, chip performance, and ML infrastructure.`, alternates: enAlternates('/blog'), openGraph: { title: `Articles | ${SITE_NAME} by ${AUTHOR_NAME}`, - description: 'AI inference benchmarking insights and chip performance analysis.', + description: 'Articles on agentic inference benchmarks, AgentX results, and chip performance.', url: `${SITE_URL}/blog`, }, }; @@ -49,7 +49,8 @@ export default async function BlogPage({

Articles

- Insights on AI inference benchmarking, chip performance, and ML infrastructure. + Articles on agentic inference, AgentX results, chip performance, and ML + infrastructure.

New to the terminology?{' '} diff --git a/packages/app/src/app/chips/[slug]/page.tsx b/packages/app/src/app/chips/[slug]/page.tsx new file mode 100644 index 000000000..9165e7f12 --- /dev/null +++ b/packages/app/src/app/chips/[slug]/page.tsx @@ -0,0 +1,113 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { ChipDetailContent, ChipVsContent } from '@/components/chips/chip-page-sections'; +import { + getAllChipRouteSlugs, + getChipPage, + getChipVsPage, + type ChipVsPage, +} from '@/lib/chip-pages'; +import { enAlternates } from '@/lib/i18n'; +import { + AUTHOR_HANDLE, + AUTHOR_NAME, + SITE_NAME, + SITE_URL, + SUPPORTERS_LINE, +} from '@semianalysisai/inferencex-constants'; + +interface Props { + params: Promise<{ slug: string }>; +} + +export const dynamicParams = false; + +export function generateStaticParams() { + return getAllChipRouteSlugs().map((slug) => ({ slug })); +} + +function vsDescription(page: ChipVsPage): string { + return `${page.a.title} vs ${page.b.title}: memory, bandwidth, FP8/FP4 compute, TDP and cloud pricing side by side, with continuously measured LLM inference benchmarks on identical workloads. ${SUPPORTERS_LINE}`; +} + +function vsKeywords(page: ChipVsPage): string[] { + const a = page.a.label; + const b = page.b.label; + return [ + `${a} vs ${b}`, + `${b} vs ${a}`, + `${a} vs ${b} benchmark`, + `${a} vs ${b} specs`, + `${a} vs ${b} inference performance`, + `${a} vs ${b} price`, + `${a} GPU comparison`, + 'AI inference benchmarks', + ]; +} + +export async function generateMetadata({ params }: Props): Promise { + const { slug } = await params; + + const chip = getChipPage(slug); + if (chip) { + const title = `${chip.title} Specs, Pricing & AI Inference Benchmarks`; + const description = `${chip.summary} ${SUPPORTERS_LINE}`; + const url = `${SITE_URL}/chips/${chip.slug}`; + return { + title, + description, + keywords: [...chip.keywords], + authors: [{ name: AUTHOR_NAME }], + alternates: enAlternates(`/chips/${chip.slug}`), + openGraph: { + title: `${title} | ${SITE_NAME}`, + description, + url, + type: 'article', + }, + twitter: { + card: 'summary_large_image', + title: `${title} | ${SITE_NAME}`, + description, + creator: AUTHOR_HANDLE, + }, + }; + } + + const vsPage = getChipVsPage(slug); + if (!vsPage) return {}; + const title = `${vsPage.a.label} vs ${vsPage.b.label}: Specs, Power & AI Inference Comparison`; + const description = vsDescription(vsPage); + const url = `${SITE_URL}/chips/${vsPage.slug}`; + return { + title, + description, + keywords: vsKeywords(vsPage), + authors: [{ name: AUTHOR_NAME }], + alternates: enAlternates(`/chips/${vsPage.slug}`), + openGraph: { + title: `${title} | ${SITE_NAME}`, + description, + url, + type: 'article', + }, + twitter: { + card: 'summary_large_image', + title: `${title} | ${SITE_NAME}`, + description, + creator: AUTHOR_HANDLE, + }, + }; +} + +export default async function ChipRoutePage({ params }: Props) { + const { slug } = await params; + + const chip = getChipPage(slug); + if (chip) return ; + + const vsPage = getChipVsPage(slug); + if (!vsPage) notFound(); + return ; +} diff --git a/packages/app/src/app/chips/page.tsx b/packages/app/src/app/chips/page.tsx new file mode 100644 index 000000000..bfb646858 --- /dev/null +++ b/packages/app/src/app/chips/page.tsx @@ -0,0 +1,39 @@ +import type { Metadata } from 'next'; + +import { ChipsIndexContent } from '@/components/chips/chip-page-sections'; +import { enAlternates } from '@/lib/i18n'; +import { SITE_NAME, SITE_URL, SUPPORTERS_LINE } from '@semianalysisai/inferencex-constants'; + +const TITLE = 'AI Chips for LLM Inference: Specs, Pricing & Benchmarks'; +const DESCRIPTION = `Specs, cloud pricing and continuously measured LLM inference benchmarks for NVIDIA Hopper and Blackwell (H100, H200, B200, B300, GB200 NVL72, GB300 NVL72) and AMD Instinct (MI300X, MI325X, MI355X) chips. ${SUPPORTERS_LINE}`; + +export const metadata: Metadata = { + title: TITLE, + description: DESCRIPTION, + keywords: [ + 'AI chips', + 'AI GPU comparison', + 'GPU specs', + 'GPU price per hour', + 'LLM inference hardware', + 'NVIDIA vs AMD GPU', + 'data center GPU pricing', + 'AI accelerator comparison', + ], + alternates: enAlternates('/chips'), + openGraph: { + title: `${TITLE} | ${SITE_NAME}`, + description: DESCRIPTION, + url: `${SITE_URL}/chips`, + type: 'website', + }, + twitter: { + card: 'summary_large_image', + title: `${TITLE} | ${SITE_NAME}`, + description: DESCRIPTION, + }, +}; + +export default function ChipsIndexPage() { + return ; +} diff --git a/packages/app/src/app/compare-per-dollar/[slug]/[scenario]/page.tsx b/packages/app/src/app/compare-per-dollar/[slug]/[scenario]/page.tsx new file mode 100644 index 000000000..ad17ae939 --- /dev/null +++ b/packages/app/src/app/compare-per-dollar/[slug]/[scenario]/page.tsx @@ -0,0 +1,37 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { type ScenarioSegment, sequenceForScenarioSegment } from '@/lib/compare-scenario-route'; + +import { buildPerDollarMetadata, renderPerDollarPage } from '../page'; + +/** + * `/compare-per-dollar//` — the same comparison with its workload pinned by + * the path instead of `?i_seq=`. Both routes render through the parent's + * exported body, so the two URLs cannot drift. + */ +export const dynamic = 'force-dynamic'; + +interface Props { + params: Promise<{ slug: string; scenario: string }>; + searchParams: Promise>; +} + +/** Reject unknown segments rather than silently falling back to the pair's + * default, which would serve 200s for arbitrary junk paths. */ +function assertScenario(scenario: string): ScenarioSegment { + if (!sequenceForScenarioSegment(scenario)) notFound(); + return scenario.toLowerCase() as ScenarioSegment; +} + +export async function generateMetadata({ params }: Props): Promise { + const { slug, scenario } = await params; + return buildPerDollarMetadata(slug, { scenarioSegment: assertScenario(scenario) }); +} + +export default async function PerDollarScenarioPage({ params, searchParams }: Props) { + const { slug, scenario } = await params; + return renderPerDollarPage(slug, await searchParams, { + scenarioSegment: assertScenario(scenario), + }); +} diff --git a/packages/app/src/app/compare-per-dollar/[slug]/opengraph-image.tsx b/packages/app/src/app/compare-per-dollar/[slug]/opengraph-image.tsx index 21bbd5642..b4f666157 100644 --- a/packages/app/src/app/compare-per-dollar/[slug]/opengraph-image.tsx +++ b/packages/app/src/app/compare-per-dollar/[slug]/opengraph-image.tsx @@ -1,201 +1,32 @@ -/** - * Per-dollar compare OG image — identical circuit tile sidebar layout to the - * /compare OG and the blog OG, but the eyebrow and bottom banner are swapped - * to surface the "Performance per Dollar" framing. - */ -import { readFile } from 'node:fs/promises'; -import { join } from 'node:path'; - import { notFound } from 'next/navigation'; -import { ImageResponse } from 'next/og'; import { getAllComparableCompareSlugs } from '@/lib/compare-availability'; +import { renderCompareOg } from '@/lib/compare-og'; import { canonicalCompareSlug, compareDisplayLabel, parseCompareSlug } from '@/lib/compare-slug'; export const alt = 'Chip performance-per-dollar inference benchmark comparison'; export const size = { width: 1200, height: 630 }; export const contentType = 'image/png'; -const BLUE = '#0B86D1'; -const BG = '#131416'; -const PANEL_BG = '#0F1214'; - -// Same tile grid as the /compare and blog OG so all three OG types read as a -// family. -const TILE_GRID: ({ file: string; rotate?: number } | null)[] = [ - { file: 'teal-chevron.png', rotate: 180 }, - { file: 'gold-diagonal.png' }, - { file: 'teal-circuit.png' }, - null, - { file: 'gold-wavy.png' }, - { file: 'teal-chip.png' }, - { file: 'teal-chevron.png', rotate: 90 }, - { file: 'teal-organic.png' }, - null, - { file: 'gold-circuit.png' }, - { file: 'teal-circuit.png', rotate: 180 }, - { file: 'teal-organic.png', rotate: 180 }, -]; - export async function generateStaticParams() { const slugs = await getAllComparableCompareSlugs(); return slugs.map(({ modelSlug, a, b }) => ({ slug: canonicalCompareSlug(modelSlug, a, b) })); } -let logoSrcPromise: Promise | undefined; -function getLogoSrc(): Promise { - if (!logoSrcPromise) { - logoSrcPromise = readFile(join(process.cwd(), 'public/brand/logo-color.png')) - .then((buf) => `data:image/png;base64,${buf.toString('base64')}`) - .catch(() => null); - } - return logoSrcPromise; -} - -let tilesPromise: Promise<({ src: string; rotate?: number } | null)[]> | undefined; -function getTiles(): Promise<({ src: string; rotate?: number } | null)[]> { - if (!tilesPromise) { - const uniqueFiles = [...new Set(TILE_GRID.filter(Boolean).map((t) => t!.file))]; - tilesPromise = Promise.all( - uniqueFiles.map(async (f) => { - const src = await readFile(join(process.cwd(), 'public/brand/og-tiles', f)) - .then((buf) => `data:image/png;base64,${buf.toString('base64')}`) - .catch(() => null); - return [f, src] as const; - }), - ).then((loaded) => { - const cache = Object.fromEntries(loaded); - return TILE_GRID.map((t) => { - if (!t) return null; - const src = cache[t.file]; - return src ? { src, rotate: t.rotate } : null; - }); - }); - } - return tilesPromise; -} - export default async function OgImage({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params; const parsed = parseCompareSlug(slug); if (!parsed) notFound(); - const [logoSrc, tiles] = await Promise.all([getLogoSrc(), getTiles()]); const title = compareDisplayLabel(parsed.a, parsed.b); - const eyebrow = `${parsed.model.label} · Performance per Dollar`; const titleSize = title.length > 26 ? 80 : title.length > 18 ? 96 : 112; - return new ImageResponse( -

-
- {tiles.map((tile, i) => { - if (!tile) return null; - const row = Math.floor(i / 2); - const col = i % 2; - return ( - - ); - })} -
-
- -
-
-
- {eyebrow} -
- -
- {title} -
-
- -
- - AI inference benchmark · cost per million tokens - - {logoSrc && } -
-
-
, + return renderCompareOg({ + eyebrow: `${parsed.model.label} · Performance per Dollar`, + title, + titleSize, + footer: 'AI inference benchmark · cost per million tokens', + language: 'en', size, - ); + }); } diff --git a/packages/app/src/app/compare-per-dollar/[slug]/page-client.tsx b/packages/app/src/app/compare-per-dollar/[slug]/page-client.tsx index 3fa64bfba..44633ff7b 100644 --- a/packages/app/src/app/compare-per-dollar/[slug]/page-client.tsx +++ b/packages/app/src/app/compare-per-dollar/[slug]/page-client.tsx @@ -3,35 +3,45 @@ import Link from 'next/link'; import { useEffect, useMemo } from 'react'; -import type { GPUDataPoint, InterpolatedResult } from '@/components/calculator/types'; +import { TCO_SOURCE_TITLE, TCO_SOURCE_URL } from '@semianalysisai/inferencex-constants'; + +import type { GPUDataPoint } from '@/components/calculator/types'; import { useThroughputData } from '@/components/calculator/useThroughputData'; -import { CompareInterpolatedTable } from '@/components/compare/compare-interpolated-table'; -import { useGlobalFilters, GlobalFilterProvider } from '@/components/GlobalFilterContext'; +import { + CompareTableRenderer, + type CompareTableData, +} from '@/components/compare/compare-table-renderer'; +import { + GlobalFilterProvider, + useGlobalFilterAvailability, + useGlobalFilterRun, + useGlobalFilterSelection, +} from '@/components/GlobalFilterContext'; import { InferenceProvider } from '@/components/inference/InferenceContext'; import InferenceChartDisplay from '@/components/inference/ui/ChartDisplay'; import { Card } from '@/components/ui/card'; import { track } from '@/lib/analytics'; +import type { BenchmarkRow } from '@/lib/api'; +import { toCalculatorBenchmarkRows } from '@/lib/benchmark-api-view'; import { toModel, toPrecisions, toSequence } from '@/lib/compare-enum-coerce'; - -interface SsrTableData { - defaultTargets: number[]; - ssrRows: { target: number; a: InterpolatedResult | null; b: InterpolatedResult | null }[]; - interactivityRange: { min: number; max: number }; -} +import type { AgenticScenarioIntro } from '@/lib/compare-ssr'; /** Only show Cost + Concurrency in the interpolated table — the rest of the * metric rows (Throughput, tok/s/MW) live on the sibling /compare page. */ const PER_DOLLAR_TABLE_METRICS = ['Cost ($/M tok)', 'Concurrency']; -/** Rename "Cost ($/M tok)" to the full-English "Dollar per Million Tokens" - * in the per-dollar table so the cell reads in line with the page's - * "Performance per Dollar" framing and surfaces the SEO term verbatim. */ +/** Localized display labels keyed by the table's stable English metric key. */ const PER_DOLLAR_LABEL_OVERRIDES = { - 'Cost ($/M tok)': 'Dollar per Million Tokens', -}; + en: { + 'Cost ($/M tok)': 'Dollar per Million Tokens', + }, + zh: { + 'Cost ($/M tok)': '每百万 token 美元成本', + }, +} as const; /** y_costh = Cost per Million Total Tokens (Owning - Hyperscaler). Defined in - * packages/app/src/components/inference/inference-chart-config.json. */ + * packages/app/src/components/inference/metric-registry.ts. */ const PER_DOLLAR_DEFAULT_Y_AXIS = 'y_costh'; const STRINGS = { @@ -54,10 +64,10 @@ const STRINGS = { fullComparisonLinkText: '查看完整延迟与吞吐量对比 →', caveatSeqFallback: '序列', caveatPrecFallback: '精度', - pricingLabel: 'Chip 定价(所属云服务商):', + pricingLabel: '芯片定价(所属云服务商):', pricingSource: '来源:', emptyState: - '当前默认模型在此 Chip 组合上没有可用的插值每 token 成本数据。请使用下方图表控件选择一个两款 Chip 均有基准测试数据的模型和精度。', + '当前默认模型在此芯片组合上没有可用的插值每 token 成本数据。请使用下方图表控件选择一个两款芯片均有基准测试数据的模型和精度。', }, } as const; @@ -72,13 +82,17 @@ interface ComparePerDollarPageClientProps { defaultModel: string; defaultSequence: string | null; defaultPrecision: string | null; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; + initialBenchmarkRows?: BenchmarkRow[]; /** One SSR-rendered prose paragraph per interpolated-table row (default * interactivity target). Each paragraph picks a template variant * deterministically from the slug so prose stays stable across renders * but varies across pages in the catalog. Empty array when there's no * comparable data. */ narrative: string[]; + /** Set only when the page is rendering the agentic workload. Explains + * what AgentX measures before the head-to-head numbers. */ + agenticIntro?: AgenticScenarioIntro | null; aLabel: string; bLabel: string; aVendor: string; @@ -105,7 +119,9 @@ export default function ComparePerDollarPageClient({ defaultSequence, defaultPrecision, ssrTableData, + initialBenchmarkRows, narrative, + agenticIntro = null, aLabel, bLabel, aVendor, @@ -125,6 +141,14 @@ export default function ComparePerDollarPageClient({ const initialModel = toModel(defaultModel); const initialSequence = toSequence(defaultSequence); const initialPrecisions = toPrecisions(defaultPrecision); + const benchmarkQueryScope = `compare-pair:${a}:${b}`; + const initialCalculatorRows = useMemo( + () => + initialBenchmarkRows && defaultSequence + ? toCalculatorBenchmarkRows(initialBenchmarkRows, defaultSequence) + : undefined, + [defaultSequence, initialBenchmarkRows], + ); const t = STRINGS[locale]; const isZh = locale === 'zh'; @@ -139,6 +163,9 @@ export default function ComparePerDollarPageClient({ initialActiveHwTypes={[a, b]} compareGpuPair={compareGpuPair} initialYAxisMetric={PER_DOLLAR_DEFAULT_Y_AXIS} + benchmarkQueryScope={benchmarkQueryScope} + initialBenchmarkModel={initialModel} + initialBenchmarkRows={initialBenchmarkRows} >
@@ -175,6 +202,21 @@ export default function ComparePerDollarPageClient({ .

)} + {agenticIntro && ( +

+ {agenticIntro.paragraph}{' '} + + {agenticIntro.linkLabel} → + +

+ )} {narrative.length > 0 && (
0 ? `$${bCostPerGpuHr.toFixed(2)}/chip/hr` : '—'}.{' '} {t.pricingSource}{' '} track('compare_per_dollar_tco_source_clicked', { slug })} > - SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model + {TCO_SOURCE_TITLE} .

@@ -258,7 +300,11 @@ export default function ComparePerDollarPageClient({ aLabel={aLabel} bLabel={bLabel} ssrTableData={ssrTableData} + initialModel={initialModel} + initialSequence={initialSequence} + initialCalculatorRows={initialCalculatorRows} emptyStateText={t.emptyState} + isZh={isZh} /> @@ -274,23 +320,48 @@ function CompareTableSection({ aLabel, bLabel, ssrTableData, + initialModel, + initialSequence, + initialCalculatorRows, emptyStateText, + isZh, }: { a: string; b: string; aLabel: string; bLabel: string; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; + initialModel: ReturnType; + initialSequence: ReturnType; + initialCalculatorRows?: BenchmarkRow[]; emptyStateText: string; + isZh: boolean; }) { - const { effectiveSequence, effectivePrecisions, selectedRunDate, selectedModel } = - useGlobalFilters(); + const { effectiveSequence, effectivePrecisions, selectedModel, sequenceResolved } = + useGlobalFilterSelection(); + const { availableDates } = useGlobalFilterAvailability(); + const { selectedRunDate } = useGlobalFilterRun(); + const latestAvailableDate = availableDates.at(-1) ?? ''; + const calculatorRunDate = + selectedRunDate && selectedRunDate === latestAvailableDate ? '' : selectedRunDate; + const initialRows = + initialCalculatorRows && + sequenceResolved && + selectedModel === initialModel && + effectiveSequence === initialSequence && + calculatorRunDate === '' + ? initialCalculatorRows + : undefined; const { gpuDataByGroupKey, ranges, hasData } = useThroughputData( selectedModel, effectiveSequence, effectivePrecisions, - selectedRunDate, + calculatorRunDate, + undefined, + undefined, + initialRows, + sequenceResolved, ); const { pointsA, pointsB } = useMemo(() => { @@ -306,25 +377,17 @@ function CompareTableSection({ const clientRange = hasData ? ranges.interactivity : ssrTableData.interactivityRange; - if (ssrTableData.defaultTargets.length === 0) { - return ( -
- {emptyStateText} -
- ); - } - return ( - ); } diff --git a/packages/app/src/app/compare-per-dollar/[slug]/page.tsx b/packages/app/src/app/compare-per-dollar/[slug]/page.tsx index 0b3584ff2..e85cf15de 100644 --- a/packages/app/src/app/compare-per-dollar/[slug]/page.tsx +++ b/packages/app/src/app/compare-per-dollar/[slug]/page.tsx @@ -9,8 +9,12 @@ import { } from '@semianalysisai/inferencex-constants'; import { JsonLd } from '@/components/json-ld'; +import { + isAgenticSequence, + type ScenarioSegment, + sequenceForScenarioSegment, +} from '@/lib/compare-scenario-route'; import { languageAlternates } from '@/lib/i18n'; -import { pickPairDefaults } from '@/lib/compare-pair-defaults'; import { canonicalCompareSlug, compareDisplayLabel, @@ -19,23 +23,40 @@ import { } from '@/lib/compare-slug'; import { getGpuSpecs } from '@/lib/constants'; import { + AGENTIC_SCENARIO_INTRO, buildBreadcrumbJsonLd, buildJsonLd, compareTableNarrative, - computeCompareTableData, - dateRangeForPair, - getCachedBenchmarks, KNOWN_MODELS, KNOWN_PRECISIONS, KNOWN_SEQUENCES, pickString, - summarize, } from '@/lib/compare-ssr'; +import { + getComparePageDerivedData, + initialCompareBenchmarkRows, +} from '@/lib/compare-page-data.server'; import ComparePerDollarPageClient from './page-client'; export const dynamic = 'force-dynamic'; +/** + * `/compare-per-dollar//` renders this same page with the workload + * pinned by the path. The segment is threaded through rather than duplicated + * in a parallel route file so the body — redirects, JSON-LD, metadata, client + * props — is written once and cannot drift between the two URLs. + */ +export interface ScenarioOptions { + scenarioSegment?: ScenarioSegment; +} + +function scenarioPath(canonical: string, scenarioSegment?: ScenarioSegment): string { + return scenarioSegment + ? `/compare-per-dollar/${canonical}/${scenarioSegment}` + : `/compare-per-dollar/${canonical}`; +} + interface Props { params: Promise<{ slug: string }>; searchParams: Promise>; @@ -43,11 +64,26 @@ interface Props { export async function generateMetadata({ params }: Props): Promise { const { slug } = await params; + return buildPerDollarMetadata(slug, {}); +} + +export function buildPerDollarMetadata( + slug: string, + { scenarioSegment }: ScenarioOptions, +): Metadata { const parsed = parseCompareSlug(slug); if (!parsed) return {}; const fullLabel = compareModelDisplayLabel(parsed.model, parsed.a, parsed.b); const gpuLabel = compareDisplayLabel(parsed.a, parsed.b); - const url = `${SITE_URL}/compare-per-dollar/${canonicalCompareSlug(parsed.model.slug, parsed.a, parsed.b)}`; + const canonical = canonicalCompareSlug(parsed.model.slug, parsed.a, parsed.b); + // The scenario segments are views of one comparison, so the bare slug URL + // stays the indexable representative and every segment canonicalizes to it. + // Without this, `/…//` and `/…/` would serve + // byte-identical pages, each claiming to be canonical. + const routePath = scenarioPath(canonical, scenarioSegment); + const canonicalPath = scenarioPath(canonical); + const url = `${SITE_URL}${routePath}`; + const canonicalUrl = `${SITE_URL}${canonicalPath}`; // Description leads with the searched GPU pair + model, then the trust // signals (verified/reproducible, what InferenceX is, named supporters) // before weaving the per-dollar SEO terms ("performance per dollar", "cost @@ -57,10 +93,8 @@ export async function generateMetadata({ params }: Props): Promise { title: `${fullLabel} — Performance per Dollar`, description, alternates: { - canonical: url, - languages: languageAlternates( - `/compare-per-dollar/${canonicalCompareSlug(parsed.model.slug, parsed.a, parsed.b)}`, - ), + canonical: canonicalUrl, + languages: languageAlternates(canonicalPath), }, openGraph: { title: `${fullLabel} — Performance per Dollar | ${SITE_NAME}`, @@ -78,11 +112,17 @@ export async function generateMetadata({ params }: Props): Promise { export default async function ComparePerDollarPage({ params, searchParams }: Props) { const { slug } = await params; + return renderPerDollarPage(slug, await searchParams, {}); +} + +export async function renderPerDollarPage( + slug: string, + sp: Record, + { scenarioSegment }: ScenarioOptions, +) { const parsed = parseCompareSlug(slug); if (!parsed) notFound(); - const sp = await searchParams; - // Same one-hop 308 normalization as /compare/[slug] — bare-slug fallback, // alias model resolution, GPU alphabetical order — but redirect target lives // under /compare-per-dollar/. Query string is preserved across the hop. @@ -98,37 +138,46 @@ export default async function ComparePerDollarPage({ params, searchParams }: Pro }) .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`) .join('&'); - permanentRedirect(`/compare-per-dollar/${canonical}${qs ? `?${qs}` : ''}`); + // Keep the scenario segment across the redirect — dropping it would send + // the reader to the pair's default workload instead. + permanentRedirect(`${scenarioPath(canonical, scenarioSegment)}${qs ? `?${qs}` : ''}`); } - const rows = await getCachedBenchmarks(parsed.model.dbKeys); - const summaryA = summarize(rows, parsed.a); - const summaryB = summarize(rows, parsed.b); - const { sequence: pickedSequence, precision: pickedPrecision } = pickPairDefaults( - rows, - parsed.a, - parsed.b, - ); + const fallbackSequence = null; const urlSeq = pickString(sp.i_seq); const urlPrec = pickString(sp.i_prec); const urlModel = pickString(sp.g_model); - const effectiveSequence = urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : pickedSequence; - const effectivePrecision = urlPrec && KNOWN_PRECISIONS.has(urlPrec) ? urlPrec : pickedPrecision; const effectiveModel = urlModel && KNOWN_MODELS.has(urlModel) ? urlModel : parsed.model.displayName; - - const { defaultTargets, ssrRows, interactivityRange } = computeCompareTableData( - rows, + // Path beats query beats the pair's default: a scenario segment is an + // explicit address for one workload, so it outranks a stale `?i_seq=`. + const pathSequence = scenarioSegment ? sequenceForScenarioSegment(scenarioSegment) : null; + const requestedSequence = pathSequence ?? (urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : null); + const requestedPrecision = urlPrec && KNOWN_PRECISIONS.has(urlPrec) ? urlPrec : null; + const { + sequence: effectiveSequence, + precision: effectivePrecision, + summaryA, + summaryB, + defaultTargets, + ssrRows, + interactivityRange, + oldest, + newest, + initialPairBenchmarkRows, + } = await getComparePageDerivedData( + parsed.model.dbKeys, parsed.a, parsed.b, - effectiveSequence, - effectivePrecision, + requestedSequence, + requestedPrecision, + fallbackSequence, ); - const url = `${SITE_URL}/compare-per-dollar/${canonical}`; + const routePath = scenarioPath(canonical, scenarioSegment); + const url = `${SITE_URL}${routePath}`; const imageUrl = `${url}/performance-per-dollar.png`; - const { oldest, newest } = dateRangeForPair(rows, parsed.a, parsed.b); const jsonLd = buildJsonLd( 'per-dollar', parsed.model, @@ -182,7 +231,13 @@ export default async function ComparePerDollarPage({ params, searchParams }: Pro defaultSequence={effectiveSequence} defaultPrecision={effectivePrecision} ssrTableData={{ defaultTargets, ssrRows, interactivityRange }} + initialBenchmarkRows={initialCompareBenchmarkRows( + parsed.model.displayName, + effectiveModel, + initialPairBenchmarkRows, + )} narrative={narrative} + agenticIntro={isAgenticSequence(effectiveSequence) ? AGENTIC_SCENARIO_INTRO : null} aLabel={aLabel} bLabel={bLabel} aVendor={aMeta?.vendor ?? ''} diff --git a/packages/app/src/app/compare-per-dollar/[slug]/performance-per-dollar.png/route.tsx b/packages/app/src/app/compare-per-dollar/[slug]/performance-per-dollar.png/route.tsx index 9d3fad942..bc00c051b 100644 --- a/packages/app/src/app/compare-per-dollar/[slug]/performance-per-dollar.png/route.tsx +++ b/packages/app/src/app/compare-per-dollar/[slug]/performance-per-dollar.png/route.tsx @@ -1,8 +1,3 @@ -import { readFile } from 'node:fs/promises'; -import { join } from 'node:path'; - -import { ImageResponse } from 'next/og'; - import { HW_REGISTRY } from '@semianalysisai/inferencex-constants'; import { trackServer } from '@/lib/analytics-server'; @@ -12,99 +7,13 @@ import { computeCompareImageRows, computeCompareTableData, getCachedBenchmarks, - type SsrInterpolatedRow, } from '@/lib/compare-ssr'; +import { getLogoSrc } from '@/lib/og-assets'; +import { renderComparePngChart } from '@/lib/png-chart'; export const dynamic = 'force-dynamic'; export const runtime = 'nodejs'; -// Render natively at high-DPI. CSS `transform: scale()` causes Satori to rasterize -// SVG/text at the source size and bitmap-upsample, which produces a blurry chart. -// Multiplying every pixel value by R keeps glyphs and strokes as vectors at full res. -const R = 2; -const SIZE = { width: 1200 * R, height: 675 * R }; -const CHART_FRAME = { left: 0, top: 18 * R, width: 746 * R, height: 382 * R }; -const CHART = { left: 96 * R, top: 42 * R, width: 630 * R, height: 260 * R }; -const COLORS = { - background: '#0d1117', - panel: '#121a23', - border: '#23303d', - muted: '#9aa7b5', - faint: '#5f6e7d', - text: '#f3f7fb', - a: '#38d9a9', - b: '#f7b041', - grid: '#263544', - blue: '#0b86d1', -}; - -interface Point { - x: number; - y: number; -} - -interface TargetedPoint extends Point { - target: number; -} - -function money(value: number): string { - if (value >= 10) return `$${value.toFixed(1)}`; - if (value >= 1) return `$${value.toFixed(2)}`; - return `$${value.toFixed(3)}`; -} - -/** Decimals chosen from the tick step so every label in the axis prints with - * the same precision (no $0.000/$9.01/$18.0 mix). */ -function decimalsForStep(step: number): number { - if (step >= 1) return 0; - return Math.max(0, Math.ceil(-Math.log10(step))); -} - -function moneyForStep(value: number, step: number): string { - return `$${value.toFixed(decimalsForStep(step))}`; -} - -/** "Nice" step in the 1/2/5 × 10ⁿ family, the same convention d3 uses. */ -function niceStep(span: number, targetCount: number): number { - const rawStep = span / Math.max(1, targetCount - 1); - const mag = 10 ** Math.floor(Math.log10(rawStep)); - const normalized = rawStep / mag; - if (normalized < 1.5) return mag; - if (normalized < 3) return 2 * mag; - if (normalized < 7) return 5 * mag; - return 10 * mag; -} - -function niceAxis( - min: number, - max: number, - targetCount = 5, -): { min: number; max: number; step: number; ticks: number[] } { - if (max <= min) return { min, max: min + 1, step: 1, ticks: [min] }; - const step = niceStep(max - min, targetCount); - const niceMin = Math.floor(min / step) * step; - const niceMax = Math.ceil(max / step) * step; - const ticks: number[] = []; - for (let t = niceMin; t <= niceMax + step * 1e-6; t += step) { - ticks.push(Number(t.toFixed(10))); - } - return { min: niceMin, max: niceMax, step, ticks }; -} - -function pointsPath(points: Point[]): string { - return points.map((point, index) => `${index === 0 ? 'M' : 'L'} ${point.x} ${point.y}`).join(' '); -} - -let logoSrcPromise: Promise | undefined; -function getLogoSrc(): Promise { - if (!logoSrcPromise) { - logoSrcPromise = readFile(join(process.cwd(), 'public/brand/logo-color.png')) - .then((buf) => `data:image/png;base64,${buf.toString('base64')}`) - .catch(() => null); - } - return logoSrcPromise; -} - export async function GET( _request: Request, { params }: { params: Promise<{ slug: string }> }, @@ -138,431 +47,30 @@ export async function GET( sequence, precision, interactivityRange, - plottedRows.map((r) => r.target), + plottedRows.map((row) => row.target), ).filter((row) => row.a || row.b); const curveRows = imageRows.length > 0 ? imageRows : plottedRows; const aLabel = HW_REGISTRY[parsed.a]?.label ?? parsed.a.toUpperCase(); const bLabel = HW_REGISTRY[parsed.b]?.label ?? parsed.b.toUpperCase(); - const costs = curveRows - .flatMap((row) => [row.a?.cost, row.b?.cost]) - .filter((cost): cost is number => typeof cost === 'number' && Number.isFinite(cost)); - const costMin = costs.length > 0 ? Math.min(...costs) : 0; - const costMax = costs.length > 0 ? Math.max(...costs) : 1; - const yAxis = niceAxis(Math.min(0, costMin), costMax); - const yMin = yAxis.min; - const yMax = yAxis.max; - const yStep = yAxis.step; - const xMin = curveRows.at(0)?.target ?? 0; - const xMax = curveRows.at(-1)?.target ?? 100; - const matchedMin = plottedRows.at(0)?.target ?? xMin; - const matchedMax = plottedRows.at(-1)?.target ?? xMax; - const hasLeftExtension = matchedMin - xMin >= 0.5; - const hasRightExtension = xMax - matchedMax >= 0.5; - const scaleX = (value: number) => - CHART.left + (xMax === xMin ? CHART.width / 2 : ((value - xMin) / (xMax - xMin)) * CHART.width); - const scaleY = (value: number) => - CHART.top + - CHART.height - - (yMax === yMin ? CHART.height / 2 : ((value - yMin) / (yMax - yMin)) * CHART.height); - - function buildSeriesPoints(getCost: (row: SsrInterpolatedRow) => number | null): TargetedPoint[] { - return curveRows - .map((row) => ({ target: row.target, cost: getCost(row) })) - .filter((p): p is { target: number; cost: number } => p.cost !== null) - .map((p) => ({ x: scaleX(p.target), y: scaleY(p.cost), target: p.target })); - } - - function splitByMatchRange(points: TargetedPoint[]) { - return { - matched: points.filter((p) => p.target >= matchedMin && p.target <= matchedMax), - leftExt: points.filter((p) => p.target <= matchedMin), - rightExt: points.filter((p) => p.target >= matchedMax), - }; - } - - const aSeries = splitByMatchRange(buildSeriesPoints((r) => r.a?.cost ?? null)); - const bSeries = splitByMatchRange(buildSeriesPoints((r) => r.b?.cost ?? null)); - const aHighlightPoints = plottedRows - .filter((row) => row.a) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.a!.cost) })); - const bHighlightPoints = plottedRows - .filter((row) => row.b) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.b!.cost) })); const workload = [sequence, precision?.toUpperCase()].filter(Boolean).join(' / '); - const showRangeEndpoints = hasLeftExtension || hasRightExtension; - const svgWidth = 760 * R; - const svgHeight = 406 * R; - - function renderSeriesPath(points: Point[], stroke: string, dashed: boolean) { - if (points.length < 2) return null; - return ( - - ); - } try { - return new ImageResponse( -
-
-
-
- InferenceX Performance per Dollar -
-
- {parsed.model.label} -
-
- {aLabel} vs {bLabel} | Cost per Million Tokens -
-
-
-
- DEFAULT WORKLOAD -
-
- {workload || 'Default comparison'} -
-
- Lower cost is better -
-
-
- -
-
- - - {yAxis.ticks.map((tick) => { - const y = scaleY(tick); - return ( - - ); - })} - {plottedRows.map((row) => { - const x = scaleX(row.target); - return ( - - ); - })} - {renderSeriesPath(aSeries.leftExt, COLORS.a, true)} - {renderSeriesPath(aSeries.rightExt, COLORS.a, true)} - {renderSeriesPath(aSeries.matched, COLORS.a, false)} - {renderSeriesPath(bSeries.leftExt, COLORS.b, true)} - {renderSeriesPath(bSeries.rightExt, COLORS.b, true)} - {renderSeriesPath(bSeries.matched, COLORS.b, false)} - {aHighlightPoints.map((point, index) => ( - - ))} - {bHighlightPoints.map((point, index) => ( - - ))} - - {logoSrc && ( - - )} - {yAxis.ticks.map((tick) => ( -
- {moneyForStep(tick, yStep)} -
- ))} - {plottedRows.map((row) => ( -
- {row.target} -
- ))} - {showRangeEndpoints && hasLeftExtension && ( -
- {Math.round(xMin)} -
- )} - {showRangeEndpoints && hasRightExtension && ( -
- {Math.round(xMax)} -
- )} -
- Interactivity (tok/s/user) -
- {showRangeEndpoints && ( -
- Dashed segments extend to each SKU's operating envelope, where cost rises steeply -
- )} -
- -
-
- Matched Interactivity -
-
- - - {aLabel} - - - - {bLabel} - -
- {plottedRows.length > 0 ? ( - plottedRows.map((row) => ( -
-
- {row.target} tok/s/user -
-
- - {row.a ? money(row.a.cost) : 'N/A'} - - - {row.b ? money(row.b.cost) : 'N/A'} - -
-
- )) - ) : ( -
- No matched cost data available. -
- )} -
-
- -
- - Owning-hyperscaler TCO | interpolated from benchmark results - - - inferencex.semianalysis.com - -
-
, - { - ...SIZE, - headers: { - 'Cache-Control': 'public, s-maxage=3600, stale-while-revalidate=86400', - }, - }, - ); + return renderComparePngChart({ + curveRows, + plottedRows, + logoSrc, + aLabel, + bLabel, + eyebrow: 'InferenceX Performance per Dollar', + title: parsed.model.label, + subtitle: `${aLabel} vs ${bLabel} | Cost per Million Tokens`, + workload, + rangeNote: + "Dashed segments extend to each SKU's operating envelope, where cost rises steeply", + footer: 'Owning-hyperscaler TCO | interpolated from benchmark results', + }); } catch (error) { - // Satori can throw on a font fetch failure, malformed JSX layout, or a - // dataset that produces NaN/Infinity geometry. Capture which slug broke so - // we can find broken categories before a crawler hits them — without this, - // failures only surface as opaque Vercel 500s. const message = error instanceof Error ? error.message : String(error); trackServer('compare_per_dollar_png_render_failed', { slug, @@ -574,9 +82,6 @@ export async function GET( error_name: error instanceof Error ? error.name : 'Unknown', error_message: message.slice(0, 500), }); - // 502 (not 500): the route itself is reachable, the downstream renderer - // failed. Short cache so a retry within the hour pulls a fixed render - // instead of pinning the failure. return new Response('PNG render failed', { status: 502, headers: { 'Cache-Control': 'public, s-maxage=60' }, diff --git a/packages/app/src/app/compare-per-dollar/layout.tsx b/packages/app/src/app/compare-per-dollar/layout.tsx index c4e14ab1a..4ff2b2492 100644 --- a/packages/app/src/app/compare-per-dollar/layout.tsx +++ b/packages/app/src/app/compare-per-dollar/layout.tsx @@ -1,20 +1,5 @@ -import { UnofficialRunProvider } from '@/components/unofficial-run-provider'; +import { ComparePageLayout } from '@/components/compare/compare-page-layout'; -/** - * Wraps `/compare-per-dollar/*` pages in UnofficialRunProvider but skips - * DashboardShell so we get a focused single-purpose page (no TabNav). Mirrors - * the structure of `/compare/layout.tsx` — the GlobalFilterProvider is mounted - * inside the page's client component so it can seed initial precision/sequence - * based on which combos actually have data for the GPU pair from the slug. - */ export default function ComparePerDollarLayout({ children }: { children: React.ReactNode }) { - return ( - -
-
- {children} -
-
-
- ); + return {children}; } diff --git a/packages/app/src/app/compare-per-dollar/page.tsx b/packages/app/src/app/compare-per-dollar/page.tsx index 1a2e24ed2..de359e045 100644 --- a/packages/app/src/app/compare-per-dollar/page.tsx +++ b/packages/app/src/app/compare-per-dollar/page.tsx @@ -13,6 +13,7 @@ import { enAlternates } from '@/lib/i18n'; import { ComparePairCardLink } from '@/components/compare/compare-pair-card-link'; import { JsonLd } from '@/components/json-ld'; import { Card } from '@/components/ui/card'; +import { comparisonPairHref, comparisonScenarioForModel } from '@/lib/compare-agentx'; import { getComparablePairsByModelSlug } from '@/lib/compare-availability'; import { type ComparePair, COMPARE_MODEL_SLUGS, type CompareModelSlug } from '@/lib/compare-slug'; import { bucketComparePairsByVendor, formatModelList } from '@/lib/compare-ssr'; @@ -108,6 +109,11 @@ export default async function ComparePerDollarIndexPage() { each page renders the cost-per-token chart and an interpolated dollars-per-million comparison table so you can pick the cheaper SKU at any target interactivity level.

+

+ Models with AgentX data open long-context, multi-turn trace replay results. Models not + yet covered by AgentX open the controlled 8K→1K workload. Each card identifies its + scenario. +

{ const pairs = comparablePairsByModel.get(model.slug) ?? []; const groups = groupPairsByVendorForModel(model, pairs); + // Same scenario split /compare uses: models with AgentX data open the + // agentic trace replay, the rest open the fixed 8K→1K workload. + const scenario = comparisonScenarioForModel(model); return (
@@ -170,10 +179,11 @@ export default async function ComparePerDollarIndexPage() { return ( ); })} diff --git a/packages/app/src/app/compare-precision/[slug]/[scenario]/page.tsx b/packages/app/src/app/compare-precision/[slug]/[scenario]/page.tsx new file mode 100644 index 000000000..f709b74a6 --- /dev/null +++ b/packages/app/src/app/compare-precision/[slug]/[scenario]/page.tsx @@ -0,0 +1,37 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { type ScenarioSegment, sequenceForScenarioSegment } from '@/lib/compare-scenario-route'; + +import { buildPrecisionMetadata, renderPrecisionPage } from '../page'; + +/** + * `/compare-precision//` — the same comparison with its workload pinned by + * the path instead of `?i_seq=`. Both routes render through the parent's + * exported body, so the two URLs cannot drift. + */ +export const dynamic = 'force-dynamic'; + +interface Props { + params: Promise<{ slug: string; scenario: string }>; + searchParams: Promise>; +} + +/** Reject unknown segments rather than silently falling back to the pair's + * default, which would serve 200s for arbitrary junk paths. */ +function assertScenario(scenario: string): ScenarioSegment { + if (!sequenceForScenarioSegment(scenario)) notFound(); + return scenario.toLowerCase() as ScenarioSegment; +} + +export async function generateMetadata({ params }: Props): Promise { + const { slug, scenario } = await params; + return buildPrecisionMetadata(slug, { scenarioSegment: assertScenario(scenario) }); +} + +export default async function PrecisionScenarioPage({ params, searchParams }: Props) { + const { slug, scenario } = await params; + return renderPrecisionPage(slug, await searchParams, { + scenarioSegment: assertScenario(scenario), + }); +} diff --git a/packages/app/src/app/compare-precision/[slug]/opengraph-image.tsx b/packages/app/src/app/compare-precision/[slug]/opengraph-image.tsx index b6a65d259..3b54414ce 100644 --- a/packages/app/src/app/compare-precision/[slug]/opengraph-image.tsx +++ b/packages/app/src/app/compare-precision/[slug]/opengraph-image.tsx @@ -1,20 +1,14 @@ -/** - * Precision compare OG image — identical circuit tile sidebar layout to the - * /compare OG and the blog OG, but the eyebrow and bottom banner surface - * the "Precision Comparison" framing. - */ import { notFound } from 'next/navigation'; -import { ImageResponse } from 'next/og'; import { HW_REGISTRY } from '@semianalysisai/inferencex-constants'; +import { renderCompareOg } from '@/lib/compare-og'; import { getAllComparablePrecisionSlugs } from '@/lib/compare-variant-availability'; import { canonicalPrecisionCompareSlug, parsePrecisionCompareSlug, precisionDisplayLabel, } from '@/lib/compare-variant-slug'; -import { BG, BLUE, getLogoSrc, getTiles, PANEL_BG } from '@/lib/og-assets'; export const alt = 'Chip precision inference benchmark comparison'; export const size = { width: 1200, height: 630 }; @@ -30,9 +24,6 @@ export async function generateStaticParams() { export default async function OgImage({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params; const parsed = parsePrecisionCompareSlug(slug); - // Reject non-canonical slugs (reversed order, aliases, mixed case) so OG - // URLs can't render labels that disagree with the canonical page — same - // strictness as the sibling PNG route. if ( !parsed || canonicalPrecisionCompareSlug(parsed.model.slug, parsed.gpu, parsed.precA, parsed.precB) !== @@ -40,127 +31,19 @@ export default async function OgImage({ params }: { params: Promise<{ slug: stri ) { notFound(); } - const [logoSrc, tiles] = await Promise.all([getLogoSrc(), getTiles()]); - const gpuMeta = HW_REGISTRY[parsed.gpu]; - const gpuLabel = gpuMeta?.label ?? parsed.gpu.toUpperCase(); + const gpuLabel = HW_REGISTRY[parsed.gpu]?.label ?? parsed.gpu.toUpperCase(); const aLabel = precisionDisplayLabel(parsed.precA); const bLabel = precisionDisplayLabel(parsed.precB); const title = `${gpuLabel}: ${aLabel} vs ${bLabel}`; - const eyebrow = `${parsed.model.label} · Precision Comparison`; const titleSize = title.length > 26 ? 80 : title.length > 18 ? 96 : 112; - return new ImageResponse( -
-
- {tiles.map((tile, i) => { - if (!tile) return null; - const row = Math.floor(i / 2); - const col = i % 2; - return ( - - ); - })} -
-
- -
-
-
- {eyebrow} -
- -
- {title} -
-
- -
- - AI inference benchmark · precision comparison - - {logoSrc && } -
-
-
, + return renderCompareOg({ + eyebrow: `${parsed.model.label} · Precision Comparison`, + title, + titleSize, + footer: 'AI inference benchmark · precision comparison', + language: 'en', size, - ); + }); } diff --git a/packages/app/src/app/compare-precision/[slug]/page-client.tsx b/packages/app/src/app/compare-precision/[slug]/page-client.tsx index f7cd67409..11e2704f4 100644 --- a/packages/app/src/app/compare-precision/[slug]/page-client.tsx +++ b/packages/app/src/app/compare-precision/[slug]/page-client.tsx @@ -3,21 +3,23 @@ import Link from 'next/link'; import { useEffect, useMemo } from 'react'; -import type { GPUDataPoint, InterpolatedResult } from '@/components/calculator/types'; +import type { GPUDataPoint } from '@/components/calculator/types'; import { useThroughputData } from '@/components/calculator/useThroughputData'; -import { CompareInterpolatedTable } from '@/components/compare/compare-interpolated-table'; -import { useGlobalFilters, GlobalFilterProvider } from '@/components/GlobalFilterContext'; +import { + CompareTableRenderer, + type CompareTableData, +} from '@/components/compare/compare-table-renderer'; +import { + GlobalFilterProvider, + useGlobalFilterRun, + useGlobalFilterSelection, +} from '@/components/GlobalFilterContext'; import { InferenceProvider } from '@/components/inference/InferenceContext'; import InferenceChartDisplay from '@/components/inference/ui/ChartDisplay'; import { Card } from '@/components/ui/card'; import { track } from '@/lib/analytics'; import { toModel, toSequence } from '@/lib/compare-enum-coerce'; - -interface SsrTableData { - defaultTargets: number[]; - ssrRows: { target: number; a: InterpolatedResult | null; b: InterpolatedResult | null }[]; - interactivityRange: { min: number; max: number }; -} +import type { AgenticScenarioIntro } from '@/lib/compare-ssr'; const STRINGS = { en: { @@ -50,8 +52,11 @@ interface ComparePrecisionPageClientProps { defaultSequence: string | null; precA: string; precB: string; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; narrative: string[]; + /** Set only when the page is rendering the agentic workload. Explains + * what AgentX measures before the head-to-head numbers. */ + agenticIntro?: AgenticScenarioIntro | null; gpuLabel: string; gpuVendor: string; gpuArch: string; @@ -71,6 +76,7 @@ export default function ComparePrecisionPageClient({ precB, ssrTableData, narrative, + agenticIntro = null, gpuLabel, gpuVendor, gpuArch, @@ -135,6 +141,21 @@ export default function ComparePrecisionPageClient({ .

)} + {agenticIntro && ( +

+ {agenticIntro.paragraph}{' '} + + {agenticIntro.linkLabel} → + +

+ )} {narrative.length > 0 && (
- {emptyStateText} -
- ); - } - return ( - ); } diff --git a/packages/app/src/app/compare-precision/[slug]/page.tsx b/packages/app/src/app/compare-precision/[slug]/page.tsx index 23f4922b9..b0f8b0e78 100644 --- a/packages/app/src/app/compare-precision/[slug]/page.tsx +++ b/packages/app/src/app/compare-precision/[slug]/page.tsx @@ -9,8 +9,18 @@ import { } from '@semianalysisai/inferencex-constants'; import { JsonLd } from '@/components/json-ld'; +import { + isAgenticSequence, + type ScenarioSegment, + sequenceForScenarioSegment, +} from '@/lib/compare-scenario-route'; import { languageAlternates } from '@/lib/i18n'; -import { getCachedBenchmarks, KNOWN_SEQUENCES, pickString } from '@/lib/compare-ssr'; +import { + AGENTIC_SCENARIO_INTRO, + getCachedBenchmarks, + KNOWN_SEQUENCES, + pickString, +} from '@/lib/compare-ssr'; import { canonicalPrecisionCompareSlug, parsePrecisionCompareSlug, @@ -30,6 +40,22 @@ import ComparePrecisionPageClient from './page-client'; export const dynamic = 'force-dynamic'; +/** + * `/compare-precision//` renders this same page with the workload + * pinned by the path. The segment is threaded through rather than duplicated + * in a parallel route file so the body — redirects, JSON-LD, metadata, client + * props — is written once and cannot drift between the two URLs. + */ +export interface ScenarioOptions { + scenarioSegment?: ScenarioSegment; +} + +function scenarioPath(canonical: string, scenarioSegment?: ScenarioSegment): string { + return scenarioSegment + ? `/compare-precision/${canonical}/${scenarioSegment}` + : `/compare-precision/${canonical}`; +} + interface Props { params: Promise<{ slug: string }>; searchParams: Promise>; @@ -37,6 +63,13 @@ interface Props { export async function generateMetadata({ params }: Props): Promise { const { slug } = await params; + return buildPrecisionMetadata(slug, {}); +} + +export function buildPrecisionMetadata( + slug: string, + { scenarioSegment }: ScenarioOptions, +): Metadata { const parsed = parsePrecisionCompareSlug(slug); if (!parsed) return {}; const gpuMeta = HW_REGISTRY[parsed.gpu]; @@ -49,14 +82,21 @@ export async function generateMetadata({ params }: Props): Promise { parsed.precA, parsed.precB, ); - const url = `${SITE_URL}/compare-precision/${canonical}`; + // The scenario segments are views of one comparison, so the bare slug URL + // stays the indexable representative and every segment canonicalizes to it. + // Without this, `/…//` and `/…/` would serve + // byte-identical pages, each claiming to be canonical. + const routePath = scenarioPath(canonical, scenarioSegment); + const canonicalPath = scenarioPath(canonical); + const url = `${SITE_URL}${routePath}`; + const canonicalUrl = `${SITE_URL}${canonicalPath}`; const description = `${gpuLabel} precision comparison of ${aLabel} versus ${bLabel} on ${parsed.model.label}: verified, reproducible results from InferenceX, the independent open-source benchmark by SemiAnalysis. ${SUPPORTERS_LINE} See which quantization level delivers better throughput and cost at every interactivity level.`; return { title: `${parsed.model.label} — ${gpuLabel} ${aLabel} vs ${bLabel} — Precision Comparison`, description, alternates: { - canonical: url, - languages: languageAlternates(`/compare-precision/${canonical}`), + canonical: canonicalUrl, + languages: languageAlternates(canonicalPath), }, openGraph: { title: `${parsed.model.label} — ${gpuLabel} ${aLabel} vs ${bLabel} — Precision Comparison | ${SITE_NAME}`, @@ -74,11 +114,17 @@ export async function generateMetadata({ params }: Props): Promise { export default async function ComparePrecisionPage({ params, searchParams }: Props) { const { slug } = await params; + return renderPrecisionPage(slug, await searchParams, {}); +} + +export async function renderPrecisionPage( + slug: string, + sp: Record, + { scenarioSegment }: ScenarioOptions, +) { const parsed = parsePrecisionCompareSlug(slug); if (!parsed) notFound(); - const sp = await searchParams; - // 308 redirect to canonical — normalizes alias models and precision order. const canonical = canonicalPrecisionCompareSlug( parsed.model.slug, @@ -95,7 +141,9 @@ export default async function ComparePrecisionPage({ params, searchParams }: Pro }) .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`) .join('&'); - permanentRedirect(`/compare-precision/${canonical}${qs ? `?${qs}` : ''}`); + // Keep the scenario segment across the redirect — dropping it would send + // the reader to the pair's default workload instead. + permanentRedirect(`${scenarioPath(canonical, scenarioSegment)}${qs ? `?${qs}` : ''}`); } const rows = await getCachedBenchmarks(parsed.model.dbKeys); @@ -111,7 +159,11 @@ export default async function ComparePrecisionPage({ params, searchParams }: Pro ); const urlSeq = pickString(sp.i_seq); - const effectiveSequence = urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : pickedSequence; + // Path beats query beats the pair's default: a scenario segment is an + // explicit address for one workload, so it outranks a stale `?i_seq=`. + const pathSequence = scenarioSegment ? sequenceForScenarioSegment(scenarioSegment) : null; + const effectiveSequence = + pathSequence ?? (urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : pickedSequence); const { defaultTargets, ssrRows, interactivityRange } = computeVariantCompareTableData( rows, @@ -129,7 +181,7 @@ export default async function ComparePrecisionPage({ params, searchParams }: Pro const aLabel = precisionDisplayLabel(parsed.precA); const bLabel = precisionDisplayLabel(parsed.precB); - const url = `${SITE_URL}/compare-precision/${canonical}`; + const url = `${SITE_URL}${scenarioPath(canonical, scenarioSegment)}`; const imageUrl = `${url}/precision-comparison.png`; const { oldest, newest } = dateRangeForVariantPair(rows, parsed.gpu, sideA, sideB); const jsonLd = buildVariantJsonLd( @@ -172,6 +224,7 @@ export default async function ComparePrecisionPage({ params, searchParams }: Pro precB={parsed.precB} ssrTableData={{ defaultTargets, ssrRows, interactivityRange }} narrative={narrative} + agenticIntro={isAgenticSequence(effectiveSequence) ? AGENTIC_SCENARIO_INTRO : null} gpuLabel={gpuLabel} gpuVendor={gpuMeta?.vendor ?? ''} gpuArch={gpuMeta?.arch ?? ''} diff --git a/packages/app/src/app/compare-precision/[slug]/precision-comparison.png/route.tsx b/packages/app/src/app/compare-precision/[slug]/precision-comparison.png/route.tsx index 95552ac1c..f081f33d4 100644 --- a/packages/app/src/app/compare-precision/[slug]/precision-comparison.png/route.tsx +++ b/packages/app/src/app/compare-precision/[slug]/precision-comparison.png/route.tsx @@ -1,5 +1,3 @@ -import { ImageResponse } from 'next/og'; - import { HW_REGISTRY } from '@semianalysisai/inferencex-constants'; import { trackServer } from '@/lib/analytics-server'; @@ -15,19 +13,7 @@ import { precisionDisplayLabel, } from '@/lib/compare-variant-slug'; import { getLogoSrc } from '@/lib/og-assets'; -import { - buildSeriesPoints, - CHART, - CHART_FRAME, - COLORS, - money, - moneyForStep, - niceAxis, - R, - renderSeriesPath, - SIZE, - splitByMatchRange, -} from '@/lib/png-chart'; +import { renderComparePngChart } from '@/lib/png-chart'; export const dynamic = 'force-dynamic'; export const runtime = 'nodejs'; @@ -38,8 +24,8 @@ export async function GET( ): Promise { const { slug } = await params; const parsed = parsePrecisionCompareSlug(slug); - // Compare case-insensitively — the HTML pages lowercase before their 308, - // so a mixed-case PNG URL should serve rather than 404. + // Compare case-insensitively. The HTML pages lowercase before their 308, so + // a mixed-case PNG URL should serve rather than 404. if ( !parsed || canonicalPrecisionCompareSlug(parsed.model.slug, parsed.gpu, parsed.precA, parsed.precB) !== @@ -50,12 +36,10 @@ export async function GET( const sideA = { precision: parsed.precA }; const sideB = { precision: parsed.precB }; - const [rows, logoSrc] = await Promise.all([ getCachedBenchmarks(parsed.model.dbKeys), getLogoSrc(), ]); - const { sequence } = pickVariantPairDefaults('precision', rows, parsed.gpu, sideA, sideB); const { ssrRows, interactivityRange } = computeVariantCompareTableData( rows, @@ -72,7 +56,7 @@ export async function GET( sideA, sideB, interactivityRange, - plottedRows.map((r) => r.target), + plottedRows.map((row) => row.target), ).filter((row) => row.a || row.b); const curveRows = imageRows.length > 0 ? imageRows : plottedRows; @@ -80,397 +64,21 @@ export async function GET( const gpuLabel = gpuMeta?.label ?? parsed.gpu.toUpperCase(); const aLabel = precisionDisplayLabel(parsed.precA); const bLabel = precisionDisplayLabel(parsed.precB); - const costs = curveRows - .flatMap((row) => [row.a?.cost, row.b?.cost]) - .filter((cost): cost is number => typeof cost === 'number' && Number.isFinite(cost)); - const costMin = costs.length > 0 ? Math.min(...costs) : 0; - const costMax = costs.length > 0 ? Math.max(...costs) : 1; - const yAxis = niceAxis(Math.min(0, costMin), costMax); - const yMin = yAxis.min; - const yMax = yAxis.max; - const yStep = yAxis.step; - const xMin = curveRows.at(0)?.target ?? 0; - const xMax = curveRows.at(-1)?.target ?? 100; - const matchedMin = plottedRows.at(0)?.target ?? xMin; - const matchedMax = plottedRows.at(-1)?.target ?? xMax; - const hasLeftExtension = matchedMin - xMin >= 0.5; - const hasRightExtension = xMax - matchedMax >= 0.5; - const scaleX = (value: number) => - CHART.left + (xMax === xMin ? CHART.width / 2 : ((value - xMin) / (xMax - xMin)) * CHART.width); - const scaleY = (value: number) => - CHART.top + - CHART.height - - (yMax === yMin ? CHART.height / 2 : ((value - yMin) / (yMax - yMin)) * CHART.height); - - const aSeries = splitByMatchRange( - buildSeriesPoints(curveRows, (r) => r.a?.cost ?? null, scaleX, scaleY), - matchedMin, - matchedMax, - ); - const bSeries = splitByMatchRange( - buildSeriesPoints(curveRows, (r) => r.b?.cost ?? null, scaleX, scaleY), - matchedMin, - matchedMax, - ); - const aHighlightPoints = plottedRows - .filter((row) => row.a) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.a!.cost) })); - const bHighlightPoints = plottedRows - .filter((row) => row.b) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.b!.cost) })); - const workload = sequence ?? ''; - const showRangeEndpoints = hasLeftExtension || hasRightExtension; - const svgWidth = 760 * R; - const svgHeight = 406 * R; try { - return new ImageResponse( -
-
-
-
- InferenceX Precision Comparison -
-
- {parsed.model.label} · {gpuLabel} -
-
- {aLabel} vs {bLabel} | Cost per Million Tokens -
-
-
-
- DEFAULT WORKLOAD -
-
- {workload || 'Default comparison'} -
-
- Lower cost is better -
-
-
- -
-
- - - {yAxis.ticks.map((tick) => { - const y = scaleY(tick); - return ( - - ); - })} - {plottedRows.map((row) => { - const x = scaleX(row.target); - return ( - - ); - })} - {renderSeriesPath(aSeries.leftExt, COLORS.a, true)} - {renderSeriesPath(aSeries.rightExt, COLORS.a, true)} - {renderSeriesPath(aSeries.matched, COLORS.a, false)} - {renderSeriesPath(bSeries.leftExt, COLORS.b, true)} - {renderSeriesPath(bSeries.rightExt, COLORS.b, true)} - {renderSeriesPath(bSeries.matched, COLORS.b, false)} - {aHighlightPoints.map((point, index) => ( - - ))} - {bHighlightPoints.map((point, index) => ( - - ))} - - {logoSrc && ( - - )} - {yAxis.ticks.map((tick) => ( -
- {moneyForStep(tick, yStep)} -
- ))} - {plottedRows.map((row) => ( -
- {row.target} -
- ))} - {showRangeEndpoints && hasLeftExtension && ( -
- {Math.round(xMin)} -
- )} - {showRangeEndpoints && hasRightExtension && ( -
- {Math.round(xMax)} -
- )} -
- Interactivity (tok/s/user) -
- {showRangeEndpoints && ( -
- Dashed segments extend to each precision's operating envelope -
- )} -
- -
-
- Matched Interactivity -
-
- - - {aLabel} - - - - {bLabel} - -
- {plottedRows.length > 0 ? ( - plottedRows.map((row) => ( -
-
- {row.target} tok/s/user -
-
- - {row.a ? money(row.a.cost) : 'N/A'} - - - {row.b ? money(row.b.cost) : 'N/A'} - -
-
- )) - ) : ( -
- No matched cost data available. -
- )} -
-
- -
- - Precision comparison | interpolated from benchmark results - - - inferencex.semianalysis.com - -
-
, - { - ...SIZE, - headers: { - 'Cache-Control': 'public, s-maxage=3600, stale-while-revalidate=86400', - }, - }, - ); + return renderComparePngChart({ + curveRows, + plottedRows, + logoSrc, + aLabel, + bLabel, + eyebrow: 'InferenceX Precision Comparison', + title: `${parsed.model.label} · ${gpuLabel}`, + subtitle: `${aLabel} vs ${bLabel} | Cost per Million Tokens`, + workload: sequence ?? '', + rangeNote: "Dashed segments extend to each precision's operating envelope", + footer: 'Precision comparison | interpolated from benchmark results', + }); } catch (error) { const message = error instanceof Error ? error.message : String(error); trackServer('compare_precision_png_render_failed', { diff --git a/packages/app/src/app/compare-precision/layout.tsx b/packages/app/src/app/compare-precision/layout.tsx index 6ed48e39a..0706bbeb3 100644 --- a/packages/app/src/app/compare-precision/layout.tsx +++ b/packages/app/src/app/compare-precision/layout.tsx @@ -1,20 +1,5 @@ -import { UnofficialRunProvider } from '@/components/unofficial-run-provider'; +import { ComparePageLayout } from '@/components/compare/compare-page-layout'; -/** - * Wraps `/compare-precision/*` pages in UnofficialRunProvider but skips - * DashboardShell so we get a focused single-purpose page (no TabNav). Mirrors - * the structure of `/compare-per-dollar/layout.tsx` — the GlobalFilterProvider - * is mounted inside the page's client component so it can seed initial - * precision/sequence based on which combos actually have data from the slug. - */ export default function ComparePrecisionLayout({ children }: { children: React.ReactNode }) { - return ( - -
-
- {children} -
-
-
- ); + return {children}; } diff --git a/packages/app/src/app/compare-spec-decode/[slug]/[scenario]/page.tsx b/packages/app/src/app/compare-spec-decode/[slug]/[scenario]/page.tsx new file mode 100644 index 000000000..ed071d34c --- /dev/null +++ b/packages/app/src/app/compare-spec-decode/[slug]/[scenario]/page.tsx @@ -0,0 +1,37 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { type ScenarioSegment, sequenceForScenarioSegment } from '@/lib/compare-scenario-route'; + +import { buildSpecDecodeMetadata, renderSpecDecodePage } from '../page'; + +/** + * `/compare-spec-decode//` — the same comparison with its workload pinned by + * the path instead of `?i_seq=`. Both routes render through the parent's + * exported body, so the two URLs cannot drift. + */ +export const dynamic = 'force-dynamic'; + +interface Props { + params: Promise<{ slug: string; scenario: string }>; + searchParams: Promise>; +} + +/** Reject unknown segments rather than silently falling back to the pair's + * default, which would serve 200s for arbitrary junk paths. */ +function assertScenario(scenario: string): ScenarioSegment { + if (!sequenceForScenarioSegment(scenario)) notFound(); + return scenario.toLowerCase() as ScenarioSegment; +} + +export async function generateMetadata({ params }: Props): Promise { + const { slug, scenario } = await params; + return buildSpecDecodeMetadata(slug, { scenarioSegment: assertScenario(scenario) }); +} + +export default async function SpecDecodeScenarioPage({ params, searchParams }: Props) { + const { slug, scenario } = await params; + return renderSpecDecodePage(slug, await searchParams, { + scenarioSegment: assertScenario(scenario), + }); +} diff --git a/packages/app/src/app/compare-spec-decode/[slug]/opengraph-image.tsx b/packages/app/src/app/compare-spec-decode/[slug]/opengraph-image.tsx index aff210426..be4340a26 100644 --- a/packages/app/src/app/compare-spec-decode/[slug]/opengraph-image.tsx +++ b/packages/app/src/app/compare-spec-decode/[slug]/opengraph-image.tsx @@ -1,13 +1,8 @@ -/** - * Spec-decode compare OG image — identical circuit tile sidebar layout to the - * /compare OG and the blog OG, but the eyebrow and bottom banner are swapped - * to surface the "Speculative Decoding" framing. - */ import { notFound } from 'next/navigation'; -import { ImageResponse } from 'next/og'; import { HW_REGISTRY } from '@semianalysisai/inferencex-constants'; +import { renderCompareOg } from '@/lib/compare-og'; import { getAllComparableSpecDecodeSlugs } from '@/lib/compare-variant-availability'; import { canonicalSpecDecodeCompareSlug, @@ -15,7 +10,6 @@ import { precisionDisplayLabel, specMethodDisplayLabel, } from '@/lib/compare-variant-slug'; -import { BG, BLUE, getLogoSrc, getTiles, PANEL_BG } from '@/lib/og-assets'; export const alt = 'Chip speculative decoding inference benchmark comparison'; export const size = { width: 1200, height: 630 }; @@ -31,9 +25,6 @@ export async function generateStaticParams() { export default async function OgImage({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params; const parsed = parseSpecDecodeCompareSlug(slug); - // Reject non-canonical slugs (reversed order, aliases, mixed case) so OG - // URLs can't render labels that disagree with the canonical page — same - // strictness as the sibling PNG route. if ( !parsed || canonicalSpecDecodeCompareSlug( @@ -45,127 +36,19 @@ export default async function OgImage({ params }: { params: Promise<{ slug: stri ) { notFound(); } - const [logoSrc, tiles] = await Promise.all([getLogoSrc(), getTiles()]); - const gpuMeta = HW_REGISTRY[parsed.gpu]; - const gpuLabel = gpuMeta?.label ?? parsed.gpu.toUpperCase(); - const precLabel = precisionDisplayLabel(parsed.precision); - const aLabel = specMethodDisplayLabel(parsed.model.displayName, parsed.method); - const title = `${gpuLabel} ${precLabel}: ${aLabel} vs Off`; - const eyebrow = `${parsed.model.label} · Speculative Decoding`; + const gpuLabel = HW_REGISTRY[parsed.gpu]?.label ?? parsed.gpu.toUpperCase(); + const precisionLabel = precisionDisplayLabel(parsed.precision); + const methodLabel = specMethodDisplayLabel(parsed.model.displayName, parsed.method); + const title = `${gpuLabel} ${precisionLabel}: ${methodLabel} vs Off`; const titleSize = title.length > 34 ? 72 : title.length > 26 ? 80 : title.length > 18 ? 96 : 112; - return new ImageResponse( -
-
- {tiles.map((tile, i) => { - if (!tile) return null; - const row = Math.floor(i / 2); - const col = i % 2; - return ( - - ); - })} -
-
- -
-
-
- {eyebrow} -
- -
- {title} -
-
- -
- - AI inference benchmark · speculative decoding comparison - - {logoSrc && } -
-
-
, + return renderCompareOg({ + eyebrow: `${parsed.model.label} · Speculative Decoding`, + title, + titleSize, + footer: 'AI inference benchmark · speculative decoding comparison', + language: 'en', size, - ); + }); } diff --git a/packages/app/src/app/compare-spec-decode/[slug]/page-client.tsx b/packages/app/src/app/compare-spec-decode/[slug]/page-client.tsx index 4a9f11a6b..fff64ec8c 100644 --- a/packages/app/src/app/compare-spec-decode/[slug]/page-client.tsx +++ b/packages/app/src/app/compare-spec-decode/[slug]/page-client.tsx @@ -3,22 +3,24 @@ import Link from 'next/link'; import { useEffect, useMemo } from 'react'; -import type { GPUDataPoint, InterpolatedResult } from '@/components/calculator/types'; +import type { GPUDataPoint } from '@/components/calculator/types'; import { useThroughputData } from '@/components/calculator/useThroughputData'; -import { CompareInterpolatedTable } from '@/components/compare/compare-interpolated-table'; -import { useGlobalFilters, GlobalFilterProvider } from '@/components/GlobalFilterContext'; +import { + CompareTableRenderer, + type CompareTableData, +} from '@/components/compare/compare-table-renderer'; +import { + GlobalFilterProvider, + useGlobalFilterRun, + useGlobalFilterSelection, +} from '@/components/GlobalFilterContext'; import { InferenceProvider } from '@/components/inference/InferenceContext'; import InferenceChartDisplay from '@/components/inference/ui/ChartDisplay'; import { Card } from '@/components/ui/card'; import { track } from '@/lib/analytics'; import { toModel, toPrecisions, toSequence } from '@/lib/compare-enum-coerce'; import { SPEC_METHODS_ACTIVE } from '@/lib/compare-variant-slug'; - -interface SsrTableData { - defaultTargets: number[]; - ssrRows: { target: number; a: InterpolatedResult | null; b: InterpolatedResult | null }[]; - interactivityRange: { min: number; max: number }; -} +import type { AgenticScenarioIntro } from '@/lib/compare-ssr'; const STRINGS = { en: { @@ -62,8 +64,11 @@ interface CompareSpecDecodePageClientProps { modelDisplayName: string; defaultSequence: string | null; defaultPrecision: string | null; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; narrative: string[]; + /** Set only when the page is rendering the agentic workload. Explains + * what AgentX measures before the head-to-head numbers. */ + agenticIntro?: AgenticScenarioIntro | null; gpuLabel: string; precisionLabel: string; gpuArch: string; @@ -84,6 +89,7 @@ export default function CompareSpecDecodePageClient({ defaultPrecision, ssrTableData, narrative, + agenticIntro = null, gpuLabel, precisionLabel, gpuArch, @@ -163,6 +169,21 @@ export default function CompareSpecDecodePageClient({

+ {agenticIntro && ( +

+ {agenticIntro.paragraph}{' '} + + {agenticIntro.linkLabel} → + +

+ )} {narrative.length > 0 && (
- {emptyStateText} -
- ); - } - return ( - ); } diff --git a/packages/app/src/app/compare-spec-decode/[slug]/page.tsx b/packages/app/src/app/compare-spec-decode/[slug]/page.tsx index 352c14fb8..06b86055a 100644 --- a/packages/app/src/app/compare-spec-decode/[slug]/page.tsx +++ b/packages/app/src/app/compare-spec-decode/[slug]/page.tsx @@ -9,8 +9,18 @@ import { } from '@semianalysisai/inferencex-constants'; import { JsonLd } from '@/components/json-ld'; +import { + isAgenticSequence, + type ScenarioSegment, + sequenceForScenarioSegment, +} from '@/lib/compare-scenario-route'; import { languageAlternates } from '@/lib/i18n'; -import { getCachedBenchmarks, KNOWN_SEQUENCES, pickString } from '@/lib/compare-ssr'; +import { + AGENTIC_SCENARIO_INTRO, + getCachedBenchmarks, + KNOWN_SEQUENCES, + pickString, +} from '@/lib/compare-ssr'; import { canonicalSpecDecodeCompareSlug, parseSpecDecodeCompareSlug, @@ -32,6 +42,22 @@ import CompareSpecDecodePageClient from './page-client'; export const dynamic = 'force-dynamic'; +/** + * `/compare-spec-decode//` renders this same page with the workload + * pinned by the path. The segment is threaded through rather than duplicated + * in a parallel route file so the body — redirects, JSON-LD, metadata, client + * props — is written once and cannot drift between the two URLs. + */ +export interface ScenarioOptions { + scenarioSegment?: ScenarioSegment; +} + +function scenarioPath(canonical: string, scenarioSegment?: ScenarioSegment): string { + return scenarioSegment + ? `/compare-spec-decode/${canonical}/${scenarioSegment}` + : `/compare-spec-decode/${canonical}`; +} + interface Props { params: Promise<{ slug: string }>; searchParams: Promise>; @@ -39,6 +65,13 @@ interface Props { export async function generateMetadata({ params }: Props): Promise { const { slug } = await params; + return buildSpecDecodeMetadata(slug, {}); +} + +export function buildSpecDecodeMetadata( + slug: string, + { scenarioSegment }: ScenarioOptions, +): Metadata { const parsed = parseSpecDecodeCompareSlug(slug); if (!parsed) return {}; const gpuMeta = HW_REGISTRY[parsed.gpu]; @@ -51,15 +84,22 @@ export async function generateMetadata({ params }: Props): Promise { parsed.precision, parsed.method, ); - const url = `${SITE_URL}/compare-spec-decode/${canonical}`; + // The scenario segments are views of one comparison, so the bare slug URL + // stays the indexable representative and every segment canonicalizes to it. + // Without this, `/…//` and `/…/` would serve + // byte-identical pages, each claiming to be canonical. + const routePath = scenarioPath(canonical, scenarioSegment); + const canonicalPath = scenarioPath(canonical); + const url = `${SITE_URL}${routePath}`; + const canonicalUrl = `${SITE_URL}${canonicalPath}`; const description = `${parsed.model.label} on ${gpuLabel} ${precLabel}: ${aLabel} vs Off speculative decoding comparison. Verified, reproducible results from InferenceX, the independent open-source benchmark by SemiAnalysis. ${SUPPORTERS_LINE} See whether speculative decoding improves throughput and cost at every interactivity level.`; const title = `${parsed.model.label} — ${gpuLabel} ${precLabel}: ${aLabel} vs Off — Speculative Decoding`; return { title, description, alternates: { - canonical: url, - languages: languageAlternates(`/compare-spec-decode/${canonical}`), + canonical: canonicalUrl, + languages: languageAlternates(canonicalPath), }, openGraph: { title: `${title} | ${SITE_NAME}`, @@ -77,11 +117,17 @@ export async function generateMetadata({ params }: Props): Promise { export default async function CompareSpecDecodePage({ params, searchParams }: Props) { const { slug } = await params; + return renderSpecDecodePage(slug, await searchParams, {}); +} + +export async function renderSpecDecodePage( + slug: string, + sp: Record, + { scenarioSegment }: ScenarioOptions, +) { const parsed = parseSpecDecodeCompareSlug(slug); if (!parsed) notFound(); - const sp = await searchParams; - // 308 canonical redirect: alias model, reversed 'none-vs-mtp' form. const canonical = canonicalSpecDecodeCompareSlug( parsed.model.slug, @@ -98,7 +144,9 @@ export default async function CompareSpecDecodePage({ params, searchParams }: Pr }) .map(([k, v]) => `${encodeURIComponent(k)}=${encodeURIComponent(v)}`) .join('&'); - permanentRedirect(`/compare-spec-decode/${canonical}${qs ? `?${qs}` : ''}`); + // Keep the scenario segment across the redirect — dropping it would send + // the reader to the pair's default workload instead. + permanentRedirect(`${scenarioPath(canonical, scenarioSegment)}${qs ? `?${qs}` : ''}`); } const rows = await getCachedBenchmarks(parsed.model.dbKeys); @@ -115,7 +163,9 @@ export default async function CompareSpecDecodePage({ params, searchParams }: Pr const defaults = pickVariantPairDefaults('spec-decode', rows, parsed.gpu, sideA, sideB); const urlSeq = pickString(sp.i_seq); - const effectiveSequence = urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : defaults.sequence; + const pathSequence = scenarioSegment ? sequenceForScenarioSegment(scenarioSegment) : null; + const effectiveSequence = + pathSequence ?? (urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : defaults.sequence); const effectivePrecision = parsed.precision; const sideAFull: VariantCompareSide = { @@ -139,7 +189,7 @@ export default async function CompareSpecDecodePage({ params, searchParams }: Pr const summaryB = summarizeVariantSide(rows, parsed.gpu, sideBFull); const { oldest, newest } = dateRangeForVariantPair(rows, parsed.gpu, sideAFull, sideBFull); - const url = `${SITE_URL}/compare-spec-decode/${canonical}`; + const url = `${SITE_URL}${scenarioPath(canonical, scenarioSegment)}`; const imageUrl = `${url}/spec-decode-comparison.png`; const jsonLd = buildVariantJsonLd( @@ -185,6 +235,7 @@ export default async function CompareSpecDecodePage({ params, searchParams }: Pr defaultPrecision={effectivePrecision} ssrTableData={{ defaultTargets, ssrRows, interactivityRange }} narrative={narrative} + agenticIntro={isAgenticSequence(effectiveSequence) ? AGENTIC_SCENARIO_INTRO : null} gpuLabel={gpuLabel} precisionLabel={precLabel} gpuArch={gpuMeta?.arch ?? ''} diff --git a/packages/app/src/app/compare-spec-decode/[slug]/spec-decode-comparison.png/route.tsx b/packages/app/src/app/compare-spec-decode/[slug]/spec-decode-comparison.png/route.tsx index 529e3d4d8..2e779fc38 100644 --- a/packages/app/src/app/compare-spec-decode/[slug]/spec-decode-comparison.png/route.tsx +++ b/packages/app/src/app/compare-spec-decode/[slug]/spec-decode-comparison.png/route.tsx @@ -1,5 +1,3 @@ -import { ImageResponse } from 'next/og'; - import { HW_REGISTRY } from '@semianalysisai/inferencex-constants'; import { trackServer } from '@/lib/analytics-server'; @@ -17,19 +15,7 @@ import { specMethodDisplayLabel, } from '@/lib/compare-variant-slug'; import { getLogoSrc } from '@/lib/og-assets'; -import { - buildSeriesPoints, - CHART, - CHART_FRAME, - COLORS, - money, - moneyForStep, - niceAxis, - R, - renderSeriesPath, - SIZE, - splitByMatchRange, -} from '@/lib/png-chart'; +import { renderComparePngChart } from '@/lib/png-chart'; export const dynamic = 'force-dynamic'; export const runtime = 'nodejs'; @@ -56,14 +42,11 @@ export async function GET( getCachedBenchmarks(parsed.model.dbKeys), getLogoSrc(), ]); - - // Precision is fixed by the slug — both sides share it. + // Precision is fixed by the slug, and both sides share it. const sideA: VariantCompareSide = { specMethod: parsed.method, precision: parsed.precision }; const sideB: VariantCompareSide = { specMethod: 'none', precision: parsed.precision }; - const defaults = pickVariantPairDefaults('spec-decode', rows, parsed.gpu, sideA, sideB); - const { sequence } = defaults; + const { sequence } = pickVariantPairDefaults('spec-decode', rows, parsed.gpu, sideA, sideB); const precision = parsed.precision; - const { ssrRows, interactivityRange } = computeVariantCompareTableData( rows, parsed.gpu, @@ -79,7 +62,7 @@ export async function GET( sideA, sideB, interactivityRange, - plottedRows.map((r) => r.target), + plottedRows.map((row) => row.target), ).filter((row) => row.a || row.b); const curveRows = imageRows.length > 0 ? imageRows : plottedRows; @@ -87,398 +70,23 @@ export async function GET( const precLabel = precisionDisplayLabel(parsed.precision); const aLabel = specMethodDisplayLabel(parsed.model.displayName, parsed.method); const bLabel = 'Off'; - - const costs = curveRows - .flatMap((row) => [row.a?.cost, row.b?.cost]) - .filter((cost): cost is number => typeof cost === 'number' && Number.isFinite(cost)); - const costMin = costs.length > 0 ? Math.min(...costs) : 0; - const costMax = costs.length > 0 ? Math.max(...costs) : 1; - const yAxis = niceAxis(Math.min(0, costMin), costMax); - const yMin = yAxis.min; - const yMax = yAxis.max; - const yStep = yAxis.step; - const xMin = curveRows.at(0)?.target ?? 0; - const xMax = curveRows.at(-1)?.target ?? 100; - const matchedMin = plottedRows.at(0)?.target ?? xMin; - const matchedMax = plottedRows.at(-1)?.target ?? xMax; - const hasLeftExtension = matchedMin - xMin >= 0.5; - const hasRightExtension = xMax - matchedMax >= 0.5; - const scaleX = (value: number) => - CHART.left + (xMax === xMin ? CHART.width / 2 : ((value - xMin) / (xMax - xMin)) * CHART.width); - const scaleY = (value: number) => - CHART.top + - CHART.height - - (yMax === yMin ? CHART.height / 2 : ((value - yMin) / (yMax - yMin)) * CHART.height); - - const aSeries = splitByMatchRange( - buildSeriesPoints(curveRows, (r) => r.a?.cost ?? null, scaleX, scaleY), - matchedMin, - matchedMax, - ); - const bSeries = splitByMatchRange( - buildSeriesPoints(curveRows, (r) => r.b?.cost ?? null, scaleX, scaleY), - matchedMin, - matchedMax, - ); - const aHighlightPoints = plottedRows - .filter((row) => row.a) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.a!.cost) })); - const bHighlightPoints = plottedRows - .filter((row) => row.b) - .map((row) => ({ x: scaleX(row.target), y: scaleY(row.b!.cost) })); const workload = [sequence, precision?.toUpperCase()].filter(Boolean).join(' / '); - const showRangeEndpoints = hasLeftExtension || hasRightExtension; - const svgWidth = 760 * R; - const svgHeight = 406 * R; try { - return new ImageResponse( -
-
-
-
- InferenceX Speculative Decoding Comparison -
-
- {parsed.model.label} -
-
- {gpuLabel} {precLabel}: {aLabel} vs {bLabel} | Cost per Million Tokens -
-
-
-
- DEFAULT WORKLOAD -
-
- {workload || 'Default comparison'} -
-
- Lower cost is better -
-
-
- -
-
- - - {yAxis.ticks.map((tick) => { - const y = scaleY(tick); - return ( - - ); - })} - {plottedRows.map((row) => { - const x = scaleX(row.target); - return ( - - ); - })} - {renderSeriesPath(aSeries.leftExt, COLORS.a, true)} - {renderSeriesPath(aSeries.rightExt, COLORS.a, true)} - {renderSeriesPath(aSeries.matched, COLORS.a, false)} - {renderSeriesPath(bSeries.leftExt, COLORS.b, true)} - {renderSeriesPath(bSeries.rightExt, COLORS.b, true)} - {renderSeriesPath(bSeries.matched, COLORS.b, false)} - {aHighlightPoints.map((point, index) => ( - - ))} - {bHighlightPoints.map((point, index) => ( - - ))} - - {logoSrc && ( - - )} - {yAxis.ticks.map((tick) => ( -
- {moneyForStep(tick, yStep)} -
- ))} - {plottedRows.map((row) => ( -
- {row.target} -
- ))} - {showRangeEndpoints && hasLeftExtension && ( -
- {Math.round(xMin)} -
- )} - {showRangeEndpoints && hasRightExtension && ( -
- {Math.round(xMax)} -
- )} -
- Interactivity (tok/s/user) -
- {showRangeEndpoints && ( -
- Dashed segments extend to each config's operating envelope, where cost rises steeply -
- )} -
- -
-
- Matched Interactivity -
-
- - - {aLabel} - - - - {bLabel} - -
- {plottedRows.length > 0 ? ( - plottedRows.map((row) => ( -
-
- {row.target} tok/s/user -
-
- - {row.a ? money(row.a.cost) : 'N/A'} - - - {row.b ? money(row.b.cost) : 'N/A'} - -
-
- )) - ) : ( -
- No matched cost data available. -
- )} -
-
- -
- - Speculative decoding comparison | interpolated from benchmark results - - - inferencex.semianalysis.com - -
-
, - { - ...SIZE, - headers: { - 'Cache-Control': 'public, s-maxage=3600, stale-while-revalidate=86400', - }, - }, - ); + return renderComparePngChart({ + curveRows, + plottedRows, + logoSrc, + aLabel, + bLabel, + eyebrow: 'InferenceX Speculative Decoding Comparison', + title: parsed.model.label, + subtitle: `${gpuLabel} ${precLabel}: ${aLabel} vs ${bLabel} | Cost per Million Tokens`, + workload, + rangeNote: + "Dashed segments extend to each config's operating envelope, where cost rises steeply", + footer: 'Speculative decoding comparison | interpolated from benchmark results', + }); } catch (error) { const message = error instanceof Error ? error.message : String(error); trackServer('compare_spec_decode_png_render_failed', { diff --git a/packages/app/src/app/compare-spec-decode/layout.tsx b/packages/app/src/app/compare-spec-decode/layout.tsx index ad681bd9a..f0a2fdb0d 100644 --- a/packages/app/src/app/compare-spec-decode/layout.tsx +++ b/packages/app/src/app/compare-spec-decode/layout.tsx @@ -1,20 +1,5 @@ -import { UnofficialRunProvider } from '@/components/unofficial-run-provider'; +import { ComparePageLayout } from '@/components/compare/compare-page-layout'; -/** - * Wraps `/compare-spec-decode/*` pages in UnofficialRunProvider but skips - * DashboardShell so we get a focused single-purpose page (no TabNav). Mirrors - * the structure of `/compare-per-dollar/layout.tsx` — the GlobalFilterProvider - * is mounted inside the page's client component so it can seed initial - * precision/sequence based on which combos actually have data from the slug. - */ export default function CompareSpecDecodeLayout({ children }: { children: React.ReactNode }) { - return ( - -
-
- {children} -
-
-
- ); + return {children}; } diff --git a/packages/app/src/app/compare/[slug]/[scenario]/page.tsx b/packages/app/src/app/compare/[slug]/[scenario]/page.tsx new file mode 100644 index 000000000..3d97fd9e8 --- /dev/null +++ b/packages/app/src/app/compare/[slug]/[scenario]/page.tsx @@ -0,0 +1,37 @@ +import type { Metadata } from 'next'; +import { notFound } from 'next/navigation'; + +import { type ScenarioSegment, sequenceForScenarioSegment } from '@/lib/compare-scenario-route'; + +import { buildCompareMetadata, renderComparePage } from '../page'; + +/** + * `/compare//` — the same comparison with its workload pinned + * by the path instead of `?i_seq=`. Both routes render through the parent's + * exported body, so the two URLs cannot drift. + */ +export const dynamic = 'force-dynamic'; + +interface Props { + params: Promise<{ slug: string; scenario: string }>; + searchParams: Promise>; +} + +/** Reject unknown segments here rather than silently falling back to the + * pair's default, which would serve 200s for arbitrary junk paths. */ +function assertScenario(scenario: string): ScenarioSegment { + if (!sequenceForScenarioSegment(scenario)) notFound(); + return scenario.toLowerCase() as ScenarioSegment; +} + +export async function generateMetadata({ params }: Props): Promise { + const { slug, scenario } = await params; + return buildCompareMetadata(slug, { scenarioSegment: assertScenario(scenario) }); +} + +export default async function CompareScenarioPage({ params, searchParams }: Props) { + const { slug, scenario } = await params; + return renderComparePage(slug, await searchParams, { + scenarioSegment: assertScenario(scenario), + }); +} diff --git a/packages/app/src/app/compare/[slug]/opengraph-image.tsx b/packages/app/src/app/compare/[slug]/opengraph-image.tsx index 709b7e975..0e7762483 100644 --- a/packages/app/src/app/compare/[slug]/opengraph-image.tsx +++ b/packages/app/src/app/compare/[slug]/opengraph-image.tsx @@ -1,206 +1,32 @@ -/** - * Compare OG image — same circuit tile sidebar layout as the blog OG. - */ -import { readFile } from 'node:fs/promises'; -import { join } from 'node:path'; - import { notFound } from 'next/navigation'; -import { ImageResponse } from 'next/og'; import { getAllComparableCompareSlugs } from '@/lib/compare-availability'; +import { renderCompareOg } from '@/lib/compare-og'; import { canonicalCompareSlug, compareDisplayLabel, parseCompareSlug } from '@/lib/compare-slug'; export const alt = 'Chip inference benchmark comparison'; export const size = { width: 1200, height: 630 }; export const contentType = 'image/png'; -const BLUE = '#0B86D1'; -const BG = '#131416'; -const PANEL_BG = '#0F1214'; - -// Same tile grid as the blog OG so the two OG types read as a family. -const TILE_GRID: ({ file: string; rotate?: number } | null)[] = [ - { file: 'teal-chevron.png', rotate: 180 }, - { file: 'gold-diagonal.png' }, - { file: 'teal-circuit.png' }, - null, - { file: 'gold-wavy.png' }, - { file: 'teal-chip.png' }, - { file: 'teal-chevron.png', rotate: 90 }, - { file: 'teal-organic.png' }, - null, - { file: 'gold-circuit.png' }, - { file: 'teal-circuit.png', rotate: 180 }, - { file: 'teal-organic.png', rotate: 180 }, -]; - export async function generateStaticParams() { - // Mirror the SSR page's static params — only emit (model, pair) combos - // with benchmark data on both sides so we don't generate OG images for - // empty pages. const slugs = await getAllComparableCompareSlugs(); return slugs.map(({ modelSlug, a, b }) => ({ slug: canonicalCompareSlug(modelSlug, a, b) })); } -// Read once at module load; a missing asset must not 500 every OG route. -let logoSrcPromise: Promise | undefined; -function getLogoSrc(): Promise { - if (!logoSrcPromise) { - logoSrcPromise = readFile(join(process.cwd(), 'public/brand/logo-color.png')) - .then((buf) => `data:image/png;base64,${buf.toString('base64')}`) - .catch(() => null); - } - return logoSrcPromise; -} - -let tilesPromise: Promise<({ src: string; rotate?: number } | null)[]> | undefined; -function getTiles(): Promise<({ src: string; rotate?: number } | null)[]> { - if (!tilesPromise) { - const uniqueFiles = [...new Set(TILE_GRID.filter(Boolean).map((t) => t!.file))]; - tilesPromise = Promise.all( - uniqueFiles.map(async (f) => { - const src = await readFile(join(process.cwd(), 'public/brand/og-tiles', f)) - .then((buf) => `data:image/png;base64,${buf.toString('base64')}`) - .catch(() => null); - return [f, src] as const; - }), - ).then((loaded) => { - const cache = Object.fromEntries(loaded); - return TILE_GRID.map((t) => { - if (!t) return null; - const src = cache[t.file]; - return src ? { src, rotate: t.rotate } : null; - }); - }); - } - return tilesPromise; -} - export default async function OgImage({ params }: { params: Promise<{ slug: string }> }) { const { slug } = await params; const parsed = parseCompareSlug(slug); if (!parsed) notFound(); - const [logoSrc, tiles] = await Promise.all([getLogoSrc(), getTiles()]); const title = compareDisplayLabel(parsed.a, parsed.b); - const eyebrow = `${parsed.model.label} · Head-to-head Chip benchmark`; - // Content area is ~895px wide (1200 - 195 panel - 55*2 padding). Scale the - // title size down for longer labels so it fits without truncating. const titleSize = title.length > 26 ? 80 : title.length > 18 ? 96 : 112; - return new ImageResponse( -
- {/* Left tile panel — identical pattern to blog OG */} -
- {tiles.map((tile, i) => { - if (!tile) return null; - const row = Math.floor(i / 2); - const col = i % 2; - return ( - - ); - })} -
-
- - {/* Content */} -
-
-
- {eyebrow} -
- -
- {title} -
-
- -
- - AI inference benchmark · latency, throughput, cost - - {logoSrc && } -
-
-
, + return renderCompareOg({ + eyebrow: `${parsed.model.label} · Head-to-head Chip benchmark`, + title, + titleSize, + footer: 'AI inference benchmark · latency, throughput, cost', + language: 'en', size, - ); + }); } diff --git a/packages/app/src/app/compare/[slug]/page-client.tsx b/packages/app/src/app/compare/[slug]/page-client.tsx index 81e2010e6..6e963de1a 100644 --- a/packages/app/src/app/compare/[slug]/page-client.tsx +++ b/packages/app/src/app/compare/[slug]/page-client.tsx @@ -3,21 +3,26 @@ import Link from 'next/link'; import { useEffect, useMemo } from 'react'; -import type { GPUDataPoint, InterpolatedResult } from '@/components/calculator/types'; +import type { GPUDataPoint } from '@/components/calculator/types'; import { useThroughputData } from '@/components/calculator/useThroughputData'; -import { CompareInterpolatedTable } from '@/components/compare/compare-interpolated-table'; -import { useGlobalFilters, GlobalFilterProvider } from '@/components/GlobalFilterContext'; +import { + CompareTableRenderer, + type CompareTableData, +} from '@/components/compare/compare-table-renderer'; +import { + GlobalFilterProvider, + useGlobalFilterAvailability, + useGlobalFilterRun, + useGlobalFilterSelection, +} from '@/components/GlobalFilterContext'; import { InferenceProvider } from '@/components/inference/InferenceContext'; import InferenceChartDisplay from '@/components/inference/ui/ChartDisplay'; import { Card } from '@/components/ui/card'; import { track } from '@/lib/analytics'; +import type { BenchmarkRow } from '@/lib/api'; +import { toCalculatorBenchmarkRows } from '@/lib/benchmark-api-view'; import { toModel, toPrecisions, toSequence } from '@/lib/compare-enum-coerce'; - -interface SsrTableData { - defaultTargets: number[]; - ssrRows: { target: number; a: InterpolatedResult | null; b: InterpolatedResult | null }[]; - interactivityRange: { min: number; max: number }; -} +import type { AgenticScenarioIntro } from '@/lib/compare-ssr'; const STRINGS = { en: { @@ -30,13 +35,13 @@ const STRINGS = { 'No interpolated comparison data available for the default model. Use the chart controls below to select a model with benchmark data for both chips.', }, zh: { - eyebrowSuffix: 'Chip 对比', + eyebrowSuffix: '芯片对比', mainChartLinkText: '主推理图表', perDollarLinkText: '查看每美元性能对比 →', caveatSeqFallback: '序列', caveatPrecFallback: '精度', emptyState: - '当前默认模型没有可用的插值对比数据。请使用下方图表控件选择一个两款 Chip 均有基准测试数据的模型。', + '当前默认模型没有可用的插值对比数据。请使用下方图表控件选择一个两款芯片均有基准测试数据的模型。', }, } as const; @@ -54,13 +59,18 @@ interface ComparePageClientProps { defaultModel: string; defaultSequence: string | null; defaultPrecision: string | null; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; + initialBenchmarkRows?: BenchmarkRow[]; /** One SSR-rendered prose paragraph per interpolated-table row (default * interactivity target). Each paragraph picks a template variant * deterministically from the slug so prose stays stable across renders * but varies across pages in the catalog. Empty array when there's no * comparable data. */ narrative: string[]; + /** Set only when the page is rendering the agentic workload. Explains what + * AgentX measures before the head-to-head numbers, since a reader who + * arrived from a GPU query has no reason to expect a replayed trace. */ + agenticIntro?: AgenticScenarioIntro | null; aLabel: string; bLabel: string; aVendor: string; @@ -80,7 +90,9 @@ export default function ComparePageClient({ defaultSequence, defaultPrecision, ssrTableData, + initialBenchmarkRows, narrative, + agenticIntro = null, aLabel, bLabel, aVendor, @@ -96,6 +108,14 @@ export default function ComparePageClient({ const compareGpuPair = useMemo(() => [a, b] as const, [a, b]); const initialModel = toModel(defaultModel); const initialSequence = toSequence(defaultSequence); + const benchmarkQueryScope = `compare-pair:${a}:${b}`; + const initialCalculatorRows = useMemo( + () => + initialBenchmarkRows && defaultSequence + ? toCalculatorBenchmarkRows(initialBenchmarkRows, defaultSequence) + : undefined, + [defaultSequence, initialBenchmarkRows], + ); const initialPrecisions = toPrecisions(defaultPrecision); const t = STRINGS[locale]; const isZh = locale === 'zh'; @@ -110,6 +130,9 @@ export default function ComparePageClient({ activeTab="compare" initialActiveHwTypes={[a, b]} compareGpuPair={compareGpuPair} + benchmarkQueryScope={benchmarkQueryScope} + initialBenchmarkModel={initialModel} + initialBenchmarkRows={initialBenchmarkRows} >
@@ -142,6 +165,21 @@ export default function ComparePageClient({ .

)} + {agenticIntro && ( +

+ {agenticIntro.paragraph}{' '} + + {agenticIntro.linkLabel} → + +

+ )} {narrative.length > 0 && (
{narrative.map((para, i) => ( @@ -177,6 +215,9 @@ export default function ComparePageClient({ aLabel={aLabel} bLabel={bLabel} ssrTableData={ssrTableData} + initialModel={initialModel} + initialSequence={initialSequence} + initialCalculatorRows={initialCalculatorRows} emptyStateText={t.emptyState} /> @@ -193,23 +234,46 @@ function CompareTableSection({ aLabel, bLabel, ssrTableData, + initialModel, + initialSequence, + initialCalculatorRows, emptyStateText, }: { a: string; b: string; aLabel: string; bLabel: string; - ssrTableData: SsrTableData; + ssrTableData: CompareTableData; + initialModel: ReturnType; + initialSequence: ReturnType; + initialCalculatorRows?: BenchmarkRow[]; emptyStateText: string; }) { - const { effectiveSequence, effectivePrecisions, selectedRunDate, selectedModel } = - useGlobalFilters(); + const { effectiveSequence, effectivePrecisions, selectedModel, sequenceResolved } = + useGlobalFilterSelection(); + const { availableDates } = useGlobalFilterAvailability(); + const { selectedRunDate } = useGlobalFilterRun(); + const latestAvailableDate = availableDates.at(-1) ?? ''; + const calculatorRunDate = + selectedRunDate && selectedRunDate === latestAvailableDate ? '' : selectedRunDate; + const initialRows = + initialCalculatorRows && + sequenceResolved && + selectedModel === initialModel && + effectiveSequence === initialSequence && + calculatorRunDate === '' + ? initialCalculatorRows + : undefined; const { gpuDataByGroupKey, ranges, hasData } = useThroughputData( selectedModel, effectiveSequence, effectivePrecisions, - selectedRunDate, + calculatorRunDate, + undefined, + undefined, + initialRows, + sequenceResolved, ); // Extract GPUDataPoint arrays for just the two GPUs in the pair. @@ -229,23 +293,15 @@ function CompareTableSection({ const clientRange = hasData ? ranges.interactivity : ssrTableData.interactivityRange; - if (ssrTableData.defaultTargets.length === 0) { - return ( -
- {emptyStateText} -
- ); - } - return ( - ); } diff --git a/packages/app/src/app/compare/[slug]/page.tsx b/packages/app/src/app/compare/[slug]/page.tsx index b11011d5d..aa70cc676 100644 --- a/packages/app/src/app/compare/[slug]/page.tsx +++ b/packages/app/src/app/compare/[slug]/page.tsx @@ -4,8 +4,13 @@ import { notFound, permanentRedirect } from 'next/navigation'; import { HW_REGISTRY, SITE_NAME, SITE_URL } from '@semianalysisai/inferencex-constants'; import { JsonLd } from '@/components/json-ld'; +import { comparisonScenarioForModel } from '@/lib/compare-agentx'; +import { + isAgenticSequence, + type ScenarioSegment, + sequenceForScenarioSegment, +} from '@/lib/compare-scenario-route'; import { languageAlternates } from '@/lib/i18n'; -import { pickPairDefaults } from '@/lib/compare-pair-defaults'; import { canonicalCompareSlug, compareDisplayLabel, @@ -15,38 +20,67 @@ import { parseCompareSlug, } from '@/lib/compare-slug'; import { + AGENTIC_SCENARIO_INTRO, buildBreadcrumbJsonLd, buildJsonLd, compareMetaDescription, compareTableNarrative, - computeCompareTableData, - dateRangeForPair, - getCachedBenchmarks, KNOWN_MODELS, KNOWN_PRECISIONS, KNOWN_SEQUENCES, pickString, - summarize, } from '@/lib/compare-ssr'; +import { + getComparePageDerivedData, + initialCompareBenchmarkRows, +} from '@/lib/compare-page-data.server'; import ComparePageClient from './page-client'; export const dynamic = 'force-dynamic'; +function comparePath(canonical: string, scenarioSegment?: ScenarioSegment): string { + return scenarioSegment ? `/compare/${canonical}/${scenarioSegment}` : `/compare/${canonical}`; +} + interface Props { params: Promise<{ slug: string }>; searchParams: Promise>; } +/** + * `/compare//` renders this same page with the workload + * pinned by the path. The segment is threaded through rather than handled in + * a parallel route file so the body — redirects, JSON-LD, metadata, the + * client props — is written once and cannot drift between the two URLs. + */ +export interface ScenarioOptions { + scenarioSegment?: ScenarioSegment; +} + export async function generateMetadata({ params }: Props): Promise { const { slug } = await params; + return buildCompareMetadata(slug, {}); +} + +export async function buildCompareMetadata( + slug: string, + { scenarioSegment }: ScenarioOptions, +): Promise { const parsed = parseCompareSlug(slug); if (!parsed) return {}; const fullLabel = compareModelDisplayLabel(parsed.model, parsed.a, parsed.b); const gpuLabel = compareDisplayLabel(parsed.a, parsed.b); const modelSeoName = compareModelSeoName(parsed.model); const canonical = canonicalCompareSlug(parsed.model.slug, parsed.a, parsed.b); - const url = `${SITE_URL}/compare/${canonical}`; + // The scenario segments are views of one comparison, so the bare slug URL + // stays the indexable representative and every segment canonicalizes to it. + // Without this, `/…//` and `/…/` would serve + // byte-identical pages, each claiming to be canonical. + const routePath = comparePath(canonical, scenarioSegment); + const canonicalPath = comparePath(canonical); + const url = `${SITE_URL}${routePath}`; + const canonicalUrl = `${SITE_URL}${canonicalPath}`; // Lead the SEO title with the GPU pair — that's the phrase people search // ("b200 vs b300") and must survive Google's ~60-char SERP truncation. The @@ -54,20 +88,25 @@ export async function generateMetadata({ params }: Props): Promise { // template so the query isn't pushed off the end. const title = `${compareSeoTitle(gpuLabel, modelSeoName)} | ${SITE_NAME}`; - // Stat-led meta description built from the interpolated head-to-head numbers - // at the slug's default operating point (falls back to boilerplate for - // sparse-data pairs). Fetch is blob-cached and shared with the page render. - const rows = await getCachedBenchmarks(parsed.model.dbKeys); - const { sequence, precision } = pickPairDefaults(rows, parsed.a, parsed.b); - const { ssrRows } = computeCompareTableData(rows, parsed.a, parsed.b, sequence, precision); + // Stat-led meta description from the default interpolated head-to-head + // numbers. The compact derived payload is shared with the page render and + // invalidated with the database cache tag. + const { ssrRows } = await getComparePageDerivedData( + parsed.model.dbKeys, + parsed.a, + parsed.b, + null, + null, + comparisonScenarioForModel(parsed.model).sequence, + ); const description = compareMetaDescription(parsed.model, parsed.a, parsed.b, ssrRows); return { title: { absolute: title }, description, alternates: { - canonical: url, - languages: languageAlternates(`/compare/${canonical}`), + canonical: canonicalUrl, + languages: languageAlternates(canonicalPath), }, openGraph: { title: `${fullLabel} | ${SITE_NAME}`, @@ -85,13 +124,17 @@ export async function generateMetadata({ params }: Props): Promise { export default async function ComparePage({ params, searchParams }: Props) { const { slug } = await params; + return renderComparePage(slug, await searchParams, {}); +} + +export async function renderComparePage( + slug: string, + sp: Record, + { scenarioSegment }: ScenarioOptions, +) { const parsed = parseCompareSlug(slug); if (!parsed) notFound(); - // Await searchParams once so we can both preserve them on redirect and read - // them for URL-param overrides further down. - const sp = await searchParams; - // One-hop redirect to the fully canonical URL. Handles all three normalization // cases in a single 308: // - legacy bare slug: `h100-vs-h200` → `deepseek-r1-h100-vs-h200` @@ -118,42 +161,46 @@ export default async function ComparePage({ params, searchParams }: Props) { // all permanent decisions — using a permanent redirect lets search engines // consolidate link equity onto the canonical URL instead of keeping the // alias URL in the index alongside the canonical one. - permanentRedirect(`/compare/${canonical}${qs ? `?${qs}` : ''}`); + // Keep the scenario segment across the redirect — dropping it would send + // `/compare/h100-vs-h200/agentic` to the pair's default workload. + permanentRedirect(`${comparePath(canonical, scenarioSegment)}${qs ? `?${qs}` : ''}`); } - const rows = await getCachedBenchmarks(parsed.model.dbKeys); - const summaryA = summarize(rows, parsed.a); - const summaryB = summarize(rows, parsed.b); - const { sequence: pickedSequence, precision: pickedPrecision } = pickPairDefaults( - rows, - parsed.a, - parsed.b, - ); - + const fallbackSequence = comparisonScenarioForModel(parsed.model).sequence; // URL params win over slug-derived defaults; this baking-into-SSR avoids the // hydration flash where the client upgrades seeded defaults to URL values. // `sp` was already awaited above for the redirect-query-preservation path. const urlSeq = pickString(sp.i_seq); const urlPrec = pickString(sp.i_prec); const urlModel = pickString(sp.g_model); - const effectiveSequence = urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : pickedSequence; - const effectivePrecision = urlPrec && KNOWN_PRECISIONS.has(urlPrec) ? urlPrec : pickedPrecision; - // `?g_model=` is honored only if it matches a known model — but the slug's - // model is the canonical default. Disregard URL param if user wants to - // explicitly override (rare). const effectiveModel = urlModel && KNOWN_MODELS.has(urlModel) ? urlModel : parsed.model.displayName; - - const { defaultTargets, ssrRows, interactivityRange } = computeCompareTableData( - rows, + // Path beats query beats the pair's default: a scenario segment is an + // explicit address for one workload, so it outranks a stale `?i_seq=`. + const pathSequence = scenarioSegment ? sequenceForScenarioSegment(scenarioSegment) : null; + const requestedSequence = pathSequence ?? (urlSeq && KNOWN_SEQUENCES.has(urlSeq) ? urlSeq : null); + const requestedPrecision = urlPrec && KNOWN_PRECISIONS.has(urlPrec) ? urlPrec : null; + const { + sequence: effectiveSequence, + precision: effectivePrecision, + summaryA, + summaryB, + defaultTargets, + ssrRows, + interactivityRange, + oldest, + newest, + initialPairBenchmarkRows, + } = await getComparePageDerivedData( + parsed.model.dbKeys, parsed.a, parsed.b, - effectiveSequence, - effectivePrecision, + requestedSequence, + requestedPrecision, + fallbackSequence, ); - const url = `${SITE_URL}/compare/${canonical}`; - const { oldest, newest } = dateRangeForPair(rows, parsed.a, parsed.b); + const url = `${SITE_URL}${comparePath(canonical, scenarioSegment)}`; const jsonLd = buildJsonLd( 'full', parsed.model, @@ -200,8 +247,14 @@ export default async function ComparePage({ params, searchParams }: Props) { defaultModel={effectiveModel} defaultSequence={effectiveSequence} defaultPrecision={effectivePrecision} + initialBenchmarkRows={initialCompareBenchmarkRows( + parsed.model.displayName, + effectiveModel, + initialPairBenchmarkRows, + )} ssrTableData={{ defaultTargets, ssrRows, interactivityRange }} narrative={narrative} + agenticIntro={isAgenticSequence(effectiveSequence) ? AGENTIC_SCENARIO_INTRO : null} aLabel={aLabel} bLabel={bLabel} aVendor={aMeta?.vendor ?? ''} diff --git a/packages/app/src/app/compare/layout.tsx b/packages/app/src/app/compare/layout.tsx index e3f01ddc4..76217427d 100644 --- a/packages/app/src/app/compare/layout.tsx +++ b/packages/app/src/app/compare/layout.tsx @@ -1,20 +1,5 @@ -import { UnofficialRunProvider } from '@/components/unofficial-run-provider'; +import { ComparePageLayout } from '@/components/compare/compare-page-layout'; -/** - * Wraps `/compare/*` pages in UnofficialRunProvider but skips DashboardShell so - * we get a focused single-purpose page (no TabNav). The GlobalFilterProvider is - * mounted inside the page's client component instead — that lets the page seed - * its initial precision/sequence based on which combos actually have data for - * the GPU pair from the slug. - */ export default function CompareLayout({ children }: { children: React.ReactNode }) { - return ( - -
-
- {children} -
-
-
- ); + return {children}; } diff --git a/packages/app/src/app/compare/page.tsx b/packages/app/src/app/compare/page.tsx index 2f1a0cb0c..99e2f560c 100644 --- a/packages/app/src/app/compare/page.tsx +++ b/packages/app/src/app/compare/page.tsx @@ -1,39 +1,39 @@ import type { Metadata } from 'next'; import Link from 'next/link'; -import { - HW_REGISTRY, - SITE_NAME, - SITE_URL, - SUPPORTERS_LINE, -} from '@semianalysisai/inferencex-constants'; +import { HW_REGISTRY, SITE_NAME, SITE_URL } from '@semianalysisai/inferencex-constants'; import { enAlternates } from '@/lib/i18n'; +import { AgentXCompareHero } from '@/components/compare/agentx-compare-hero'; import { ComparePairCardLink } from '@/components/compare/compare-pair-card-link'; import { JsonLd } from '@/components/json-ld'; import { Card } from '@/components/ui/card'; +import { ModelLogo } from '@/components/ui/model-logo'; +import { comparisonPairHref, comparisonScenarioForModel } from '@/lib/compare-agentx'; import { getComparablePairsByModelSlug } from '@/lib/compare-availability'; import { type ComparePair, COMPARE_MODEL_SLUGS, type CompareModelSlug } from '@/lib/compare-slug'; import { bucketComparePairsByVendor, formatModelList } from '@/lib/compare-ssr'; +import { type Model } from '@/lib/data-mappings'; export const dynamic = 'force-dynamic'; -const DESCRIPTION = `InferenceX is the independent, open-source chip inference benchmark from SemiAnalysis, with verified, reproducible nightly results. ${SUPPORTERS_LINE} Compare latency, throughput & cost head-to-head across DeepSeek V4 Pro, DeepSeek R1, Kimi K2, MiniMax M3, GLM 5, Qwen 3.5 & more.`; +const DESCRIPTION = + 'Compare AgentX agentic inference results for Kimi K3, DeepSeek V4 Pro, MiniMax M3, Qwen 3.5, and GLM 5.3, plus fixed-sequence chip comparisons.'; export const metadata: Metadata = { - title: 'Chip Comparisons', + title: 'AgentX Inference Comparisons', description: DESCRIPTION, alternates: enAlternates('/compare'), openGraph: { - title: `Chip Comparisons | ${SITE_NAME}`, + title: `AgentX Inference Comparisons | ${SITE_NAME}`, description: DESCRIPTION, url: `${SITE_URL}/compare`, type: 'website', }, twitter: { card: 'summary_large_image', - title: `Chip Comparisons | ${SITE_NAME}`, + title: `AgentX Inference Comparisons | ${SITE_NAME}`, description: DESCRIPTION, }, }; @@ -77,7 +77,7 @@ function groupPairsByVendorForModel( const jsonLd = { '@context': 'https://schema.org', '@type': 'CollectionPage', - name: `Chip Comparisons | ${SITE_NAME}`, + name: `AgentX Inference Comparisons | ${SITE_NAME}`, description: DESCRIPTION, url: `${SITE_URL}/compare`, }; @@ -97,19 +97,27 @@ export default async function CompareIndexPage() { return ( <> -
+ + +
-

Chip Comparisons

+

+ Comparison catalog +

+

+ AgentX and 8K→1K results +

{totalUrls.toLocaleString()} head-to-head inference benchmark comparisons across{' '} - {formatModelList(modelsWithPairs)}. Each page includes interactive charts for latency, - throughput, and cost metrics, plus an interpolated comparison table. + {formatModelList(modelsWithPairs)}. Models with AgentX data open long-context, + multi-turn trace replay results. Models not yet covered by AgentX open the controlled + 8K→1K workload. Each card identifies its scenario.

Compare Chip performance per dollar
+
+ +
+

- Open Source Continuous Inference Benchmark trusted by Operators of Trillion Dollar - GigaWatt Scale Token Factories + Open-Source Continuous Agentic Inference Benchmark Trusted by Operators of + Trillion-Dollar, GigaWatt-Scale Token Factories

As the world progresses exponentially towards AGI, software development and model @@ -58,9 +53,9 @@ export default function AboutPage() {

InferenceX™ (formerly InferenceMAX) is our independent, vendor - neutral, reproducible benchmark which addresses these issues by continuously - benchmarking inference software across a wide range of AI accelerators that are - actually available to the ML community. + neutral, reproducible benchmark. It measures both fixed-sequence serving and AgentX, + our long-context, multi-turn agentic coding workload, across AI accelerators and + serving stacks available to the ML community.

Our open data & insights are widely adopted by the ML community, capacity planning @@ -176,45 +171,7 @@ export default function AboutPage() {

-
- -

Frequently Asked Questions

-
- {FAQ_ITEMS.map((item) => ( -
-
{item.question}
-
- {item.answer && ( -

- {item.answer} - {item.link && ( - <> - {' '} - - {item.link.text} - - - )} -

- )} - {item.list && ( -
    - {item.list.map((li) => ( -
  • {li}
  • - ))} -
- )} -
-
- ))} -
- -
+