diff --git a/plugins/aws-startup-advisor/agents/llm2bedrock-code-analyzer.md b/plugins/aws-startup-advisor/agents/llm2bedrock-code-analyzer.md index 6f83da7e3..127169c96 100644 --- a/plugins/aws-startup-advisor/agents/llm2bedrock-code-analyzer.md +++ b/plugins/aws-startup-advisor/agents/llm2bedrock-code-analyzer.md @@ -256,7 +256,12 @@ If `source_provider ∈ {openai, google, anthropic}` AND `same_model_family == f For `cohere` or `custom`, set `behavior_deltas: []` and skip this section. Anthropic always loads `references/anthropic-to-bedrock.md`: a version hop needs the selected target's full parameter/thinking/prefill/tool/response checks, and an exact same-model move still needs API/path compatibility checks. Emit `[]` only after those checks find no deltas. -**Exception — OpenAI same-model on mantle.** When `source_provider == openai` AND `same_model_family == true` (every target is a proprietary `openai.gpt-5*` / `openai.gpt-6*` model), do NOT emit `behavior_deltas: []` wholesale. The model is unchanged, so the parameter-surface deltas (temperature range, penalties, stop sequences) genuinely do not apply — but the **API surface** can still change. Read only the "Same-model (mantle) deltas" section of the reference's `openai-to-bedrock.md` and emit those: Chat Completions → Responses, reasoning items round-tripping, endpoint path and credential, and prompt-caching availability. Skipping these would leave a `chat.completions.create` call pointed at a model that does not verifiably accept it. +**Exception — proprietary OpenAI targets.** For GPT-6 Astra / GPT-5.x, load the +"Same-vendor GPT endpoint and API deltas" section of the OpenAI reference even when +`same_model_family == true`. It covers both bare Mantle and supported runtime CRIS forms. +Model identity does not prove API or parameter parity: preserve the selected Chat/Responses +surface, verify Astra's unprobed parameters, and keep endpoint/credential deltas. Do not apply +Claude-specific parameter changes to Astra. A same-vendor version upgrade still requires eval. 1. Read the `behavior-delta-detection` reference at the absolute path given in your context block's `behavior-delta-detection reference:` line. Call that file's directory @@ -377,7 +382,7 @@ Return ONE flat object: the typed fields and `summary` are all top-level sibling "source_provider": "openai", "source_models": ["gpt-4o"], "target_models": ["gpt-4o -> us.anthropic.claude-sonnet-4-6"], - "_comment_target_models": "example only — the real mapping comes from the migration plan, never from this illustration. Plans may also map same-provider (e.g. gpt-4o -> openai.gpt-5.6-terra on Bedrock Mantle); note that GPT-5.6 targets are Responses-API-only (no Converse, no us./global. inference profiles), so echo them bare and expect the rewrite to keep the OpenAI SDK surface", + "_comment_target_models": "example only — the real mapping comes from the migration plan, never from this illustration. Plans may also map same-provider (e.g. gpt-4o -> openai.gpt-5.6-terra on Bedrock Mantle); preserve the validated endpoint and id form: bare proprietary GPT ids select Mantle; supported GPT-5.6/Astra CRIS ids select runtime; do not add or remove a prefix to change the chosen path", "same_model_family": false, "bedrock_provider_available": true, "prompt_locations": ["app.py:42 : SYSTEM_PROMPT constant"], diff --git a/plugins/aws-startup-advisor/agents/llm2bedrock-code-rewriter.md b/plugins/aws-startup-advisor/agents/llm2bedrock-code-rewriter.md index 1a16f449e..6cfdc6979 100644 --- a/plugins/aws-startup-advisor/agents/llm2bedrock-code-rewriter.md +++ b/plugins/aws-startup-advisor/agents/llm2bedrock-code-rewriter.md @@ -146,7 +146,35 @@ Conservative scope — only universally-junk patterns; `.venv/` / `node_modules/ ## Strategy selection (check FIRST) -If your context has a `Rewrite strategy: mantle` line, use the **Mantle express lane** below. Otherwise (the line is absent — the common case, including every run where any target lacks a Mantle equivalent) use the Converse rewrite that follows. Never mix: a run is entirely Mantle or entirely Converse. +If your context has `Rewrite strategy: runtime_openai`, use the runtime OpenAI lane below. +For `Rewrite strategy: mantle`, use the Mantle express lane. Otherwise use the Converse +rewrite. The selected `Target API surface` must match what the evaluator verified. + +### Runtime OpenAI lane + +Keep the OpenAI SDK and the selected Chat/Responses surface. Use the runtime `/openai/v1` +endpoint with the exact validated profile ID or ARN, not a bare model ID or a Mantle URL. +For Python, use: + +```python +from openai import OpenAI +from aws_bedrock_token_generator import provide_token + +region = "" +client = OpenAI( + base_url=f"https://bedrock-runtime.{region}.amazonaws.com/openai/v1", + api_key=lambda: provide_token(region=region), +) +``` + +Call `chat.completions.create(model="", messages=...)` for +`Target API surface: chat_completions`, or `responses.create(model="", input=...)` +for `responses`, keeping that API's response parsing, images and tool/state mapping. +Use the equivalent OpenAI client for the application's other languages. Runtime permissions +remain `bedrock:InvokeModel*` for the selected profile and backing model, not Mantle actions. +In §12 keep `openai>=2.45.0,<3` and `aws-bedrock-token-generator>=1,<2`; do not apply an +analyzer's generic OpenAI-to-boto3 dependency replacement to this lane. Apply confirmed deltas +in §9, then skip the Converse-specific §8 rewrite guidance. ### Mantle express lane @@ -164,7 +192,7 @@ The source SDK stays. Per client, change only three things: - **Credential** → a Bedrock bearer token, NOT the original provider key. For a short CLI or one-shot script, read it from the `AWS_BEARER_TOKEN_BEDROCK` env var; for any long-running process use the auto-refreshing `provide_token` client shown below (the env-var form expires within 12 hours). Do not leave the old `api_key=os.environ["OPENAI_API_KEY"]` line in place. - **Model ID** → the Bedrock model id from the `Mantle model map` context line (the `aws_model_id` from the migration plan). -**When your context has `Same model: true`**, the target is the same model the app already used. Do NOT change model parameters (`temperature`, penalties, stop sequences) — they are unchanged, and §9 will not ask about them. Limit edits to base_url, credential, and model id, plus the Chat Completions → Responses reshape below if the source used Chat Completions. +**When your context has `Same model: true`**, avoid cross-family prompt rewrites. Preserve the selected API from `Mantle surface`; apply only confirmed behavior-delta decisions. Astra parameter acceptance is unprobed: do not claim sampling, penalties, `n`, or state behavior is unchanged. Keep unresolved compatibility checks explicit and test the rewritten application on its actual API before claiming it works. OpenAI SDK example — proprietary GPT target (note the `openai/v1` path): @@ -197,7 +225,7 @@ client = BedrockOpenAI( ) ``` -**Chat Completions → Responses (proprietary GPT targets only).** Chat Completions is unverified for these models; every AWS sample uses Responses. If the source calls `chat.completions.create`, reshape it — this is the one part of the Mantle lane that is not config-only: +**Chat Completions → Responses (selected Responses path only).** If `Mantle surface: chat_completions`, preserve `chat.completions.create` and its response parsing; Astra supports this API. Do not apply this reshape. If the saved plan instead selects Responses and the source uses Chat, apply the confirmed API change below. Older GPT-5.x targets retain their dated model/API checks: ```python # Before @@ -227,7 +255,7 @@ client = anthropic.Anthropic( ) ``` -Preserve compatible SDK request/response shapes. A model version change still requires §9 target checks, including typed response parsing for Opus 5.5. Apply the Chat Completions → Responses reshape above for proprietary GPT targets. After applying the changes above, skip the Converse-specific guidance in the rest of §8; the §9 behavior-delta application still applies normally. +Preserve compatible SDK request/response shapes. A model version change still requires §9 target checks, including typed response parsing for Opus 5.5. Apply the Chat Completions → Responses reshape only when that API change was selected. Preserve the exact validated runtime profile ID or ARN. After applying the changes above, skip the Converse-specific guidance in the rest of §8; the §9 behavior-delta application still applies normally. **Converse for a proprietary GPT target is family-dependent.** GPT-5.5 / GPT-5.4 (`openai.gpt-5.5`, `openai.gpt-5.4`) have no `bedrock-runtime` surface — a boto3 `converse()` call against them fails at runtime; if your context pairs one with the Converse path, stop and report the contradiction. GPT-5.6 and GPT-6 (Astra, Sol, Luna, 6.1 Sol) DO have a `bedrock-runtime` path via CRIS ids (`us.`/`in.`/`global.` prefixed): when the plan says `migration_path: runtime_openai_cris`, a Converse or Responses rewrite against the CRIS id is valid (base URL `bedrock-runtime.{region}.amazonaws.com/openai/v1` for the OpenAI-compatible APIs; note Guardrails are Converse-only and prompt caching Responses-only on runtime). For a Responses-based source app, the mantle express lane above remains the default and smallest change. @@ -859,7 +887,11 @@ grep -rl "from openai\|import openai\|require.*openai\|from anthropic\|import an If any files still contain source SDK references, fix them before proceeding. Test directories are NOT excluded from this scan on purpose: the source SDK package is being removed from the manifest, so a leftover `import openai` in a customer test means `pytest` ImportErrors on the customer's machine — §18.0 should have migrated those tests; if one appears here, go back and fix it. -**Mantle express lane exception:** when this run used the Mantle express lane (§8, `Rewrite strategy: mantle`), the source-SDK imports are EXPECTED to remain — Mantle keeps the original SDK, so this residual scan does NOT apply. Verify instead that every client init points at Mantle and uses a Bedrock credential in the form §8 prescribes for that client's lifetime — and that model IDs were swapped to their Mantle forms: +**OpenAI-compatible lane exception:** when this run used `Rewrite strategy: mantle` or +`Rewrite strategy: runtime_openai`, the source-SDK imports are expected to remain, so this +residual scan does not apply. Verify the selected endpoint's `base_url`, Bedrock credentials +and model identifiers instead: bare IDs for Mantle, exact validated profiles for runtime. +The runtime lane must retain the Chat/Responses API verified by the evaluator. - **Short CLI / one-shot script:** `base_url` set to the Mantle path and `api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"]`. - **Server, worker, scheduled job, ECS service, or Lambda:** `BedrockOpenAI(..., bedrock_token_provider=lambda: provide_token(...))` (or the equivalent auto-refreshing client for the SDK in use), **no** `AWS_BEARER_TOKEN_BEDROCK` read in code, and `.env.example` does **not** set it. diff --git a/plugins/aws-startup-advisor/agents/llm2bedrock-prompt-evaluator.md b/plugins/aws-startup-advisor/agents/llm2bedrock-prompt-evaluator.md index 969d4ed3d..d18e1a13d 100644 --- a/plugins/aws-startup-advisor/agents/llm2bedrock-prompt-evaluator.md +++ b/plugins/aws-startup-advisor/agents/llm2bedrock-prompt-evaluator.md @@ -39,8 +39,9 @@ Read from prompt context (forwarded from llm2bedrock-code-analyzer, llm2bedrock- - **``** — `/.saws-migrate/golden-dataset/prompts.jsonl` (from T2-2). May be empty if T2-2 took the abort / paste / vision-no-images / embeddings path. - **``** — `/.saws-migrate/golden-dataset/templates/prompt_template.txt` (from T2-2). -- **``** — Bedrock target model ID from the migration plan, validated by llm2bedrock-code-analyzer §10. Substitute it in every Bedrock call below. **It also selects the API path:** a bare `openai.gpt-5*` / `openai.gpt-6*` id (excluding `-oss`) is mantle-served and must use the Responses client in §6a; a `us.`/`in.`/`global.`-prefixed `openai.gpt-5.6-*` / `openai.gpt-6*` CRIS id is a `bedrock-runtime` target and uses `boto3.converse` like any other profile; everything else uses `boto3.converse`. See the table at the top of §6. +- **``** — the exact validated target ID or ARN from the migration plan. It identifies the endpoint, not the API: bare proprietary GPT uses Mantle; supported CRIS uses runtime. `` selects Chat, Responses or Converse within that endpoint. Preserve both through every call; never remove a profile prefix to change the endpoint. - **``** — AWS region for Bedrock (the `AWS region:` line in your context). +- **``** — copy the normalized `Target API surface` context line: `chat_completions`, `responses`, or `converse`. It applies to both Mantle and runtime. If a direct legacy dispatch omits that line, call `preflight_bedrock.normalize_api_path` with the saved path, target IDs and `runtime_api`, then derive the same surface. An absent runtime API means Converse; never discard an explicit runtime Chat/Responses choice. - **From `llm2bedrock-code-analyzer` (`AiAnalysisData`)** — key fields: - `source_provider` — `openai` / `anthropic` / `google` / `cohere` / `custom`. Drives §9 baseline gating. (Vertex AI customers are emitted as `google` here; the analyzer's `errors` field carries the `vertex AI auth detected` signal that gates baseline collection upstream — by the time you reach §9, `source_baseline_available` already reflects that.) - `source_models` — list of source-model IDs. Pass `` to the §9 baseline skill verbatim. @@ -127,17 +128,17 @@ When reporting results, clearly separate which layers passed / failed / skipped. Create the eval results directory and verify Bedrock connectivity against the target model using the SAME API path the evaluation will use. -**First, pick the API path from `` — this decides every Bedrock +**First, use `` and the validated model ID — this decides every Bedrock call in §6, §9.5 and §10:** -| `` matches | API path | Connectivity | Vision smoke | Golden eval | -| ------------------------------------------------- | ---------------- | ------------ | ------------ | ------------------------ | -| bare `openai.gpt-5*` / `openai.gpt-6*` (not `-oss`) | Mantle Responses | §6a | §9.5a | §8 loop (same-model) | -| `us.`/`in.`/`global.` + `openai.gpt-5.6-*` / `openai.gpt-6*` (CRIS) | `boto3.converse` | §6 | §9.5 | §8 loop (same-model) | -| anything else (Claude, Nova, `openai.gpt-oss-*`) | `boto3.converse` | §6 | §9.5 | §10, or §8 if same-model | +| `` matches | API path | Connectivity | Vision smoke | Golden eval | +| --------------------------------------------------------- | ------------------- | ------------ | ------------ | ------------------------------- | +| bare `openai.gpt-5*` (not `-oss`) or `openai.gpt-6*` | Selected Mantle API | §6a | §9.5a | §8 if same-model; otherwise §10 | +| Supported CRIS with selected Chat/Responses | Runtime OpenAI API | §6a | §9.5a | §8 if same-model; otherwise §10 | +| Supported CRIS with Converse (legacy default) | `boto3.converse` | §6 | §9.5 | §8 if same-model; otherwise §10 | +| anything else (Claude, Nova, `openai.gpt-oss-*`) | `boto3.converse` | §6 | §9.5 | §10, or §8 if same-model | -The proprietary OpenAI GPT models are served ONLY on the `bedrock-mantle` -endpoint. Calling `boto3.converse` against one fails, so running the Converse +The bare proprietary GPT ids select the `bedrock-mantle` endpoint; supported CRIS ids select runtime. Calling `boto3.converse` against a bare Mantle id fails, so running the Converse check below for a mantle target would abort this agent before it ever reached §8's same-model short-circuit. Route those to §6a instead. @@ -186,26 +187,35 @@ Interpret the result: `bedrock_provider_available` from the orchestrator context (it is a rewrite-strategy flag, not an account-capability flag). -# 6a. Mantle connectivity check (proprietary OpenAI GPT targets) +# 6a. OpenAI-compatible connectivity (Mantle or runtime) -Use this INSTEAD of §6 when `` matches `openai.gpt-5*` / `openai.gpt-6*` (excluding -`-oss`). Same purpose, different endpoint and SDK. +Use this INSTEAD of §6 whenever the selected surface is Chat Completions or Responses, +including explicit runtime OpenAI choices. Only a selected Converse surface uses §6. +The script classifier `preflight_bedrock.is_mantle_model` defines this endpoint split. +Astra Standard Mantle supports `us-east-1` and `us-west-2`; its `us.` / `global.` ids remain on runtime. +For a runtime OpenAI target, interpret 401/403 as credential/model-access or `bedrock:InvokeModel` +authorization failures, and 404 as an unavailable selected profile/API. Do not prescribe Mantle +actions or change endpoint/API. The Mantle-specific recovery notes below apply only to bare IDs. ```bash mkdir -p /.saws-migrate/eval-results AWS_REGION= when your context has an `AWS profile` line> uv run --project python - <<'PY' import os, sys -from aws_bedrock_token_generator import provide_token -from openai import BedrockOpenAI +sys.path.insert(0, "") +from preflight_bedrock import create_openai_client region = os.environ.get('AWS_REGION', 'us-east-1') try: - client = BedrockOpenAI(aws_region=region, - bedrock_token_provider=lambda: provide_token(region=region), - max_retries=2) - r = client.responses.create(model='', input='ping', - max_output_tokens=16, store=False) - print('OK:', r.output_text) + client = create_openai_client('', region, max_retries=2) + if '' == 'chat_completions': + r = client.chat.completions.create( + model='', messages=[{'role': 'user', 'content': 'ping'}]) + text = r.choices[0].message.content + else: + r = client.responses.create(model='', input='ping', + max_output_tokens=16, store=False) + text = r.output_text + print('OK:', text) except Exception as e: print(f'FAIL [{type(e).__name__}]: {e}', file=sys.stderr) sys.exit(1) @@ -219,10 +229,10 @@ Interpret the result: region. These models are **in-region only**, so there is no cross-region inference profile to fall back to and `resolve-bedrock-model-id`'s inference-profile ranking does not apply. Return - `{ blocked: { reason: 'model_unresolvable', detail: ' — is not available on the bedrock-mantle endpoint in . Mantle is in-region only; for GPT-5.6 a bedrock-runtime CRIS id (us./in./global. prefixed) may cover the region instead, while GPT-5.5/5.4 need a supported region or a different model.' } }`. + `{ blocked: { reason: 'model_unresolvable', detail: ' — is not available on the bedrock-mantle endpoint in . Mantle is in-region only; for Astra use Virginia/Oregon Standard Mantle or a supported us./global. runtime profile with an explicit plan change; GPT-5.6 may use its us./in./global. CRIS forms, while GPT-5.5/5.4 need a supported region or a different model.' } }`. Use `model_unresolvable` rather than a new reason — its recovery path (user picks or pastes an ID, recorded in `resolved_model_overrides`) is exactly right - here. For GPT-5.6 a `us.`/`in.`/`global.`-prefixed CRIS candidate is legitimate + here. Astra may use only supported `us.` / `global.` CRIS candidates; check its own region matrix. For GPT-5.6 a `us.`/`in.`/`global.`-prefixed CRIS candidate is legitimate (it switches the run to the Converse path); for GPT-5.5 / GPT-5.4 no prefixed form exists — do not offer one. - **HTTP 401 / 403** — distinguish the two causes from the message. If it names @@ -264,19 +274,19 @@ Mantle target into §10 because its API changes have not yet been confirmed. - Skip rubric generation and scoring (no parameter-surface drift to score against). - Just verify each prompt works on Bedrock (connectivity + response format): run each prompt, check for errors, verify response is non-empty. -- **Use the API path §6 selected, not §10's script.** For an Anthropic 1P → Claude run that is `boto3.converse`. For an `openai.gpt-5*` / `openai.gpt-6*` run it is the Mantle Responses client from §6a — `client.responses.create(model='', input=, max_output_tokens=, store=False)`, reading `r.output_text`. §10's script is Converse-only and will fail against a mantle target; do not call it here. +- **Use the API path §6 selected.** Use §6a for selected Chat/Responses calls on either endpoint, or §6 for selected Converse calls. Reuse the endpoint-aware target call from §10 when iterating prompts, but skip quality scoring for this same-model branch. Do not infer the endpoint from `same_model_family`. - Output pass / fail per prompt; count successes as `success_count`. - Compute `pass_rate = success_count / total_cases` (connectivity-only ratio) and write `failures = total_cases - success_count`. - In §14, set `live_source_baseline: false` (no live comparison ran) and add `notes` prefix `same_model_family: true — connectivity-only verification, no rubric scoring`. T2-6 reads that prefix to render the report banner with "connectivity verified" instead of "judge scored X/Y prompts". Set `source_baseline_quality: 'unknown'` (no live baseline ran). -- **§9.5 still runs when `special_patterns.vision == true`** — do NOT skip it. It is the Layer 2 gate that proves image input works before any per-case image call, and skipping it would let a vision migration report `pass_rate: 1.0` without a single image request. §9.5 selects its own API path (Converse or Responses) the same way §6 does, so it is valid for both same-model variants. +- **§9.5 still runs when `special_patterns.vision == true`** — do NOT skip it. It is the Layer 2 gate that proves image input works before any per-case image call, and skipping it would let a vision migration report `pass_rate: 1.0` without a single image request. §9.5 selects its own API path (Converse, Responses or Chat) the same way §6 does, so it is valid for the selected same-model path. - **Send image input for every golden case that has one.** If a case's `image_path` is non-null, the verification call MUST include the image, not just the prompt text — see the per-path shapes below. A vision run whose cases are sent as text-only is not a verification, and the resulting pass rate is meaningless. -- Skip to §14 (no §9 baseline, no §10 Converse eval, no §11 scoring, no §12–§13 adaptation). §10 is Converse-only, so for a mantle target it is replaced by the loop described here, not merely skipped. +- Skip to §14 after the connectivity loop (no §9 baseline, §11 scoring, or §12–§13 adaptation). For a Pro-to-Astra or other model change this shortcut does not apply; continue through the selected API's golden evaluation and quality scoring. Build the request with the shared helper — do NOT assemble these dicts inline: ```python import sys; sys.path.insert(0, "") -from image_input import converse_message, responses_message +from image_input import converse_message, responses_message, chat_message # Golden-case fields are `user_prompt`, `system_prompt`, `image_path` — NOT `prompt`. # (Canonical record shape: llm2bedrock-log-ingestor §9.) @@ -298,6 +308,13 @@ if case.get("system_prompt"): items.append(responses_message(case["user_prompt"], case.get("image_path"), raw)) r = client.responses.create(model="", input=items, max_output_tokens=512, store=False) + +# Mantle Chat target (select this instead of the Responses call above): +messages = [] +if case.get("system_prompt"): + messages.append({"role": "system", "content": case["system_prompt"]}) +messages.append(chat_message(case["user_prompt"], case.get("image_path"), raw)) +r = client.chat.completions.create(model="", messages=messages) ``` Two things the helper exists to get right, both of which fail _only_ on golden cases @@ -374,7 +391,7 @@ If `special_patterns.vision == false`, SKIP this section. **This section runs for same-model migrations too** (both Anthropic 1P → Claude and OpenAI → the same GPT model). §8 short-circuits scoring, not image verification: it is the only gate that proves image input works before per-case image calls, so skipping it would let a vision migration pass without ever sending an image. -**Pick the API path exactly as §6 did.** The script below uses `boto3.converse` and is valid for Converse targets. For a mantle-only `openai.gpt-5*` / `openai.gpt-6*` target use §9.5a instead — the GPT-5.x/GPT-6 model cards list image input as supported, but it goes through the Responses API, not Converse. +**Pick the API path exactly as §6 did.** The script below is only for a selected Converse surface. For selected Chat/Responses on either endpoint, use §9.5a instead. Otherwise, run a one-shot Bedrock call against a public Wikipedia image to prove the SDK accepts image input before §10 attempts it on every golden prompt. If the public CDN isn't reachable, the smoke is INCONCLUSIVE — do NOT attempt an inline-fixture fallback (tiny synthetic JPEGs trip Claude's minimum-dimension validators and produce false `VISION_FAIL` even when the SDK is fine): @@ -415,9 +432,9 @@ Outcomes: - **`VISION_INFRA_SKIPPED`** — image download failed (DNS / proxy / air-gapped machine). Bedrock vision was NOT exercised; the test is inconclusive at this layer. Add to `notes`: `vision_smoke_skipped: CDN unreachable — Bedrock vision SDK path not exercised at smoke layer`. Proceed to §10 — golden cases carry their own images from T2-2, which will exercise the SDK directly. - **`VISION_FAIL`** — Bedrock rejected the image (`ValidationException`, `AccessDeniedException`, etc.). Surface the exact error in your result file's `notes`, STOP — golden vision eval will fail the same way. (If the failure is an `AccessDeniedException` on model access, route it through `{ blocked: { reason: 'model_access', detail: ... } }` per §6.) -# 9.5a Mantle vision smoke test (proprietary OpenAI GPT targets) +# 9.5a OpenAI-compatible vision smoke test (Mantle or runtime) -Use this INSTEAD of §9.5 when `` matches `openai.gpt-5*` / `openai.gpt-6*` (excluding `-oss`) AND `special_patterns.vision == true`. Same purpose and same three outcomes; the image travels as a base64 data URL on the Responses API rather than as Converse image bytes. +Use this INSTEAD of §9.5 when the selected surface is Chat/Responses AND `special_patterns.vision == true`. Preserve `` on either endpoint; both variants send the image as a base64 data URL. ```bash AWS_REGION= when your context has an `AWS profile` line> uv run --project python - <<'PY' @@ -432,25 +449,28 @@ except Exception as e: print(f"VISION_INFRA_SKIPPED [{type(e).__name__}]: {e}", file=sys.stderr) sys.exit(0) -from aws_bedrock_token_generator import provide_token -from openai import BedrockOpenAI +sys.path.insert(0, "") +from preflight_bedrock import create_openai_client region = os.environ.get("AWS_REGION", "us-east-1") data_url = "data:image/jpeg;base64," + base64.b64encode(img).decode() try: - client = BedrockOpenAI(aws_region=region, - bedrock_token_provider=lambda: provide_token(region=region), - max_retries=2) - r = client.responses.create( - model="", - input=[{"role": "user", "content": [ - {"type": "input_text", "text": "Describe this image briefly."}, - # `detail` is Required on ResponseInputImageParam in the pinned SDK. - {"type": "input_image", "image_url": data_url, "detail": "auto"}, - ]}], - max_output_tokens=32, - store=False, - ) - print("VISION_OK:", r.output_text) + client = create_openai_client('', region, max_retries=2) + if "" == "chat_completions": + r = client.chat.completions.create(model="", messages=[{ + "role": "user", "content": [ + {"type": "text", "text": "Describe this image briefly."}, + {"type": "image_url", "image_url": {"url": data_url}}, + ]}]) + text = r.choices[0].message.content + else: + r = client.responses.create( + model="", + input=[{"role": "user", "content": [ + {"type": "input_text", "text": "Describe this image briefly."}, + {"type": "input_image", "image_url": data_url, "detail": "auto"}, + ]}], max_output_tokens=32, store=False) + text = r.output_text + print("VISION_OK:", text) except Exception as e: print(f"VISION_FAIL [{type(e).__name__}]: {e}", file=sys.stderr) sys.exit(1) @@ -459,15 +479,13 @@ PY Outcomes are the same as §9.5: `VISION_OK` → proceed; `VISION_INFRA_SKIPPED` → record `vision_smoke_skipped` in `notes` and proceed (the golden cases carry their own images and will exercise the path directly); `VISION_FAIL` → surface the exact error and STOP, routing a model-access denial through `{ blocked: { reason: 'model_access', ... } }` and an IAM denial through `{ blocked: { reason: 'authz', ... } }` per §6a. -**On `VISION_FAIL` specifically, do not fall back to text-only prompts.** The migration's vision capability is unverified at that point, and a text-only pass rate would misrepresent it as working. The content-block shape above is the documented OpenAI Responses form; if it is rejected for a reason other than access, report the exact error rather than guessing at an alternative shape. +**On `VISION_FAIL` specifically, do not fall back to text-only prompts.** The migration's vision capability is unverified at that point, and a text-only pass rate would misrepresent it as working. The content-block shapes above match their selected OpenAI API; if it is rejected for a reason other than access, report the exact error rather than guessing at an alternative shape. # 10. Run golden prompt evaluation -> **Converse-only.** The script below calls `boto3.converse` and is valid only for -> the API path §6 selected. A `openai.gpt-5*` / `openai.gpt-6*` target never reaches this section — -> §8's same-model short-circuit fires first and verifies prompts through the Mantle -> Responses client from §6a. If you somehow arrive here with a mantle target, -> stop and report the contradiction rather than running a call that cannot work. +> **Use the endpoint and API selected in §6.** A model change (including Pro-to-Astra) +> still requires golden evaluation and scoring on Mantle. Same-model runs use §8's +> connectivity-only branch. Never send a bare Astra id to Converse. For each prompt in the golden dataset, run the evaluation via `python` stdin (avoids the brittle nested-heredoc + escaped-quote pattern that breaks on any literal `'` inside the script): @@ -483,11 +501,22 @@ import sys import time import boto3 from botocore.exceptions import ClientError - sys.path.insert(0, "") -from image_input import converse_message +from preflight_bedrock import is_mantle_model, create_openai_client +from image_input import converse_message, responses_message, chat_message -bedrock = boto3.client("bedrock-runtime", region_name=os.environ.get("AWS_REGION", "us-east-1")) +region = os.environ.get("AWS_REGION", "us-east-1") +model_id = "" +api_surface = "" +if api_surface not in {"responses", "chat_completions", "converse"}: + raise ValueError("Resolve the selected API surface before evaluation") +openai_api = api_surface in {"responses", "chat_completions"} +if is_mantle_model(model_id) and not openai_api: + raise ValueError("A bare Mantle model cannot use Converse") +if openai_api: + client = create_openai_client(model_id, region, max_retries=0) +else: + bedrock = boto3.client("bedrock-runtime", region_name=region) gd_path = "/.saws-migrate/golden-dataset/prompts.jsonl" if not os.path.exists(gd_path) or os.path.getsize(gd_path) == 0: @@ -516,17 +545,45 @@ if done_ids: print(f"RESUME: {len(done_ids)} cases already evaluated, {len(prompts)} remaining") -def converse_with_backoff(**kwargs): - """§10 throttling rule: 429 → exponential backoff + jitter, 2s start, - double each retry, 60s cap, max 5 retries. Raises Throttled on exhaustion - so the caller stops and returns the `partial` control state.""" +def target_with_backoff(prompt): + """Preserve endpoint/API and the existing bounded retry/partial-resume contract.""" + raw = None + if prompt.get("image_path"): + with open(prompt["image_path"], "rb") as image: + raw = image.read() delay = 2.0 - for attempt in range(6): # initial try + 5 retries + for attempt in range(6): try: - return bedrock.converse(**kwargs) - except ClientError as e: - code = e.response.get("Error", {}).get("Code", "") - if code != "ThrottlingException" or attempt == 5: + if openai_api: + if api_surface == "chat_completions": + messages = [] + if prompt.get("system_prompt"): + messages.append({"role": "system", "content": prompt["system_prompt"]}) + messages.append(chat_message(prompt["user_prompt"], prompt.get("image_path"), raw)) + r = client.chat.completions.create(model=model_id, messages=messages) + return r.choices[0].message.content + args = {"model": model_id, "input": [responses_message( + prompt["user_prompt"], prompt.get("image_path"), raw)], + "max_output_tokens": 4096, "store": False} + if prompt.get("system_prompt"): + args["instructions"] = prompt["system_prompt"] + return client.responses.create(**args).output_text + system = [{"text": prompt["system_prompt"]}] if prompt.get("system_prompt") else [] + r = bedrock.converse(modelId=model_id, + messages=[converse_message(prompt["user_prompt"], prompt.get("image_path"), raw)], + system=system, inferenceConfig={"maxTokens": 4096}) + text = "".join(block["text"] for block in r["output"]["message"]["content"] if "text" in block) + stop_reason = r.get("stopReason", "unknown") + if stop_reason in ("max_tokens", "model_context_window_exceeded", "guardrail_intervened", "content_filtered", "refusal", "tool_use"): + raise ValueError(f"non_final_response (stopReason={stop_reason})") + if not text: + raise ValueError(f"no_text_response (stopReason={stop_reason})") + return text + except Exception as e: + throttled = (getattr(e, "status_code", None) == 429 or + isinstance(e, ClientError) and + e.response.get("Error", {}).get("Code") == "ThrottlingException") + if not throttled or attempt == 5: raise time.sleep(min(delay, 60) + random.uniform(0, 1)) delay *= 2 @@ -546,52 +603,18 @@ except FileNotFoundError: results = [] throttled_out = False for prompt in prompts: - # Build Bedrock request - messages = [] - if prompt.get("system_prompt"): - system = [{"text": prompt["system_prompt"]}] - else: - system = [] - try: - image_path = prompt.get("image_path") - if image_path: - with open(image_path, "rb") as image: - raw = image.read() - else: - raw = None - messages.append(converse_message(prompt["user_prompt"], image_path, raw)) - # IMPORTANT: substitute the §6-validated ID here, not the raw plan ID — if §6's - # `resolve-bedrock-model-id` skill ran, the plan ID was stale and the validated - # one is what works for converse calls. - response = converse_with_backoff( - modelId="", - messages=messages, - system=system, - inferenceConfig={"maxTokens": 4096} - ) - bedrock_output = "".join( - block["text"] for block in response["output"]["message"]["content"] if "text" in block - ) - stop_reason = response.get("stopReason", "unknown") - if stop_reason in ("max_tokens", "model_context_window_exceeded", "guardrail_intervened", "content_filtered", "refusal", "tool_use"): - status = f"error: non_final_response (stopReason={stop_reason})" - elif not bedrock_output: - status = f"error: no_text_response (stopReason={stop_reason})" - else: - status = "success" - except ClientError as e: - if e.response.get("Error", {}).get("Code", "") == "ThrottlingException": - # Retry budget exhausted — stop here; remaining prompts stay unevaluated. - # Throttling is a pacing problem, NOT a quality failure: do not record it - # as an error row that §11 would score as FAIL. + bedrock_output = target_with_backoff(prompt) + status = "success" + except Exception as e: + throttled = (getattr(e, "status_code", None) == 429 or + isinstance(e, ClientError) and + e.response.get("Error", {}).get("Code") == "ThrottlingException") + if throttled: throttled_out = True break bedrock_output = "" status = f"error: {str(e)}" - except Exception as e: - bedrock_output = "" - status = f"error: {str(e)}" # Determine which baseline to score against. Prefer live source-model # output when available; fall back to the dataset's stored @@ -629,7 +652,7 @@ if throttled_out: PY ``` -**Throttling rule (429).** The script above implements it: on `ThrottlingException` each call retries with exponential backoff + jitter (start 2s, double each retry, cap 60s, max 5 retries via `converse_with_backoff`). Do NOT treat throttling as a block or a quality failure. If stdout contains a `THROTTLED_PARTIAL completed= total=` line, the retry budget was exhausted with cases unscored: score the completed cases normally if you wish, but your RETURN must be `{ partial: { completed: , total: , reason: 'throttled' } }` instead of the normal eval object (see §14). The same backoff applies to any re-scoring calls in §12 — reuse `converse_with_backoff` there. New accounts have floor Bedrock quotas, so throttling is expected here, not exceptional. +**Throttling rule (429).** The script above implements it: on `ThrottlingException` each call retries with exponential backoff + jitter (start 2s, double each retry, cap 60s, max 5 retries via `target_with_backoff`). Do NOT treat throttling as a block or a quality failure. If stdout contains a `THROTTLED_PARTIAL completed= total=` line, the retry budget was exhausted with cases unscored: score the completed cases normally if you wish, but your RETURN must be `{ partial: { completed: , total: , reason: 'throttled' } }` instead of the normal eval object (see §14). The same backoff applies to any re-scoring calls in §12 — reuse `target_with_backoff` there. New accounts have floor Bedrock quotas, so throttling is expected here, not exceptional. If the script's stdout contains `EMPTY_DATASET`, §7's gate was missed upstream — skip §11–§13 entirely and jump to §14 with the **zero-cases payload** (per §14). Do NOT proceed to §11; there are no successful evaluations to score. diff --git a/plugins/aws-startup-advisor/agents/llm2bedrock-report-generator.md b/plugins/aws-startup-advisor/agents/llm2bedrock-report-generator.md index 02d973cf8..5232c8e5c 100644 --- a/plugins/aws-startup-advisor/agents/llm2bedrock-report-generator.md +++ b/plugins/aws-startup-advisor/agents/llm2bedrock-report-generator.md @@ -133,7 +133,9 @@ The pricing script only covers Bedrock models. For the source provider (OpenAI, | claude-3-5-sonnet (1P API) | 3.00 | 15.00 | | claude-3-haiku (1P API) | 0.25 | 1.25 | -**Same-model pair cost depends on the inference option — state it conditionally, never as a flat figure.** In-Region and Geo CRIS are priced at OpenAI's _data residency_ tier (1.10x standard), so those moves cost ~10% more — quote the increase plainly and carry the non-cost justification (commitments, governance, residency, prompt caching, no behaviour delta). **GPT-5.6 on Global CRIS is priced at the standard rate — cost parity** — applicable only when the workload has no data-residency constraint; when the plan's `migration_path` is `runtime_openai_cris` with a `global.` id, report parity, not a premium. GPT-5.5/5.4 have no CRIS, so the ~10% figure always applies to them. **If a pair's workload exceeds 272K context**, price the Bedrock side at the GPT-5.6 long-context tier — 2.0x input, 1.5x output per option — and show that line separately; GPT-5.5/5.4 have no long-context tier so their usable window is 272K. +**Astra:** use its separately dated Bedrock rates and context/inference tier. Its source price is absent from the table below; verify that price or mark the source comparison unavailable. Do not estimate Astra from a lower GPT tier or assume GPT-5.x parity/premium. + +**GPT-5.x same-model pair cost depends on the inference option — state it conditionally, never as a flat figure.** In-Region and Geo CRIS are priced at OpenAI's _data residency_ tier (1.10x standard), so those moves cost ~10% more — quote the increase plainly and carry the non-cost justification (commitments, governance, residency, prompt caching, no behaviour delta). **GPT-5.6 on Global CRIS is priced at the standard rate — cost parity** — applicable only when the workload has no data-residency constraint; when the plan's `migration_path` is `runtime_openai_cris` with a `global.` id, report parity, not a premium. GPT-5.5/5.4 have no CRIS, so the ~10% figure always applies to them. **If a pair's workload exceeds 272K context**, price the Bedrock side at the GPT-5.6 long-context tier — 2.0x input, 1.5x output per option — and show that line separately; GPT-5.5/5.4 have no long-context tier so their usable window is 272K. If the source model is not in this table (likely for any model released after the table's last update — check the provider's public pricing page if you know current rates), note it as a gap in the Risk Assessment section, label the figure "(estimated from ``)", and estimate using the closest listed model's pricing. @@ -456,7 +458,7 @@ uv run --project python /iam_policy.py \ The script handles the dual-ARN pattern: foundation-model ARNs for plain model IDs and inference-profile ARNs for geo-prefixed IDs (e.g. `us.anthropic.claude-sonnet-4-6`). The output is a ready-to-use IAM policy JSON file. -**Mantle targets get different actions.** When any target is a proprietary OpenAI GPT model (`openai.gpt-5*`), the script also emits `bedrock-mantle:CreateInference` / `Get*` / `List*` scoped to `arn:aws:bedrock-mantle:::project/*`, plus `bedrock-mantle:CallWithBearerToken` on `*` (AWS does not permit narrowing that one). `bedrock:InvokeModel` alone does **not** authorize these models. Two consequences to carry into the report: +**Mantle targets get different actions.** When any target is a bare proprietary OpenAI GPT model (GPT-5.x or `openai.gpt-6-astra`), the script also emits `bedrock-mantle:CreateInference` / `Get*` / `List*` scoped to `arn:aws:bedrock-mantle:::project/*`, plus `bedrock-mantle:CallWithBearerToken` on `*` (AWS does not permit narrowing that one). `bedrock:InvokeModel` alone does **not** authorize these models. Two consequences to carry into the report: - For an all-mantle run there is no `BedrockInvokeModelScoped` statement, because those models have no foundation-model or inference-profile ARN to scope to. That absence is correct, not a generation failure. - Mantle authorizes at project granularity, so the policy cannot be scoped to specific model IDs. If the reader needs to restrict the model set, say so explicitly and point them at a service control policy. diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/references/decision-refs/model-selection.md b/plugins/aws-startup-advisor/skills/agent-advisor/references/decision-refs/model-selection.md index 77c735ac4..426e3ae89 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/references/decision-refs/model-selection.md +++ b/plugins/aws-startup-advisor/skills/agent-advisor/references/decision-refs/model-selection.md @@ -22,7 +22,8 @@ file — it is dispatched by source provider so the path vocabulary and rules di - **Anthropic** → `scripts/anthropic_model_recommendation.py` + `references/models/anthropic-bedrock-2026-09-24.json` (paths: `mantle_messages`, `runtime_converse`, `runtime_invoke`). - **OpenAI** → `scripts/openai_model_recommendation.py` + `references/models/openai-bedrock-2026-09-24.json` - (paths: `mantle_openai_responses`, `runtime_converse`; Chat Completions reshapes to Responses). + (paths: `mantle_openai_chat`, `mantle_openai_responses`, `runtime_converse`; preserve Astra's + supported source API and apply each model's dated capability and region checks). - **Azure OpenAI / other** → generic `provider_module_pending` until a dedicated module exists. Do not restate the per-provider path rules here — the dated catalogs and the engine are diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/references/models/openai-bedrock-2026-09-24.json b/plugins/aws-startup-advisor/skills/agent-advisor/references/models/openai-bedrock-2026-09-24.json index 96a626e83..62455179b 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/references/models/openai-bedrock-2026-09-24.json +++ b/plugins/aws-startup-advisor/skills/agent-advisor/references/models/openai-bedrock-2026-09-24.json @@ -246,20 +246,136 @@ "context_window": 1050000, "output_token_ceiling": 128000, "inference_profiles": { - "us": ["us-east-1", "us-east-2", "us-west-1", "us-west-2"], - "global": ["us-east-1", "us-east-2", "us-west-1", "us-west-2"] + "us": [ + "us-east-1", + "us-east-2", + "us-west-1", + "us-west-2", + "ca-central-1" + ], + "global": [ + "us-east-1", + "us-east-2", + "us-west-1", + "us-west-2", + "ca-central-1", + "eu-central-1", + "eu-north-1", + "eu-west-1", + "eu-west-2", + "eu-west-3", + "ap-northeast-1", + "ap-northeast-2", + "ap-northeast-3", + "ap-south-1", + "ap-southeast-1", + "ap-southeast-2", + "sa-east-1" + ] }, "region_availability": { - "commercial": ["us-east-1", "us-west-2"], + "commercial": [ + "us-east-1", + "us-west-2" + ], "govcloud": [], "evidence": "AWS GPT-6 Astra model card and regional table, retrieved 2026-10-10." }, - "capabilities": ["reasoning", "tool_or_function_calling", "image_input_vision"], - "capability_source": "AWS GPT-6 Astra model card, retrieved 2026-10-10: text and image input, tool calling, 1.05M context and 128K output.", + "capabilities": [ + "reasoning", + "image_input_vision" + ], + "capability_source": "AWS model card retrieved 2026-09-09: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html. Launched 2026-09-08, Active, no scheduled EOL; EOL no sooner than 2027-09-08. Text/image input and text output. Server-side tool calling is listed on mantle; runtime lists response streaming and supports non-streaming JSON Schema output (reverified 2026-10-03) and explicitly excludes server-side tool use. These are documentation claims, not feature-level account probes. Sampling, hosted tools, and reasoning-effort values require verification on the selected API.", "paths": { - "mantle_openai_responses": {"available": true, "model_id": "openai.gpt-6-astra", "requires_cris": false, "evidence": "AWS model card: Mantle Responses in us-east-1 and us-west-2."}, - "mantle_openai_chat": {"available": true, "model_id": "openai.gpt-6-astra", "requires_cris": false, "evidence": "AWS model card: Mantle Chat Completions supported."}, - "runtime_converse": {"available": true, "model_id": "openai.gpt-6-astra", "requires_cris": true, "evidence": "AWS model card: US and Global CRIS support Responses, Chat Completions, and Converse; Invoke is unsupported."} + "mantle_openai_responses": { + "available": true, + "model_id": "openai.gpt-6-astra", + "requires_cris": false, + "supported_regions": [ + "us-east-1", + "us-west-2" + ], + "capabilities": [ + "reasoning", + "image_input_vision", + "tool_or_function_calling", + "prompt_caching" + ], + "evidence": "AWS model card 2026-09-30: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html. Responses supported at /openai/v1/responses; Standard mantle is available in us-east-1 and us-west-2. Server-side tool calling and implicit/explicit Responses caching supported; application inference profiles unsupported." + }, + "mantle_openai_chat": { + "available": true, + "model_id": "openai.gpt-6-astra", + "requires_cris": false, + "supported_regions": [ + "us-east-1", + "us-west-2" + ], + "capabilities": [ + "reasoning", + "image_input_vision" + ], + "evidence": "AWS model card 2026-09-30: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html. Chat Completions supported at /openai/v1/chat/completions in us-east-1 and us-west-2; prompt caching is Responses-only. Tool behavior on Chat Completions remains unprobed." + }, + "runtime_converse": { + "available": true, + "model_id": "openai.gpt-6-astra", + "requires_cris": true, + "supported_regions": [ + "us-east-1", + "us-east-2", + "us-west-1", + "us-west-2", + "ca-central-1", + "eu-central-1", + "eu-north-1", + "eu-west-1", + "eu-west-2", + "eu-west-3", + "ap-northeast-1", + "ap-northeast-2", + "ap-northeast-3", + "ap-south-1", + "ap-southeast-1", + "ap-southeast-2", + "sa-east-1" + ], + "inference_profile_regions": { + "us.openai.gpt-6-astra": [ + "us-east-1", + "us-east-2", + "us-west-1", + "us-west-2", + "ca-central-1" + ], + "global.openai.gpt-6-astra": [ + "us-east-1", + "us-east-2", + "us-west-1", + "us-west-2", + "ca-central-1", + "eu-central-1", + "eu-north-1", + "eu-west-1", + "eu-west-2", + "eu-west-3", + "ap-northeast-1", + "ap-northeast-2", + "ap-northeast-3", + "ap-south-1", + "ap-southeast-1", + "ap-southeast-2", + "sa-east-1" + ] + }, + "capabilities": [ + "reasoning", + "image_input_vision", + "streaming", + "structured_output_json" + ], + "evidence": "AWS model card, runtime JSON Schema reverified 2026-10-03: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html. Runtime APIs: Responses, Chat Completions, Converse; Invoke and Messages unsupported. US Geo/Global CRIS only, no in-region or EU/India Geo form. Guardrails and application inference profiles are Converse-only. Non-streaming JSON Schema outputs supported: Converse requires outputConfig.textFormat.structure.jsonSchema and additionalModelRequestFields.text.format.strict=true. Streaming structured output and server-side tool use are not evidenced. Runtime TPM accounting uses 10x output-token burndown." + } } }, "openai_gpt_6_sol": { diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/migration-plan/migration-plan.md b/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/migration-plan/migration-plan.md index a2437d67f..cb0ce6876 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/migration-plan/migration-plan.md +++ b/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/migration-plan/migration-plan.md @@ -243,6 +243,7 @@ Write the injection context to `$RUN_DIR/migration-plan-injection.json`: "endpoint_contract": "", "model": "", "api_path": "", + "invocation_model_id": "", "source": "", "services": "", "evidence": "" @@ -353,11 +354,43 @@ are not modified and gcp never sees this step: ### Step 3.5 — Validate the advisor model/path contract (advisor wins) For every matched, model-bearing unit, compare the plan's model and migration path with -`design.json.units[].model_recommendation.{model,api_path}`. +that unit's `design.json` model contract, including its separate `invocation_model_id`. +Carry the injected invocation identifier into the corresponding GCP `aws_model_id` unchanged. +For a proprietary GPT `runtime_converse` decision, GCP represents the same API as +`migration_path: "runtime_openai_cris"` with `required_client_api: "converse"` on the matched model row. +An absent `runtime_api` on a legacy runtime plan means Converse; an explicit Chat/Responses +choice is not equivalent. Do not replace a confirmed Global profile with US Geo or strip an ARN. +Read each matched model row's `migration_path` and `required_client_api`; use legacy `runtime_api` and the `code_migration` +defaults only for a single-unit legacy plan. A secondary unit must not inherit the primary's +API or invocation profile. Missing per-unit data requires re-running Design with the injection. + +Pass the resolved per-model API to `runtime_api` below. A current Design plan with no +explicit API defaults to Responses and cannot satisfy an advisor Converse decision. + +Use this comparison for each unit (including the single-unit case). It separates logical model +identity from the invocation identifier while preserving the existing exact comparison for other paths: + +```python +def matches_advisor_model(advisor, plan_model, migration_path, runtime_api=None): + model = advisor["model"] + proprietary_runtime = model.startswith(("openai.gpt-6", "openai.gpt-5.6-")) + if advisor["api_path"] == "runtime_converse" and proprietary_runtime: + invocation = advisor.get("invocation_model_id") + if not invocation: + return False + profile_id = invocation.rsplit("/", 1)[-1] + logical_model = (profile_id.split(".", 1)[1] + if profile_id.startswith(("us.", "in.", "global.")) else profile_id) + return (logical_model == model and plan_model == invocation + and migration_path == "runtime_openai_cris" + and (runtime_api or "converse") == "converse") + return plan_model == model and migration_path == advisor["api_path"] +``` -1. If both match, annotate the design block with +1. If the comparison matches, annotate the design block with `"advisor_model_contract": "validated"` and continue. -2. If either differs, do not rewrite `design.json`, the recommendation, or the report. +2. If model, API or invocation profile differs (or the required profile is unresolved), + do not rewrite `design.json`, the recommendation, or the report. Record the proposed value and rationale in the design block as `plan_model_mismatch`, then STOP with `_halt_and_inform`. Tell the user which requirement or account/region probe caused the mismatch. Resolution MUST return to Model Recommend, update diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/model-recommend/model-recommend.md b/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/model-recommend/model-recommend.md index 256f0a1e4..f2a219a7a 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/model-recommend/model-recommend.md +++ b/plugins/aws-startup-advisor/skills/agent-advisor/references/phases/model-recommend/model-recommend.md @@ -110,6 +110,10 @@ Sol/Luna/6.1 Sol support all four APIs. Every selected path remains provisional in the target account and region. Azure OpenAI remains an explicit `provider_module_pending` generic result. +For Astra, preserve a compatible explicit API preference before the source default. +Keep Astra selected and require a decision when its path, region or feature evidence +cannot satisfy the request; do not silently substitute another model. + **OpenRouter/LiteLLM-sourced OpenAI models still use the OpenAI module.** Discover records the underlying `provider` (e.g. `openai`) even when the calls transit a gateway — see `discover.md`'s gateway-detection rule. Do not treat a gateway-routed source as `unknown` or route it to the @@ -142,7 +146,7 @@ For Anthropic source paths, scan for migration-sensitive features: For OpenAI source paths, scan instead for OpenAI feature codes: - `tool_or_function_calling`, `structured_output_json`, `streaming`, `image_input_vision` -- `reasoning`, `sampling_params`, `max_tokens`, `multiple_candidates_n` +- `reasoning`, `sampling_params`, `max_tokens`, `multiple_candidates_n`, `prompt_caching` - `web_search`, `file_search_retrieval`, `files_api`, `vector_stores`, `assistants_threads` - `audio_modality`, `embeddings`, `images`, `conversation_state` diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/openai_model_recommendation.py b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/openai_model_recommendation.py index 69c140edb..a7f1289cf 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/openai_model_recommendation.py +++ b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/openai_model_recommendation.py @@ -130,7 +130,7 @@ def _delta(code, category, description): def _converse_tier_for_source(source): sid = _primary_source_id(source).lower() - if "5.6-sol" in sid: + if "5.6-sol" in sid or "6-astra" in sid: return "anthropic_claude_opus_5_5" if "5.6-luna" in sid: return "anthropic_claude_haiku_4_5" @@ -227,6 +227,8 @@ def _catalog_model_for_path(catalog, path, detected_features=None, requirements= path_config = model["paths"].get(path, {}) if path_config.get("available") is not True: continue + if "capabilities" in path_config: + model = {**model, "capabilities": path_config["capabilities"]} availability = model.get("region_availability") if path.startswith("mantle_") and region and region != "unknown" and availability: supported_regions = set(availability.get("commercial", [])) @@ -235,6 +237,7 @@ def _catalog_model_for_path(catalog, path, detected_features=None, requirements= continue if ( path == "runtime_converse" + and not path_config.get("inference_profile_regions") and path_config.get("requires_cris") and _hard_profile_requirement(requirements or {}) and _resolve_invocation_model_id( @@ -359,6 +362,8 @@ def _target_accepts_sampling(target_model): """ if target_model is None or target_model.get("generation") == "bedrock_native": return None + if target_model.get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)": + return None version = _version_tuple(target_model.get("version")) if version is None: return None @@ -493,6 +498,15 @@ def _reasoning_findings(source, requirements, target_model, path, detected_featu "Verify sampling acceptance for the exact target model and surface.", ) ) + elif target_model and target_model.get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)": + tuning.append( + _finding( + "sampling_params_unverified", "[TUNE]", + "Astra's model card does not specify accepted sampling parameters.", + "Probe sampling and reasoning settings on the selected API; do not " + "inherit GPT-5.x parameter acceptance or rejection rules.", + ) + ) return blocks, tuning, deltas @@ -523,7 +537,7 @@ def _chat_to_responses_deltas(): ] -def _feature_findings(detected_features, requirements, path=None): +def _feature_findings(detected_features, requirements, path=None, target_model=None): """Structured output, n, tools, state, and hosted-tool/modality impacts. Findings are path-aware: Mantle Responses keeps verified capabilities @@ -536,7 +550,20 @@ def _feature_findings(detected_features, requirements, path=None): blocks, tuning, deltas, impacts = [], [], [], [] if "structured_output_json" in detected: - if is_responses: + if path == "runtime_converse" and (target_model or {}).get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)": + deltas.append( + _delta( + "structured_output_text_format", + "feature", + "Astra runtime JSON Schema output is documented for non-streaming calls. " + "Set outputConfig.textFormat.type=json_schema and put name plus schema " + "(encoded as a JSON string) in outputConfig.textFormat.structure.jsonSchema. " + "Also set additionalModelRequestFields.text.format.strict=true. Use an " + "object schema with all fields required and additionalProperties=false; " + "validate the schema and check refusal/incomplete responses before parsing.", + ) + ) + elif is_responses: # Verified on Mantle: responses.parse(text_format=Model) returns a typed object. deltas.append( _delta( @@ -557,15 +584,26 @@ def _feature_findings(detected_features, requirements, path=None): "Responses parse() helper is not available.", ) ) - if (requirements.get("uses_n") or "multiple_candidates_n" in detected) and not is_chat: - deltas.append( - _delta( - "responses_no_n", - "feature", - "The Responses API does not support n; request multiple candidates with " - "repeated calls.", + if requirements.get("uses_n") or "multiple_candidates_n" in detected: + if path == "mantle_openai_chat": + if target_model and target_model.get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)": + tuning.append( + _finding( + "multiple_candidates_n_unverified", "[TUNE]", + "Astra Chat Completions support for n has not been verified.", + "Probe n on the selected model and endpoint before preserving it; " + "if unsupported, request multiple candidates with repeated calls.", + ) + ) + else: + deltas.append( + _delta( + "responses_no_n", + "feature", + "The Responses API does not support n; request multiple candidates with " + "repeated calls.", + ) ) - ) if "tool_or_function_calling" in detected: if is_chat: tool_description = ( @@ -580,7 +618,17 @@ def _feature_findings(detected_features, requirements, path=None): ) deltas.append(_delta("tool_result_shape", "feature", tool_description)) if "conversation_state" in detected or requirements.get("uses_hosted_state"): - if is_responses: + if target_model and target_model.get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)": + deltas.append( + _delta( + "conversation_state_ownership", + "path", + "Astra's hosted state behavior has not been probed on this API. Verify " + "retention and continuation semantics; use application-owned history " + "until those requirements are confirmed.", + ) + ) + elif is_responses: deltas.append( _delta( "conversation_state_ownership", @@ -606,7 +654,7 @@ def _feature_findings(detected_features, requirements, path=None): _delta( "conversation_state_ownership", "path", - "A Bedrock-native Converse target has no hosted Responses state; carry " + "This path does not use hosted Responses state; carry " "conversation history in an application-owned store.", ) ) @@ -698,6 +746,8 @@ def _unsupported_required_capabilities(target_model, detected_features, requirem "reasoning", } ) + if target_model.get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)" and "prompt_caching" in needed: + checkable.add("prompt_caching") return sorted(checkable - catalog_caps) @@ -709,6 +759,8 @@ def _compatibility(detected_features, requirements, path, target_model=None): text_features = detected.intersection( {"tool_or_function_calling", "structured_output_json", "streaming", "image_input_vision", "reasoning"} ) + if (target_model or {}).get("display_name") == "OpenAI GPT-6 Astra (on Bedrock)" and "prompt_caching" in detected: + text_features.add("prompt_caching") catalog_caps = set((target_model or {}).get("capabilities") or []) native = sorted(text_features.intersection(catalog_caps)) if target_model else [] unsupported = sorted(text_features - catalog_caps) if target_model else [] @@ -788,13 +840,26 @@ def _verification(region, catalog, path, requires_cris, invocation_model_id, sel def _decision_options(catalog, workload, region): - """Two-sided option set: Mantle continuity vs runtime Converse governance.""" + """Two-sided option set: the selected Mantle API vs runtime Converse governance.""" options = [] - detected = workload.get("detected_features") or [] + detected = [feature for feature, status in _feature_assessment(workload).items() + if status == "detected"] + astra = _same_model_runtime_key(workload["source"]) == "openai_gpt_6_astra" mantle_path = _mantle_path(workload["source"], workload["requirements"]) + if astra: + preferred = workload["requirements"].get("preferred_api_path") + if preferred in {"mantle_openai_chat", "mantle_openai_responses"}: + mantle_path = preferred + elif workload["source"].get("api_surface") == "chat_completions": + mantle_path = "mantle_openai_chat" + # A decision may offer changing an incompatible Chat preference to the + # documented Responses caching path; it must not silently select it. + needed = set(detected) | set(workload["requirements"].get("critical_features") or []) + if "prompt_caching" in needed: + mantle_path = _mantle_path(workload["source"], workload["requirements"]) mantle, _ = _catalog_model_for_path( catalog, mantle_path, detected, workload["requirements"], - candidate_order=_mantle_candidate_order(catalog, workload["source"]), + candidate_order=["openai_gpt_6_astra"] if astra else _mantle_candidate_order(catalog, workload["source"]), region=region, ) mantle_requires_region_change = False @@ -859,9 +924,9 @@ def _decision_options(catalog, workload, region): path_config, invocation_model_id ), "reason": ( - "SAME-MODEL governance path: this OpenAI target runs on bedrock-runtime via " - "a CRIS id — Guardrails (Converse API only), invocation logging, and " - "same-model behavior without a family change." + "SAME-MODEL governance path: this OpenAI target runs on bedrock-runtime via a " + "CRIS id — Guardrails (Converse API only) and invocation logging without a " + "model change. Check the model's region/profile matrix and dated prices." if model_key == _same_model_runtime_key(workload["source"]) else "Uses Bedrock-native Converse request/response shapes and a " "cross-family model; requires rewriting the OpenAI integration." @@ -1014,6 +1079,35 @@ def _unresolved( "evaluated before capability and numeric fallbacks." ) + # Astra's new paths must not inherit the older GPT-5.x defaults. Keep this + # source on Astra and fail closed when the required path lacks evidence. + if _same_model_runtime_key(source) == "openai_gpt_6_astra": + candidate_order = ["openai_gpt_6_astra"] + preferred = requirements.get("preferred_api_path") + if preferred: + available = catalog["models"]["openai_gpt_6_astra"]["paths"].get(preferred, {}) + if (available.get("available") is not True + or runtime_required and preferred != "runtime_converse" + or continuity_required and preferred == "runtime_converse"): + return _unresolved( + "The explicit Astra API preference conflicts with the available paths " + "or required API/governance constraints.", [], + _finding( + "preferred_api_path_conflict", "[BLOCKS]", + f"The requested {preferred} path cannot satisfy this workload.", + "Choose a supported API or revise the conflicting requirements, " + "then rerun Model Recommend.", + ), preferred, + ) + path = preferred + elif not runtime_required and surface == "chat_completions": + path = "mantle_openai_chat" + rationale_head = ( + "Keep GPT-6 Astra on Bedrock using the compatible explicit API preference, " + "otherwise the source-compatible API or required runtime Converse path; " + "apply Astra's own region and feature evidence." + ) + # Evidence-driven selection: pick the first candidate on the path whose catalog # capabilities cover the required/detected text features (a newer model without # feature-level evidence falls back to an older probed one). G02: fail closed @@ -1071,6 +1165,19 @@ def _unresolved( path, ) + needed = set(detected) | set(requirements.get("critical_features") or []) + if (model_key == "openai_gpt_6_astra" and path == "runtime_converse" + and {"structured_output_json", "streaming"} <= needed): + return _unresolved( + rationale_head, [], + _finding( + "structured_output_streaming_unverified", "[BLOCKS]", + "Astra runtime JSON Schema output is documented only for non-streaming calls.", + "Use non-streaming structured output or separate the streaming workload, " + "then update requirements and rerun Model Recommend.", + ), path, + ) + numeric_conflicts = _numeric_requirement_conflict(model, requirements) if numeric_conflicts: return _unresolved( @@ -1089,11 +1196,41 @@ def _unresolved( invocation_model_id = _resolve_invocation_model_id( path_config["model_id"], path_config["requires_cris"], requirements, model, region ) + regions = path_config.get("supported_regions") + profile_regions = path_config.get("inference_profile_regions") + if regions is not None and region not in regions: + return _unresolved( + rationale_head, [], + _finding( + "model_region_unavailable", "[BLOCKS]", + f"{model['display_name']} is not listed for {path} in {region}.", + "Choose a supported region or a supported CRIS path with explicit " + "data-residency consent, or evaluate a different model.", + ), path, + ) + if profile_regions is not None and (invocation_model_id is not None or _hard_profile_requirement(requirements)): + profile_key = invocation_model_id + if invocation_model_id and invocation_model_id.startswith("arn:"): + match = re.fullmatch( + r"arn:aws:bedrock:([^:]+):[0-9]{12}:inference-profile/([^/]+)", + invocation_model_id, + ) + profile_key = match.group(2) if match and match.group(1) == region else None + if region not in profile_regions.get(profile_key, []): + return _unresolved( + rationale_head, [], + _finding( + "inference_profile_unverified", "[BLOCKS]", + f"{invocation_model_id} has no catalog evidence for calls from {region}.", + "Select a cataloged profile available from this region, or verify an " + "application inference profile in the target account before proceeding.", + ), path, + ) source_analysis = _source_analysis(source, model) # --- Findings (target- and path-derived) --- r_blocks, r_tuning, r_deltas = _reasoning_findings(source, requirements, model, path, detected) - f_blocks, f_tuning, f_deltas, impacts = _feature_findings(detected, requirements, path) + f_blocks, f_tuning, f_deltas, impacts = _feature_findings(detected, requirements, path, model) blocks = r_blocks + f_blocks tuning = r_tuning + f_tuning deltas = list(r_deltas) @@ -1131,12 +1268,33 @@ def _unresolved( _finding( "chat_completions_to_responses_required", "[BLOCKS]", - "GPT-5.x on Mantle rejects Chat Completions; the source must reshape to the " - "Responses API.", - "Apply the request/response/tool/state reshape deltas before cutover; do not " - "target mantle_openai_chat for GPT-5.x.", + ( + "The selected Astra Responses path requires reshaping the Chat source." + if model_key == "openai_gpt_6_astra" + else "GPT-5.x on Mantle rejects Chat Completions; the source must reshape " + "to the Responses API." + ), + "Apply the request/response/tool/state reshape deltas before cutover.", ) ) + if path == "mantle_openai_chat" and surface == "responses": + deltas.extend([ + _delta( + "responses_to_chat_request", "path", + "Map Responses input/instructions to Chat messages/system content and " + "verify the selected model's output-budget parameters.", + ), + _delta( + "responses_to_chat_response", "path", + "Read choices[0].message.content instead of output_text; preserve Chat " + "messages and tool_call_id rather than Responses output items.", + ), + ]) + blocks.append(_finding( + "responses_to_chat_completions_required", "[BLOCKS]", + "The selected Astra Chat path requires reshaping the Responses source.", + "Apply the request/response/tool/state mapping before cutover.", + )) deltas.extend(f_deltas) if path == "runtime_converse": diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_openai_model_recommendation.py b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_openai_model_recommendation.py index d9baf837e..ef92d7291 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_openai_model_recommendation.py +++ b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_openai_model_recommendation.py @@ -933,3 +933,254 @@ def test_same_model_governance_still_recommended_end_to_end(): assert rec["decision_status"] == "recommended" assert rec["primary_model"] == "openai.gpt-5.6-luna" assert rec["api_path"] == "runtime_converse" + + +@pytest.mark.parametrize("surface,path", [ + ("responses", "mantle_openai_responses"), + ("chat_completions", "mantle_openai_chat"), +]) +@pytest.mark.parametrize("region", ["us-east-1", "us-west-2"]) +def test_astra_keeps_source_model_and_api_on_mantle(surface, path, region): + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"], "api_surface": surface}), + region, OPENAI_CATALOG, + ) + assert rec["decision_status"] == "recommended" + assert rec["primary_model"] == "openai.gpt-6-astra" + assert rec["api_path"] == path + assert rec["source_analysis"]["source_family"] == "reasoning" + assert rec["source_analysis"]["version_changed"] is False + assert "chat_completions_to_responses_required" not in _codes(rec["blocks"]) + assert "sampling_params_accepted" not in _codes(rec["tuning"]) + assert "sampling_params_unverified" in _codes(rec["tuning"]) + + +@pytest.mark.parametrize("region", ["us-east-2", "eu-west-1"]) +@pytest.mark.parametrize("surface", ["responses", "chat_completions"]) +def test_astra_mantle_outside_supported_regions_requires_decision(region, surface): + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"], "api_surface": surface}), + region, OPENAI_CATALOG, + ) + assert rec["decision_status"] == "decision_required" + assert rec["primary_model"] is None + assert "mantle_region_unavailable" in _codes(rec["blocks"]) + + +@pytest.mark.parametrize("surface,preferred,expected,delta", [ + ("chat_completions", "mantle_openai_responses", "mantle_openai_responses", + "chat_to_responses_request"), + ("chat_completions", "mantle_openai_chat", "mantle_openai_chat", None), + ("responses", "mantle_openai_chat", "mantle_openai_chat", "responses_to_chat_request"), + ("responses", "mantle_openai_responses", "mantle_openai_responses", None), + ("chat_completions", "runtime_converse", "runtime_converse", "openai_sdk_to_converse"), +]) +def test_astra_explicit_api_preference_reaches_validated_output(surface, preferred, expected, delta): + data = { + "schema_version": 2, "region": "us-west-2", "primary_unit": "chat-svc", + "workloads": [_workload( + source={"model_ids": ["gpt-6-astra"], "api_surface": surface}, + requirements={"preferred_api_path": preferred, "data_residency": "global_allowed"}, + )], + } + jsonschema.validate(data, json.loads((SCRIPTS / "schemas/model-recommendation-input.json").read_text())) + result = model_recommendation.recommend(data) + jsonschema.validate(result, json.loads((SCRIPTS / "schemas/model-recommendation.json").read_text())) + rec = result["workloads"]["chat-svc"] + assert rec["decision_status"] == "recommended" + assert rec["api_path"] == expected + assert rec["primary_model"] == "openai.gpt-6-astra" + if delta: + assert delta in _delta_codes(rec) + assert not any("GPT-5.x" in block["message"] for block in rec["blocks"]) + + +@pytest.mark.parametrize("preferred,constraints", [ + ("mantle_openai_chat", {"governance": ["guardrails"]}), + ("mantle_openai_responses", {"governance": ["guardrails"]}), + ("runtime_converse", {"api_continuity": "required"}), + ("runtime_invoke", {}), + ("mantle_messages", {}), +]) +def test_astra_api_preference_conflicts_require_an_explicit_decision(preferred, constraints): + rec = oai.recommend_openai_workload( + _workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={"preferred_api_path": preferred, **constraints}, + ), "us-west-2", OPENAI_CATALOG, + ) + assert rec["decision_status"] == "decision_required" + assert rec["api_path"] is None + assert "preferred_api_path_conflict" in _codes(rec["blocks"]) + + +@pytest.mark.parametrize("region", ["us-east-1", "us-west-2"]) +@pytest.mark.parametrize("surface,preferred,expected", [ + ("chat_completions", None, None), + ("responses", "mantle_openai_chat", None), + ("responses", None, "mantle_openai_responses"), + ("chat_completions", "mantle_openai_responses", "mantle_openai_responses"), +]) +def test_astra_required_caching_is_checked_on_the_selected_api(region, surface, preferred, expected): + requirements = {"critical_features": ["prompt_caching"]} + if preferred: + requirements["preferred_api_path"] = preferred + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"], "api_surface": surface}, + requirements=requirements), + region, OPENAI_CATALOG, + ) + assert rec["api_path"] == expected + if expected is None: + assert rec["decision_status"] == "decision_required" + assert "unverified_capability" in _codes(rec["blocks"]) + assert "prompt_caching" in rec["blocks"][0]["message"] + assert rec["decision_options"][0]["api_path"] == "mantle_openai_responses" + else: + assert rec["decision_status"] == "recommended" + assert "prompt_caching" in rec["compatibility"]["native"] + + +def test_astra_decision_options_ignore_an_explicitly_absent_cache_signal(): + workload = _workload( + source={"model_ids": ["gpt-6-astra"], "api_surface": "chat_completions"}, + requirements={"preferred_api_path": "mantle_openai_chat"}, + detected_features=["prompt_caching"], + feature_status={"prompt_caching": "absent"}, + ) + options = oai._decision_options(OPENAI_CATALOG, workload, "us-west-2") + assert options[0]["api_path"] == "mantle_openai_chat" + + +@pytest.mark.parametrize("signals", [ + {"requirements": {"uses_n": True}}, + {"detected_features": ["multiple_candidates_n"]}, +]) +def test_astra_chat_n_requires_verification_without_responses_reshape(signals): + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"], "api_surface": "chat_completions"}, + **signals), + "us-west-2", OPENAI_CATALOG, + ) + assert rec["api_path"] == "mantle_openai_chat" + assert "multiple_candidates_n_unverified" in _codes(rec["tuning"]) + assert "responses_no_n" not in _delta_codes(rec) + assert "chat_completions_to_responses_required" not in _codes(rec["blocks"]) + + +def test_astra_runtime_preserves_model_with_sourced_streaming_and_limits(): + rec = _recommend(_workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={ + "governance": ["guardrails"], "critical_features": ["streaming"], + "data_residency": "global_allowed", "min_context_tokens": 1050000, + "expected_output_tokens": 128000, + }, + )) + assert rec["decision_status"] == "recommended" + assert rec["primary_model"] == "openai.gpt-6-astra" + assert rec["api_path"] == "runtime_converse" + assert rec["invocation_model_id"] == "global.openai.gpt-6-astra" + assert "streaming" in rec["compatibility"]["native"] + + +@pytest.mark.parametrize("region,geography", [ + ("eu-west-1", "eu"), + ("eu-west-1", "us"), + ("ap-south-1", "in"), +]) +def test_astra_rejects_unsupported_geo_profile_or_caller_region(region, geography): + rec = oai.recommend_openai_workload( + _workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={ + "governance": ["guardrails"], "data_residency": "geo_required", + "cris_geography": geography, + }, + ), region, OPENAI_CATALOG, + ) + assert rec["decision_status"] == "decision_required" + assert "inference_profile_unverified" in _codes(rec["blocks"]) + + +def test_astra_runtime_does_not_inherit_mantle_tools(): + rec = _recommend(_workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={"governance": ["guardrails"], + "critical_features": ["tool_or_function_calling"]}, + )) + assert rec["decision_status"] == "decision_required" + assert rec["primary_model"] is None + assert "unverified_capability" in _codes(rec["blocks"]) + + +def test_astra_runtime_json_schema_retains_model_and_documented_configuration(): + data = { + "schema_version": 2, "region": "us-west-2", "primary_unit": "chat-svc", + "workloads": [_workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={"governance": ["guardrails"], "data_residency": "global_allowed", + "critical_features": ["structured_output_json"]}, + )], + } + jsonschema.validate(data, json.loads((SCRIPTS / "schemas/model-recommendation-input.json").read_text())) + result = model_recommendation.recommend(data) + jsonschema.validate(result, json.loads((SCRIPTS / "schemas/model-recommendation.json").read_text())) + rec = result["workloads"]["chat-svc"] + assert rec["decision_status"] == "recommended" + assert rec["api_path"] == "runtime_converse" + assert rec["invocation_model_id"] == "global.openai.gpt-6-astra" + assert rec["verification"]["availability_claim"] == "provisional" + assert "structured_output_json" in rec["compatibility"]["native"] + guidance = next(d["description"] for d in rec["migration_deltas"] + if d["code"] == "structured_output_text_format") + assert "outputConfig.textFormat.structure.jsonSchema" in guidance + assert "additionalModelRequestFields.text.format.strict=true" in guidance + assert "JSON string" in guidance and "non-streaming" in guidance + + +@pytest.mark.parametrize("status,expected", [ + ("detected", "decision_required"), ("absent", "recommended"), +]) +def test_astra_json_schema_does_not_claim_streaming_support(status, expected): + rec = _recommend(_workload( + source={"model_ids": ["gpt-6-astra"]}, + requirements={"governance": ["guardrails"], "data_residency": "global_allowed", + "critical_features": ["structured_output_json"]}, + detected_features=["streaming"], feature_status={"streaming": status}, + )) + assert rec["decision_status"] == expected + if expected == "decision_required": + assert "structured_output_streaming_unverified" in _codes(rec["blocks"]) + + +@pytest.mark.parametrize("profile", [ + "arn:aws:bedrock:us-east-1:111122223333:inference-profile/us.openai.gpt-6-astra", + "arn:aws-cn:bedrock:us-west-2:111122223333:inference-profile/us.openai.gpt-6-astra", + "arn:aws:bedrock:us-west-2:bad-account:inference-profile/us.openai.gpt-6-astra", + "arn:aws:bedrock:us-west-2:111122223333:application-inference-profile/example", + "arn:aws:bedrock:us-west-2:111122223333:inference-profile/eu.openai.gpt-6-astra", + "arn:aws:bedrock:us-west-2:111122223333:inference-profile/us.openai.gpt-5.6-sol", +]) +def test_astra_rejects_unverified_profile_arn_or_region(profile): + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"]}, + requirements={"governance": ["guardrails"], "inference_profile_id": profile}), + "us-west-2", OPENAI_CATALOG, + ) + assert rec["decision_status"] == "decision_required" + assert "inference_profile_unverified" in _codes(rec["blocks"]) + + +@pytest.mark.parametrize("requirement,limit", [ + ("min_context_tokens", 1050001), + ("expected_output_tokens", 128001), +]) +def test_astra_rejects_capacity_above_documented_limit(requirement, limit): + rec = oai.recommend_openai_workload( + _workload(source={"model_ids": ["gpt-6-astra"]}, + requirements={requirement: limit}), + "us-west-2", OPENAI_CATALOG, + ) + assert rec["decision_status"] == "decision_required" + assert "unverified_capacity" in _codes(rec["blocks"]) diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_opus55_generation_contracts.py b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_opus55_generation_contracts.py index 15534c4b8..4ad45801f 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_opus55_generation_contracts.py +++ b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_opus55_generation_contracts.py @@ -162,7 +162,7 @@ def test_complete_evaluator_script_handles_typed_output(tmp_path, monkeypatch, b data.mkdir(parents=True) (tmp_path / ".saws-migrate/eval-results").mkdir() (data / "prompts.jsonl").write_text(json.dumps({"id": "case", "user_prompt": "Prompt", "assistant_response": "Answer"}) + "\n") - exec(compile(code.replace("", str(tmp_path)), "", "exec"), {}) # nosec B102 + exec(compile(code.replace("", str(tmp_path)).replace("", "converse").replace("", "us.anthropic.claude-opus-5-5"), "", "exec"), {}) # nosec B102 result = json.loads((tmp_path / ".saws-migrate/eval-results/raw_results.jsonl").read_text()) assert result["status"].startswith(expected) assert "Private reasoning" not in result["bedrock_response"] @@ -313,7 +313,7 @@ def send(*args): monkeypatch.setitem(sys.modules, "botocore", fake_core) monkeypatch.setitem(sys.modules, "botocore.exceptions", fake_exceptions) code = block(EVALUATOR, 'gd_path = "/.saws-migrate/golden-dataset/prompts.jsonl"') - exec(compile(code.replace("", str(tmp_path)), "", "exec"), {}) # nosec B102 + exec(compile(code.replace("", str(tmp_path)).replace("", "converse").replace("", "us.anthropic.claude-opus-5-5"), "", "exec"), {}) # nosec B102 evaluated = json.loads((outputs / "raw_results.jsonl").read_text()) assert evaluated["source_baseline_source"] == "static-unknown" assert evaluated["source_response"] == "Complete stored answer" @@ -332,7 +332,7 @@ def test_same_gpt_pending_api_delta_keeps_the_mantle_connectivity_loop(): assert "If `same_model_family: true`:" in section assert "no unresolved behavior changes" not in section assert "C4 later confirms user-visible deltas" in section - code = block(EVALUATOR, 'from image_input import converse_message, responses_message') + code = block(EVALUATOR, '# Golden-case fields are') nodes = ast.parse(code).body start = next(i for i, node in enumerate(nodes) if isinstance(node, ast.Assign) and isinstance(node.targets[0], ast.Name) and node.targets[0].id == "items") @@ -348,7 +348,9 @@ def test_same_gpt_pending_api_delta_keeps_the_mantle_connectivity_loop(): }, } # The literal model placeholder is replaced exactly as the evaluator does. - source = ast.unparse(ast.Module(body=nodes[start:], type_ignores=[])) + end = next(i for i, node in enumerate(nodes[start:], start) if isinstance(node, ast.Assign) + and isinstance(node.targets[0], ast.Name) and node.targets[0].id == "messages") + source = ast.unparse(ast.Module(body=nodes[start:end], type_ignores=[])) exec(compile(source.replace("", "openai.gpt-5.5"), "", "exec"), namespace) # nosec B102 assert calls[0]["model"] == "openai.gpt-5.5" @@ -381,7 +383,7 @@ def test_comparative_loop_preserves_each_golden_image(tmp_path, monkeypatch, ima monkeypatch.setitem(sys.modules, "boto3", fake_boto) monkeypatch.setitem(sys.modules, "botocore", fake_core) monkeypatch.setitem(sys.modules, "botocore.exceptions", fake_exceptions) - exec(compile(code.replace("", str(tmp_path)), "", "exec"), {}) # nosec B102 + exec(compile(code.replace("", str(tmp_path)).replace("", "converse").replace("", "us.anthropic.claude-opus-5-5"), "", "exec"), {}) # nosec B102 result = json.loads((tmp_path / ".saws-migrate/eval-results/raw_results.jsonl").read_text()) if image_kind in {"missing", "unsupported"}: assert result["status"].startswith("error:") and not client.requests diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_scoring.py b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_scoring.py index 956292931..a2e507428 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_scoring.py +++ b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_scoring.py @@ -721,6 +721,7 @@ def test_profile_is_well_formed(profile): "claude_opus_5_5": "claude-opus-5-5", "claude_sonnet_5": "claude-sonnet-5", "claude_haiku_4_5": "claude-haiku-4-5", + "openai_gpt_6_astra": "gpt-6-astra", "openai_gpt_5_6_sol": "gpt-5.6-sol", "openai_gpt_5_6_terra": "gpt-5.6-terra", "openai_gpt_5_6_luna": "gpt-5.6-luna", diff --git a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_verify_model_path.py b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_verify_model_path.py index 210d50e13..96f53ce3b 100644 --- a/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_verify_model_path.py +++ b/plugins/aws-startup-advisor/skills/agent-advisor/scripts/test_verify_model_path.py @@ -217,6 +217,62 @@ def __init__(self, responses=None, chat=None): self.chat = chat or FakeChat() +@pytest.mark.parametrize("region", ["us-east-1", "us-west-2"]) +@pytest.mark.parametrize("surface", ["chat_completions", "responses"]) +def test_astra_recommendation_probes_selected_api_in_supported_regions(region, surface): + data = _openai_input() + data["region"] = region + data["workloads"][0]["source"].update( + model_ids=["gpt-6-astra"], api_surface=surface + ) + recommendation = model_recommendation.recommend(data) + completions = FakeResponses(response={"model": "openai.gpt-6-astra"}) + client = type("ChatClient", (), { + "chat": type("Chat", (), {"completions": completions})(), + "responses": completions, + })() + result = verify_model_path.verify_recommendation( + recommendation, now=NOW, + openai_responses_client_factory=lambda region: client, + openai_chat_client_factory=lambda region: client, + ) + verification = result["workloads"]["openai-svc"] + assert verification["status"] == "passed" + assert verification["api_path"] == ( + "mantle_openai_chat" if surface == "chat_completions" else "mantle_openai_responses" + ) + assert verification["region"] == region + assert completions.calls[0]["model"] == "openai.gpt-6-astra" + if surface == "chat_completions": + assert completions.calls[0]["messages"][0]["role"] == "user" + else: + assert completions.calls[0]["input"] == verify_model_path.PROMPT + + +@pytest.mark.parametrize("prefix", ["us", "global"]) +@pytest.mark.parametrize("as_arn", [False, True]) +def test_astra_system_profile_representation_reaches_verifier_unchanged(prefix, as_arn): + model_id = f"{prefix}.openai.gpt-6-astra" + if as_arn: + model_id = f"arn:aws:bedrock:us-west-2:111122223333:inference-profile/{model_id}" + data = _openai_input({"governance": ["guardrails"], "inference_profile_id": model_id}) + data["region"] = "us-west-2" + data["workloads"][0]["source"]["model_ids"] = ["gpt-6-astra"] + schemas = pathlib.Path(model_recommendation.__file__).parent / "schemas" + jsonschema.validate(data, json.loads((schemas / "model-recommendation-input.json").read_text())) + recommendation = model_recommendation.recommend(data) + jsonschema.validate(recommendation, json.loads((schemas / "model-recommendation.json").read_text())) + selected = recommendation["workloads"]["openai-svc"] + assert selected["decision_status"] == "recommended" + assert selected["invocation_model_id"] == model_id + client = FakeRuntimeClient() + result = verify_model_path.verify_recommendation( + recommendation, now=NOW, runtime_client_factory=lambda region: client, + ) + assert result["workloads"]["openai-svc"]["status"] == "passed" + assert client.converse_calls[0]["modelId"] == model_id + + def _openai_input(requirements=None, source=None, region="us-east-2"): source_data = { "provider": "openai", @@ -536,6 +592,7 @@ def test_mantle_responses_probe_calls_responses_create_with_exact_id(): result = _verify_openai( {"api_continuity": "required"}, openai_responses_client_factory=lambda region: client, + openai_chat_client_factory=lambda region: client, ) verification = result["workloads"]["openai-svc"] @@ -551,6 +608,7 @@ def test_mantle_responses_failure_does_not_substitute_model(): result = _verify_openai( {"api_continuity": "required"}, openai_responses_client_factory=lambda region: client, + openai_chat_client_factory=lambda region: client, ) verification = result["workloads"]["openai-svc"] diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design-ai.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design-ai.md index 2db0721fb..9d3509b5e 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design-ai.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design-ai.md @@ -158,7 +158,7 @@ flat rates from `references/vendored/ai/bedrock-pricing-cache.md` throughout the do not apply the GPT multiplier. Verify other models' tiers individually. **Quota risk** (per `references/vendored/ai/bedrock-quotas.md`): high/very_high volume → -`quota_risk: "high"` + a pre-migration quota-increase flag; medium + Claude (5× burndown) → +`quota_risk: "high"` + a pre-migration quota-increase flag; medium + Claude (5× output burndown) or Astra runtime (10× output burndown) → `"medium"`; else `"low"`. Record on each `bedrock_models[]` entry. ## Part 1B: Volume-based strategy @@ -204,6 +204,8 @@ Map Azure AI infrastructure to AWS: ## Part 5: Code migration plan +Preserve `mantle_openai_chat` when the selected API is Chat Completions. + Generate before/after examples per detected `integration.pattern` and `ai_source`. For an `azure_openai`/`openai` source with the model on Bedrock, the **Mantle Responses API** is the primary path (`migration_path: "mantle_openai_responses"`, `model_change: false`): the app @@ -230,6 +232,12 @@ surface — `converse` for Astra; `converse` / `chat_completions` / `invoke` (ne for 6.1 Sol and the other non-Astra GPT-6/5.6 CRIS models. Do not leave the client on `responses` when Guardrails is required.** `Responses` stays the default for a region-only CRIS move with no Guardrails need. + +An Azure-specific client (`AzureOpenAI`, deployment/API-version configuration) must be adapted +to the selected Bedrock client; do not claim that setting OPENAI_BASE_URL alone changes that client. +Keep `model_change: false` only when the exact underlying model is unchanged; opaque Azure +deployment names need verified model identity, not a guessed GPT family. + Anthropic-SDK sources → boto3 Converse client swap. Record `code_migration.openrouter_path` when a router was detected (`same_model_mantle|direct|litellm|keep_openrouter`). diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design.md index 5b4f62bab..1ccb36c7a 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/design/design.md @@ -82,7 +82,7 @@ _postconditions: _on_failure: _halt_and_inform - _assert: "WHEN aws-design-ai.json exists: every design_blocks[] row has exactly one of target_bedrock_model / target_aws_service non-null (XOR), and every document_extraction/image_analysis/speech_transcription row has target_bedrock_model null, a non-null target_aws_service, and honest_assessment 'not_applicable' — per references/shared/schema-design-aws-ai.md" _on_failure: _halt_and_inform - - _assert: "WHEN aws-design-ai.json exists AND summary.ai_source is azure_openai or openai: every source model that is available on Bedrock and carried by the target region maps to itself with model_change false — not to a Claude/Nova substitute; and no proprietary GPT-5.5 or GPT-5.4 model ID is paired with a converse/bedrock-runtime migration path (GPT-5.6 and GPT-6 models MAY use migration_path: runtime_openai_cris on bedrock-runtime — this is not a violation)" + - _assert: "WHEN aws-design-ai.json exists AND summary.ai_source is azure_openai or openai: every source model that is available on Bedrock and carried by the target region maps to itself with model_change false — not to a Claude/Nova substitute; and no proprietary GPT-5.5 or GPT-5.4 model ID is paired with a converse/bedrock-runtime migration path (bare proprietary openai.gpt-* IDs use Mantle only; runtime_openai_cris accepts documented Astra us./global. and GPT-5.6/GPT-6 profiles with caller-region checks and the selected API; GPT-5.6 and GPT-6 models MAY use migration_path: runtime_openai_cris on bedrock-runtime — this is not a violation)" _on_failure: _halt_and_inform - _assert: "WHEN aws-design-ai.json exists: regional_warnings[] and multi_model_warnings[] are present (empty is fine); agentic_design is present only when agentic_profile.is_agentic is true; and App Runner appears nowhere" _on_failure: _halt_and_inform diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/estimate/estimate-ai.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/estimate/estimate-ai.md index 70431f289..059892283 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/estimate/estimate-ai.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/estimate/estimate-ai.md @@ -298,18 +298,23 @@ Do **not** repeat these as "costs" in the user-facing summary. Present the monthly and annual cost difference between current Azure AI spend and projected Bedrock cost: -- **If the model is unchanged** (`model_change: false`): projected cost is **about 10% higher**, - not the same, UNLESS the chosen path is Global CRIS (`migration_path: "runtime_openai_cris"` - with a `global.`-prefixed model ID) — Global CRIS is priced at OpenAI's standard list price, - i.e. cost PARITY with the source, not a premium. In-Region and Geo CRIS (and any GPT-5.5/GPT-5.4 - move, which has no CRIS at all) remain at the 1.10x data-residency-tier premium over OpenAI / - Azure OpenAI standard. Quote whichever applies plainly and make the case on non-cost grounds. If - any workload exceeds 272K context, price it at the long-context tier and show that separately; - it can dominate the comparison. +- **If the model is unchanged** (`model_change: false`): derive the difference from the selected + model's inference option/context tier and verified source rate. For Astra use its dedicated + short/long-context In-Region/Geo/Global rows. A $60 Global target versus $60 source is 0%, + not a fixed premium. If the source rate is unverified, mark the source comparison unavailable. - **If Bedrock is cheaper:** present monthly and annual savings clearly. - **If Bedrock is more expensive:** state clearly, justify with non-cost benefits, or note "not justified if cost is the only priority". +When both monthly totals are established for the same workload volume: + +```python +monthly_difference = bedrock_monthly - source_monthly +percentage_difference = monthly_difference / source_monthly * 100 if source_monthly > 0 else None +``` + +A zero source total has no percentage comparison; show its dollar difference only. + Reference `aws-design-ai.json` → `honest_assessment`. If `"recommend_stay"`, present prominently along with `honest_assessment_reason`. @@ -317,7 +322,8 @@ along with `honest_assessment_reason`. IAM/VPC/PrivateLink/KMS/CloudTrail governance, in-region processing for data residency, prompt caching (Claude, and GPT-5.6 at 90% off cached input with cached tokens exempt from the input-TPM quota), model flexibility (100+ models), AWS ecosystem (Guardrails, Knowledge Bases, AgentCore), -and — for a same-model move — the elimination of behavior-delta and prompt-regression risk. +and — for a same-model move — reduced model-change risk, while endpoint/API compatibility +still requires evaluation. **Pricing source caveat for OpenAI models:** the AWS Price List API does not carry the proprietary GPT-5.x models. A missing or empty price-list result is **not** evidence the model is @@ -366,7 +372,7 @@ equivalent of `estimate-infra.md` Part 8. | Condition | Verdict | `recommendation.path` | | --------------------------------------------------------------------------------------------------------------------------------- | ------------------- | --------------------- | -| **Same model on Bedrock** (`model_change: false`) — ~10% higher, short-context; non-cost benefits carry it | Migrate with caveat | `migrate_optimized` | +| **Same model on Bedrock** (`model_change: false`) — use the verified Part 5 cost delta and applicable non-cost benefits | Migrate with caveat | `migrate_optimized` | | Bedrock cheaper AND capabilities match | Migrate | `migrate_optimized` | | Bedrock more expensive BUT non-cost benefits justify (vendor diversification, Guardrails, multi-model) AND user priority ≠ `cost` | Migrate with caveat | `migrate_optimized` | | Bedrock more expensive AND user priority = `cost` AND no compelling non-cost reason | Stay | `stay` | @@ -392,13 +398,12 @@ equivalent of `estimate-infra.md` Part 8. state why in `rationale`. - If `honest_assessment` from `aws-design-ai.json` says `recommend_stay`, `recommendation.path` MUST be `stay` regardless of cost numbers. -- **A same-model move is a modest cost increase, not parity.** When - `bedrock_models[].model_change` is `false`, Bedrock in-region costs ~10% more than OpenAI / - Azure OpenAI standard for the same model. Report that figure rather than "no savings - identified", and argue the case on commitments, governance, residency, prompt caching, and - eliminated behavior-delta risk. A ~10% increase alone should not route to `stay` unless - `ai_priority = cost` and no non-cost driver applies; a long-context workload at the 1M tier is a - different matter and may legitimately favour staying. +- **Same-model cost is the computed Part 5 result, not a fixed premium.** The final + `recommendation.rationale` must use the selected inference option/context tier and verified + source baseline. Report parity, savings or an increase as actually calculated. When the + source comparison is unavailable, say so, set confidence appropriately and do not invent a + percentage. Apply the decision table to those actual costs and the user's non-cost drivers; + the `honest_assessment: recommend_stay` override still takes precedence. - For multi-workload runs: if some workloads favor migration and others don't, use `migrate_phased` and list which workloads to migrate vs. keep in `rationale`. diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate-artifacts-ai.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate-artifacts-ai.md index 4b9d8b3bb..f901f650a 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate-artifacts-ai.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate-artifacts-ai.md @@ -27,7 +27,7 @@ Read `aws-design-ai.json → ai_architecture.code_migration.migration_path` and | `migration_path` / framework | Emits (in addition to the always-on set) | | ----------------------------------------------------------------- | ------------------------------------------------------------------------- | -| `mantle_openai_responses` | `ai-migration/migrate_to_mantle.sh` — **skip the provider adapter** | +| `mantle_openai_responses` / `mantle_openai_chat` | `ai-migration/migrate_to_mantle.sh` — **skip the provider adapter** | | `runtime_openai_cris` | `ai-migration/migrate_to_mantle.sh` (CRIS variant — `bedrock-runtime` base URL + CRIS id) — skip the provider adapter | | `direct` (or absent) or `converse` | `ai-migration/provider_adapter.{py,js,go}` — `converse` is a Converse client swap, not a Mantle script | | `gpt-oss` | `ai-migration/provider_adapter.{py,js,go}` targeting gpt-oss via Converse | @@ -61,7 +61,9 @@ a Converse call against the `bedrock-runtime` CRIS id — rather than defaulting `responses.create`. Absent or `"responses"` keeps the Responses surface (the region-only-move default). -## Step 1: `provider_adapter.{py,js,go}` (direct / gpt-oss path) +Adapt Azure-specific client/deployment/API-version settings to the Bedrock client rather than relying on OPENAI_BASE_URL alone. + +## Step 1: `provider_adapter.{py,js,go}` (direct / gpt-oss / runtime_openai_cris path) Feature-flagged on `AI_PROVIDER` (values `azure_openai` | `bedrock` | `shadow`; **default `azure_openai`**). Emits methods gated on `integration.capabilities_summary`: `text_generation`→ @@ -150,6 +152,8 @@ is AWS-only and ports from gcp unchanged. path emitted the adapter and NO mantle script. - [ ] `bedrock_monitoring.tf` present on every path; budget is a computed integer ≥ 10. - [ ] No secret VALUE in any emitted file; credentials are env-var references. +- [ ] Bare proprietary GPT ids use Mantle; Astra runtime calls retain the validated `us.` / + `global.` CRIS id. Chat/Responses and runtime artifact selection match the saved design. - [ ] No **GPT-5.5 or GPT-5.4** model ID paired with a Converse/`bedrock-runtime` path (those two models only — mantle-only, no CRIS). GPT-5.6 and GPT-6 models MAY use `migration_path: "runtime_openai_cris"` on `bedrock-runtime` — this is NOT a violation. diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate.md index 722dd832b..be0ac3018 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/phases/generate/generate.md @@ -73,7 +73,7 @@ _postconditions: _on_failure: _halt_and_inform - _assert: "no secret VALUE from the inventory appears in any generated artifact; secrets are emitted as Secrets Manager references" _on_failure: _halt_and_inform - - _assert: "WHEN aws-design-ai.json exists AND run_mode is decide_and_execute: generation-ai.json exists and validates, its rollback_plan.mechanism is 'feature_flag' with flag_name 'AI_PROVIDER' and default_value 'azure_openai'; an ai-migration/ directory was produced (setup_bedrock.sh, test_comparison.py, bedrock_monitoring.tf on every path; migrate_to_mantle.sh for the mantle path OR provider_adapter.* for the direct/gpt-oss path, not both); and no proprietary GPT-5.5 or GPT-5.4 model ID is paired with a converse/bedrock-runtime path (GPT-5.6 and GPT-6 models MAY use migration_path: runtime_openai_cris on bedrock-runtime — this is not a violation). When aws-design-ai.json is absent this is vacuously satisfied" + - _assert: "WHEN aws-design-ai.json exists AND run_mode is decide_and_execute: generation-ai.json exists and validates, its rollback_plan.mechanism is 'feature_flag' with flag_name 'AI_PROVIDER' and default_value 'azure_openai'; an ai-migration/ directory was produced (setup_bedrock.sh, test_comparison.py, bedrock_monitoring.tf on every path; migrate_to_mantle.sh for the mantle path OR provider_adapter.* for the direct/gpt-oss path, not both); and no proprietary GPT-5.5 or GPT-5.4 model ID is paired with a converse/bedrock-runtime path (bare proprietary openai.gpt-* IDs use Mantle only; runtime_openai_cris accepts documented Astra us./global. and GPT-5.6/GPT-6 profiles with caller-region checks and the selected API; GPT-5.6 and GPT-6 models MAY use migration_path: runtime_openai_cris on bedrock-runtime — this is not a violation). When aws-design-ai.json is absent this is vacuously satisfied" _on_failure: _halt_and_inform _forbids_files: - azure-resource-inventory.json diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/openai-on-bedrock.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/openai-on-bedrock.md index 12455bfcb..0a207f058 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/openai-on-bedrock.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/openai-on-bedrock.md @@ -56,6 +56,31 @@ is on this list have no same-model landing target — see `ai-openai-to-bedrock. ## Access Paths — Split by Family +**GPT-6 Astra has two endpoints; regions reverified 2026-09-30:** + +| Endpoint | Reach | Model ID | Supported APIs | +| ----------------- | ----------------------------- | ----------------------------------------------------- | ------------------------------------- | +| `bedrock-mantle` | Standard in-region, **us-east-1 / us-west-2** | `openai.gpt-6-astra` | Responses, Chat Completions | +| `bedrock-runtime` | CRIS only, no in-region form | `us.openai.gpt-6-astra` / `global.openai.gpt-6-astra` | Responses, Chat Completions, Converse | + +Use `/openai/v1` on either endpoint for the OpenAI-compatible APIs. Astra does **not** support +Invoke or Messages. On runtime, Guardrails and application inference profiles are **Converse-only**; +server-side tool use is unsupported. Runtime supports non-streaming JSON Schema output (reverified +2026-10-03): Chat uses `response_format.json_schema`, Responses uses `text.format`, and Converse +uses `outputConfig.textFormat.type: "json_schema"` with `name` and a JSON-encoded schema string in +`outputConfig.textFormat.structure.jsonSchema`, plus **`additionalModelRequestFields.text.format.strict=true`**. +For Chat/Responses set the format type to `json_schema` and include `name`, `schema` and `strict: true`. +Use an object schema, require every field, set `additionalProperties: false`, validate before calling, +and check refusal/incomplete responses before parsing. Do not claim streamed JSON Schema support. +On mantle, server-side tool calling +is supported, implicit/explicit prompt caching is **Responses-only**, and application inference +profiles are unsupported. Do not copy GPT-5.6's API or feature matrix onto Astra. + +The card specifies **1,050,000 context tokens and 128,000 maximum output tokens**, with text/image +input and text output. The launch announcement rounds the input context to 1M; use the card's +exact limits for capacity checks. Browser/computer-use capability does not establish parity with +OpenAI-hosted tools; probe the application's specific tool flow. + **GPT-5.5 and GPT-5.4 are `bedrock-mantle`-only and in-region only.** Their model cards list a single Programmatic Access row (`bedrock-mantle`, Geo/Global "Not supported") and In-Region pricing only. @@ -136,7 +161,7 @@ including `bedrock-mantle:CreateInference` and `bedrock-mantle:CallWithBearerTok authorize mantle calls. On the GPT-5.6 `bedrock-runtime` path the usual `bedrock:InvokeModel*` against the CRIS inference-profile ARN applies, as for any other runtime model. -**Reasoning effort:** all five accept `none`, `low`, `medium`, `high`, `xhigh`, `max`. Because these models reason +**Reasoning effort:** the five GPT-5.x models accept `none`, `low`, `medium`, `high`, `xhigh`, `max`. Because these models reason before responding, the model's output items (which may include reasoning items) must be passed back in the next request for multi-turn and tool-calling flows. @@ -160,6 +185,11 @@ request for multi-turn and tool-calling flows. Terra and Luna reached AWS GovCloud (US-West, US-East) in August 2026 — newer than the rest of this matrix. +**Astra runtime availability:** US Geo CRIS can be called from `us-east-1`, `us-east-2`, `us-west-1`, +`us-west-2`, and `ca-central-1`. Global CRIS additionally supports `eu-central-1`, `eu-north-1`, +`eu-west-1`, `eu-west-2`, `eu-west-3`, `ap-northeast-1`, `ap-northeast-2`, `ap-northeast-3`, +`ap-south-1`, `ap-southeast-1`, `ap-southeast-2`, and `sa-east-1`. No EU/India Geo profile or +GovCloud region is listed on the Astra card. A Canada caller using US Geo still routes to the US. **GPT-6 Astra's mantle reach is us-east-1 and us-west-2; GPT-6 Sol, GPT-6 Luna, and GPT-6.1 Sol are us-east-1 only.** All four additionally reach broad commercial regions via `bedrock-runtime` CRIS (`us.` / `global.` inference profiles), same as GPT-5.6 — a region outside the mantle matrix does not block the same-model path for @@ -180,8 +210,8 @@ for these models). Sol rates reflect the Aug 21, 2026 reduction (−20% input / the AWS What's New announcement lists as promotional through at least Nov 21, 2026. **Pricing now has an inference-option dimension:** -- **In-Region and Geo CRIS: 1.10x OpenAI's standard list price** (parity with OpenAI's _data residency_ tier). -- **Global CRIS: OpenAI's standard list price** — cost parity, available for GPT-5.6 only, and only when the +- **GPT-5.x In-Region and Geo CRIS: 1.10x OpenAI's standard list price** (parity with OpenAI's _data residency_ tier). +- **Global CRIS: OpenAI's standard list price** — cost parity for the recorded GPT-5.6 rates, and only when the workload has no data-residency constraint. So the honest cost statement is conditional, not flat: a same-model GPT-5.6 move on Global CRIS is @@ -219,6 +249,9 @@ Context window is **1.05M tokens** for all four models (confirmed on every model context must be priced at this long-context tier. Max output tokens: 128,000 (Astra, Sol, Luna) / 131,072 (6.1 Sol). +Astra cache-write rates use a **30-minute cache-write** window. Verify the source +provider rate before claiming savings or parity for the selected context tier. + ### GPT-5.6 — short context (272K) | Model | In-Region / Geo (in · out) | Global CRIS (in · out) | Cache write / read (In-Region) | @@ -250,7 +283,9 @@ A workload above 272K context must be priced at the long-context tier. > Separately, the **AWS Price List API still carries no GPT-5.x rows** (checked 2026-08-04): > a missing or empty price-list result must not be read as "model unavailable." -### Prompt caching — GPT-5.6 and GPT-6 only +### Prompt caching — GPT-5.6 details + +Astra has separate Responses-only caching evidence above. Do not inherit GPT-5.6 prefix, breakpoint or quota-exemption rules. Listed as a supported feature on the GPT-5.6 Sol/Terra/Luna model cards and on all four GPT-6 model cards (Astra, Sol, Luna, 6.1 Sol — confirmed by the cache-write/cache-read columns in the GPT-6 pricing tables above). The @@ -274,6 +309,8 @@ count against the input-TPM quota**, which compounds the benefit at scale. ## Quotas +**Astra runtime:** use input TPM + 10 × output TPM. Do not apply the GPT-5.x Mantle quota rules below to Astra without verification. + Inference on `bedrock-mantle` is governed by **two per-model, per-region quotas: input tokens per minute and output tokens per minute. There is no requests-per-minute quota.** Exceeding a TPM quota returns HTTP 429. diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/schema-design-aws-ai.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/schema-design-aws-ai.md index b1c183d59..2f148a1b7 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/schema-design-aws-ai.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/shared/schema-design-aws-ai.md @@ -106,7 +106,8 @@ Traditional-AI capabilities (AWS AI service, not Bedrock): `document_extraction` `design_blocks[]` rows with `target_aws_service`. - **`capability_mapping`** — per capability that is `true` in the profile's `integration.capabilities_summary`: `{ parity: "full|partial|none", notes }`. -- **`code_migration`** — `primary_pattern` (matches profile `integration.pattern`), +- **`code_migration`** — `migration_path` (the selected API/endpoint path consumed by Generate), + `primary_pattern` (matches profile `integration.pattern`), `framework`, `files_to_modify[]`, `dependency_changes`, and — when the source is Azure OpenAI / OpenAI — `openrouter_path` when a router was detected (`same_model_mantle|direct|litellm|keep_openrouter`). @@ -115,6 +116,7 @@ Traditional-AI capabilities (AWS AI service, not Bedrock): `document_extraction` ## migration_path vocabulary +`mantle_openai_chat` preserves a selected GPT-6 Chat Completions API on Mantle. `mantle_openai_responses` (Azure OpenAI / OpenAI source, model on Bedrock, region carries it — keep the SDK, `model_change: false`) · `runtime_openai_cris` (GPT-5.6/GPT-6 same-model move served via `bedrock-runtime` CRIS, `us.`/`global.` prefixed, `model_change: false`) · `converse` @@ -190,6 +192,8 @@ rubric file (`ai.md`, `ai-azure-openai-to-bedrock.md`, or a `vendored/ai/*` ref) `price_comparison`. - [ ] `honest_assessment` overall is the weakest across `bedrock_models[]`; if `recommend_stay`, `honest_assessment_reason` names a non-cost blocker. +- [ ] Bare proprietary GPT ids use Mantle; runtime Astra uses a supported `us.` / `global.` + CRIS id and its caller-region gate. The recorded API and `model_change` match the design. - [ ] No `bedrock_models[]` entry pairs **GPT-5.5 or GPT-5.4** with a `converse`/`bedrock-runtime` migration path (those two models only — mantle-only, no CRIS). GPT-5.6 and GPT-6 models MAY use `migration_path: "runtime_openai_cris"` on `bedrock-runtime` — this is NOT a violation. diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-migration-guardrails.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-migration-guardrails.md index 45d18b24a..bf7f5ee1d 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-migration-guardrails.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-migration-guardrails.md @@ -48,7 +48,7 @@ Inference on `bedrock-mantle` for these models is governed by **two per-model, p | Medium | Medium | Monitor 429s against **token** throughput, not request rate; enable prompt caching | | High | High | Enable prompt caching first (cached input is exempt from input TPM), then request a quota increase | -**The `bedrock-runtime` fallback exists only for GPT-5.6 and GPT-6 (Astra, Sol, Luna, 6.1 Sol).** GPT-5.5 and GPT-5.4 are `bedrock-mantle` only and in-region only — for them, "switch to `bedrock-runtime`" requires moving to a different model (Bedrock-native or `gpt-oss`), a model change with its own eval cost. GPT-5.6 Sol/Terra/Luna and GPT-6 Astra/Sol/Luna/6.1 Sol DO have a `bedrock-runtime` path via CRIS inference profiles (`us.`/`in.`/`global.` prefixed ids; the model cards recommend runtime for new applications) — a legitimate endpoint option with its own quota family, and on Global CRIS it is also the cost-parity option. Scaling levers on the mantle path, in order: +**The `bedrock-runtime` fallback exists only for GPT-5.6 and GPT-6 (Astra, Sol, Luna, 6.1 Sol).** GPT-5.5 and GPT-5.4 are `bedrock-mantle` only and in-region only — for them, "switch to `bedrock-runtime`" requires moving to a different model (Bedrock-native or `gpt-oss`), a model change with its own eval cost. GPT-5.6 Sol/Terra/Luna and GPT-6 Astra/Sol/Luna/6.1 Sol DO have a `bedrock-runtime` path via CRIS inference profiles (GPT-5.6 `us.`/`in.`/`global.`, GPT-6 `us.`/`global.` prefixed ids; the model cards recommend runtime for new applications) — a legitimate endpoint option with its own quota family, and on Global CRIS it is also the cost-parity option. Scaling levers on the mantle path, in order: 1. Prompt caching (GPT-5.6 and GPT-6 only) — 90% off cached input and exempt from the input-TPM quota 2. Exponential backoff with a bounded retry count (`max_retries` on the OpenAI SDK) @@ -60,6 +60,19 @@ See `references/shared/openai-on-bedrock.md` for the endpoint, region matrix, an **Source:** [Get started with GPT-5.6 on Amazon Bedrock — Quotas and scaling](https://aws.amazon.com/blogs/machine-learning/get-started-with-openai-gpt-5-6-sol-terra-and-luna-on-amazon-bedrock/) +### GPT-6 Astra — verify the selected endpoint's quota + +Astra Standard mantle is available in `us-east-1` and `us-west-2` (region availability verified 2026-09-30). Its model card lists implicit/explicit caching on +Responses and 30-minute cache-write prices, but does not establish GPT-5.6's cached-input quota +exemption, minimum prefix, or breakpoint rules. Verify account quotas before promising throughput. + +Astra runtime uses **10× output-token burndown**: +`required_TPM = peak_input_TPM + 10 × peak_output_TPM`. Use the selected US Geo or Global CRIS quota; +do not multiply total source TPM by ten or carry this rule over to mantle. An endpoint change also +requires residency consent and API/capability checks from the canonical catalog. + +**Source:** [Astra model card, verified 2026-09-16](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html) + ### Other models on Mantle For non-OpenAI models served through Mantle, verify current quota dimensions and any shared-account limits in the [Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) before advising on throughput. Do not carry the OpenAI TPM-only model over to other providers without checking, and do not assume previously documented shared-RPM behavior still applies. @@ -78,7 +91,9 @@ Claude models on Mantle have an additional **output TPM cap** that differs by mo **Impact for migration decisions:** - For Claude migrations at medium/high volume: the 2M output TPM cap on Claude 4.7+ is the binding constraint -- For OpenAI proprietary GPT targets: this Claude cap does not apply. Their mantle constraint is the per-model input/output TPM quota described above; for GPT-5.6 the `bedrock-runtime`/CRIS path is an available alternative (its own quota family), while GPT-5.5/5.4 have no runtime path +- For OpenAI proprietary GPT targets: this Claude cap does not apply. Use the GPT-5.x mantle quotas or + Astra-specific verification above. Astra / GPT-5.6 have runtime CRIS alternatives with their own + quotas; GPT-5.5/5.4 have no runtime path - For `gpt-oss` targets: these do run on `bedrock-runtime`, so standard account TPM limits and the Converse-path mitigations apply - When output-heavy workloads (long JSON, tool outputs, multi-step reasoning) are detected, flag the relevant cap prominently; recommend `bedrock-runtime` for production **only** when the target model actually has a `bedrock-runtime` path diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-model-lifecycle.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-model-lifecycle.md index 11042e780..adaf2a939 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-model-lifecycle.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-model-lifecycle.md @@ -73,6 +73,14 @@ On each run, compute `days_to_eol = EOL date − today` for every model in the L --- +## GPT-6 Astra — Active (verified September 9, 2026) + +The [Astra model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html) +lists launch **September 8, 2026**, lifecycle **Active**, and no scheduled EOL date. Its EOL is +**no sooner than September 8, 2027**, with a Legacy period of at least six months. The earliest +possible EOL is not an announced retirement date. Existing GPT-5.x entries are not superseded. +See `openai-on-bedrock.md` for access paths, regions, and pricing. + ## Legacy / EOL Models (as of October 5, 2026) For models launched before 2026-09-07, the [legacy lifecycle table](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html) is authoritative. For models launched on or after that date, the model card and the runtime `modelLifecycle.status` field are authoritative — they will not appear in the table below. The table captures pre-policy-change models referenced elsewhere in this plugin. **Recompute the Status column on each run** using `days_to_eol = EOL date − today`. diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-openai-to-bedrock.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-openai-to-bedrock.md index 6edb373f3..2503256b5 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-openai-to-bedrock.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/ai-openai-to-bedrock.md @@ -30,20 +30,22 @@ For an explicitly requested 5.5 assessment, read its [Bedrock model card](https: ## Key Insight: OpenAI's Own Models Run on Bedrock **The old premise of this guide — "OpenAI models are unavailable on AWS, so migrating means switching model families" -— is obsolete.** GPT-5.6 Sol / Terra / Luna, GPT-5.5, and GPT-5.4 are generally available on Bedrock, at OpenAI's -data-residency-tier rates, counting toward existing AWS commitments. +— is obsolete.** GPT-6 Astra, GPT-5.6 Sol / Terra / Luna, GPT-5.5, and GPT-5.4 are generally available on Bedrock, +counting toward existing AWS commitments. Pricing depends on the model and inference option; +see `shared/openai-on-bedrock.md`. Two consequences that invert the previous decision logic: 1. **When the source model is on Bedrock, keep it.** A same-model move has no behavior delta to validate, no prompt re-engineering, and no eval regression risk — the migration is an endpoint and credential change, not a model change. This is the default recommendation. -2. **The cost of a same-model move depends on the inference option — state it conditionally.** In-Region and Geo +2. **The cost of a same-model move depends on the inference option — state it conditionally.** GPT-5.x In-Region and Geo CRIS are priced at OpenAI's _data residency_ tier, exactly 1.10x the standard list price, so those moves cost ~10% more. **GPT-5.6 on Global CRIS is priced at the standard list price — cost parity** — available only when the workload has no data-residency constraint (GPT-5.5 / GPT-5.4 have no CRIS at all). See `shared/openai-on-bedrock.md`. The non-cost case — AWS commitments, IAM/VPC/CloudTrail governance, residency, prompt caching, one vendor relationship — carries the in-region path; never present that path as free or neutral. + For Astra, use its separately dated Bedrock rates and verify the source provider's rate before comparing. Cross-family mapping (to Claude / Nova / DeepSeek) is still the right answer in two situations: the source model has no Bedrock equivalent, or the user's priority is cost and is willing to accept a model change to get it. Both are @@ -62,7 +64,7 @@ GPT-5.4, the target is **the same model on Bedrock**. | Source model | Bedrock target | Model ID (mantle / runtime CRIS) | Assessment | | ------------- | -------------- | ------------------------------------------------------- | --------------------------------------------------------------- | -| GPT-6 Astra | GPT-6 Astra | `openai.gpt-6-astra` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | +| GPT-6 Astra | GPT-6 Astra | `openai.gpt-6-astra` / `us.` `global.` prefixed | `strong_migrate` after the Astra region/API gate in `shared/openai-on-bedrock.md`; price the chosen inference option | | GPT-6 Sol | GPT-6 Sol | `openai.gpt-6-sol` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | | GPT-6 Luna | GPT-6 Luna | `openai.gpt-6-luna` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | | GPT-6.1 Sol | GPT-6.1 Sol | `openai.gpt-6.1-sol` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | @@ -73,8 +75,9 @@ GPT-5.4, the target is **the same model on Bedrock**. | GPT-5.4 | GPT-5.4 | `openai.gpt-5.4` (mantle only) | `strong_migrate` — same model, ~10% over OpenAI std | Then apply the **region gate** below. Record `model_change: false` and the chosen path in -`aws-design-ai.json` → `ai_architecture.code_migration`: `migration_path: "mantle_openai_responses"` for the -in-region mantle endpoint, or `migration_path: "runtime_openai_cris"` when a GPT-5.6 target is served via +`aws-design-ai.json` → `ai_architecture.code_migration`: `migration_path: "mantle_openai_chat"` for +Astra sources using Chat Completions, or `"mantle_openai_responses"` for a selected Mantle Responses +path. Use `migration_path: "runtime_openai_cris"` when a GPT-6 Astra / GPT-5.6 target is served via `bedrock-runtime` with a CRIS id (the model cards recommend runtime for new applications). Report the assessment honestly: `strong_migrate` here means "low-risk, well-supported move", not "cheaper". @@ -83,6 +86,11 @@ Report the assessment honestly: `strong_migrate` here means "low-risk, well-supp The gate differs by family (see `shared/openai-on-bedrock.md` § Regional Availability): +**GPT-6 Astra:** apply its own mantle and runtime matrices from the canonical catalog. For strict +in-region residency, the mantle region must match. Otherwise offer a supported runtime CRIS path +with explicit routing consent; a Global-only caller region cannot satisfy a Geo-only requirement. +If neither path meets the constraints, offer a supported region or a model change under Tier 1. +Do not assume GPT-5.6's broader regional footprint applies to Astra. **GPT-6 Astra / Sol / Luna / 6.1 Sol:** same shape as GPT-5.6 — if the target region is in the mantle in-region matrix (narrower than GPT-5.6's: us-east-1 + us-west-2 for Astra only, us-east-1 only for Sol/Luna/6.1 Sol — see `openai-on-bedrock.md` § Regional Availability), either endpoint works; otherwise the `bedrock-runtime` CRIS path @@ -122,15 +130,19 @@ let the user choose.** Do not silently default to either. Option A tier mapping: -| Source tier | Bedrock GPT target | -| -------------------------------------------------------- | ------------------ | -| Flagship reasoning (o1-pro, o3-pro, GPT-5 Pro, `*-Pro`) | GPT-5.6 Sol | -| Flagship general (GPT-4o, GPT-4.1, GPT-5/5.1/5.2, GPT-4) | GPT-5.6 Terra | -| Mid / reasoning (o1, o3, o4-mini) | GPT-5.6 Terra | -| Fast / cheap (`*-mini`, `*-nano`, GPT-3.5 Turbo) | GPT-5.6 Luna | +| Source tier | Bedrock GPT target | +| -------------------------------------------------------- | ------------------------------------------------------ | +| Flagship reasoning (o1-pro, o3-pro, GPT-5 Pro, `*-Pro`) | GPT-6 Astra; GPT-5.6 Sol as the lower-cost alternative | +| Flagship general (GPT-4o, GPT-4.1, GPT-5/5.1/5.2, GPT-4) | GPT-5.6 Terra | +| Mid / reasoning (o1, o3, o4-mini) | GPT-5.6 Terra | +| Fast / cheap (`*-mini`, `*-nano`, GPT-3.5 Turbo) | GPT-5.6 Luna | Option A is subject to the same region gate as Tier 0. +Astra is the capability-first choice for this reasoning tier, not a mandatory upgrade for an +existing Bedrock model. At the recorded rates it costs 2.5x Sol for the same inference option and +context tier; evaluate quality and cost before choosing between them. + ### Tier 2 — Cost priority override If `ai_constraints.ai_priority == "cost"`, present the cheapest adequate Bedrock target **as an explicit @@ -231,7 +243,7 @@ frontier tier. Recommend them when the user wants OpenAI-architecture models on ## Migration Decision Framework -**Migrate to Bedrock, same model (Tier 0), if:** the source is GPT-5.6 / 5.5 / 5.4 and the target region carries it. +**Migrate to Bedrock, same model (Tier 0), if:** the source is GPT-6 Astra / GPT-5.6 / 5.5 / 5.4 and the selected endpoint, API, and region support it. This is the common case and needs no cost justification. **Migrate to Bedrock, model change (Tier 1/2), if:** @@ -239,8 +251,8 @@ This is the common case and needs no cost justification. - The source model is not on Bedrock (see the region and catalog lists in `shared/openai-on-bedrock.md`) - The user's priority is cost and they accept a model change — note that the cheapest option is often Nova, and that GPT-5.6 Luna already beats Claude Haiku 4.5 on price -- Bedrock-native features are required that mantle does not expose (Guardrails, Knowledge Bases, invocation logging, - Converse-based tooling) — these need a Bedrock-native model or gpt-oss +- Required features cannot be served by a supported same-model path. Astra / GPT-5.6 have runtime CRIS + options for applicable governance features; check their capability matrix before proposing a model change **Consider staying on OpenAI's own API only if:** @@ -268,7 +280,7 @@ rather than dismissing it. For a long-context (>272K) workload the gap is much w | ------------------------------------------ | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | OpenAI SDK (direct) | Same model on mantle Responses API | Base URL + credential + model ID change; see `shared/openai-on-bedrock.md` for the `/openai/v1` path | | Responses API | Mantle Responses API | Closest to a drop-in; still verify the path and model ID | -| Chat Completions | Reshape to Responses | Unverified for GPT-5.x — probe before committing | +| Chat Completions | Preserve supported Chat when selected | Reshape only for a selected API change; verify the model and endpoint | | Function calling | Supported on GPT-5.x via mantle; Claude tools via Converse | Same-model keeps tool semantics identical | | Reasoning effort | `reasoning={"effort": ...}` on mantle | `none` / `low` / `medium` / `high` / `xhigh` / `max` | | Prompt caching | GPT-5.6 only (90% off cached input) | Not listed for GPT-5.5 / 5.4; Claude has its own caching | diff --git a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/bedrock-quotas.md b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/bedrock-quotas.md index 5d6916bc3..9ca28857e 100644 --- a/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/bedrock-quotas.md +++ b/plugins/aws-startup-advisor/skills/azure-to-aws/references/vendored/ai/bedrock-quotas.md @@ -16,10 +16,14 @@ OpenAI and Gemini APIs have high default rate limits for paying customers (OpenA --- -## The 5× Burndown Trap +## Output-Token Burndown Depends on the Model and Endpoint Claude models (3.7+) on Bedrock consume quota at **5× the rate for output tokens**. This is the single most common surprise for teams migrating from OpenAI (which has no output multiplier). +**GPT-6 Astra on `bedrock-runtime` uses 10× output-token burndown**: one output token consumes ten quota +tokens. This is runtime-specific; do not apply it to mantle or infer Astra mantle quotas from GPT-5.6. +Source: [Astra model card, verified 2026-09-16](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html). + **Effective TPM by workload type** (at 200K TPM quota on Claude): | Workload Type | Input:Output Ratio | Effective TPM | vs OpenAI 2M TPM | @@ -30,6 +34,9 @@ Claude models (3.7+) on Bedrock consume quota at **5× the rate for output token Formula: `Effective TPM = TPM_quota / (input_ratio + output_ratio × 5)` +For Astra runtime, replace 5 with 10. A 60:40 workload with a hypothetical 200K TPM quota supports +about 43.5K actual tokens/minute: `200K / (0.6 + 0.4 × 10)`. Check the account's actual quota. + Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Output-heavy workloads that hit Claude quota limits may benefit from routing to these models for applicable tasks. --- @@ -38,12 +45,14 @@ Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Apply during Design phase after model selection. Uses `ai_token_volume` from `preferences.json` and the selected model family. -| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | -| ------------------------- | ---------------------------------- | ------------ | ----------------------------------------------------------------------------- | -| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | -| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | -| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | -| `"low"` | Any | `"low"` | No action | +| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | +| ------------------------- | -------------------------------------- | ------------ | ------------------------------------------------------------------------------- | +| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | +| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | +| `"medium"` | Astra runtime (10× output burndown) | `"medium"` | "Calculate weighted peak TPM and verify the selected CRIS quota before cutover" | +| `"medium"` | Verified 1× models (e.g. Nova / Llama) | `"low"` | No action | +| `"medium"` | Unverified model/endpoint quota | `"medium"` | "Verify quota dimensions before estimating throughput" | +| `"low"` | Any | `"low"` | No action | Include `quota_risk` in `aws-design-ai.json` → `ai_architecture`. @@ -99,10 +108,14 @@ Include this guidance in migration artifacts when `quota_risk` ≥ `"medium"`. U **How to calculate the target value:** ``` -Target TPM = (peak_tokens_per_minute_on_source_provider) × burndown_multiplier +Target TPM = peak_input_tokens_per_minute + peak_output_tokens_per_minute × output_burndown_multiplier ``` -Where `burndown_multiplier` = 5 for Claude models (because output tokens consume 5× quota), 1 for all others. If the user doesn't know their peak TPM, estimate from daily volume: +Use `output_burndown_multiplier` = 10 for Astra runtime, 5 for the Claude runtime models above, +and 1 only where verified for the selected model/endpoint. The multiplier applies to output tokens, +not the total source TPM. For separate mantle input/output quotas, size each quota independently. +If the user doesn't know their peak TPM, estimate total volume below and split it using the measured +input:output ratio: ``` Estimated peak TPM = (daily_tokens / active_hours / 60) × 3 (3× for peak headroom) diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/clarify/clarify-ai-only.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/clarify/clarify-ai-only.md index 1da0f91fe..a6003aca1 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/clarify/clarify-ai-only.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/clarify/clarify-ai-only.md @@ -201,7 +201,13 @@ _Skip when:_ `models[].model_id` is populated in `ai-workload-profile.json` **wi | o-series | GPT-5.6 Sol/Terra; or Sonnet 5 | Not on Bedrock — offer both | | Claude (any) | Same model on Bedrock | Client swap only — no model change | -**Same-model rows first.** GPT-5.6 / 5.5 / 5.4 run on Bedrock, so those sources map to themselves. Cost is ~10% ABOVE OpenAI standard (Bedrock in-region is priced at OpenAI's data-residency tier), so the case is AWS commitments, governance, and residency — not savings, and not parity. They are `bedrock-mantle` / Responses-only and in-region only; see `references/shared/openai-on-bedrock.md`. For sources with no Bedrock equivalent, present both a same-vendor upgrade and a cross-family option rather than pre-picking. +**Same-model rows first.** GPT-6 Astra / GPT-5.6 / 5.5 / 5.4 sources map to themselves. +Apply the model-specific API and region gate in `references/shared/openai-on-bedrock.md`. +Astra Standard mantle supports `us-east-1` and `us-west-2`; runtime uses US Geo / Global CRIS. GPT-5.6 also has runtime CRIS; +GPT-5.5 / 5.4 remain mantle-only. GPT-5.x in-region costs about 10% above OpenAI standard, +while GPT-5.6 Global CRIS uses the standard rate. For Astra, compare its separately dated +inference/context tier against the source price before claiming savings or parity. +For sources without a Bedrock equivalent, present a same-vendor upgrade and a cross-family option. Override examples: GPT-4 + Q2=cost → Haiku; Flash + Q10=extended thinking → Sonnet; GPT-4o + Q10=speech → Nova 2 Sonic; GPT-5.5 + Q2=cost → Sonnet 5. diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/design/design-ai.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/design/design-ai.md index 3b7f6cf74..3b235258a 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/design/design-ai.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/design/design-ai.md @@ -221,6 +221,9 @@ otherwise use the cost-comparison cases below with `model_change` reflecting the `"strong_migrate"`. Set `model_change: false`. The rationale is risk (no behavior delta) plus AWS commitments, governance, and residency — **not** an assumed cost saving. Price the actual model and inference option using the model-specific rules below; same identity does not imply cost parity or a universal premium. + + If source pricing is unavailable, retain that unavailable comparison. Derive the + difference from verified source and selected target inference/context-tier rates. - Source model on Bedrock but the target region does not carry it → treat as no-same-model below, and record the region as the cause in `regional_warnings[]`. Do not report this as a model-quality judgment. - Bedrock cheaper → `"strong_migrate"` @@ -256,7 +259,9 @@ After selecting models, assess quota risk based on `ai_token_volume` from `prefe | ------------------------- | ---------------------------------- | ------------ | --------------------------------------------------------------------------------- | | `"high"` or `"very_high"` | Any | `"high"` | Flag: "Request Bedrock quota increase before migration (allow 1–5 business days)" | | `"medium"` | Claude (5× burndown) | `"medium"` | Flag: "Monitor TPM usage; quota increase may be needed at peak" | -| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | +| `"medium"` | Astra runtime (10× output burndown) | `"medium"` | Flag: "Verify input TPM + 10 × output TPM against the selected CRIS quota" | +| `"medium"` | Verified 1× models (e.g. Nova / Llama) | `"low"` | No action | +| `"medium"` | Unverified model/endpoint quota | `"medium"` | Flag: "Verify quota dimensions before estimating throughput" | | `"low"` | Any | `"low"` | No action | Include `quota_risk` in `aws-design-ai.json` → `ai_architecture` alongside `honest_assessment`. @@ -378,19 +383,25 @@ For each detected `integration.pattern` and `ai_source`, generate before/after m **Patterns to include (matched to detected language and source):** -| Pattern | Source | Target | Key Change | -| ------------------------------- | ------------------------- | -------------------- | ---------------------------------------------------------------------------------------------------- | -| Direct SDK (OpenAI), same model | OpenAI | Mantle Responses API | Base URL → `.../openai/v1`, Bedrock credential, `openai.gpt-*` model ID, `bedrock-mantle:*` IAM | -| Direct SDK | Vertex AI | boto3 Converse API | `generate_content()` → `converse()` | -| Direct SDK | OpenAI | boto3 Converse API | `completions.create()` → `converse()` (use if Mantle region unavailable or Converse features needed) | -| Direct SDK | Anthropic | boto3 Converse API | `messages.create()` → `converse()` with Claude model ID on Bedrock | -| LangChain | ChatVertexAI / ChatOpenAI | ChatBedrock | Swap import and model_id | -| LlamaIndex | Vertex / OpenAI LLM | BedrockConverse | Swap import | -| LLM Router (LiteLLM) | Any | Config change | `model="bedrock/"` (1 line) | -| Embeddings | TextEmbeddingModel | Titan Embeddings v2 | `invoke_model` with JSON body | -| Streaming | `stream=True` | `converse_stream` | Event loop over `contentBlockDelta` | - -**Mantle Responses API (primary path for OpenAI sources):** If `ai_source = "openai"` and the source model is on Bedrock, this is the primary option. The application keeps the OpenAI SDK and the same model; only the base URL, credential, model ID, and IAM actions change. Read `references/shared/openai-on-bedrock.md` for the exact values — three details break naive implementations: the path is `/openai/v1/responses` (not `/v1/responses`), the credential must be a Bedrock API key or token provider (not an OpenAI key), and IAM needs `bedrock-mantle:*` actions (not `bedrock:InvokeModel`). Record `migration_path: "mantle_openai_responses"` and `model_change: false`. +| Pattern | Source | Target | Key Change | +| ------------------------------- | ------------------------- | ------------------- | ---------------------------------------------------------------------------------------------------- | +| Direct SDK (OpenAI), same model | OpenAI | Selected Mantle API | Base URL → `.../openai/v1`, Bedrock credential, `openai.gpt-*` model ID, `bedrock-mantle:*` IAM | +| Direct SDK | Vertex AI | boto3 Converse API | `generate_content()` → `converse()` | +| Direct SDK | OpenAI | boto3 Converse API | `completions.create()` → `converse()` (use if Mantle region unavailable or Converse features needed) | +| Direct SDK | Anthropic | boto3 Converse API | `messages.create()` → `converse()` with Claude model ID on Bedrock | +| LangChain | ChatVertexAI / ChatOpenAI | ChatBedrock | Swap import and model_id | +| LlamaIndex | Vertex / OpenAI LLM | BedrockConverse | Swap import | +| LLM Router (LiteLLM) | Any | Config change | `model="bedrock/"` (1 line) | +| Embeddings | TextEmbeddingModel | Titan Embeddings v2 | `invoke_model` with JSON body | +| Streaming | `stream=True` | `converse_stream` | Event loop over `contentBlockDelta` | + +**OpenAI model/API selection:** preserve the model and the supported API chosen in the +mapping guide. For Astra on Standard Mantle in `us-east-1` or `us-west-2`, a Chat source selects +`migration_path: "mantle_openai_chat"`; a Responses source selects `"mantle_openai_responses"`. +Keep the OpenAI SDK, use `/openai/v1`, Bedrock credentials and Mantle IAM. Do not reshape +Astra Chat solely because it is proprietary GPT. Record `model_change: false` only when +the exact source model is retained; a Pro-to-Astra upgrade is a model change requiring eval. +GPT-5.x retains its separately dated API evidence and any selected Chat-to-Responses reshape. **A Chat Completions source is a supported near-drop-in for GPT-6 and GPT-5.6 — not a mandatory reshape.** All four GPT-6 models and the GPT-5.6 family list Chat Completions as a supported API on both mantle and runtime CRIS (per `openai-on-bedrock.md` § Access Paths); if `integration` evidence shows `chat.completions.create` for one of these models, the design may keep `chat.completions.create`/`choices[0].message` parsing — reshape to `responses.create` is optional, only for a Responses-only feature (runtime prompt caching, reasoning-item round-trips) or a demonstrated constraint. **GPT-5.5 and GPT-5.4 are the exception**: Chat Completions support is unconfirmed on their model cards (Responses on mantle is the only verified surface), so a Chat Completions source on either model still needs a planned reshape, probed against the target account before committing. @@ -415,7 +426,18 @@ generated client uses a Guardrails-capable surface — `converse` for Astra; `co CRIS models. Do not leave the client on `responses` when Guardrails is required.** `Responses` stays the default for a region-only CRIS move with no Guardrails need. -**Mantle throughput (medium/high volume):** quotas on `bedrock-mantle` are **per-model, per-region input TPM and output TPM — there is no RPM quota**. For `ai_token_volume = "medium"` or `"high"`, note: "Mantle enforces per-model input/output TPM quotas per region; 429s indicate a TPM ceiling, not a request-rate cap. Mitigate with exponential backoff, spreading load across minutes, and prompt caching (cached input is exempt from the input-TPM quota). There is no `bedrock-runtime` fallback for these models, so sustained growth needs a quota increase." See `references/vendored/ai/ai-migration-guardrails.md`. +When the advisor supplied a +per-unit model contract, retain its exact `invocation_model_id` in that unit's `aws_model_id` +(including a system-profile ARN when supplied), and translate `runtime_converse` to +`runtime_openai_cris` plus `required_client_api: "converse"`. A missing invocation ID requires resolution; +do not manufacture a profile from the region. Keep each unit's model/API/profile together. +For multiple advisor units, write `migration_path` and `required_client_api` on each corresponding +`bedrock_models[]` row with its `aws_model_id`; `code_migration` remains the primary-unit default. + +**Throughput:** load `references/vendored/ai/ai-migration-guardrails.md` and apply the selected +model/endpoint's quota rules. Astra runtime uses input TPM plus 10 times output TPM. +Astra Mantle quotas and cache exemptions must be verified independently; do not copy GPT-5.6's +rules or promise runtime fallback for a model that has none. **gpt-oss migration path:** If `ai_source = "openai"` and the user wants OpenAI-architecture models on the Bedrock-native runtime surface, offer `gpt-oss` as an additional path. Unlike the proprietary GPT models, gpt-oss **does** support `bedrock-runtime` (Converse / InvokeModel), so it is the option when Guardrails or invocation logging are required and an OpenAI-lineage model is preferred. It sits a capability class below the GPT-5.x frontier tier. Record `migration_path: "gpt-oss"`. @@ -456,7 +478,7 @@ Write `aws-design-ai.json` to `$MIGRATION_DIR/`. | `ai_architecture.tiered_strategy` | object/null | Tiered model routing (null for low/medium volume) | | `ai_architecture.bedrock_models` | array | Per-model: `gcp_model_id`, `aws_model_id`, `capabilities_matched[]`, `capability_gaps[]`, `honest_assessment`, `source_provider_price`, `bedrock_price`, `price_comparison`, `migration_complexity` | | `ai_architecture.capability_mapping` | object | Per-capability: `parity` (full/partial/none), `notes` | -| `ai_architecture.code_migration` | object | `primary_pattern`, `framework`, `files_to_modify[]`, `dependency_changes` | +| `ai_architecture.code_migration` | object | `primary_pattern`, `framework`, `files_to_modify[]`, `dependency_changes`, `migration_path`; per-model `required_client_api` selects the runtime API | | `ai_architecture.infrastructure` | array | GCP resource → AWS equivalent mappings with confidence | | `ai_architecture.services_to_migrate` | array | GCP service → AWS service with effort and notes | | `regional_warnings` | array | Per-service: `service`, `target_region`, `nearest_available`, `impact` (empty array if all services available) | diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/discover/discover-preview.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/discover/discover-preview.md index 1d79f839a..9600dca92 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/discover/discover-preview.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/discover/discover-preview.md @@ -84,9 +84,10 @@ from `references/shared/pricing-cache.md` (Source Provider Pricing) and `referen **Source model → Bedrock equivalent mapping:** -**Same-model rows first.** OpenAI's proprietary GPT models run on Bedrock, so these sources map to themselves and -the comparison is a ~10% premium (Bedrock in-region is at OpenAI's data-residency tier, 1.10x standard — see -`references/shared/openai-on-bedrock.md`). Match these before falling through to the cross-family rows. +**Same-model rows first.** Match GPT-6 Astra and the GPT-5.x sources below before cross-family rows. +For each same-model row, derive the price direction from verified source rates and the selected Bedrock +inference option and context tier. Do not assume a fixed premium or parity. +See `references/shared/openai-on-bedrock.md` for the model-specific pricing rules. | Source model pattern | Bedrock equivalent | Bedrock model ID | | -------------------------------------- | ------------------ | ---------------------- | @@ -112,26 +113,31 @@ prefixes). At Discover time the target region may not be known — record the same-model mapping and let Design apply the region gate. See `references/shared/openai-on-bedrock.md`. -**Cross-family rows** — for sources with no Bedrock equivalent: - -| Source model pattern | Bedrock equivalent | Bedrock model ID | -| ------------------------------------------------------- | -------------------------------- | ------------------------------------------ | -| `gpt-4o`, `gpt-4.1`, `gpt-5`/`5.1`/`5.2` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | -| `gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5.*-mini` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | -| `gpt-3.5-turbo`, `gpt-4.1-nano`, `gpt-5.*-nano` | Amazon Nova Micro | `amazon.nova-micro-v1:0` | -| `gpt-*-pro` (GPT-5.x Pro), `o1-pro`, `o3-pro` | Amazon Nova 2 Pro | `amazon.nova-2-pro-v1:0` | -| `o3`, `o4-mini`, reasoning models | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | -| `gemini-2.5-pro`, `gemini-3.*-pro` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | -| `gemini-2.5-flash`, `gemini-2.0-flash` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | -| `gemini-2.0-flash-lite` | Amazon Nova Lite | `amazon.nova-lite-v1:0` | -| `claude-3-5-sonnet`, `claude-sonnet-*` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | -| `claude-3-5-haiku`, `claude-haiku-*` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | -| `claude-3-opus`, `claude-opus-*` | Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` | -| `text-embedding-*`, `*-embedding-*` | Amazon Titan Embeddings v2 | `amazon.titan-embed-text-v2:0` | -| `dall-e-*`, `gpt-image-*`, `imagen-*`, image generation | Stability AI — Stable Image Core | `stability.stable-image-core-v1:0` | -| `whisper-*`, speech-to-text | Amazon Transcribe | (non-token service — note separately) | -| `tts-*`, text-to-speech | Amazon Polly | (non-token service — note separately) | -| Unknown / other | Amazon Nova Pro | `amazon.nova-pro-v1:0` | +**Pro sources need a choice.** For `gpt-*-pro`, `o1-pro`, and `o3-pro`, offer GPT-6 Astra as the +capability-first same-vendor upgrade and GPT-5.6 Sol as the lower-cost GPT alternative, alongside Nova 2 Pro +below. Astra is 2.5x Sol at the recorded matching context/inference tier. This is a model change requiring eval; +record the choice in `key_decisions_ahead` instead of silently selecting Nova. Apply the Astra region gate. + +**Alternatives for sources with no Bedrock equivalent** — the Pro row offers both vendor paths: + +| Source model pattern | Bedrock equivalent | Bedrock model ID | +| ------------------------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------------- | +| `gpt-4o`, `gpt-4.1`, `gpt-5`/`5.1`/`5.2` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | +| `gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5.*-mini` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `gpt-3.5-turbo`, `gpt-4.1-nano`, `gpt-5.*-nano` | Amazon Nova Micro | `amazon.nova-micro-v1:0` | +| `gpt-*-pro` (GPT-5.x Pro), `o1-pro`, `o3-pro` | GPT-6 Astra; Sol for lower cost; or Nova 2 Pro | `openai.gpt-6-astra` / `openai.gpt-5.6-sol` / `amazon.nova-2-pro-v1:0` | +| `o3`, `o4-mini`, reasoning models | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | +| `gemini-2.5-pro`, `gemini-3.*-pro` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | +| `gemini-2.5-flash`, `gemini-2.0-flash` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `gemini-2.0-flash-lite` | Amazon Nova Lite | `amazon.nova-lite-v1:0` | +| `claude-3-5-sonnet`, `claude-sonnet-*` | Claude Sonnet 5 | `anthropic.claude-sonnet-5` | +| `claude-3-5-haiku`, `claude-haiku-*` | Claude Haiku 4.5 | `anthropic.claude-haiku-4-5-20251001-v1:0` | +| `claude-3-opus`, `claude-opus-*` | Claude Opus 4.6 | `anthropic.claude-opus-4-6-v1` | +| `text-embedding-*`, `*-embedding-*` | Amazon Titan Embeddings v2 | `amazon.titan-embed-text-v2:0` | +| `dall-e-*`, `gpt-image-*`, `imagen-*`, image generation | Stability AI — Stable Image Core | `stability.stable-image-core-v1:0` | +| `whisper-*`, speech-to-text | Amazon Transcribe | (non-token service — note separately) | +| `tts-*`, text-to-speech | Amazon Polly | (non-token service — note separately) | +| Unknown / other | Amazon Nova Pro | `amazon.nova-pro-v1:0` | For each mapped model pair, record `source_model`, `bedrock_equivalent`, both per-token prices, and `cost_direction` (`"higher"`, `"lower"`, or `"comparable"` — Bedrock relative diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md index bb6c80cca..d198496fa 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/estimate/estimate-ai.md @@ -101,6 +101,11 @@ family — long context table, Global column) is $135. Bedrock Global CRIS long- (parity); Bedrock In-Region/Geo-CRIS long-context ($4.40 input / $16.50 output) is $148.50 — a ~10% premium, not the 112% a flat-rate comparison would produce. +**Astra context and inference option:** use its dedicated table in `shared/pricing-cache.md`: +short-context at up to 272K input tokens, long-context above that boundary, with separate +In-Region/Geo and Global input/output rates. Do not use a short-context row for a long-context +workload or apply a Claude-specific surcharge to Astra. + **Comparison table columns:** Model, Bedrock Monthly, vs Source Provider ($ and %), vs Current GCP, Quality, Capabilities Match (checked against `ai_capabilities_required`). Include source provider pricing from `aws-design-ai.json` → `bedrock_models[].source_provider_price`. @@ -146,13 +151,22 @@ Do **not** repeat these as "costs" in the user-facing summary. Present the monthly and annual cost difference between current GCP AI spend and projected Bedrock cost: -- **If the model is unchanged** (`model_change: false`): projected cost is **about 10% higher**, not the same, UNLESS the chosen path is Global CRIS (`migration_path: "runtime_openai_cris"` with a `global.`-prefixed model ID) — Global CRIS is priced at OpenAI's standard list price, i.e. cost PARITY with the source, not a premium. In-Region and Geo CRIS (and any GPT-5.5/GPT-5.4 move, which has no CRIS at all) remain at the 1.10x data-residency-tier premium over OpenAI standard. Quote whichever applies plainly and make the case on non-cost grounds. If any workload exceeds 272K context, price it at the long-context tier (2.0x input / 1.5x output) and show that separately; it can dominate the comparison. +- **If the model is unchanged** (`model_change: false`): compute the difference from the verified source rate and the selected model's context tier and inference option. Do not impose a fixed premium. A verified $60 source versus $60 Astra Global comparison is 0%; an applicable $60 source versus $66 In-Region/Geo comparison is 10%. If the source rate is unverified, mark the source comparison unavailable and do not infer a percentage from another tier. For Astra inputs above 272K, select its long-context input/output rates for that inference option before calculating totals. - **If Bedrock is cheaper**: present monthly and annual savings clearly - **If Bedrock is more expensive**: state clearly, justify with non-cost benefits or note "not justified if cost is the only priority" +When both monthly totals are established for the same workload volume, calculate: + +```python +monthly_difference = bedrock_monthly - source_monthly +percentage_difference = monthly_difference / source_monthly * 100 if source_monthly > 0 else None +``` + +A zero source total has no defined percentage comparison; show the dollar difference instead. + Reference `aws-design-ai.json` → `honest_assessment`. If `"recommend_stay"`, present prominently along with `honest_assessment_reason`. -**Non-cost benefits to present:** usage counting toward existing AWS commitments, IAM/VPC/PrivateLink/KMS/CloudTrail governance, in-region processing for data residency, prompt caching (Claude, and GPT-5.6 at 90% off cached input with cached tokens exempt from the input-TPM quota), model flexibility (100+ models), AWS ecosystem (Guardrails, Knowledge Bases, AgentCore), and — for a same-model move — the elimination of behavior-delta and prompt-regression risk. +**Non-cost benefits to present:** usage counting toward existing AWS commitments, IAM/VPC/PrivateLink/KMS/CloudTrail governance, in-region processing for data residency, prompt caching (Claude, and GPT-5.6 at 90% off cached input with cached tokens exempt from the input-TPM quota), model flexibility (100+ models), AWS ecosystem (Guardrails, Knowledge Bases, AgentCore), and — for a same-model move — reduced model-change risk, while endpoint/API compatibility still requires evaluation. **Pricing source caveat (all providers):** a `references/vendored/ai/bedrock-pricing-cache.md` cell marked `_unverified_` is **blocking for any quoted Bedrock figure, whatever the provider** — resolve it from the Bedrock pricing page or the model card before the row enters `model_comparison` or the ROI table; never substitute a guess or a same-tier sibling's rate. An `_unverified_` Bedrock cache cell is not evidence the model is unavailable or free — it means the rate was not confirmed when the cache was last updated. See `shared/openai-on-bedrock.md`. @@ -250,7 +264,7 @@ Produce a clear migrate/stay/optimize verdict for the AI workload migration. Thi | Condition | Verdict | `recommendation.path` | | --------------------------------------------------------------------------------------------------------------------------------- | ------------------- | --------------------- | -| **Same model on Bedrock** (`model_change: false`) — ~10% higher, short-context; non-cost benefits carry it | Migrate with caveat | `migrate_optimized` | +| **Same model on Bedrock** (`model_change: false`) — use the verified Part 5 cost delta and applicable non-cost benefits | Migrate with caveat | `migrate_optimized` | | Bedrock cheaper AND capabilities match | Migrate | `migrate_optimized` | | Bedrock more expensive BUT non-cost benefits justify (vendor diversification, Guardrails, multi-model) AND user priority ≠ `cost` | Migrate with caveat | `migrate_optimized` | | Bedrock more expensive AND user priority = `cost` AND no compelling non-cost reason | Stay | `stay` | @@ -274,7 +288,7 @@ Produce a clear migrate/stay/optimize verdict for the AI workload migration. Thi - MUST emit `recommendation` — never omit. If data is insufficient, set `confidence: "low"` and state why in `rationale`. - If `honest_assessment` from `aws-design-ai.json` says `recommend_stay`, `recommendation.path` MUST be `stay` regardless of cost numbers. -- **A same-model move is a modest cost increase, not parity.** When `bedrock_models[].model_change` is `false`, Bedrock in-region costs ~10% more than OpenAI standard for the same model. Report that figure rather than "no savings identified", and argue the case on commitments, governance, residency, prompt caching, and eliminated behavior-delta risk. A ~10% increase alone should not route to `stay` unless `ai_priority = cost` and no non-cost driver applies; a long-context workload at the 1M tier is a different matter and may legitimately favour staying. +- **Same-model cost is the computed Part 5 result, not a fixed premium.** The final `recommendation.rationale` must use the selected inference option/context tier and verified source baseline. Report parity, savings or an increase as actually calculated. When the source comparison is unavailable, say so, set confidence appropriately and do not invent a percentage. Apply the decision table to those actual costs and the user's non-cost drivers; the `honest_assessment: recommend_stay` override still takes precedence. - For multi-workload runs: if some workloads favor migration and others don't, use `migrate_phased` and list which workloads to migrate vs. keep in `rationale`. --- diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-ai.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-ai.md index a08122427..2571e0def 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-ai.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-ai.md @@ -28,7 +28,7 @@ Check `preferences.json` → `ai_constraints.ai_framework` to determine the migr | Voice Platform (Vapi, Bland.ai) | Check native Bedrock support, update dashboard | Dashboard config | | Framework (LangChain, LlamaIndex) | Swap provider import (e.g., `ChatBedrock` for `ChatVertexAI`) | 1-5 lines of code | -**OpenAI SDK users via Mantle (config path for a Responses source; small reshape for a Chat Completions source)** — `ai_framework` = `direct` AND `ai_source` = `openai` AND (`migration_path` **starts with** `mantle` (`mantle`, `mantle_openai_responses`) OR `migration_path` == `"runtime_openai_cris"`): +**OpenAI SDK users via Mantle (config path for a Responses source; small reshape for a Chat Completions source)** — `ai_framework` = `direct` AND `ai_source` = `openai` AND (`migration_path` **starts with** `mantle` (`mantle`, `mantle_openai_responses`, `mantle_openai_chat`) OR `migration_path` == `"runtime_openai_cris"`): - `mantle` / `mantle_openai_responses`: point the client at `https://bedrock-mantle.{region}.api.aws/openai/v1` — note the `openai/v1` segment for proprietary GPT models; a bare `/v1` 404s. Swap the credential for a Bedrock API key or the auto-refreshing token provider — **not** the existing OpenAI key. Update the model string to the bare Bedrock model ID (`openai.gpt-*` for a same-model move). Grant the `bedrock-mantle:*` actions; `bedrock:InvokeModel` does not authorize these models. - `runtime_openai_cris` (GPT-5.6/GPT-6 via `bedrock-runtime` CRIS): point the client at `https://bedrock-runtime.{region}.amazonaws.com/openai/v1` instead — **not** the mantle URL. Update the model string to the `us.`/`in.`/`global.`-prefixed CRIS inference-profile ID, not the bare ID. Grant standard `bedrock:InvokeModel*` actions against the CRIS inference-profile ARN — **not** `bedrock-mantle:*`. For a GPT-6 CRIS id, also grant `bedrock:InvokeModel` on `arn:aws:bedrock:{region}:{account}:project/default` alongside the inference-profile ARN (per the GPT-6 model cards). Check `aws-design-ai.json` → `ai_architecture.bedrock_models[].required_client_api`: if non-Responses, emit that surface (e.g. Converse) instead of defaulting to `responses.create` — absent/`responses` keeps the Responses surface for a region-only move. @@ -67,7 +67,7 @@ Based on `ai-workload-profile.json` → `integration.pattern` and `integration.l For each detected language and pattern, generate before/after code examples using actual model IDs from `aws-design-ai.json`. -Include streaming migration (`converse_stream`) if `capabilities_summary.streaming = true`. +If `capabilities_summary.streaming = true`, use the selected API's streaming surface: OpenAI Chat/Responses streaming on Mantle, or `converse_stream` for Converse. Do not switch the recorded endpoint to obtain streaming. Include embeddings migration (Titan Embeddings v2 via `invoke_model`) if `capabilities_summary.embeddings = true`. @@ -106,7 +106,7 @@ Include embeddings migration (Titan Embeddings v2 via `invoke_model`) if `capabi ## Part 5: Production Readiness Checklist - [ ] Bedrock model access enabled -- [ ] IAM role with `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` +- [ ] IAM matches the selected endpoint: Mantle project/bearer actions for bare proprietary GPT ids, or runtime InvokeModel actions with foundation-model/profile resources for CRIS - [ ] Provider adapter deployed and tested in staging - [ ] A/B test with >= 100 representative prompts - [ ] Response quality >= 90% of source baseline diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-ai.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-ai.md index 25f425fdb..9bbd2f72f 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-ai.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-ai.md @@ -14,7 +14,11 @@ Generate migration artifacts from the AI migration plan and design. Artifacts va - `ai-migration/test_comparison.py` — A/B test harness (always Python) - `ai-migration/bedrock_monitoring.tf` — Bedrock cost budget, anomaly detection, inference profiles (always) -**Outputs (direct SDK users — `ai_framework` = `"direct"`):** +**Outputs (direct SDK users, selected Mantle path):** + +- `ai-migration/migrate_to_mantle.sh` — Selected Chat/Responses client setup; no provider adapter + +**Outputs (direct SDK users, selected runtime path):** - `ai-migration/provider_adapter.{py,js,go}` — Provider abstraction with feature flag @@ -45,7 +49,7 @@ If any required file is missing: **STOP**. Output: "Missing required artifact: [ Check `preferences.json` → `ai_constraints.ai_framework.value` and `aws-design-ai.json` → `ai_architecture.code_migration.migration_path`: -- `migration_path` = `"mantle"` or `"mantle_openai_responses"` → Generate Mantle setup (Step 1M) + setup (Step 3) + test harness (Step 2). Skip provider adapter (Step 1). Design records `"mantle_openai_responses"`; treat that spelling as this same Mantle path. +- `migration_path` starts with `"mantle"` (`"mantle"`, `"mantle_openai_chat"`, `"mantle_openai_responses"`) → Generate Mantle setup (Step 1M) + setup (Step 3) + test harness (Step 2). Skip provider adapter (Step 1). Design records `"mantle_openai_responses"`; treat that spelling as this same Mantle path. - `migration_path` = `"runtime_openai_cris"` → Generate the CRIS-aware Mantle-equivalent script (Step 1M, using the `us.`/`global.`-prefixed inference-profile ID and the `bedrock-runtime` base URL instead of `bedrock-mantle`) + setup (Step 3) + test harness (Step 2). Skip provider @@ -54,6 +58,14 @@ Check `preferences.json` → `ai_constraints.ai_framework.value` and `aws-design - `"converse"`, `"direct"`, or absent → Generate provider adapter (Step 1) + setup (Step 3) + test harness (Step 2). `"converse"` is a Converse client swap (Design records it when a proprietary model cannot stay on Mantle); do NOT generate a Mantle script for it. - `"llm_router"`, `"api_gateway"`, `"voice_platform"`, or `"framework"` → Skip Step 1, generate gateway config (Step 3B) instead. Step 3B's OpenRouter subroute can still replace that gateway config with Step 1M (`same_model_mantle`) or the provider adapter (`direct`); do not decide those two from `migration_path` alone. +For `runtime_openai_cris`, read the matched `bedrock_models[].required_client_api`. +Use `responses` for current Design plans without that field; only legacy plans fall back +to the row or `code_migration.runtime_api` and their Converse default. Use the resolved +API consistently for the generated client, comparison and setup calls. +Preserve the exact `aws_model_id` string, whether a short profile ID or a system-profile ARN. +When building the IAM policy, use a supplied profile ARN directly and resolve its backing +foundation-model resources; do not interpolate the ARN into another ARN. + **Step 3F runs for ALL paths** (direct, mantle, gpt-oss, gateway, harness, strands) — generate `bedrock_monitoring.tf` regardless of migration path. **Determine language** (direct SDK users only): Read `ai-workload-profile.json` → `integration.languages` array. Use the first entry: `"python"` → `.py`, `"javascript"`/`"typescript"` → `.js`, `"go"` → `.go`, other/unknown → `.py`. @@ -67,13 +79,12 @@ Generate `ai-migration/migrate_to_mantle.sh` — a shell script that configures **Requirements:** - Dry-run by default (`--execute` flag to apply) -- Print the environment variables to set: `OPENAI_BASE_URL=https://bedrock-mantle.{region}.api.aws/openai/v1` (the `/openai/v1` segment is load-bearing — a bare `/v1` 404s for the proprietary GPT models that route through this path; see `references/shared/openai-on-bedrock.md` for the per-model base paths), `OPENAI_API_KEY=` +- For proprietary GPT targets, print `OPENAI_BASE_URL=https://bedrock-mantle.{region}.api.aws/openai/v1`. Use the model-specific base path from `shared/openai-on-bedrock.md`; generic `/v1` returns 404 for proprietary GPT models. Credentials must be Bedrock credentials, not the source provider's API key; prefer the auto-refreshing `BedrockOpenAI` token provider shown in that reference. - Print the model string change: current model ID → Bedrock model ID from `aws-design-ai.json` -- Include a quick verification call using the OpenAI SDK against the Mantle endpoint -- **Add a warning comment** about `max_tokens`: OpenAI SDK users migrating via Mantle carry over their existing `max_tokens` value unchanged. If the existing value is 4096 (OpenAI default), this reduces Bedrock concurrency by 5–8x on TPM quota. Instruct users to audit `max_tokens` before production and include the workload-type lookup table (see Step 1M max_tokens guidance above). -- Note: "No code changes required. Your existing OpenAI SDK calls work unchanged." +- Include a quick verification call using the selected Mantle API: `chat.completions.create` for `mantle_openai_chat`, `responses.create` for `mantle_openai_responses`. Reuse the target-call shape in Step 2; never call Converse with a bare Astra ID. +- Preserve request/response shapes only when the selected API matches the source. Otherwise include the planned API reshape. Do not claim all existing SDK calls work unchanged; parameters and tool/state behavior still need the selected model/API's verification. - Reference [Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) for API key generation -- **Set `max_tokens` default to 1024** in the generated migration script with an explanatory comment. Do NOT inherit the caller's existing `max_tokens` value (often 4096 from OpenAI defaults) — this reduces concurrency by 5–8x on Bedrock's TPM quota. Include a workload-type lookup table as a comment so users tune before production: +- Include the output-budget lookup table below as a starting heuristic. Send a budget parameter only after verifying its name and behavior for the selected API; `MAX_TOKENS` is a local tuning value, not permission to send a `max_tokens` kwarg to every model. Do not apply Claude/runtime quota multipliers or GPT-5.x cache exemptions to Astra Mantle. ```bash # max_tokens guidance — tune before production: @@ -90,7 +101,8 @@ Generate `ai-migration/migrate_to_mantle.sh` — a shell script that configures **CRIS variant client surface:** When `migration_path` is `"runtime_openai_cris"`, check `aws-design-ai.json` → `ai_architecture.bedrock_models[].required_client_api`. If it is non-Responses (`converse`, `chat_completions`, or `invoke`), the generated CRIS client/adapter MUST emit that surface — e.g. a `client.converse(...)` call against the `bedrock-runtime` CRIS id — rather than defaulting to `responses.create`. If absent or `"responses"`, keep the Responses surface (the region-only-move default). -Skip Step 1 (provider adapter) when this step runs — the OpenAI SDK is the adapter. Step 2's test harness unconditionally imports `from provider_adapter import get_provider`, so when Step 1 is skipped (every `mantle`/`runtime_openai_cris` run), also emit a minimal `ai-migration/provider_adapter.py` stub here that exposes `get_provider()` returning a thin wrapper around the Mantle/CRIS OpenAI-SDK client configured by this script — this keeps Step 2 shared and path-agnostic without rewriting its import. +Skip Step 1 when this step runs. Step 2 uses the selected client directly for both +Mantle and runtime CRIS; do not emit or import a provider-adapter stub on these paths. --- @@ -106,7 +118,7 @@ Generate `ai-migration/provider_adapter.{py,js,go}` — an abstraction layer tha - `streaming: true` → `generate_stream(prompt) → Iterator[str]` - `embeddings: true` → `embed(text) → list[float]` - **Source provider class**: Use SDK imports from `ai-workload-profile.json` → `integration.sdk_imports`. Use model IDs from `ai-workload-profile.json` → `models[].model_id`. -- **Bedrock provider class**: Use `boto3` Converse API (`converse` for generate, `converse_stream` for streaming, `invoke_model` for embeddings with Titan). Use inference profile ARNs from `bedrock_monitoring.tf` output `bedrock_inference_profile_arns` as `modelId` — this enables cost attribution in Cost Explorer. Fall back to raw model IDs from `aws-design-ai.json` → `ai_architecture.bedrock_models[].aws_model_id` if profiles are not yet deployed. Use region from `preferences.json` → `design_constraints.target_region`. +- **Bedrock provider class**: For a Converse plan, use `boto3` (`converse` for generate, `converse_stream` for streaming, `invoke_model` for separately selected Titan embeddings). For `runtime_openai_cris`, preserve the runtime API selected in the design: Converse uses this adapter shape; an OpenAI-compatible Chat/Responses selection uses the OpenAI SDK at `https://bedrock-runtime.{region}.amazonaws.com/openai/v1` with Bedrock credentials and the exact CRIS ID. Never send a bare Astra ID to runtime or replace a validated `global.` profile with `us.`. Use an application inference profile only when the selected model/API supports it; otherwise retain `aws-design-ai.json` → `ai_architecture.bedrock_models[].aws_model_id`. Use region from `preferences.json` → `design_constraints.target_region`. ```python # Use inference profile ARN for cost attribution (preferred) @@ -120,7 +132,7 @@ Generate `ai-migration/provider_adapter.{py,js,go}` — an abstraction layer tha - **Shadow mode**: Send requests to both providers, return source response, log Bedrock response for comparison. - Include error handling and logging for API calls. -For JS: use `@aws-sdk/client-bedrock-runtime` + `@google-cloud/vertexai`. For Go: use `github.com/aws/aws-sdk-go-v2/service/bedrockruntime` + `cloud.google.com/go/aiplatform`. +For JS/Go, choose the source SDK from the recorded imports rather than assuming Vertex. Use the AWS runtime SDK for a Converse target, or the matching OpenAI client for a selected OpenAI-compatible runtime API. --- @@ -131,15 +143,59 @@ Generate `ai-migration/test_comparison.py` — always Python regardless of adapt **Requirements:** - Accept prompts from a JSON file (`--prompts`) or use built-in defaults (`--quick`) +- Support `--target-only` for the setup smoke test; this mode needs no source credentials. - Run each prompt against both the source provider and Bedrock - Measure per-prompt: latency (ms), success/failure, response text (truncated to 500 chars) - Compute summary statistics: p50/p95/mean latency per provider, quality score (trait matching against expected traits), pass/fail criteria - Pass criteria: Bedrock latency ≤ 2x source latency, mean quality score ≥ 0.9 - Output structured JSON to `--output` (default: `comparison_results.json`) - Built-in test prompts: include 3-5 prompts based on `ai-workload-profile.json` → `models[].usage_context` covering the primary use case -- **Source/target client setup — conditional on the migration path (Step 0):** - - **Non-Mantle paths** (`direct`, `gpt-oss`, `converse`, and gateway paths that emit the adapter): import the provider adapter via `from provider_adapter import get_provider` — those paths generate `provider_adapter.{py,js,go}` in Step 1, so the import resolves. - - **Mantle path** (`migration_path` = `"mantle"` / `"mantle_openai_responses"`, or the Step 3B `same_model_mantle` OpenRouter subroute): do NOT import `provider_adapter` — the Mantle path skips Step 1 and emits no adapter module, so that import would break. Instead set up the OpenAI SDK pointed at the Mantle `/openai/v1` base URL with a Bedrock token provider and call `client.responses.create(...)` (the client-setup pattern in `references/shared/openai-on-bedrock.md` § Client setup) for the Bedrock side, and the source provider's existing OpenAI SDK client for the source side. The harness stays self-contained and does not depend on the omitted adapter. +- **Adapter paths** (`direct`, `converse`, `gpt-oss`, or a gateway subroute that emits an adapter): use `from provider_adapter import get_provider` from the emitted provider adapter. +- **Mantle / runtime CRIS / `same_model_mantle` paths:** do not import `provider_adapter` because Step 0 intentionally skips it. Call the source using its recorded SDK and call the target directly with the selected Mantle API. Use this target function for proprietary GPT plans, substituting the saved design values: + + ```python + def call_bedrock(prompt): + from aws_bedrock_token_generator import provide_token + from openai import BedrockOpenAI + + region = "{target_region}" + model_id = "{aws_model_id}" + api_path = "{migration_path}" + required_client_api = "{required_client_api}" + if api_path == "runtime_openai_cris": + if required_client_api == "converse": + import boto3 + response = boto3.client("bedrock-runtime", region_name=region).converse( + modelId=model_id, messages=[{"role": "user", "content": [{"text": prompt}]}] + ) + return "".join(block["text"] for block in response["output"]["message"]["content"] if "text" in block) + if required_client_api not in {"responses", "chat_completions"}: + raise ValueError("Generate the model-specific selected runtime API before comparison") + from openai import OpenAI + client = OpenAI( + base_url=f"https://bedrock-runtime.{region}.amazonaws.com/openai/v1", + api_key=lambda: provide_token(region=region), max_retries=2, + ) + else: + client = BedrockOpenAI( + aws_region=region, + bedrock_token_provider=lambda: provide_token(region=region), + max_retries=2, + ) + if api_path == "mantle_openai_chat" or (api_path == "runtime_openai_cris" and required_client_api == "chat_completions"): + response = client.chat.completions.create( + model=model_id, messages=[{"role": "user", "content": prompt}] + ) + return response.choices[0].message.content + response = client.responses.create(model=model_id, input=prompt) + return response.output_text + ``` + + Substitute the resolved `required_client_api` for runtime plans. A selected `invoke` + surface needs its model-specific Invoke harness before setup; never substitute Converse. + Mantle uses the recorded migration path. + + Install `openai>=2.45.0,<3` and `aws-bedrock-token-generator>=1,<2` in the generated project's environment. The setup smoke test invokes this same harness with `--target-only --quick`, so setup and comparison cannot choose different APIs. --- @@ -151,15 +207,12 @@ Generate `ai-migration/setup_bedrock.sh`. - Dry-run by default (`--execute` flag to run for real) - Step 1 — Request model access: List each model from `aws-design-ai.json` → `bedrock_models[].aws_model_id` and the embedding model -- Step 2 — Create IAM role: Trust policy for the compute platform (Lambda, ECS, or EC2 based on `aws-design.json` if present). Bedrock policy — **conditional on the migration path (Step 0):** - - **Non-Mantle paths** (`direct`, `gpt-oss`, `converse`, gateway): grant `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` scoped to `arn:aws:bedrock:*::foundation-model/*`. These are correct for Converse/`bedrock-runtime` traffic. For a `runtime_openai_cris` target whose CRIS id is a GPT-6 model (Astra, Sol, Luna, 6.1 Sol), the Bedrock policy must additionally grant `bedrock:InvokeModel` on `arn:aws:bedrock:{region}:{account}:project/default` alongside the CRIS inference-profile ARN — both are required per the GPT-6 model cards' "Call the model" section. This does not affect the separate `bedrock-mantle:*` policy for bare mantle GPT ids. - - **Mantle path** (`migration_path` = `"mantle"` / `"mantle_openai_responses"`, or the `same_model_mantle` OpenRouter subroute): `bedrock:InvokeModel` does NOT authorize Mantle calls. Grant the Mantle inference actions `bedrock-mantle:CreateInference` (plus `bedrock-mantle:CallWithBearerToken` for bearer-token auth), or attach the managed policy `AmazonBedrockMantleInferenceAccess`. See `references/shared/openai-on-bedrock.md` § Client setup (IAM) for the authoritative action/policy names. +- Step 2 — Create IAM role: Trust policy for the compute platform (Lambda, ECS, or EC2 based on `aws-design.json` if present). Match inference permissions to the selected endpoint. Mantle needs `bedrock-mantle:CreateInference` scoped to `arn:aws:bedrock-mantle:{region}:{account_id}:project/*`, plus `bedrock-mantle:CallWithBearerToken` on `*` when using bearer credentials. Runtime needs `bedrock:InvokeModel` / `bedrock:InvokeModelWithResponseStream` against the selected foundation-model ARN and, for CRIS, the exact inference-profile ARN. For GPT-6 CRIS, also grant `bedrock:InvokeModel` on `arn:aws:bedrock:{region}:{account}:project/default`. Runtime actions do not authorize Mantle. +- If the runtime client uses `BEDROCK_INFERENCE_PROFILE_ARN` from the generated monitoring output, also authorize that exact `application-inference-profile/…` ARN, retaining the backing model/CRIS resources needed for the selected model and fallback. Resolve it from the same output used by the client; do not grant every application profile. Re-run setup after enabling or changing this override so the effective invocation target and IAM remain aligned. See [inference-profile IAM prerequisites](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html). - Compliance carry-through: if `preferences.json` → `design_constraints.compliance` is set and not `none`/`unknown` (full-flow Q2 / AI-only Q1.5), the script and its printed notes MUST reflect the design's Step 0.7 constraints — `AWS_REGION` from the compliance-constrained region (GovCloud for fedramp, EU for gdpr), `eu.` inference-profile model IDs for gdpr, and for hipaa a printed warning that Bedrock invocation logs retain original content (KMS-encrypt + restrict IAM on the log group before enabling) - Step 3 — Print required environment variables: `AWS_REGION`, `AI_PROVIDER=bedrock`, model IDs - Step 4 — Check quota: Query current TPM quota for the primary model via `aws service-quotas get-service-quota`. If `aws-design-ai.json` → `ai_architecture.quota_risk` is `"high"` or `"medium"`, print warning: "⚠️ Your token volume may exceed default Bedrock quotas. Request a quota increase via Service Quotas console (allow 1–5 business days)." Include the `aws service-quotas request-service-quota-increase` command template. -- Step 5 — Verification — **conditional on the migration path (Step 0):** - - **Non-Mantle paths** (`direct`, `gpt-oss`, `converse`, gateway): test Bedrock access with a simple `converse` call using the primary model. `converse` is a `bedrock-runtime` surface and is correct for these paths. - - **Mantle path** (`migration_path` = `"mantle"` / `"mantle_openai_responses"`, or the `same_model_mantle` OpenRouter subroute): the proprietary GPT models on Mantle are not reached via `converse`. Verify against the Mantle `/openai/v1` endpoint using an API the model supports — Responses (`client.responses.create(...)`) or Chat Completions — with the OpenAI SDK and a Bedrock token provider (the pattern in `references/shared/openai-on-bedrock.md` § Client setup). Do NOT use a `converse` call on this path. +- Step 5 — Verification: Run `test_comparison.py --target-only --quick` using the same selected API, model ID and region as the generated client. A Mantle selection must never run a Converse probe; a runtime selection must preserve its validated profile ID. - If `$MIGRATION_DIR/terraform/` exists, print coordination note: "Ensure the IAM role is referenced in compute.tf task definitions" - Use region from `preferences.json` → `design_constraints.target_region` @@ -595,6 +648,8 @@ print(f"Next step: Validate model quality, then migrate to Strands.") Creates `ai-migration/bedrock_monitoring.tf` — standalone Terraform for AI-only migrations that have no `baseline.tf` from the infra track. +Budget and anomaly monitoring apply to every path. **Application inference profiles are conditional:** omit the profile resource and its output for Mantle targets, including Astra, and for any model/API without documented support. Astra supports application inference profiles only on runtime Converse, not on runtime Chat/Responses. An unsupported profile must not prevent the applicable budget/anomaly resources from being emitted. + > **These are detective controls, not spend caps.** Bedrock has no native spend limit. > Anomaly Detection has ~24h data lag — you'll know within a day, not at month-end invoice. > AI-only migrations (infra stays on GCP) have zero cost guardrails without this file. @@ -713,22 +768,22 @@ resource "aws_ce_anomaly_subscription" "bedrock_alert" { } # Application inference profiles — cost attribution by environment/app +# AGENT: omit this resource and its output for Mantle or any unsupported model/API. # Tags on these profiles appear in Cost Explorer; tags on API calls do not. # copy_from must be a full foundation-model ARN or cross-region inference profile ARN. # For Claude models, use the cross-region inference profile ARN format: # arn:aws:bedrock:{region}:{account_id}:inference-profile/us.anthropic.claude-sonnet-5 # For Amazon Nova models, use: # arn:aws:bedrock:{region}:{account_id}:inference-profile/us.amazon.nova-pro-v1:0 -# [AGENT: generate one block per model in aws-design-ai.json → ai_architecture.bedrock_models[]] +# [AGENT: generate one block per eligible runtime model/API in the saved design] resource "aws_bedrock_inference_profile" "primary" { name = "${var.project_name}-${var.environment}-primary" model_source { - # Use cross-region inference profile ARN, not bare model ID - copy_from = "arn:aws:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:inference-profile/{cross_region_inference_profile_id}" - # AGENT: replace {cross_region_inference_profile_id} with the correct ID from - # aws-design-ai.json bedrock_models[].aws_model_id, prefixed with "us." for US regions - # e.g., us.anthropic.claude-sonnet-5 or us.amazon.nova-pro-v1:0 + copy_from = "{verified_model_source_arn}" + # AGENT: use the exact foundation-model or CRIS ARN resolved for this model/API. + # Preserve the chosen profile's geography; never manufacture a "us." prefix + # or replace a validated global profile with a regional one. } tags = { diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-report.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-report.md index 4b42c56ad..b24ad7da4 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-report.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-report.md @@ -248,7 +248,7 @@ Open any JSON file with a text editor or `cat | python3 -m json.tool` | Artifact | Description | | ----------------------------------------------------------- | --------------------------------------------------------------------------------- | -| `ai-migration/bedrock_monitoring.tf` | Bedrock budget alerts, anomaly detection, inference profiles for cost attribution | +| `ai-migration/bedrock_monitoring.tf` | Bedrock budget alerts and anomaly detection; add profile attribution only when an eligible profile was generated | | `ai-migration/STARTUP_PROGRAMS.md` or `STARTUP_PROGRAMS.md` | AWS Activate credit tiers, application URLs, eligibility guidance | | `ai-migration/setup_bedrock.sh` | Bedrock model access setup script | | `ai-migration/test_comparison.py` | A/B comparison harness for source vs Bedrock quality | @@ -322,7 +322,8 @@ Full security baseline capabilities. Executive summary shows a teaser; this appe | ------------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------- | ------------ | | Bedrock budget (1.5× projected) | Hard spend alert at 150% of estimated AI costs — fires before month-end | Runaway token spend from buggy retry loops | GCP Budgets (free, but no per-service scoping) | $0 | | Cost anomaly detection | Daily digest when AI spend deviates from baseline (~24h data lag) | Gradual cost creep, unexpected model-price changes | None (no GCP per-service anomaly equivalent) | $0 | -| Inference profiles (tagged) | Per-model cost attribution in Cost Explorer | Invisible cost distribution across models | None | $0 | + +Append an **Inference profiles (tagged)** row for per-model Cost Explorer attribution only when the actual generated `bedrock_monitoring.tf` contains an enabled `aws_bedrock_inference_profile` resource and its `bedrock_inference_profile_arns` output, and the selected model/API supports that profile. A file's existence, a comment mentioning profiles, or a disabled resource is not evidence of this control. Omit the row for Astra Mantle Chat/Responses and runtime OpenAI Chat/Responses; retain budget and anomaly rows. A supported runtime Converse pack that emits the profile retains the attribution row. Apply this same condition to the artifact description in Appendix E. These are detective controls, not spend caps. You will know within ~24 hours if something goes wrong — not at invoice time. diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/shared/openai-on-bedrock.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/shared/openai-on-bedrock.md index 429acfb7c..016e9fa0b 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/shared/openai-on-bedrock.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/shared/openai-on-bedrock.md @@ -44,6 +44,9 @@ share a generation with a future `Sol`. **GPT-6** carries forward `Sol` and `Lun **GPT-6.1 Sol** is a point-release upgrade to GPT-6 Sol (same tier, improved capability, a deeper cache-read discount) — not a new tier name (cite [openai.com/index/introducing-gpt-6-1-sol/](https://openai.com/index/introducing-gpt-6-1-sol/)). +GPT-6 Astra is the newest, most capable OpenAI model in this catalog. Sol remains an Active, +lower-cost reasoning alternative; Astra does not make existing GPT-5.x sources require a model upgrade. + > **Context-window conflict (resolved):** the GPT-5.6 launch blog states 272K for all three variants; all three > model cards state 1M. **The model cards are authoritative** — use 1M for GPT-5.6. GPT-5.5 and GPT-5.4 are 272K on > both sources. Re-check on refresh; if AWS corrects the blog, the cards still win. @@ -57,6 +60,31 @@ discount) — not a new tier name (cite [openai.com/index/introducing-gpt-6-1-so ## Access Paths — Split by Family +**GPT-6 Astra has two endpoints; regions reverified 2026-09-30:** + +| Endpoint | Reach | Model ID | Supported APIs | +| ----------------- | ----------------------------- | ----------------------------------------------------- | ------------------------------------- | +| `bedrock-mantle` | Standard in-region, **us-east-1 / us-west-2** | `openai.gpt-6-astra` | Responses, Chat Completions | +| `bedrock-runtime` | CRIS only, no in-region form | `us.openai.gpt-6-astra` / `global.openai.gpt-6-astra` | Responses, Chat Completions, Converse | + +Use `/openai/v1` on either endpoint for the OpenAI-compatible APIs. Astra does **not** support +Invoke or Messages. On runtime, Guardrails and application inference profiles are **Converse-only**; +server-side tool use is unsupported. Runtime supports non-streaming JSON Schema output (reverified +2026-10-03): Chat uses `response_format.json_schema`, Responses uses `text.format`, and Converse +uses `outputConfig.textFormat.type: "json_schema"` with `name` and a JSON-encoded schema string in +`outputConfig.textFormat.structure.jsonSchema`, plus **`additionalModelRequestFields.text.format.strict=true`**. +For Chat/Responses set the format type to `json_schema` and include `name`, `schema` and `strict: true`. +Use an object schema, require every field, set `additionalProperties: false`, validate before calling, +and check refusal/incomplete responses before parsing. Do not claim streamed JSON Schema support. +On mantle, server-side tool calling +is supported, implicit/explicit prompt caching is **Responses-only**, and application inference +profiles are unsupported. Do not copy GPT-5.6's API or feature matrix onto Astra. + +The card specifies **1,050,000 context tokens and 128,000 maximum output tokens**, with text/image +input and text output. The launch announcement rounds the input context to 1M; use the card's +exact limits for capacity checks. Browser/computer-use capability does not establish parity with +OpenAI-hosted tools; probe the application's specific tool flow. + **GPT-5.5 and GPT-5.4 are `bedrock-mantle`-only and in-region only.** Their model cards list a single Programmatic Access row (`bedrock-mantle`, Geo/Global "Not supported") and In-Region pricing only. @@ -137,7 +165,8 @@ including `bedrock-mantle:CreateInference` and `bedrock-mantle:CallWithBearerTok authorize mantle calls. On the GPT-5.6 `bedrock-runtime` path the usual `bedrock:InvokeModel*` against the CRIS inference-profile ARN applies, as for any other runtime model. -**Reasoning effort:** all five accept `none`, `low`, `medium`, `high`, `xhigh`, `max`. Because these models reason +**Reasoning effort:** the five GPT-5.x models accept `none`, `low`, `medium`, `high`, `xhigh`, `max`. Astra's card +does not enumerate these values; verify its accepted settings rather than inheriting the GPT-5.x list. Because these models reason before responding, the model's output items (which may include reasoning items) must be passed back in the next request for multi-turn and tool-calling flows. @@ -145,7 +174,7 @@ request for multi-turn and tool-calling flows. ## Regional Availability — Endpoint-Aware -**The mantle in-region matrix** (the only reach for GPT-5.5 / GPT-5.4, and the in-region option for GPT-5.6): +**The mantle in-region matrix** (the only reach for GPT-5.5 / GPT-5.4, and the in-region option for GPT-6 Astra / GPT-5.6): | Model | us-east-1 | us-east-2 | us-west-2 | us-gov-west-1 | us-gov-east-1 | | ------------- | --------- | --------- | --------- | ------------- | ------------- | @@ -161,6 +190,11 @@ request for multi-turn and tool-calling flows. Terra and Luna reached AWS GovCloud (US-West, US-East) in August 2026 — newer than the rest of this matrix. +**Astra runtime availability:** US Geo CRIS can be called from `us-east-1`, `us-east-2`, `us-west-1`, +`us-west-2`, and `ca-central-1`. Global CRIS additionally supports `eu-central-1`, `eu-north-1`, +`eu-west-1`, `eu-west-2`, `eu-west-3`, `ap-northeast-1`, `ap-northeast-2`, `ap-northeast-3`, +`ap-south-1`, `ap-southeast-1`, `ap-southeast-2`, and `sa-east-1`. No EU/India Geo profile or +GovCloud region is listed on the Astra card. A Canada caller using US Geo still routes to the US. **GPT-6 Astra's mantle reach is us-east-1 and us-west-2; GPT-6 Sol, GPT-6 Luna, and GPT-6.1 Sol are us-east-1 only.** All four additionally reach broad commercial regions via `bedrock-runtime` CRIS (`us.` / `global.` inference profiles), same as GPT-5.6 — a region outside the mantle matrix does not block the same-model path for @@ -176,13 +210,13 @@ Verify current footprints per model card / `aws___get_regional_availability` (AW ## Pricing -Read off the model cards, 2026-08-31. All rates per 1M tokens, Standard tier (Priority and Flex are NOT supported +GPT-5.x rates read off the model cards, 2026-08-31; Astra rates verified 2026-09-09. All rates per 1M tokens, Standard tier (Priority and Flex are NOT supported for these models). Sol rates reflect the Aug 21, 2026 reduction (−20% input / −33% output vs launch rates), which the AWS What's New announcement lists as promotional through at least Nov 21, 2026. **Pricing now has an inference-option dimension:** -- **In-Region and Geo CRIS: 1.10x OpenAI's standard list price** (parity with OpenAI's _data residency_ tier). -- **Global CRIS: OpenAI's standard list price** — cost parity, available for GPT-5.6 only, and only when the +- **GPT-5.x In-Region and Geo CRIS: 1.10x OpenAI's standard list price** (parity with OpenAI's _data residency_ tier). +- **Global CRIS: OpenAI's standard list price** — cost parity for the GPT-5.6 rates verified above, and only when the workload has no data-residency constraint. So the honest cost statement is conditional, not flat: a same-model GPT-5.6 move on Global CRIS is @@ -220,6 +254,9 @@ Context window is **1.05M tokens** for all four models (confirmed on every model context must be priced at this long-context tier. Max output tokens: 128,000 (Astra, Sol, Luna) / 131,072 (6.1 Sol). +Astra cache-write rates use a **30-minute cache-write** window. Verify the source +provider rate before claiming savings or parity for the selected context tier. + ### GPT-5.6 — short context (272K) | Model | In-Region / Geo (in · out) | Global CRIS (in · out) | Cache write / read (In-Region) | @@ -251,7 +288,10 @@ A workload above 272K context must be priced at the long-context tier. > Separately, the **AWS Price List API still carries no GPT-5.x rows** (checked 2026-08-04): > a missing or empty price-list result must not be read as "model unavailable." -### Prompt caching — GPT-5.6 and GPT-6 only +### Prompt caching — GPT-5.6 details + +Astra separately lists implicit and explicit caching on mantle Responses, with rates above. +Its card does not establish the GPT-5.6 minimum-prefix, breakpoint, or quota-exemption rules below. Listed as a supported feature on the GPT-5.6 Sol/Terra/Luna model cards and on all four GPT-6 model cards (Astra, Sol, Luna, 6.1 Sol — confirmed by the cache-write/cache-read columns in the GPT-6 pricing tables above). The @@ -275,6 +315,9 @@ count against the input-TPM quota**, which compounds the benefit at scale. ## Quotas +**Astra runtime:** the model card specifies TPM accounting with **10x output-token burndown**. +Do not reuse the GPT-5.x mantle quota accounting below for Astra without verifying its quota rules. + Inference on `bedrock-mantle` is governed by **two per-model, per-region quotas: input tokens per minute and output tokens per minute. There is no requests-per-minute quota.** Exceeding a TPM quota returns HTTP 429. diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-migration-guardrails.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-migration-guardrails.md index 45d18b24a..bf7f5ee1d 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-migration-guardrails.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-migration-guardrails.md @@ -48,7 +48,7 @@ Inference on `bedrock-mantle` for these models is governed by **two per-model, p | Medium | Medium | Monitor 429s against **token** throughput, not request rate; enable prompt caching | | High | High | Enable prompt caching first (cached input is exempt from input TPM), then request a quota increase | -**The `bedrock-runtime` fallback exists only for GPT-5.6 and GPT-6 (Astra, Sol, Luna, 6.1 Sol).** GPT-5.5 and GPT-5.4 are `bedrock-mantle` only and in-region only — for them, "switch to `bedrock-runtime`" requires moving to a different model (Bedrock-native or `gpt-oss`), a model change with its own eval cost. GPT-5.6 Sol/Terra/Luna and GPT-6 Astra/Sol/Luna/6.1 Sol DO have a `bedrock-runtime` path via CRIS inference profiles (`us.`/`in.`/`global.` prefixed ids; the model cards recommend runtime for new applications) — a legitimate endpoint option with its own quota family, and on Global CRIS it is also the cost-parity option. Scaling levers on the mantle path, in order: +**The `bedrock-runtime` fallback exists only for GPT-5.6 and GPT-6 (Astra, Sol, Luna, 6.1 Sol).** GPT-5.5 and GPT-5.4 are `bedrock-mantle` only and in-region only — for them, "switch to `bedrock-runtime`" requires moving to a different model (Bedrock-native or `gpt-oss`), a model change with its own eval cost. GPT-5.6 Sol/Terra/Luna and GPT-6 Astra/Sol/Luna/6.1 Sol DO have a `bedrock-runtime` path via CRIS inference profiles (GPT-5.6 `us.`/`in.`/`global.`, GPT-6 `us.`/`global.` prefixed ids; the model cards recommend runtime for new applications) — a legitimate endpoint option with its own quota family, and on Global CRIS it is also the cost-parity option. Scaling levers on the mantle path, in order: 1. Prompt caching (GPT-5.6 and GPT-6 only) — 90% off cached input and exempt from the input-TPM quota 2. Exponential backoff with a bounded retry count (`max_retries` on the OpenAI SDK) @@ -60,6 +60,19 @@ See `references/shared/openai-on-bedrock.md` for the endpoint, region matrix, an **Source:** [Get started with GPT-5.6 on Amazon Bedrock — Quotas and scaling](https://aws.amazon.com/blogs/machine-learning/get-started-with-openai-gpt-5-6-sol-terra-and-luna-on-amazon-bedrock/) +### GPT-6 Astra — verify the selected endpoint's quota + +Astra Standard mantle is available in `us-east-1` and `us-west-2` (region availability verified 2026-09-30). Its model card lists implicit/explicit caching on +Responses and 30-minute cache-write prices, but does not establish GPT-5.6's cached-input quota +exemption, minimum prefix, or breakpoint rules. Verify account quotas before promising throughput. + +Astra runtime uses **10× output-token burndown**: +`required_TPM = peak_input_TPM + 10 × peak_output_TPM`. Use the selected US Geo or Global CRIS quota; +do not multiply total source TPM by ten or carry this rule over to mantle. An endpoint change also +requires residency consent and API/capability checks from the canonical catalog. + +**Source:** [Astra model card, verified 2026-09-16](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html) + ### Other models on Mantle For non-OpenAI models served through Mantle, verify current quota dimensions and any shared-account limits in the [Mantle documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/bedrock-mantle.html) before advising on throughput. Do not carry the OpenAI TPM-only model over to other providers without checking, and do not assume previously documented shared-RPM behavior still applies. @@ -78,7 +91,9 @@ Claude models on Mantle have an additional **output TPM cap** that differs by mo **Impact for migration decisions:** - For Claude migrations at medium/high volume: the 2M output TPM cap on Claude 4.7+ is the binding constraint -- For OpenAI proprietary GPT targets: this Claude cap does not apply. Their mantle constraint is the per-model input/output TPM quota described above; for GPT-5.6 the `bedrock-runtime`/CRIS path is an available alternative (its own quota family), while GPT-5.5/5.4 have no runtime path +- For OpenAI proprietary GPT targets: this Claude cap does not apply. Use the GPT-5.x mantle quotas or + Astra-specific verification above. Astra / GPT-5.6 have runtime CRIS alternatives with their own + quotas; GPT-5.5/5.4 have no runtime path - For `gpt-oss` targets: these do run on `bedrock-runtime`, so standard account TPM limits and the Converse-path mitigations apply - When output-heavy workloads (long JSON, tool outputs, multi-step reasoning) are detected, flag the relevant cap prominently; recommend `bedrock-runtime` for production **only** when the target model actually has a `bedrock-runtime` path diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-model-lifecycle.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-model-lifecycle.md index 11042e780..adaf2a939 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-model-lifecycle.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-model-lifecycle.md @@ -73,6 +73,14 @@ On each run, compute `days_to_eol = EOL date − today` for every model in the L --- +## GPT-6 Astra — Active (verified September 9, 2026) + +The [Astra model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html) +lists launch **September 8, 2026**, lifecycle **Active**, and no scheduled EOL date. Its EOL is +**no sooner than September 8, 2027**, with a Legacy period of at least six months. The earliest +possible EOL is not an announced retirement date. Existing GPT-5.x entries are not superseded. +See `openai-on-bedrock.md` for access paths, regions, and pricing. + ## Legacy / EOL Models (as of October 5, 2026) For models launched before 2026-09-07, the [legacy lifecycle table](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle-legacy.html) is authoritative. For models launched on or after that date, the model card and the runtime `modelLifecycle.status` field are authoritative — they will not appear in the table below. The table captures pre-policy-change models referenced elsewhere in this plugin. **Recompute the Status column on each run** using `days_to_eol = EOL date − today`. diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-openai-to-bedrock.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-openai-to-bedrock.md index 6edb373f3..2503256b5 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-openai-to-bedrock.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/ai-openai-to-bedrock.md @@ -30,20 +30,22 @@ For an explicitly requested 5.5 assessment, read its [Bedrock model card](https: ## Key Insight: OpenAI's Own Models Run on Bedrock **The old premise of this guide — "OpenAI models are unavailable on AWS, so migrating means switching model families" -— is obsolete.** GPT-5.6 Sol / Terra / Luna, GPT-5.5, and GPT-5.4 are generally available on Bedrock, at OpenAI's -data-residency-tier rates, counting toward existing AWS commitments. +— is obsolete.** GPT-6 Astra, GPT-5.6 Sol / Terra / Luna, GPT-5.5, and GPT-5.4 are generally available on Bedrock, +counting toward existing AWS commitments. Pricing depends on the model and inference option; +see `shared/openai-on-bedrock.md`. Two consequences that invert the previous decision logic: 1. **When the source model is on Bedrock, keep it.** A same-model move has no behavior delta to validate, no prompt re-engineering, and no eval regression risk — the migration is an endpoint and credential change, not a model change. This is the default recommendation. -2. **The cost of a same-model move depends on the inference option — state it conditionally.** In-Region and Geo +2. **The cost of a same-model move depends on the inference option — state it conditionally.** GPT-5.x In-Region and Geo CRIS are priced at OpenAI's _data residency_ tier, exactly 1.10x the standard list price, so those moves cost ~10% more. **GPT-5.6 on Global CRIS is priced at the standard list price — cost parity** — available only when the workload has no data-residency constraint (GPT-5.5 / GPT-5.4 have no CRIS at all). See `shared/openai-on-bedrock.md`. The non-cost case — AWS commitments, IAM/VPC/CloudTrail governance, residency, prompt caching, one vendor relationship — carries the in-region path; never present that path as free or neutral. + For Astra, use its separately dated Bedrock rates and verify the source provider's rate before comparing. Cross-family mapping (to Claude / Nova / DeepSeek) is still the right answer in two situations: the source model has no Bedrock equivalent, or the user's priority is cost and is willing to accept a model change to get it. Both are @@ -62,7 +64,7 @@ GPT-5.4, the target is **the same model on Bedrock**. | Source model | Bedrock target | Model ID (mantle / runtime CRIS) | Assessment | | ------------- | -------------- | ------------------------------------------------------- | --------------------------------------------------------------- | -| GPT-6 Astra | GPT-6 Astra | `openai.gpt-6-astra` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | +| GPT-6 Astra | GPT-6 Astra | `openai.gpt-6-astra` / `us.` `global.` prefixed | `strong_migrate` after the Astra region/API gate in `shared/openai-on-bedrock.md`; price the chosen inference option | | GPT-6 Sol | GPT-6 Sol | `openai.gpt-6-sol` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | | GPT-6 Luna | GPT-6 Luna | `openai.gpt-6-luna` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | | GPT-6.1 Sol | GPT-6.1 Sol | `openai.gpt-6.1-sol` / `us.` `global.` prefixed | `strong_migrate` — same model; parity on Global CRIS, else +10% | @@ -73,8 +75,9 @@ GPT-5.4, the target is **the same model on Bedrock**. | GPT-5.4 | GPT-5.4 | `openai.gpt-5.4` (mantle only) | `strong_migrate` — same model, ~10% over OpenAI std | Then apply the **region gate** below. Record `model_change: false` and the chosen path in -`aws-design-ai.json` → `ai_architecture.code_migration`: `migration_path: "mantle_openai_responses"` for the -in-region mantle endpoint, or `migration_path: "runtime_openai_cris"` when a GPT-5.6 target is served via +`aws-design-ai.json` → `ai_architecture.code_migration`: `migration_path: "mantle_openai_chat"` for +Astra sources using Chat Completions, or `"mantle_openai_responses"` for a selected Mantle Responses +path. Use `migration_path: "runtime_openai_cris"` when a GPT-6 Astra / GPT-5.6 target is served via `bedrock-runtime` with a CRIS id (the model cards recommend runtime for new applications). Report the assessment honestly: `strong_migrate` here means "low-risk, well-supported move", not "cheaper". @@ -83,6 +86,11 @@ Report the assessment honestly: `strong_migrate` here means "low-risk, well-supp The gate differs by family (see `shared/openai-on-bedrock.md` § Regional Availability): +**GPT-6 Astra:** apply its own mantle and runtime matrices from the canonical catalog. For strict +in-region residency, the mantle region must match. Otherwise offer a supported runtime CRIS path +with explicit routing consent; a Global-only caller region cannot satisfy a Geo-only requirement. +If neither path meets the constraints, offer a supported region or a model change under Tier 1. +Do not assume GPT-5.6's broader regional footprint applies to Astra. **GPT-6 Astra / Sol / Luna / 6.1 Sol:** same shape as GPT-5.6 — if the target region is in the mantle in-region matrix (narrower than GPT-5.6's: us-east-1 + us-west-2 for Astra only, us-east-1 only for Sol/Luna/6.1 Sol — see `openai-on-bedrock.md` § Regional Availability), either endpoint works; otherwise the `bedrock-runtime` CRIS path @@ -122,15 +130,19 @@ let the user choose.** Do not silently default to either. Option A tier mapping: -| Source tier | Bedrock GPT target | -| -------------------------------------------------------- | ------------------ | -| Flagship reasoning (o1-pro, o3-pro, GPT-5 Pro, `*-Pro`) | GPT-5.6 Sol | -| Flagship general (GPT-4o, GPT-4.1, GPT-5/5.1/5.2, GPT-4) | GPT-5.6 Terra | -| Mid / reasoning (o1, o3, o4-mini) | GPT-5.6 Terra | -| Fast / cheap (`*-mini`, `*-nano`, GPT-3.5 Turbo) | GPT-5.6 Luna | +| Source tier | Bedrock GPT target | +| -------------------------------------------------------- | ------------------------------------------------------ | +| Flagship reasoning (o1-pro, o3-pro, GPT-5 Pro, `*-Pro`) | GPT-6 Astra; GPT-5.6 Sol as the lower-cost alternative | +| Flagship general (GPT-4o, GPT-4.1, GPT-5/5.1/5.2, GPT-4) | GPT-5.6 Terra | +| Mid / reasoning (o1, o3, o4-mini) | GPT-5.6 Terra | +| Fast / cheap (`*-mini`, `*-nano`, GPT-3.5 Turbo) | GPT-5.6 Luna | Option A is subject to the same region gate as Tier 0. +Astra is the capability-first choice for this reasoning tier, not a mandatory upgrade for an +existing Bedrock model. At the recorded rates it costs 2.5x Sol for the same inference option and +context tier; evaluate quality and cost before choosing between them. + ### Tier 2 — Cost priority override If `ai_constraints.ai_priority == "cost"`, present the cheapest adequate Bedrock target **as an explicit @@ -231,7 +243,7 @@ frontier tier. Recommend them when the user wants OpenAI-architecture models on ## Migration Decision Framework -**Migrate to Bedrock, same model (Tier 0), if:** the source is GPT-5.6 / 5.5 / 5.4 and the target region carries it. +**Migrate to Bedrock, same model (Tier 0), if:** the source is GPT-6 Astra / GPT-5.6 / 5.5 / 5.4 and the selected endpoint, API, and region support it. This is the common case and needs no cost justification. **Migrate to Bedrock, model change (Tier 1/2), if:** @@ -239,8 +251,8 @@ This is the common case and needs no cost justification. - The source model is not on Bedrock (see the region and catalog lists in `shared/openai-on-bedrock.md`) - The user's priority is cost and they accept a model change — note that the cheapest option is often Nova, and that GPT-5.6 Luna already beats Claude Haiku 4.5 on price -- Bedrock-native features are required that mantle does not expose (Guardrails, Knowledge Bases, invocation logging, - Converse-based tooling) — these need a Bedrock-native model or gpt-oss +- Required features cannot be served by a supported same-model path. Astra / GPT-5.6 have runtime CRIS + options for applicable governance features; check their capability matrix before proposing a model change **Consider staying on OpenAI's own API only if:** @@ -268,7 +280,7 @@ rather than dismissing it. For a long-context (>272K) workload the gap is much w | ------------------------------------------ | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | OpenAI SDK (direct) | Same model on mantle Responses API | Base URL + credential + model ID change; see `shared/openai-on-bedrock.md` for the `/openai/v1` path | | Responses API | Mantle Responses API | Closest to a drop-in; still verify the path and model ID | -| Chat Completions | Reshape to Responses | Unverified for GPT-5.x — probe before committing | +| Chat Completions | Preserve supported Chat when selected | Reshape only for a selected API change; verify the model and endpoint | | Function calling | Supported on GPT-5.x via mantle; Claude tools via Converse | Same-model keeps tool semantics identical | | Reasoning effort | `reasoning={"effort": ...}` on mantle | `none` / `low` / `medium` / `high` / `xhigh` / `max` | | Prompt caching | GPT-5.6 only (90% off cached input) | Not listed for GPT-5.5 / 5.4; Claude has its own caching | diff --git a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/bedrock-quotas.md b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/bedrock-quotas.md index 5d6916bc3..9ca28857e 100644 --- a/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/bedrock-quotas.md +++ b/plugins/aws-startup-advisor/skills/gcp-to-aws/references/vendored/ai/bedrock-quotas.md @@ -16,10 +16,14 @@ OpenAI and Gemini APIs have high default rate limits for paying customers (OpenA --- -## The 5× Burndown Trap +## Output-Token Burndown Depends on the Model and Endpoint Claude models (3.7+) on Bedrock consume quota at **5× the rate for output tokens**. This is the single most common surprise for teams migrating from OpenAI (which has no output multiplier). +**GPT-6 Astra on `bedrock-runtime` uses 10× output-token burndown**: one output token consumes ten quota +tokens. This is runtime-specific; do not apply it to mantle or infer Astra mantle quotas from GPT-5.6. +Source: [Astra model card, verified 2026-09-16](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html). + **Effective TPM by workload type** (at 200K TPM quota on Claude): | Workload Type | Input:Output Ratio | Effective TPM | vs OpenAI 2M TPM | @@ -30,6 +34,9 @@ Claude models (3.7+) on Bedrock consume quota at **5× the rate for output token Formula: `Effective TPM = TPM_quota / (input_ratio + output_ratio × 5)` +For Astra runtime, replace 5 with 10. A 60:40 workload with a hypothetical 200K TPM quota supports +about 43.5K actual tokens/minute: `200K / (0.6 + 0.4 × 10)`. Check the account's actual quota. + Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Output-heavy workloads that hit Claude quota limits may benefit from routing to these models for applicable tasks. --- @@ -38,12 +45,14 @@ Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Apply during Design phase after model selection. Uses `ai_token_volume` from `preferences.json` and the selected model family. -| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | -| ------------------------- | ---------------------------------- | ------------ | ----------------------------------------------------------------------------- | -| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | -| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | -| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | -| `"low"` | Any | `"low"` | No action | +| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | +| ------------------------- | -------------------------------------- | ------------ | ------------------------------------------------------------------------------- | +| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | +| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | +| `"medium"` | Astra runtime (10× output burndown) | `"medium"` | "Calculate weighted peak TPM and verify the selected CRIS quota before cutover" | +| `"medium"` | Verified 1× models (e.g. Nova / Llama) | `"low"` | No action | +| `"medium"` | Unverified model/endpoint quota | `"medium"` | "Verify quota dimensions before estimating throughput" | +| `"low"` | Any | `"low"` | No action | Include `quota_risk` in `aws-design-ai.json` → `ai_architecture`. @@ -99,10 +108,14 @@ Include this guidance in migration artifacts when `quota_risk` ≥ `"medium"`. U **How to calculate the target value:** ``` -Target TPM = (peak_tokens_per_minute_on_source_provider) × burndown_multiplier +Target TPM = peak_input_tokens_per_minute + peak_output_tokens_per_minute × output_burndown_multiplier ``` -Where `burndown_multiplier` = 5 for Claude models (because output tokens consume 5× quota), 1 for all others. If the user doesn't know their peak TPM, estimate from daily volume: +Use `output_burndown_multiplier` = 10 for Astra runtime, 5 for the Claude runtime models above, +and 1 only where verified for the selected model/endpoint. The multiplier applies to output tokens, +not the total source TPM. For separate mantle input/output quotas, size each quota independently. +If the user doesn't know their peak TPM, estimate total volume below and split it using the measured +input:output ratio: ``` Estimated peak TPM = (daily_tokens / active_hours / 60) × 3 (3× for peak headroom) diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/SKILL.md b/plugins/aws-startup-advisor/skills/llm-to-bedrock/SKILL.md index e268efe8b..3748e0180 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/SKILL.md +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/SKILL.md @@ -789,7 +789,7 @@ reliably visible, and even that must be materialized to the file for the baselin ### B4 — Bedrock preflight ```bash -uv run --project $SCRIPTS python $SCRIPTS/preflight_bedrock.py --region $REGION --models --dataset-size 200 +uv run --project $SCRIPTS python $SCRIPTS/preflight_bedrock.py --region $REGION --models --dataset-size 200 --runtime-api ``` (`--dataset-size 200` matches the golden-dataset cap, so the quota warning reflects the worst case. Prefix with `AWS_PROFILE=$AWS_PROFILE_CHOICE` if B2 chose a non-default profile.) @@ -799,8 +799,9 @@ first failing model) plus `failing_models` (all failing ids); per-model verdicts - `ok == false` + `reason: credentials` → show the detail (configure/refresh credentials), stop; user re-runs after fixing. - `ok == false` + `reason: model_access` → model access not enabled in the Bedrock console (NOT an IAM problem): point the user at the console Model access page for the failing models, stop; re-run B4 after they enable it. -- `ok == false` + `reason: authz` → IAM denies inference. For a Converse/InvokeModel target the action to grant is `bedrock:InvokeModel`; for a mantle-only `openai.gpt-5*` target it is the `bedrock-mantle:*` set (see B4a). The `detail` names which. Tell the user the action to grant; stop. +- `ok == false` + `reason: authz` → IAM denies inference. For a Converse/InvokeModel target the action to grant is `bedrock:InvokeModel`; for a bare proprietary GPT target (GPT-5.x or `openai.gpt-6-astra`) it is the `bedrock-mantle:*` set (see B4a). The `detail` names which. Tell the user the action to grant; stop. - `ok == false` + `reason: mantle_deps_missing` → the pinned scripts environment lacks `openai` / `aws-bedrock-token-generator`, so a mantle-only target could not be probed at all. This is an environment fault, not a Bedrock verdict: tell the user to re-sync (`uv sync --project $SCRIPTS`) and stop. Do NOT proceed — access was never verified. +- `ok == false` + `reason: openai_deps_missing` → the selected runtime Chat/Responses API could not be probed because its SDK is missing. Re-sync the same scripts environment and stop; do not substitute Converse. - `ok == false` + `reason: model_unavailable` → Read the `resolve-bedrock-model-id` reference at `$HELPERS/resolve-bedrock-model-id/resolve-bedrock-model-id.md` and follow its procedure with each ID from `failing_models` + region. AskUserQuestion with the candidates: "Use `` (cross-region inference profile)" / "Paste a different model ID" / "Abort". On a choice, replace the ID in `$TARGET_MODELS` and re-run B4. - `ok == false` + any other `reason` → show `detail` and stop. - `ok == true` → proceed. Surface any `quota_warning`, and any model whose `reason` is @@ -839,6 +840,33 @@ uv run --project $SCRIPTS python $SCRIPTS/validate_result.py --schema ") +from preflight_bedrock import normalize_api_path +resolved_api_path = normalize_api_path( + plan_migration_path, validated_target_model_ids, plan_runtime_api) +``` + +Read `plan_runtime_api` from the selected model row's `required_client_api`. For older +plans, fall back to that row's `runtime_api`, then `ai_architecture.code_migration.runtime_api`. +A current Design CRIS plan without an explicit API uses `responses`; only a legacy +Converse plan retains `None`. An explicit `invoke` surface is unsupported by this +execution lane: stop for an explicit supported API decision rather than treating it as Converse. A `runtime_openai_cris` +plan resolves to `runtime_openai_chat` or `runtime_openai_responses` when that API is explicit, +and retains the Converse default otherwise. Do not ignore this field on dispatch or resume. +The legacy `migration_path: "mantle"` alias with bare proprietary GPT targets resolves to +`mantle_openai_responses`, so evaluator and rewriter keep the documented Responses default. +Other explicit `migration_path` values are preserved. With no path, an all-bare proprietary GPT target +set (including Astra) resolves to `mantle_openai_responses`; runtime targets use `converse`. +A mixed legacy target set is `model_unresolvable`: stop before evaluation and request an +explicit plan instead of guessing. Recompute the same value on every dispatch/resume from +the same saved inputs; the existing target-model and `assess_design_sha256` checks detect +changed inputs. Do not let C3 and C5 choose separate missing-path defaults. + Build this exact line format (agents parse the labels). Omit lines marked optional when empty: ``` @@ -846,6 +874,8 @@ Repository: <$REPO> AWS region: <$REGION> AWS profile (pass as --profile / AWS_PROFILE= inline on every aws/boto3 invocation): <$AWS_PROFILE_CHOICE — omit line if default> Target Bedrock model(s): +Target API path: +Target API surface: Migration plan dir: <$MIGRATION_DIR> Resolved target model id: Scripts directory (pinned uv toolchain): <$SCRIPTS> @@ -995,7 +1025,7 @@ prior-phase file paths), then validate its output file: - `source_key_auth` → user supplies a new key (re-run B3) or sets baseline unavailable - `authz` → IAM denies inference. The `detail` names the action set to grant: `bedrock:InvokeModel*` for a Converse target, or the `bedrock-mantle:*` actions - (`CreateInference`, `CallWithBearerToken`) for a mantle-only `openai.gpt-5*` target. + (`CreateInference`, `CallWithBearerToken`) for a bare proprietary GPT target (GPT-5.x or `openai.gpt-6-astra`). User fixes IAM; nothing fingerprinted changes, so re-dispatch the blocked phase only. Do NOT route this to `model_access` — the console Model access page is the wrong fix for an IAM denial and vice versa. @@ -1032,17 +1062,13 @@ Below, AskUserQuestion: invalidates ANALYSIS/EVAL and execution resumes at C1). Cap: 2 retries. - **Abort** → stop, no code touched. -**Gate (a.5) — Rewrite strategy (from migration plan).** Read `migration_path` from -`$MIGRATION_DIR/aws-design-ai.json` → `ai_architecture.code_migration.migration_path`. -If the value **starts with** `"mantle"` (`"mantle"`, `"mantle_openai_responses"`), set -`rewrite_strategy = "mantle"`. Otherwise (value is `"converse"`, `"gpt-oss"`, or the field is -absent), set `rewrite_strategy = "converse"`. -No user question needed — the decision was already made during the Assess/Design phase. - -Match on the prefix, not on equality: Design writes the more specific -`"mantle_openai_responses"` for a same-model OpenAI migration, and an equality check against -`"mantle"` would silently route those runs down the Converse path — rewriting working -same-model code into a boto3 Converse client against a model that has no Converse surface. +**Gate (a.5) — Rewrite strategy (from the normalized API path).** Reuse +`resolved_api_path` from the context-block normalization above. If it starts with `mantle`, +set `rewrite_strategy = "mantle"`. For `runtime_openai_chat` or `runtime_openai_responses`, +set `rewrite_strategy = "runtime_openai"`; otherwise set `rewrite_strategy = "converse"`. +An absent legacy plan field is not a second Converse default here: a validated bare Astra +plan without that field has already resolved to Mantle Responses before C3. Explicit +`mantle_openai_chat`, `mantle_openai_responses`, and runtime CRIS choices stay unchanged. **Gate (b) — Behavior-delta resolution.** For each `analysis.behavior_deltas[]` with `user_visible == true`, AskUserQuestion with the options from the `behavior-delta-detection` @@ -1071,14 +1097,27 @@ When `rewrite_strategy == "mantle"`, C5's context block ALSO includes: signal for the default Converse path) - `Mantle model map: -> ` — sourced from the plan's `ai_architecture.bedrock_models[]` entries (each `source_model` → `aws_model_id` pair). -- `Mantle surface: responses` and `Mantle base path: /openai/v1` when any mapped - `aws_model_id` is a proprietary GPT model (`openai.gpt-5*`). These are served only on the - `/openai/v1` path via the Responses API — distinct from the `v1` path other mantle models - use — so the rewriter must not emit a `/v1` base URL or a Chat Completions call for them. - See `$ASSESS_BASE/references/shared/openai-on-bedrock.md`. -- `Same model: true` when `bedrock_models[].model_change` is `false`. Signals the rewriter to - keep model parameters untouched and limit changes to the endpoint, credential, model id, and - (if the source used Chat Completions) the surface reshape. +- `Mantle base path: /openai/v1` for bare proprietary GPT targets (`openai.gpt-5*` + excluding gpt-oss, or `openai.gpt-6*`). Do not use `/v1` for Astra. +- `Mantle surface: chat_completions` when `resolved_api_path` is `mantle_openai_chat`; + otherwise `Mantle surface: responses` for the proprietary GPT Mantle path. Astra supports + both surfaces on Standard Mantle in Virginia and Oregon. Preserve the selected API; do not reshape Astra Chat solely + because it is a proprietary GPT model. Older GPT-5.x targets retain their dated API checks. +- `Same model: true` only when every mapping keeps the exact source model (after removing + the Bedrock provider/profile prefix for identity comparison; for a validated system-profile + ARN, first take its resource ID after `/`). A Pro-to-Astra upgrade is + `false` and still requires quality evaluation. This flag does not establish parameter parity; + Astra sampling, `n`, and hosted-state behavior require the selected API's evidence. + +When `rewrite_strategy == "runtime_openai"`, add `Rewrite strategy: runtime_openai`. +C5 must use the shared `Target API surface` and exact target profile at the runtime +`/openai/v1` endpoint; it must not use the Mantle client or a Converse rewrite. + +Rebuild these context lines from the same normalized path on every dispatch/resume. The existing +`assess_design_sha256` run-context check detects a changed model/API decision. If a bare +Astra id is paired with a runtime path, or a CRIS id with Mantle, stop and correct the plan; +do not switch endpoints or invent another profile. See the resolver's Case C and +`gcp-to-aws/references/shared/openai-on-bedrock.md`. ### C7 — Render summary diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/behavior-delta-detection.md b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/behavior-delta-detection.md index d82a787ca..bbba9276f 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/behavior-delta-detection.md +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/behavior-delta-detection.md @@ -11,17 +11,16 @@ The motivation is to prevent silent UX changes during migration. Example: OpenAI `same_model_family` means exact model identity/version, verified with `scripts/model_identity.py` for every source→validated-target pair. It does not mean the same provider. Anthropic version changes (including Opus 4.8 → 5.5) load all target checks in `references/anthropic-to-bedrock.md`; exact same-model moves still check the target API/path. Custom OpenAI-compatible providers (Together, Fireworks, etc.) remain outside these recipes — emit `behavior_deltas: []`. -**OpenAI → the same GPT model on Bedrock is also `same_model_family: true`.** When every source/target pair is the identical GPT model and the target is on `bedrock-mantle` (`openai.gpt-5.6-sol` / `-terra` / `-luna`, `openai.gpt-5.5`, `openai.gpt-5.4`), the model is unchanged, so the model-parameter deltas in `references/openai-to-bedrock.md` — temperature range, penalty parameters, stop-sequence limits — **do not apply**. Applying them would prompt the user to accept range changes that are not happening. - -That case is not delta-free, though: the **API surface** can change (Chat Completions → Responses) and reasoning models require reasoning items to be echoed back across turns. Read only the "Same-model (mantle) deltas" section of `references/openai-to-bedrock.md` for those, and skip the parameter-surface blocks. - -Decide from the resolved target model id, not from `source_provider`: - -| Source | Target model id | What to load | -| ------ | ---------------------------------------- | --------------------------------------------------------- | -| openai | Identical `openai.gpt-5*` model (mantle) | Same-model (mantle) deltas ONLY | -| openai | Claude / Nova / DeepSeek | Full parameter-surface deltas (cross-family) | -| openai | `openai.gpt-oss-*` | Full parameter-surface deltas — different model, Converse | +| Source | Resolved target | What to load | +| ------ | ---------------------------------------------------------------------- | --------------------------------------------------------------- | +| openai | Bare `openai.gpt-6-astra` or proprietary `openai.gpt-5*` (not gpt-oss) | Same-vendor GPT endpoint and API deltas | +| openai | Supported Astra `us.` / `global.` or GPT-5.6 CRIS id | Same-vendor GPT endpoint and API deltas, using the runtime path | +| openai | Claude / Nova / DeepSeek / gpt-oss | Cross-family parameter-surface deltas | + +The OpenAI reference's **Same-vendor GPT endpoint and API deltas** section is authoritative +for those targets. Astra Chat does not require a Responses reshape unless the plan selected +Responses. Sampling, penalties, `n`, and hosted state remain unverified until checked for the +selected model/API; do not infer unchanged parameters from unchanged model identity. ## Choose the right reference diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/references/openai-to-bedrock.md b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/references/openai-to-bedrock.md index f64148094..96ab4f4f7 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/references/openai-to-bedrock.md +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/behavior-delta-detection/references/openai-to-bedrock.md @@ -10,10 +10,20 @@ In the `detect_grep` recipes below, `` is the repository path supplied in --- -## Same-model (mantle) deltas +## Same-vendor GPT endpoint and API deltas **Read this section INSTEAD of the Claude parameter-surface sections below when the resolved target is a proprietary GPT model on `bedrock-mantle`** (`openai.gpt-5.6-sol` / `-terra` / `-luna`, `openai.gpt-5.5`, `openai.gpt-5.4`, `openai.gpt-6-astra`, `openai.gpt-6-sol`, `openai.gpt-6-luna`, `openai.gpt-6.1-sol`). Only an exact identity match from `scripts/model_identity.py` establishes that the model is unchanged. For a GPT version change, retain comparative evaluation and verify the selected GPT version's parameters; do not apply Claude ranges or claim identical behavior. The deltas here are about the API surface and the endpoint, not about model behavior. +Preserving the model avoids a vendor change but does not prove parameter parity across endpoints. +For Astra, sampling, penalties, `n`, and hosted-state behavior are unprobed: verify on the exact selected +API before preserving them. A Pro-to-Astra or other version upgrade is a model change and requires +quality evaluation even though the vendor is unchanged. + +Astra runtime supports OpenAI-compatible Responses / Chat Completions and Converse. Keep a supported +source API where possible; a Converse selection requires request/response mapping and capability checks, +not Claude-specific temperature or penalty rewrites. Use `bedrock-runtime.{region}.amazonaws.com/openai/v1` +for OpenAI-compatible runtime calls, CRIS ids, and runtime IAM permissions. Astra Standard mantle supports `us-east-1` and `us-west-2`. + ## chat-completions-to-responses - `resolution_kind`: `ux_choice` @@ -33,7 +43,8 @@ grep -rEn 'choices\[0\]\.(message|delta)' --include="*.py" --include="*.j ## reasoning-items-must-round-trip - `resolution_kind`: `impl_path` -- These models reason before responding. In multi-turn and tool-calling flows the model's output items — which may include reasoning items — must be appended to the next request's `input`. Dropping them degrades multi-step and tool-use quality without raising an error, so this fails silently. +- On Responses, preserve output items (including reasoning items) when replaying multi-turn or tool-calling + flows. On Chat Completions, preserve messages and tool-call ids instead; do not apply Responses item shapes. - Detection: any Responses-API call that builds the next turn's `input` from message text alone rather than appending `response.output`. ### detect_grep @@ -46,9 +57,10 @@ grep -rEn 'function_call_output|tool_call' --include="*.py" --include="*. ## endpoint-path-and-credential - `resolution_kind`: `impl_path` -- Base URL must be `https://bedrock-mantle.{region}.api.aws/openai/v1` — the `openai/v1` segment is required and differs from the `v1` path other mantle models use. A hardcoded `/v1` returns 404. +- For a mantle selection, base URL must be `https://bedrock-mantle.{region}.api.aws/openai/v1` — the `openai/v1` segment is required and differs from the `v1` path other mantle models use. A hardcoded `/v1` returns 404. - The API key must be a Bedrock API key or an auto-refreshing token provider, **not** an existing OpenAI key. A long-lived `OPENAI_API_KEY` read from the environment will fail authentication. -- IAM must grant `bedrock-mantle:*` actions; `bedrock:InvokeModel` does not authorize these models. +- Mantle IAM uses project-scoped inference actions and bearer-token authorization; `bedrock:InvokeModel` + does not authorize mantle calls. Runtime CRIS selections use runtime permissions instead. - Not user-visible — always `user_visible: false`, `resolution_kind: impl_path`. Apply without prompting. ## prompt-caching-availability @@ -56,6 +68,10 @@ grep -rEn 'function_call_output|tool_call' --include="*.py" --include="*. - `resolution_kind`: `impl_path` - Prompt caching is listed as supported on GPT-5.6 and GPT-6 only. Do not emit caching configuration for `openai.gpt-5.5` or `openai.gpt-5.4`. +- Astra lists implicit and explicit prompt caching on mantle Responses, with **30-minute cache-write** rates. + Do not inherit GPT-5.6's minimum-prefix, breakpoint, or cached-input quota-exemption rules. + Verify caching on the selected API; do not transfer mantle behavior to runtime or Chat. + --- ## Cross-family parameter-surface deltas diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/resolve-bedrock-model-id/resolve-bedrock-model-id.md b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/resolve-bedrock-model-id/resolve-bedrock-model-id.md index 2b4c71daa..f64dc9fef 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/resolve-bedrock-model-id/resolve-bedrock-model-id.md +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/references/helpers/resolve-bedrock-model-id/resolve-bedrock-model-id.md @@ -47,8 +47,8 @@ aws bedrock list-foundation-models \ **OpenAI proprietary GPT IDs.** -**Check this before Step 1.** The proprietary GPT models split into two cases (verified 2026-08-21; see -`$ASSESS_BASE/references/shared/openai-on-bedrock.md`): +**Check this before Step 1.** Route by family and endpoint (GPT-5 verified 2026-08-21; Astra regions verified 2026-09-30; see +`gcp-to-aws/references/shared/openai-on-bedrock.md`): **Case A — GPT-5.5 / GPT-5.4 (`openai.gpt-5.5`, `openai.gpt-5.4`): mantle-only, no inference profile.** The inference-profile path below cannot resolve them — `list-inference-profiles` never returns them, and Step 3's token @@ -108,7 +108,26 @@ follow Case A's catalog-lookup procedure when the plan targets the bare/mantle e `bedrock-runtime` (`migration_path: runtime_openai_cris` or an already-CRIS-prefixed id), they follow this Case's inference-profile procedure instead — same routing rule as GPT-5.6, just a different id family. -Non-`openai.gpt-5*` / non-`openai.gpt-6*` ids continue to Step 1 unchanged after checking the London exception above. +**Case C — GPT-6 Astra: `openai.gpt-6-astra` on mantle, or `us.openai.gpt-6-astra` / +`global.openai.gpt-6-astra` on runtime.** Preserve the plan's endpoint and residency choice: + +- A supplied system-profile ARN uses runtime. Validate its `arn:aws:bedrock:::inference-profile/` + form and apply the checks below to its profile ID. In Step 2 require the full ARN to match + `inferenceProfileArn` in the live listing, then return the original ARN unchanged. + Do not reduce it to a short ID, manufacture another ARN or accept an unverified application profile. +- Bare id → Standard mantle in `us-east-1` or `us-west-2`. Validate the exact id with the Case A catalog query. + A different region returns `blocked` with `reason: model_unresolvable`; offer Virginia, Oregon, or a supported + runtime CRIS path through the orchestrator. Do not silently add a prefix or change endpoints. +- `us.` / `global.` id → Step 1, then require an exact live profile match in the caller region. + Check the Astra runtime matrix in `gcp-to-aws/references/shared/openai-on-bedrock.md`; + a Global-only caller region does not satisfy US Geo residency. +- An `in.` / `eu.` prefix, version suffix, partial id, or conflicting endpoint/id pair → `blocked`. + Do not substitute GPT-5.6 or manufacture an Astra profile. + +Astra's bare id needs mantle project permissions; its CRIS ids need runtime foundation-model and +inference-profile permissions. Resolution does not verify account access or API behavior; run preflight. + +Other model ids continue to Step 1 unchanged. ### Step 1: List live inference profiles @@ -116,15 +135,15 @@ Non-`openai.gpt-5*` / non-`openai.gpt-6*` ids continue to Step 1 unchanged after aws bedrock list-inference-profiles \ --region \ when your context has an `AWS profile` line> \ - --query 'inferenceProfileSummaries[].[inferenceProfileId,inferenceProfileName]' \ + --query 'inferenceProfileSummaries[].[inferenceProfileId,inferenceProfileName,inferenceProfileArn]' \ --output json ``` -Parse the JSON. Each entry is a `[id, name]` pair. +Parse the JSON. Each entry is an `[id, name, arn]` triple; ranking still uses the ID and name. ### Step 2: Try exact match -If `plan_model_id` appears verbatim in the list, return it. No user prompt +If `plan_model_id` exactly matches a listed ID or ARN, return that original value. No user prompt needed. ### Step 3: Token-based ranking when no exact match @@ -199,7 +218,7 @@ stops on abort. ID will hit Step 0 (documented bare-ID route) or Step 2 (inference profile) and return immediately. - Steps 1–5 assume the target is a `bedrock-runtime` model reachable through an inference profile. Mantle-only ids (GPT-5.5/5.4, and GPT-5.6 when the plan - targets the mantle endpoint), and London in-region Opus 5.5, are handled entirely in Step 0; GPT-5.6 CRIS ids + targets the mantle endpoint), and London in-region Opus 5.5, are handled entirely in Step 0; GPT-5.6/GPT-6 CRIS ids flow through Steps 1–5 like any other inference profile. See `$ASSESS_BASE/references/shared/openai-on-bedrock.md` for the authoritative family split. diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/bedrock_pricing.py b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/bedrock_pricing.py index 0f8065742..87e005dae 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/bedrock_pricing.py +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/bedrock_pricing.py @@ -280,6 +280,11 @@ def display_name_guess(model_id: str) -> str: def lookup(region: str, model_id: str) -> dict: + profile_arn = re.fullmatch( + r"arn:aws:bedrock:[^:]+:[0-9]{12}:inference-profile/([^/]+)", model_id + ) + if profile_arn: + model_id = profile_arn.group(1) if "claude-opus-5" in model_id: return _opus55_pricing(region, model_id) # Curated static table is the primary source — the Pricing API keys models @@ -289,8 +294,11 @@ def lookup(region: str, model_id: str) -> dict: fb["note"] = ("static pricing table (verified 2026-10-05 against " "aws.amazon.com/bedrock/pricing, the Bedrock model cards' " "Geo/Global inference ID tables, and the vendored pricing cache)") + if model_id.endswith("openai.gpt-6-astra"): + fb["note"] += ("; Standard tier, <=272K input tokens. For >272K, " + "use 2x input / 1.5x output rates.") return fb - if is_mantle_gpt(model_id): + if is_mantle_gpt(model_id) or re.match(r"^[^.]+\.openai\.gpt-[56]", model_id): # Short-circuit: the PriceList API carries no rows for the proprietary GPT # models, so a live lookup would burn a round trip and still return nothing — # and a generic "unavailable" would read as "this model doesn't exist". diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/iam_policy.py b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/iam_policy.py index d3ee5f1e1..5b56d2fd2 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/iam_policy.py +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/iam_policy.py @@ -13,7 +13,9 @@ def is_inference_profile(model_id: str) -> bool: """True when the model ID uses a geo-prefix (cross-region inference profile).""" - return bool(_GEO_PREFIX.match(model_id)) + return bool(_GEO_PREFIX.match(model_id) or re.fullmatch( + r"arn:aws:bedrock:[^:]+:[0-9]{12}:inference-profile/[^/]+", model_id + )) def is_mantle_model(model_id: str, region: str = "") -> bool: @@ -49,6 +51,8 @@ def foundation_model_arn(model_id: str, region: str = "") -> str: def inference_profile_arn(model_id: str, region: str, account_id: str) -> str: """ARN for a cross-region inference profile.""" + if model_id.startswith("arn:"): + return model_id return f"arn:{_partition(region)}:bedrock:{region}:{account_id}:inference-profile/{model_id}" @@ -67,7 +71,7 @@ def is_gpt6_cris_model(model_id: str) -> bool: match — that path is covered by `is_mantle_model`, not this predicate.""" if not is_inference_profile(model_id): return False - base_id = _GEO_PREFIX.sub("", model_id).lower() + base_id = _GEO_PREFIX.sub("", model_id.rsplit("/", 1)[-1]).lower() return base_id.startswith("openai.gpt-6") @@ -94,7 +98,7 @@ def generate_policy(model_ids: list[str], region: str, account_id: str) -> dict: for mid in sorted(set(runtime_ids)): if is_inference_profile(mid): resources.append(inference_profile_arn(mid, region, account_id)) - base_id = _GEO_PREFIX.sub("", mid) + base_id = _GEO_PREFIX.sub("", mid.rsplit("/", 1)[-1]) resources.append(foundation_model_arn(base_id, region)) # A GPT-6 CRIS id's model card "Call the model" section grants # InvokeModel on the inference-profile ARN AND the default project diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/image_input.py b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/image_input.py index 05fe5bab6..e8ec6567e 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/image_input.py +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/image_input.py @@ -1,4 +1,4 @@ -"""Build image-bearing request payloads for Converse and the Responses API. +"""Build image-bearing request payloads for Converse, Responses and Chat Completions. Two mistakes are easy to make by hand and both produce a confusing failure mode where the vision smoke test passes (it hardcodes a known-good jpeg) while every @@ -13,7 +13,7 @@ declares `ResponseInputImageParam.detail` as `Required`, so omitting it is a contract violation even though a TypedDict will not catch it at runtime. -Callers should use `converse_message` / `responses_message` rather than assembling +Callers should use `converse_message` / `responses_message` / `chat_message` rather than assembling these dicts inline, so the shape is decided in one tested place. """ from pathlib import Path @@ -88,3 +88,15 @@ def responses_message(prompt: str, image_path: str | None = None, "image_url": data_url(image_path, raw), "detail": detail}) return {"role": "user", "content": content} + + +def chat_message(prompt: str, image_path: str | None = None, + raw: bytes | None = None) -> dict: + """One Chat Completions message; retain the same image MIME normalization.""" + content: list[dict] = [{"type": "text", "text": prompt}] + if image_path is not None: + if raw is None: + raise ValueError("raw image bytes are required when image_path is given") + content.append({"type": "image_url", "image_url": { + "url": data_url(image_path, raw)}}) + return {"role": "user", "content": content} diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/preflight_bedrock.py b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/preflight_bedrock.py index ca8120746..2a4c031bb 100644 --- a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/preflight_bedrock.py +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/preflight_bedrock.py @@ -6,12 +6,13 @@ Always prints JSON — including for missing credentials / bad region — so the caller can parse the verdict instead of a traceback. On failure the top level carries `reason`/`detail`/`failing_models` lifted from the first failing model. -Chat models are probed via Converse; embedding models (which don't speak +Chat models default to Converse; --runtime-api preserves an explicit proprietary +GPT Chat/Responses selection. Embedding models (which don't speak Converse) via InvokeModel with their family's request body. Bare OpenAI proprietary GPT ids are mantle-served and probed via the OpenAI Responses API; -GPT-5.6 CRIS profile ids (us./in./global. prefixed) are bedrock-runtime targets -and take the normal Converse probe. -The 1-token probe costs a fraction of a cent (noted in output). +GPT-6 Astra (us./global.) and GPT-5.6 (us./in./global.) CRIS ids are bedrock-runtime targets +take the selected runtime API probe. +Probe output budgets and cost depend on the selected API. """ import argparse, json, sys @@ -69,12 +70,77 @@ def is_mantle_model(model_id: str) -> bool: return (mid.startswith("openai.gpt-5") or mid.startswith("openai.gpt-6")) and "oss" not in mid + +def normalize_api_path(plan_path: str | None, model_ids: list[str], + runtime_api: str | None = None) -> str: + """Resolve a missing legacy path once for evaluator and rewriter dispatch. + + Explicit plans retain their chosen API. Legacy bare proprietary GPT targets + use Mantle Responses; runtime targets retain the existing Converse default. + A mixed legacy target set needs an explicit plan rather than an unsafe guess. + """ + if plan_path == "mantle" and model_ids and all(is_mantle_model(mid) for mid in model_ids): + return "mantle_openai_responses" + if plan_path == "runtime_openai_cris": + paths = {"converse": plan_path, "responses": "runtime_openai_responses", + "chat_completions": "runtime_openai_chat"} + if runtime_api is not None and runtime_api not in paths: + raise ValueError("Unknown runtime API; resolve the saved migration plan") + return paths[runtime_api or "converse"] + if plan_path: + return plan_path + if not model_ids: + raise ValueError("No validated target model IDs to resolve an API path") + mantle = [is_mantle_model(mid) for mid in model_ids] + if all(mantle): + return "mantle_openai_responses" + if any(mantle): + raise ValueError("Mixed Mantle and runtime targets require an explicit migration plan") + return "converse" + + +def create_openai_client(model_id: str, region: str, max_retries: int = 2): + """Build the selected endpoint's client without changing the invocation ID.""" + from aws_bedrock_token_generator import provide_token + if is_mantle_model(model_id): + from openai import BedrockOpenAI + return BedrockOpenAI( + aws_region=region, bedrock_token_provider=lambda: provide_token(region=region), + max_retries=max_retries, + ) + from openai import OpenAI + return OpenAI( + base_url=f"https://bedrock-runtime.{region}.amazonaws.com/openai/v1", + api_key=lambda: provide_token(region=region), max_retries=max_retries, + ) + + +def probe_runtime_openai_model(model_id: str, region: str, api_surface: str) -> dict: + """Probe the selected runtime Chat/Responses API, not a Converse substitute.""" + try: + client = create_openai_client(model_id, region, max_retries=0) + if api_surface == "chat_completions": + client.chat.completions.create( + model=model_id, messages=[{"role": "user", "content": "ping"}]) + else: + client.responses.create(model=model_id, input="ping", max_output_tokens=16) + return {"ok": True, "reason": "ok", "detail": f"Runtime {api_surface} API authorized."} + except ImportError as e: + return {"ok": False, "reason": "openai_deps_missing", + "detail": f"Cannot verify the runtime OpenAI API: {e}. Re-sync scripts/pyproject.toml."} + except Exception as e: + code = {401: "UnrecognizedClientException", 403: "AccessDeniedException", + 404: "ResourceNotFoundException", 429: "ThrottlingException"}.get( + getattr(e, "status_code", None), "unknown") + return classify_invoke_error(code, str(e)) + + def classify_mantle_error(status: int | None, message: str) -> dict: """Pure: map an OpenAI-SDK HTTP status from the mantle endpoint to a verdict. Mirrors classify_invoke_error's contract but for the mantle surface, where the remedies differ: IAM needs bedrock-mantle:* actions (not bedrock:InvokeModel), - and mantle itself has no cross-region form — though for GPT-5.6 a + and mantle itself has no cross-region form — though for Astra / GPT-5.6 a bedrock-runtime CRIS id can cover the region instead.""" if status in (401, 403): lowered = message.lower() @@ -92,7 +158,9 @@ def classify_mantle_error(status: int | None, message: str) -> dict: return {"ok": False, "reason": "model_unavailable", "detail": f"Model not available at this mantle endpoint/region — {message}. " f"Mantle is in-region only; for GPT-5.6 try a bedrock-runtime CRIS " - f"id (us./in./global. prefixed) instead, for GPT-5.5/5.4 switch to " + f"id (us./in./global. prefixed); for GPT-6 Astra use us-east-1/us-west-2 " + f"mantle or a supported us./global. runtime CRIS path. " + f"For GPT-5.5/5.4 switch to " f"a supported region or a different model."} if status == 429: # Reaching a token-per-minute ceiling still proves we are authorized. @@ -244,6 +312,8 @@ def main(argv=None) -> int: ap.add_argument("--region", required=True) ap.add_argument("--models", required=True, help="comma-separated model ids") ap.add_argument("--dataset-size", type=int, default=0) + ap.add_argument("--runtime-api", choices=["converse", "responses", "chat_completions"], + default="converse") args = ap.parse_args(argv) model_ids = [m.strip() for m in args.models.split(",") if m.strip()] @@ -278,11 +348,25 @@ def main(argv=None) -> int: "Mantle quotas are per-model input/output tokens per minute; there is no RPM " "quota. Pace on token throughput, and note that prompt-cached input tokens are " "exempt from the input-TPM quota.") + if model_id.lower() == "openai.gpt-6-astra": + verdict["quota_note"] = ( + f"Verify Astra's mantle quotas in the target account and {args.region}. " + "Do not assume GPT-5.6's cached-input quota exemption or RPM rules " + "apply; Astra's documented 10x output burndown is runtime-specific.") else: - verdict = probe_model(client, model_id) + if (args.runtime_api != "converse" and ".openai.gpt-" in model_id + and "gpt-oss" not in model_id): + verdict = probe_runtime_openai_model(model_id, args.region, args.runtime_api) + else: + verdict = probe_model(client, model_id) rpm = quota_rpm(quotas, model_id) verdict["model_id"] = model_id verdict["rpm_quota"] = rpm + if model_id.rsplit("/", 1)[-1] in ("us.openai.gpt-6-astra", "global.openai.gpt-6-astra"): + verdict["quota_note"] = ( + "Astra runtime TPM usage = input tokens + 10 * output tokens. " + "Check the selected CRIS profile's token quota; an RPM check alone " + "does not establish throughput capacity.") if rpm is not None and args.dataset_size > rpm: verdict["quota_warning"] = ( f"Dataset ({args.dataset_size}) exceeds ~{rpm} RPM quota — " @@ -291,7 +375,7 @@ def main(argv=None) -> int: results.append(verdict) out = {"ok": all_ok, "region": args.region, - "probe_cost_note": "1-token InvokeModel probe per model (~$0.00001 each)", + "probe_cost_note": "Minimal model/API permission probes; output budgets and cost vary by API.", "models": results} out.update(aggregate_failure(results)) print(json.dumps(out, indent=2)) diff --git a/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/test_astra_execution_contract.py b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/test_astra_execution_contract.py new file mode 100644 index 000000000..244677d1b --- /dev/null +++ b/plugins/aws-startup-advisor/skills/llm-to-bedrock/scripts/test_astra_execution_contract.py @@ -0,0 +1,546 @@ +"""Execute the actual evaluator snippets and validate Astra handoff contracts.""" +import json +import re +import sys +import textwrap +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import Mock + +import pytest + +import image_input +import preflight_bedrock +import bedrock_pricing +import iam_policy + +PLUGIN = Path(__file__).resolve().parents[3] +EVALUATOR = PLUGIN / "agents/llm2bedrock-prompt-evaluator.md" + + +def snippet(section, end): + text = EVALUATOR.read_text().split(section, 1)[1].split(end, 1)[0] + return re.search(r"python - <<'PY'\n(.*?)\nPY\n", text, re.S).group(1) + + +def clients(monkeypatch): + responses = Mock(return_value=SimpleNamespace(output_text="answer")) + chat = Mock(return_value=SimpleNamespace(choices=[ + SimpleNamespace(message=SimpleNamespace(content="answer"))])) + runtime = Mock(return_value={"output": {"message": {"content": [{"text": "answer"}]}}}) + client = SimpleNamespace(responses=SimpleNamespace(create=responses), + chat=SimpleNamespace(completions=SimpleNamespace(create=chat))) + monkeypatch.setitem(sys.modules, "openai", SimpleNamespace( + BedrockOpenAI=lambda **k: client, OpenAI=lambda **k: client)) + monkeypatch.setitem(sys.modules, "aws_bedrock_token_generator", + SimpleNamespace(provide_token=lambda **k: "test-token")) + import boto3 + monkeypatch.setattr(boto3, "client", lambda *a, **k: SimpleNamespace(converse=runtime)) + return responses, chat, runtime + + +@pytest.mark.parametrize("surface", ["responses", "chat_completions"]) +@pytest.mark.parametrize("mid", ["openai.gpt-6-astra", "us.openai.gpt-6-astra", + "global.openai.gpt-6-astra"]) +def test_astra_connectivity_uses_selected_openai_api(surface, mid, monkeypatch): + responses, chat, runtime = clients(monkeypatch) + code = snippet("# 6a. OpenAI-compatible connectivity", "# 7. Load golden") + code = code.replace("", mid).replace("", surface) + monkeypatch.setenv("AWS_REGION", "us-west-2") + exec(compile(code, str(EVALUATOR), "exec"), {}) # nosec B102 - committed test fixture + selected = chat if surface == "chat_completions" else responses + assert selected.call_args.kwargs["model"] == mid + assert selected.call_count == 1 + runtime.assert_not_called() + (responses if surface == "chat_completions" else chat).assert_not_called() + + +@pytest.mark.parametrize("mid,surface,endpoint", [ + ("openai.gpt-6-astra", "responses", "responses"), + ("openai.gpt-6-astra", "chat_completions", "chat"), + ("us.openai.gpt-6-astra", "converse", "runtime"), + ("global.openai.gpt-6-astra", "converse", "runtime"), + ("us.openai.gpt-6-astra", "responses", "responses"), + ("global.openai.gpt-6-astra", "chat_completions", "chat"), + ("openai.gpt-5.6-sol", "responses", "responses"), + ("openai.gpt-6-astra", "legacy_mantle", "responses"), + ("openai.gpt-5.6-sol", "legacy_mantle", "responses"), + ("us.openai.gpt-5.6-sol", "converse", "runtime"), +]) +def test_golden_loop_preserves_endpoint_images_and_resume(mid, surface, endpoint, monkeypatch, tmp_path): + responses, chat, runtime = clients(monkeypatch) + if surface == "legacy_mantle": + path = preflight_bedrock.normalize_api_path("mantle", [mid]) + surface = ("chat_completions" if path.endswith("_openai_chat") else + "responses" if path.endswith("_openai_responses") else "converse") + calls = {"responses": responses, "chat": chat, "runtime": runtime} + data = tmp_path / ".saws-migrate/golden-dataset" + out = tmp_path / ".saws-migrate/eval-results" + data.mkdir(parents=True) + out.mkdir(parents=True) + image = tmp_path / "case.jpg" + image.write_bytes(b"image-bytes") + (data / "prompts.jsonl").write_text(json.dumps({ + "id": "case-1", "user_prompt": "describe", "system_prompt": "be concise", + "image_path": str(image), "assistant_response": "source answer"}) + "\n") + code = snippet("# 10. Run golden prompt evaluation", "# 11. Score") + code = (code.replace("", mid).replace("", surface) + .replace("", str(Path(__file__).parent)).replace("", str(tmp_path))) + monkeypatch.setenv("AWS_REGION", "us-west-2") + exec(compile(code, str(EVALUATOR), "exec"), {}) # nosec B102 - committed test fixture + result = json.loads((out / "raw_results.jsonl").read_text()) + assert result["status"] == "success" and result["bedrock_response"] == "answer" + call = calls[endpoint].call_args.kwargs + assert call.get("model", call.get("modelId")) == mid + if endpoint == "responses": + assert call["input"][0]["content"][1]["image_url"].startswith("data:image/jpeg;base64,") + assert call["instructions"] == "be concise" + elif endpoint == "chat": + assert call["messages"][0] == {"role": "system", "content": "be concise"} + assert call["messages"][1]["content"][1]["image_url"]["url"].startswith("data:image/jpeg;base64,") + else: + assert call["messages"][0]["content"][0]["image"]["format"] == "jpeg" + for name, spy in calls.items(): + assert spy.call_count == (1 if name == endpoint else 0) + exec(compile(code, str(EVALUATOR), "exec"), {}) # nosec B102 - exercise actual resume guard + assert calls[endpoint].call_count == 1 + + +@pytest.mark.parametrize("mid", ["openai.gpt-6-astra", "global.openai.gpt-6-astra"]) +@pytest.mark.parametrize("surface", ["responses", "chat_completions"]) +def test_golden_openai_throttle_returns_partial_without_scoring_error(mid, surface, monkeypatch, tmp_path): + responses, chat, runtime = clients(monkeypatch) + selected = chat if surface == "chat_completions" else responses + error = RuntimeError("throttled") + error.status_code = 429 + selected.side_effect = error + import time + monkeypatch.setattr(time, "sleep", lambda seconds: None) + data = tmp_path / ".saws-migrate/golden-dataset" + out = tmp_path / ".saws-migrate/eval-results" + data.mkdir(parents=True) + out.mkdir(parents=True) + (data / "prompts.jsonl").write_text(json.dumps({"id": "p", "user_prompt": "ping"}) + "\n") + code = snippet("# 10. Run golden prompt evaluation", "# 11. Score") + code = (code.replace("", mid) + .replace("", surface) + .replace("", str(Path(__file__).parent)).replace("", str(tmp_path))) + ns = {} + exec(compile(code, str(EVALUATOR), "exec"), ns) # nosec B102 - committed test fixture + assert ns["throttled_out"] is True + assert selected.call_count == 6 + assert (out / "raw_results.jsonl").read_text() == "" + runtime.assert_not_called() + + +@pytest.mark.parametrize("surface,path", [ + (None, "runtime_openai_cris"), ("converse", "runtime_openai_cris"), + ("responses", "runtime_openai_responses"), ("chat_completions", "runtime_openai_chat"), +]) +def test_explicit_runtime_api_survives_normalization(surface, path): + assert preflight_bedrock.normalize_api_path( + "runtime_openai_cris", ["global.openai.gpt-6-astra"], surface) == path + with pytest.raises(ValueError, match="Unknown runtime API"): + preflight_bedrock.normalize_api_path("runtime_openai_cris", [], "unknown") + + +@pytest.mark.parametrize("surface", ["responses", "chat_completions"]) +def test_runtime_preflight_uses_selected_api(surface, monkeypatch, capsys): + responses, chat, runtime = clients(monkeypatch) + monkeypatch.setattr(preflight_bedrock, "fetch_bedrock_quotas", lambda region: []) + mid = "global.openai.gpt-6-astra" + assert preflight_bedrock.main( + ["--region", "us-west-2", "--models", mid, "--runtime-api", surface]) == 0 + selected = chat if surface == "chat_completions" else responses + assert selected.call_args.kwargs["model"] == mid + runtime.assert_not_called() + assert json.loads(capsys.readouterr().out)["ok"] + + +@pytest.mark.parametrize("surface", ["responses", "chat_completions"]) +@pytest.mark.parametrize("mid", ["openai.gpt-6-astra", "global.openai.gpt-6-astra"]) +def test_vision_smoke_preserves_openai_endpoint_and_surface(surface, mid, monkeypatch): + import io + import urllib.request + responses, chat, runtime = clients(monkeypatch) + monkeypatch.setattr(urllib.request, "urlopen", lambda *a, **k: io.BytesIO(b"test-image")) + code = snippet("# 9.5a OpenAI-compatible vision", "# 10. Run golden") + code = code.replace("", mid).replace("", surface) + exec(compile(code, str(EVALUATOR), "exec"), {}) # nosec B102 - committed smoke snippet + selected = chat if surface == "chat_completions" else responses + call = selected.call_args.kwargs + assert call["model"] == mid + if surface == "responses": + assert call["input"][0]["content"][1]["image_url"].startswith("data:image/jpeg;base64,") + else: + assert call["messages"][0]["content"][1]["image_url"]["url"].startswith("data:image/jpeg;base64,") + runtime.assert_not_called() + + +@pytest.mark.parametrize("surface,endpoint", [ + ("responses", "responses"), ("chat_completions", "chat/completions"), +]) +def test_runtime_openai_client_uses_real_sdk_wire_contract(surface, endpoint, monkeypatch): + import httpx + import openai + import aws_bedrock_token_generator + requests = [] + + def transport(request): + requests.append(request) + if surface == "chat_completions": + result = {"id": "chat-fixture", "object": "chat.completion", "created": 0, + "model": "global.openai.gpt-6-astra", "choices": [{ + "index": 0, "finish_reason": "stop", + "message": {"role": "assistant", "content": "answer"}}]} + else: + result = {"id": "response-fixture", "object": "response", "created_at": 0, + "model": "global.openai.gpt-6-astra", "status": "completed", "output": [{ + "id": "message-fixture", "type": "message", "role": "assistant", + "status": "completed", "content": [{ + "type": "output_text", "text": "answer", "annotations": []}]}]} + return httpx.Response(200, json=result) + + real_client = openai.OpenAI + monkeypatch.setattr(aws_bedrock_token_generator, "provide_token", lambda **k: "fixture-token") + with httpx.Client(transport=httpx.MockTransport(transport)) as http_client: + monkeypatch.setattr(openai, "OpenAI", + lambda **kwargs: real_client(http_client=http_client, **kwargs)) + assert preflight_bedrock.probe_runtime_openai_model( + "global.openai.gpt-6-astra", "us-west-2", surface)["ok"] + assert len(requests) == 1 + assert str(requests[0].url) == f"https://bedrock-runtime.us-west-2.amazonaws.com/openai/v1/{endpoint}" + assert requests[0].headers["Authorization"] == "Bearer fixture-token" + body = json.loads(requests[0].content) + assert body["model"] == "global.openai.gpt-6-astra" + assert ("messages" in body) is (surface == "chat_completions") + + +def test_dispatch_and_rewriter_preserve_runtime_openai_strategy(): + skill = (PLUGIN / "skills/llm-to-bedrock/SKILL.md").read_text() + rewriter = (PLUGIN / "agents/llm2bedrock-code-rewriter.md").read_text() + assert "validated_target_model_ids, plan_runtime_api" in skill + assert 'set `rewrite_strategy = "runtime_openai"`' in skill + assert "Target API surface:" in skill + lane = rewriter.split("### Runtime OpenAI lane", 1)[1].split("### Mantle express lane", 1)[0] + assert "bedrock-runtime.{region}.amazonaws.com/openai/v1" in lane + assert "chat.completions.create" in lane and "responses.create" in lane + assert "OpenAI-to-boto3 dependency replacement" in lane + + +def test_chat_image_helper_validates_missing_bytes(): + with pytest.raises(ValueError, match="raw image bytes"): + image_input.chat_message("describe", "case.jpg") + assert image_input.chat_message("text")["content"] == [{"type": "text", "text": "text"}] + + +@pytest.mark.parametrize("source,target,expected", [(60, 60, 0), (60, 66, 10), (120, 60, -50), (0, 60, None)]) +@pytest.mark.parametrize("skill", ["gcp-to-aws", "azure-to-aws"]) +def test_roi_executes_rate_derived_formula(source, target, expected, skill): + text = (PLUGIN / "skills" / skill / "references/phases/estimate/estimate-ai.md").read_text() + roi = text.split("## Part 5: ROI Analysis", 1)[1].split("## Part 6:", 1)[0] + code = re.search(r"```python\n(.*?)\n```", roi, re.S).group(1) + ns = {"source_monthly": source, "bedrock_monthly": target} + exec(compile(code, "ROI formula", "exec"), ns) # nosec B102 - committed formula + assert ns["percentage_difference"] == expected + assert ns["monthly_difference"] == target - source + assert "mark the source comparison unavailable" in roi + assert "projected cost is **about 10% higher**" not in roi + + +def test_api_handoff_paths_and_real_evaluator_classifier_are_consistent(): + helper = (PLUGIN / "skills/llm-to-bedrock/references/helpers/behavior-delta-detection/behavior-delta-detection.md").read_text() + reference = (PLUGIN / "skills/llm-to-bedrock/references/helpers/behavior-delta-detection/references/openai-to-bedrock.md").read_text() + assert "Same-vendor GPT endpoint and API deltas" in helper + assert "## Same-vendor GPT endpoint and API deltas" in reference + for rel in ["skills/llm-to-bedrock/SKILL.md", "skills/gcp-to-aws/references/phases/design/design-ai.md", + "skills/gcp-to-aws/references/phases/generate/generate-ai.md", + "skills/gcp-to-aws/references/vendored/ai/ai-openai-to-bedrock.md"]: + assert "mantle_openai_chat" in (PLUGIN / rel).read_text(), rel + assert preflight_bedrock.is_mantle_model("openai.gpt-6-astra") + assert not preflight_bedrock.is_mantle_model("global.openai.gpt-6-astra") + assert "from preflight_bedrock import is_mantle_model" in EVALUATOR.read_text() + + +@pytest.mark.parametrize("path,ids,expected", [ + (None, ["openai.gpt-6-astra"], "mantle_openai_responses"), + ("", ["openai.gpt-5.6-sol"], "mantle_openai_responses"), + (None, ["us.openai.gpt-6-astra"], "converse"), + (None, ["global.openai.gpt-6-astra"], "converse"), + ("mantle_openai_chat", ["openai.gpt-6-astra"], "mantle_openai_chat"), + ("mantle_openai_responses", ["openai.gpt-6-astra"], "mantle_openai_responses"), + ("runtime_openai_cris", ["us.openai.gpt-6-astra"], "runtime_openai_cris"), + ("mantle_messages", ["anthropic.claude-sonnet-5"], "mantle_messages"), +]) +def test_dispatch_and_resume_share_api_default(path, ids, expected): + selected = preflight_bedrock.normalize_api_path(path, ids) + assert selected == expected + assert preflight_bedrock.normalize_api_path(path, ids) == selected + + +def test_mixed_or_empty_legacy_targets_require_resolution(): + with pytest.raises(ValueError, match="Mixed Mantle and runtime"): + preflight_bedrock.normalize_api_path(None, ["openai.gpt-6-astra", "amazon.nova-lite-v1:0"]) + with pytest.raises(ValueError, match="No validated target"): + preflight_bedrock.normalize_api_path(None, []) + + +def test_c3_and_c5_use_shared_normalization_instructions(): + skill = (PLUGIN / "skills/llm-to-bedrock/SKILL.md").read_text() + before_dispatch = skill.split("### C0", 1)[0] + gate = skill.split("**Gate (a.5)", 1)[1].split("**Gate (b)", 1)[0] + assert "normalize_api_path" in before_dispatch + assert "Target API path: 272K) workload the gap is much w | ------------------------------------------ | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | | OpenAI SDK (direct) | Same model on mantle Responses API | Base URL + credential + model ID change; see `shared/openai-on-bedrock.md` for the `/openai/v1` path | | Responses API | Mantle Responses API | Closest to a drop-in; still verify the path and model ID | -| Chat Completions | Reshape to Responses | Unverified for GPT-5.x — probe before committing | +| Chat Completions | Preserve supported Chat when selected | Reshape only for a selected API change; verify the model and endpoint | | Function calling | Supported on GPT-5.x via mantle; Claude tools via Converse | Same-model keeps tool semantics identical | | Reasoning effort | `reasoning={"effort": ...}` on mantle | `none` / `low` / `medium` / `high` / `xhigh` / `max` | | Prompt caching | GPT-5.6 only (90% off cached input) | Not listed for GPT-5.5 / 5.4; Claude has its own caching | diff --git a/plugins/aws-startup-advisor/skills/shared/ai/bedrock-quotas.md b/plugins/aws-startup-advisor/skills/shared/ai/bedrock-quotas.md index 5d6916bc3..9ca28857e 100644 --- a/plugins/aws-startup-advisor/skills/shared/ai/bedrock-quotas.md +++ b/plugins/aws-startup-advisor/skills/shared/ai/bedrock-quotas.md @@ -16,10 +16,14 @@ OpenAI and Gemini APIs have high default rate limits for paying customers (OpenA --- -## The 5× Burndown Trap +## Output-Token Burndown Depends on the Model and Endpoint Claude models (3.7+) on Bedrock consume quota at **5× the rate for output tokens**. This is the single most common surprise for teams migrating from OpenAI (which has no output multiplier). +**GPT-6 Astra on `bedrock-runtime` uses 10× output-token burndown**: one output token consumes ten quota +tokens. This is runtime-specific; do not apply it to mantle or infer Astra mantle quotas from GPT-5.6. +Source: [Astra model card, verified 2026-09-16](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-6-astra.html). + **Effective TPM by workload type** (at 200K TPM quota on Claude): | Workload Type | Input:Output Ratio | Effective TPM | vs OpenAI 2M TPM | @@ -30,6 +34,9 @@ Claude models (3.7+) on Bedrock consume quota at **5× the rate for output token Formula: `Effective TPM = TPM_quota / (input_ratio + output_ratio × 5)` +For Astra runtime, replace 5 with 10. A 60:40 workload with a hypothetical 200K TPM quota supports +about 43.5K actual tokens/minute: `200K / (0.6 + 0.4 × 10)`. Check the account's actual quota. + Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Output-heavy workloads that hit Claude quota limits may benefit from routing to these models for applicable tasks. --- @@ -38,12 +45,14 @@ Nova, Llama, DeepSeek, and Mistral models have **1× burndown** (no multiplier). Apply during Design phase after model selection. Uses `ai_token_volume` from `preferences.json` and the selected model family. -| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | -| ------------------------- | ---------------------------------- | ------------ | ----------------------------------------------------------------------------- | -| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | -| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | -| `"medium"` | Nova / Llama / other (1× burndown) | `"low"` | No action | -| `"low"` | Any | `"low"` | No action | +| `ai_token_volume` | Model Family | `quota_risk` | Surface to User | +| ------------------------- | -------------------------------------- | ------------ | ------------------------------------------------------------------------------- | +| `"high"` or `"very_high"` | Any | `"high"` | "Request Bedrock quota increase before migration (allow 1–5 business days)" | +| `"medium"` | Claude (5× burndown) | `"medium"` | "Monitor TPM usage during parallel run; quota increase may be needed at peak" | +| `"medium"` | Astra runtime (10× output burndown) | `"medium"` | "Calculate weighted peak TPM and verify the selected CRIS quota before cutover" | +| `"medium"` | Verified 1× models (e.g. Nova / Llama) | `"low"` | No action | +| `"medium"` | Unverified model/endpoint quota | `"medium"` | "Verify quota dimensions before estimating throughput" | +| `"low"` | Any | `"low"` | No action | Include `quota_risk` in `aws-design-ai.json` → `ai_architecture`. @@ -99,10 +108,14 @@ Include this guidance in migration artifacts when `quota_risk` ≥ `"medium"`. U **How to calculate the target value:** ``` -Target TPM = (peak_tokens_per_minute_on_source_provider) × burndown_multiplier +Target TPM = peak_input_tokens_per_minute + peak_output_tokens_per_minute × output_burndown_multiplier ``` -Where `burndown_multiplier` = 5 for Claude models (because output tokens consume 5× quota), 1 for all others. If the user doesn't know their peak TPM, estimate from daily volume: +Use `output_burndown_multiplier` = 10 for Astra runtime, 5 for the Claude runtime models above, +and 1 only where verified for the selected model/endpoint. The multiplier applies to output tokens, +not the total source TPM. For separate mantle input/output quotas, size each quota independently. +If the user doesn't know their peak TPM, estimate total volume below and split it using the measured +input:output ratio: ``` Estimated peak TPM = (daily_tokens / active_hours / 60) × 3 (3× for peak headroom) diff --git a/plugins/aws-startup-advisor/tests/test_generate_mantle_dispatch.py b/plugins/aws-startup-advisor/tests/test_generate_mantle_dispatch.py index d36d44079..92722d8fd 100644 --- a/plugins/aws-startup-advisor/tests/test_generate_mantle_dispatch.py +++ b/plugins/aws-startup-advisor/tests/test_generate_mantle_dispatch.py @@ -72,7 +72,7 @@ def test_step2_harness_import_conditional_on_mantle_path() -> None: ) # The import must now be conditional, not unconditional: the Mantle path has # its own no-adapter client-setup branch. - assert "Mantle path" in harness and "do NOT import `provider_adapter`" in harness, ( + assert "Mantle / runtime CRIS" in harness and "do not import `provider_adapter`" in harness, ( "Step 2 does not scope the provider_adapter import away from the Mantle " "path (finding 2a) — the Mantle path emits no adapter module" ) @@ -96,32 +96,19 @@ def test_step3_mantle_iam_and_test_api_are_conditional() -> None: "Step 3's Mantle branch does not grant bedrock-mantle:CreateInference " "(finding 2c)" ) - assert "AmazonBedrockMantleInferenceAccess" in step3, ( - "Step 3's Mantle branch does not name the AmazonBedrockMantleInferenceAccess " - "managed policy (finding 2c)" - ) + assert "bedrock-mantle:CallWithBearerToken" in step3 + assert "project/*" in step3 # Non-Mantle paths keep the runtime action. assert "bedrock:InvokeModel" in step3, ( "Step 3 dropped bedrock:InvokeModel for the non-Mantle paths, which is " "wrong — those paths still need it (finding 2c)" ) - # The IAM and verification choices must be path-conditional. - assert step3.count("conditional on the migration path") >= 2, ( - "Step 3's IAM and verification are not both scoped as conditional on " - "the migration path (finding 2c)" - ) - # Mantle verification uses Responses/Chat Completions, not converse. - assert "client.responses.create" in step3, ( - "Step 3's Mantle verification does not use the Responses API (finding 2c)" - ) - assert "Do NOT use a `converse` call on this path" in step3, ( - "Step 3's Mantle verification does not forbid the converse call (finding 2c)" - ) - # Non-Mantle verification keeps converse. - assert "converse" in step3, ( - "Step 3 dropped the converse verification for the non-Mantle paths " - "(finding 2c)" - ) + # Setup invokes the same selected client as the comparison harness. + assert "Match inference permissions to the selected endpoint" in step3 + assert "test_comparison.py --target-only --quick" in step3 + assert "same selected API, model ID and region" in step3 + assert "A Mantle selection must never run a Converse probe" in step3 + assert "runtime selection must preserve its validated profile ID" in step3 def test_azure_generate_mantle_branch_still_correct() -> None: diff --git a/plugins/aws-startup-advisor/tests/test_gpt6_round4_findings.py b/plugins/aws-startup-advisor/tests/test_gpt6_round4_findings.py index 36990aae8..716ddda3c 100644 --- a/plugins/aws-startup-advisor/tests/test_gpt6_round4_findings.py +++ b/plugins/aws-startup-advisor/tests/test_gpt6_round4_findings.py @@ -111,14 +111,16 @@ def test_gcp_generate_ai_routes_cris_through_bedrock_runtime_not_mantle() -> Non assert "not** `bedrock-mantle:*`" in normalized or "not `bedrock-mantle:*`" in normalized.lower() -def test_gcp_generate_artifacts_ai_step1m_emits_provider_adapter_stub() -> None: +def test_gcp_step1m_and_harness_agree_on_adapter_free_paths() -> None: text = GCP_GENERATE_ARTIFACTS_AI_MD.read_text(encoding="utf-8") - assert "provider_adapter.py" in text and "stub" in text.lower(), ( - "generate-artifacts-ai.md Step 1M must emit a provider_adapter stub for mantle/CRIS " - "paths so Step 2's unconditional `from provider_adapter import get_provider` import " - "keeps working" - ) - assert "get_provider" in text + step1m = text.split("## Step 1M:", 1)[1].split("## Step 1:", 1)[0] + harness = text.split("## Step 2:", 1)[1].split("## Step 3:", 1)[0] + assert "do not emit or import a provider-adapter stub" in step1m + assert "Mantle / runtime CRIS" in harness + assert "do not import `provider_adapter`" in harness + assert "from provider_adapter import get_provider" in harness + assert "client.responses.create" in harness + assert "client.chat.completions.create" in harness # --- Finding 0.3: canonical guide + vendored guardrails name full GPT-6 set --- @@ -157,7 +159,8 @@ def test_both_estimators_carve_out_global_cris_parity() -> None: assert "model_change: false" in text assert "Global CRIS" in text, f"{f}: no Global CRIS carve-out near model_change:false rule" assert "parity" in normalized.lower() - assert "UNLESS" in text, f"{f}: 10% premium must not be stated as unconditional" + assert "computed Part 5 result, not a fixed premium" in text + assert "source comparison is unavailable" in text # --- Finding 1 (prose): bare GPT-6 recognized by resolver / agents ---