Repository navigation
Conversation
The model catalog now owns activation: clay.routing_mode (legacy by default, or class_routes) is written by the provisioning writer in the same transaction as the model's classes, and setting it back to legacy rolls the model back. class_routes requires a standard class and nothing else; a class may carry its own realtime_inflight_limit. Class aliases (synonyms) are retired and refused, and an activated model cannot redirect traffic. The migration adds model_serving_classes.realtime_inflight_limit, a config notification on class changes, and realtime_inflight_class_overrides for per-account limits on non-standard classes. Nothing routes on any of it yet.
A class other than standard is accepted when the model catalog declares it: checked offline against the catalog and at startup against the stored classes. Its realtime limit goes to realtime_inflight_class_overrides (standard keeps realtime_inflight_limit_overrides and its key), its prices are reconciled like standard's, and prices under a class the file no longer declares are retired.
A model whose routing_mode is class_routes is served as one gateway target per serving class instead of its composite: the model's alias for standard, alias:<class> for any other, each with the class's single upstream. Every class target admits the model's keys and carries the model's deny rules, its own realtime limit (the class default, else the model's) and account overrides (standard from the existing table, other classes from realtime_inflight_class_overrides). Batch is capped on standard only. Account policies apply model-wide and narrow per class. A model with no standard class, or with a redirect rule, keeps its legacy route and logs an error. For class-routed models, the free-model admission check also counts class prices.
Account-catalog presets are keyed by the gateway target a request names: the model's alias for standard, alias:<class> for any other. A model-wide preset applies to every class; a standard-class preset stays on standard.
…del's class Flex and background requests naming a class route (alias:<class>) are refused with serving_class_realtime_only, and batch lines naming one are refused at upload; other suffixes keep today's silent drop. Analytics record a class route as the model's alias with the class as requested and resolved class, and billing prices it from the model's public class prices for class-routed models. An account's class-less deal now covers the standard class only, in both the batcher and effective_model_tariff: any other class is priced by the account's deal for that class, else the model's.
A class-routed model serves realtime by class (alias to standard's upstream, alias:fast to fast's), refuses flex and batch lines on fast, and records and prices each request as the model in its class.
/v1/models lists alias:<class> beside a class-routed model for each class other than standard (without prices for now). A class route inherits its model's prompt-cache enablement, with its own cache scope.
A class-routed model's targets carry no completions pool, so continuation leaves them alone. Documents activation, class limits, account settings per class and the billing rule that an account's class-less price covers the standard class only.
Deploying control-layer with
|
| Latest commit: |
7c19bb7
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c0eaf6a1.control-layer.pages.dev |
| Branch Preview URL: | https://feat-class-routing.control-layer.pages.dev |
|
Running ultrareview automatically — this PR appears to change payment, billing, or quota logic, where a missed bug can charge customers the wrong amount, so a deeper multi-pass review is worth running. I'll post findings when complete. |
Contributor
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
Ultrareview ran automatically — this PR appears to change payment, billing, or quota logic, where a missed bug can charge customers the wrong amount, so a deeper multi-pass review is worth running.
11 issues found across 35 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="docs/src/conceptual-guides/how-billing-works.md">
<violation number="1" location="docs/src/conceptual-guides/how-billing-works.md:145">
P3: This omits classless models, which also receive the account all-class multiplier. Document the scope as applying to standard and classless models.</violation>
</file>
<file name="dwctl/src/inference/rejections.rs">
<violation number="1" location="dwctl/src/inference/rejections.rs:52">
P2: This marks every configured colon-bearing alias as a class route, but catalog validation allows `:` in legacy aliases. A flex/background request matching such a target is rejected instead of following the existing suffix handling; track class routes explicitly rather than inferring them from punctuation.</violation>
</file>
<file name="model-provisioning.schema.json">
<violation number="1" location="model-provisioning.schema.json:171">
P2: The schema accepts `routing_mode: class_routes` with an empty `class_routes` map or one lacking `standard`, despite this description. Add conditional `required` constraints so schema-based catalogue checks reject configurations that runtime validation refuses.</violation>
</file>
<file name="dwctl/src/request_logging/batcher.rs">
<violation number="1" location="dwctl/src/request_logging/batcher.rs:872">
P2: Queued `alias:class` records become unresolvable as soon as rollback sets `routing_mode` to `legacy`. The outbox stores `metrics.request_model` and enrichment runs later, so this lookup misses, `batch_lookup_models_with_tariffs` finds no alias, and successful usage is left unbilled. Resolve historical route identities independently of the current mode or persist the alias/class in the outbox.</violation>
</file>
<file name="dwctl/src/api/handlers/ai_models.rs">
<violation number="1" location="dwctl/src/api/handlers/ai_models.rs:387">
P2: Each `model:<class>` entry inherits the base alias's reasoning efforts even though class routes can target a different inference endpoint. Derive the advertised efforts from that class's endpoint so clients aren't told unsupported efforts are available or shown a narrower set than the upstream supports.</violation>
</file>
<file name="docs/src/reference/model-provisioning.md">
<violation number="1" location="docs/src/reference/model-provisioning.md:205">
P2: Class routes still receive the composite's configured `PoolSpec.fallback` and retry/backoff settings, so this says those settings are discarded. Remove `fallback` from the exclusion or document that configured retries remain.</violation>
<violation number="2" location="docs/src/reference/model-provisioning.md:230">
P1: This rolls back routing only on a release that understands the field: `routing_mode: legacy` still leaves it in YAML, which older releases reject at startup. State that the catalogue must remove `routing_mode` before deploying an older image.</violation>
<violation number="3" location="docs/src/reference/model-provisioning.md:231">
P2: Add `cache_tariff` to the legacy fields kept for rollback; class caching inherits enablement and minimum-prefix settings from the model-level cache tariff, so dropping it loses that cache configuration.</violation>
</file>
<file name="dwctl/src/sync/onwards_config/mod.rs">
<violation number="1" location="dwctl/src/sync/onwards_config/mod.rs:998">
P2: This paid-access gate treats class tariffs as active even when `convert_class_routes` rejects activation and serves the legacy route. A zero-balance account can therefore lose access to a legacy-free model because of a tariff on an inactive class; make both tariff gates depend on successful class-route activation.</violation>
</file>
<file name="dwctl/src/inference/middleware.rs">
<violation number="1" location="dwctl/src/inference/middleware.rs:188">
P2: A class-route request with `service_tier: "flex"` on `/embeddings` is downgraded to realtime before this guard and reaches the class upstream instead of being rejected. Check the requested tier for class routes before the downgrade to enforce the realtime-only contract.</violation>
<violation number="2" location="dwctl/src/inference/middleware.rs:198">
P3: Aliases may contain `:`, so `split_once` truncates the suggested standard route (for example, `foo:bar` becomes `foo`). Use `rsplit_once` because the class key is the final component.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
| added outside the catalogue) falls back to the legacy route and is logged as | ||
| `invalid_class_activation`. | ||
|
|
||
| To roll back, set `routing_mode: legacy` and redeploy. Keep the legacy |
There was a problem hiding this comment.
P1: This rolls back routing only on a release that understands the field: routing_mode: legacy still leaves it in YAML, which older releases reject at startup. State that the catalogue must remove routing_mode before deploying an older image.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/src/reference/model-provisioning.md, line 230:
<comment>This rolls back routing only on a release that understands the field: `routing_mode: legacy` still leaves it in YAML, which older releases reject at startup. State that the catalogue must remove `routing_mode` before deploying an older image.</comment>
<file context>
@@ -179,51 +179,78 @@ clay:
+added outside the catalogue) falls back to the legacy route and is logged as
+`invalid_class_activation`.
+
+To roll back, set `routing_mode: legacy` and redeploy. Keep the legacy
+`deployments`, `routing` and `tariffs` in place while a model is activated so the
+rollback has a route to return to. The catalogue owns the column: it is written
</file context>
Suggested change
| To roll back, set `routing_mode: legacy` and redeploy. Keep the legacy | |
| To roll back, set `routing_mode: legacy` and redeploy. Before using an older image, remove `routing_mode` from the catalogue and redeploy it. |
| /// Whether `model` names a class route of a class-routed model | ||
| /// (`alias:<class>`), which serves realtime requests only. | ||
| pub fn is_class_route(&self, model: &str) -> bool { | ||
| model.contains(':') && self.targets.targets.contains_key(model) |
There was a problem hiding this comment.
P2: This marks every configured colon-bearing alias as a class route, but catalog validation allows : in legacy aliases. A flex/background request matching such a target is rejected instead of following the existing suffix handling; track class routes explicitly rather than inferring them from punctuation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/inference/rejections.rs, line 52:
<comment>This marks every configured colon-bearing alias as a class route, but catalog validation allows `:` in legacy aliases. A flex/background request matching such a target is rejected instead of following the existing suffix handling; track class routes explicitly rather than inferring them from punctuation.</comment>
<file context>
@@ -45,6 +45,12 @@ impl Recorder {
+ /// Whether `model` names a class route of a class-routed model
+ /// (`alias:<class>`), which serves realtime requests only.
+ pub fn is_class_route(&self, model: &str) -> bool {
+ model.contains(':') && self.targets.targets.contains_key(model)
+ }
}
</file context>
| }, | ||
| "class_routes": { | ||
| "description": "Explicit class destinations and prices, staged independently of legacy routing.\nDeclaring these does not activate routes or advertise additional model names.", | ||
| "description": "Class destinations and prices. Declaring them alone changes nothing;\n`routing_mode: class_routes` activates them. `standard` is required.", |
There was a problem hiding this comment.
P2: The schema accepts routing_mode: class_routes with an empty class_routes map or one lacking standard, despite this description. Add conditional required constraints so schema-based catalogue checks reject configurations that runtime validation refuses.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At model-provisioning.schema.json, line 171:
<comment>The schema accepts `routing_mode: class_routes` with an empty `class_routes` map or one lacking `standard`, despite this description. Add conditional `required` constraints so schema-based catalogue checks reject configurations that runtime validation refuses.</comment>
<file context>
@@ -156,14 +156,19 @@
},
"class_routes": {
- "description": "Explicit class destinations and prices, staged independently of legacy routing.\nDeclaring these does not activate routes or advertise additional model names.",
+ "description": "Class destinations and prices. Declaring them alone changes nothing;\n`routing_mode: class_routes` activates them. `standard` is required.",
"type": "object",
"additionalProperties": {
</file context>
| SELECT dm.alias, c.class_key | ||
| FROM model_serving_classes c | ||
| JOIN deployed_models dm ON dm.id = c.deployed_model_id | ||
| WHERE dm.routing_mode = 'class_routes' |
There was a problem hiding this comment.
P2: Queued alias:class records become unresolvable as soon as rollback sets routing_mode to legacy. The outbox stores metrics.request_model and enrichment runs later, so this lookup misses, batch_lookup_models_with_tariffs finds no alias, and successful usage is left unbilled. Resolve historical route identities independently of the current mode or persist the alias/class in the outbox.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/request_logging/batcher.rs, line 872:
<comment>Queued `alias:class` records become unresolvable as soon as rollback sets `routing_mode` to `legacy`. The outbox stores `metrics.request_model` and enrichment runs later, so this lookup misses, `batch_lookup_models_with_tariffs` finds no alias, and successful usage is left unbilled. Resolve historical route identities independently of the current mode or persist the alias/class in the outbox.</comment>
<file context>
@@ -830,6 +847,42 @@ where
+ SELECT dm.alias, c.class_key
+ FROM model_serving_classes c
+ JOIN deployed_models dm ON dm.id = c.deployed_model_id
+ WHERE dm.routing_mode = 'class_routes'
+ AND c.class_key <> 'standard'
+ AND dm.alias || ':' || c.class_key = ANY($1)
</file context>
| object: "model".to_string(), | ||
| created: model.created, | ||
| owned_by: model.owned_by.clone(), | ||
| supported_reasoning_efforts: model.supported_reasoning_efforts.clone(), |
There was a problem hiding this comment.
P2: Each model:<class> entry inherits the base alias's reasoning efforts even though class routes can target a different inference endpoint. Derive the advertised efforts from that class's endpoint so clients aren't told unsupported efforts are available or shown a narrower set than the upstream supports.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/api/handlers/ai_models.rs, line 387:
<comment>Each `model:<class>` entry inherits the base alias's reasoning efforts even though class routes can target a different inference endpoint. Derive the advertised efforts from that class's endpoint so clients aren't told unsupported efforts are available or shown a narrower set than the upstream supports.</comment>
<file context>
@@ -335,29 +351,50 @@ pub async fn list_ai_models<P: PoolProvider>(
+ object: "model".to_string(),
+ created: model.created,
+ owned_by: model.owned_by.clone(),
+ supported_reasoning_efforts: model.supported_reasoning_efforts.clone(),
+ tariffs: None,
+ cache_pricing: None,
</file context>
Comment on lines
+205
to
+206
| endpoint and upstream name, without the composite's members, fallback or stream | ||
| continuation. `self_hosted_only` does not apply: the class names its upstream. |
There was a problem hiding this comment.
P2: Class routes still receive the composite's configured PoolSpec.fallback and retry/backoff settings, so this says those settings are discarded. Remove fallback from the exclusion or document that configured retries remain.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/src/reference/model-provisioning.md, line 205:
<comment>Class routes still receive the composite's configured `PoolSpec.fallback` and retry/backoff settings, so this says those settings are discarded. Remove `fallback` from the exclusion or document that configured retries remain.</comment>
<file context>
@@ -179,51 +179,78 @@ clay:
+
+With `routing_mode: class_routes`, the gateway serves one route per class: `model`
+for `standard` and `model:<class>` for the rest. Each route forwards to its class's
+endpoint and upstream name, without the composite's members, fallback or stream
+continuation. `self_hosted_only` does not apply: the class names its upstream.
+Every route shares the model's deny rules, and only `standard` carries the
</file context>
Suggested change
| endpoint and upstream name, without the composite's members, fallback or stream | |
| continuation. `self_hosted_only` does not apply: the class names its upstream. | |
| endpoint and upstream name, without the composite's members or stream | |
| continuation; configured retry settings still apply. `self_hosted_only` does not apply: the class names its upstream. |
| SELECT 1 FROM model_tariffs mt | ||
| WHERE mt.deployed_model_id = cm.id AND mt.user_id IS NULL AND mt.serving_class IS NULL | ||
| WHERE mt.deployed_model_id = cm.id AND mt.user_id IS NULL | ||
| AND (mt.serving_class IS NULL OR cm.routing_mode = 'class_routes') |
There was a problem hiding this comment.
P2: This paid-access gate treats class tariffs as active even when convert_class_routes rejects activation and serves the legacy route. A zero-balance account can therefore lose access to a legacy-free model because of a tariff on an inactive class; make both tariff gates depend on successful class-route activation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/sync/onwards_config/mod.rs, line 998:
<comment>This paid-access gate treats class tariffs as active even when `convert_class_routes` rejects activation and serves the legacy route. A zero-balance account can therefore lose access to a legacy-free model because of a tariff on an inactive class; make both tariff gates depend on successful class-route activation.</comment>
<file context>
@@ -908,7 +994,8 @@ async fn load_composite_models_from_db(db: &PgPool, escalation_models: &[String]
SELECT 1 FROM model_tariffs mt
- WHERE mt.deployed_model_id = cm.id AND mt.user_id IS NULL AND mt.serving_class IS NULL
+ WHERE mt.deployed_model_id = cm.id AND mt.user_id IS NULL
+ AND (mt.serving_class IS NULL OR cm.routing_mode = 'class_routes')
AND mt.valid_until IS NULL
AND (mt.input_price_per_token > 0 OR mt.output_price_per_token > 0)
</file context>
| // A class other than standard (`alias:<class>` on a class-routed model) | ||
| // serves realtime only: asynchronous work is scheduled on the standard | ||
| // class by its deadline. | ||
| if is_daemon_processed |
There was a problem hiding this comment.
P2: A class-route request with service_tier: "flex" on /embeddings is downgraded to realtime before this guard and reaches the class upstream instead of being rejected. Check the requested tier for class routes before the downgrade to enforce the realtime-only contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/inference/middleware.rs, line 188:
<comment>A class-route request with `service_tier: "flex"` on `/embeddings` is downgraded to realtime before this guard and reaches the class upstream instead of being rejected. Check the requested tier for class routes before the downgrade to enforce the realtime-only contract.</comment>
<file context>
@@ -182,6 +182,24 @@ pub async fn inference_middleware<P: PoolProvider + Clone + Send + Sync + 'stati
+ // A class other than standard (`alias:<class>` on a class-routed model)
+ // serves realtime only: asynchronous work is scheduled on the standard
+ // class by its deadline.
+ if is_daemon_processed
+ && let Some(model) = request_value["model"].as_str()
+ && parts
</file context>
Suggested change
| if is_daemon_processed | |
| if matches!(requested_tier, ServiceTier::Flex | ServiceTier::Background) |
|
|
||
| Cache multipliers are selected independently by account + resolved class → | ||
| account all-class → general model, at the same timestamp. They have no | ||
| account all-class (standard only) → public class → general model, at the same |
There was a problem hiding this comment.
P3: This omits classless models, which also receive the account all-class multiplier. Document the scope as applying to standard and classless models.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/src/conceptual-guides/how-billing-works.md, line 145:
<comment>This omits classless models, which also receive the account all-class multiplier. Document the scope as applying to standard and classless models.</comment>
<file context>
@@ -132,7 +142,8 @@ rolls back atomically. Historical org token/cache prices prevent hard account
Cache multipliers are selected independently by account + resolved class →
-account all-class → general model, at the same timestamp. They have no
+account all-class (standard only) → public class → general model, at the same
+timestamp. They have no
purpose/window dimension. A general cache tariff is required for Control Layer
</file context>
| let model = model.to_owned(); | ||
| let message = format!( | ||
| "Model '{model}' is available for realtime requests only; send flex and background requests to '{}'.", | ||
| model.split_once(':').map_or(model.as_str(), |(alias, _)| alias) |
There was a problem hiding this comment.
P3: Aliases may contain :, so split_once truncates the suggested standard route (for example, foo:bar becomes foo). Use rsplit_once because the class key is the final component.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dwctl/src/inference/middleware.rs, line 198:
<comment>Aliases may contain `:`, so `split_once` truncates the suggested standard route (for example, `foo:bar` becomes `foo`). Use `rsplit_once` because the class key is the final component.</comment>
<file context>
@@ -182,6 +182,24 @@ pub async fn inference_middleware<P: PoolProvider + Clone + Send + Sync + 'stati
+ let model = model.to_owned();
+ let message = format!(
+ "Model '{model}' is available for realtime requests only; send flex and background requests to '{}'.",
+ model.split_once(':').map_or(model.as_str(), |(alias, _)| alias)
+ );
+ return invalid_request(&parts, Some(&model), &message, "serving_class_realtime_only", "model");
</file context>
Suggested change
| model.split_once(':').map_or(model.as_str(), |(alias, _)| alias) | |
| model.rsplit_once(':').map_or(model.as_str(), |(alias, _)| alias) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Serving classes become routable. A model that opts in with
clay.routing_mode: class_routesservesmodelfrom itsstandardclass andmodel:<class>from each other class it declares inclay.class_routes. Each class has its own upstream (endpoint and upstream model name), realtime prices and default realtime in-flight limit. Accounts can set priority, scheduling hints, toleration presets, limits and prices per class.Nothing changes for a model until its catalogue entry sets
routing_mode: class_routes. Every existing model stays on the legacy route.What changes
Catalogue (
model_provisioning)clay.routing_mode(legacydefault |class_routes). The provisioning writer setsdeployed_models.routing_modefrom it, so activation and rollback are catalogue edits.class_routesrequiresstandard. Other classes are optional (previouslystandardandfastwere both required).class_routes.<class>.realtime_inflight_limit. When omitted, the class uses the model's limit.aliases) are retired and must be empty. Class names are alwaysmodelandmodel:<class>.standardor with redirect traffic rules.Migration (additive):
model_serving_classes.realtime_inflight_limit, therealtime_inflight_class_overridestable (account × non-standard class limits), notify triggers, andeffective_model_tariffupdated to the billing rule below.Gateway sync
modelfor standard,model:<class>otherwise.model, so existing limiter keys and the batch lane are unchanged.invalid_class_activation.Account files
classesmay name any class the model declares, not onlystandard.Requests
model:<class>gets 400serving_class_realtime_only, and batch lines naming one are refused at upload.model:<class>under the model's name, withrequested_serving_classandresolved_serving_classset to the class. The request is priced as that class./v1/modelslistsmodel:<class>for each non-standard class.Docs: model provisioning reference (class routing, account files per class) and the billing precedence guide.
Decisions to review
standardclass only. On another class, the account pays its price for that class if it has one, otherwise the model's public price for that class. A discount on the standard tier therefore never undercuts a premium class. Cache multipliers follow the same rule./v1/modelsare a follow-up.self_hosted_onlydoes not apply to class routes. The class names its upstream directly.routing_mode: legacyplus a redeploy. Keep the legacydeployments/routing/tariffswhile a model is activated. Older releases reject therouting_modefield, so an image rollback needs the catalogue rolled back first.Follow-ups (not in this PR)
/v1/models.classlabel on existing gateway metrics.Testing
test::serving_classes): realtime requests reach each class's upstream and are billed at that class's price; flex and batch naming a class are refused; discovery lists the class.just lint rustandjust test rustpass locally.🤖 Generated with Claude Code