[Breaking]: Drop gateway-based routers

gateway/pyproject.toml

@@ -15,6 +15,7 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
+# TODO: drop, unused by the server since 0.21.0
 sglang = ["sglang-router==0.3.2"]
 
 [tool.setuptools.package-data]

mkdocs/blog/posts/pd-disaggregation.md

@@ -27,7 +27,7 @@ For inference, `dstack` provides a [services](../../docs/concepts/services.md) a
 > If you’re new to Prefill–Decode disaggregation, see the official [SGLang docs](https://docs.sglang.io/advanced_features/pd_disaggregation.html).
 
 !!! note "Deprecation notice"
-    Configuring the SGLang router in a gateway is deprecated and will be disallowed in a future release. To run router and workers as separate replica groups, see [SGLang PD disaggregation (router as replica group)](../../docs/examples/inference/sglang.md#pd-disaggregation).
+    The router configuration approach described in this blog post is no longer supported. For the modern approach, see [SGLang PD disaggregation (router as replica group)](../../docs/examples/inference/sglang.md#pd-disaggregation).
 
 ## Services
 

mkdocs/blog/posts/sglang-router.md

@@ -125,6 +125,9 @@ After that, if you configure [replicas and scaling](../../docs/concepts/services
 
 ### Router
 
+!!! note "Deprecation notice"
+    The router configuration approach described in this section is no longer supported. For the modern approach, see [SGLang PD disaggregation (router as replica group)](../../docs/examples/inference/sglang.md#pd-disaggregation).
+
 By default, the gateway uses its built-in load balancer to route traffic across replicas. With the latest release, you can instead delegate traffic routing to the [SGLang Model Gateway](https://docs.sglang.ai/advanced_features/router.html) by setting the `router` property to `sglang`:
 
 <div editor-title="gateway.dstack.yml">

mkdocs/docs/concepts/gateways.md

@@ -76,54 +76,6 @@ You can create gateways with the `aws`, `azure`, `gcp`, or `kubernetes` backends
     Gateways in `kubernetes` backend require an external load balancer. Managed Kubernetes solutions usually include a load balancer.
     For self-hosted Kubernetes, you must provide a load balancer by yourself.
 
-### Router
-
-> In previous releases, `dstack` allowed configuring `router` the gateway, which was required for PD disaggregation. Since 0.20.17, the `router` configuration has moved to [services](services.md#pd-disaggregation), and the gateway no longer needs to configure router.
-
-<!-- ### Router
-
-By default, the gateway uses its own load balancer to route traffic between replicas. However, you can delegate this responsibility to a specific router by setting the `router` property. Currently, the only supported external router is `sglang`.
-
-#### SGLang
-
-The `sglang` router delegates routing logic to the [SGLang Model Gateway](https://docs.sglang.ai/advanced_features/router.html#).
-
-To enable it, set `type` field under `router` to `sglang`:
-
-<div editor-title="gateway.dstack.yml">
-
-```yaml
-type: gateway
-name: sglang-gateway
-
-backend: aws
-region: eu-west-1
-
-domain: example.com
-
-router:
-  type: sglang
-  policy: cache_aware
-```
-
-</div>
-
-If you configure the `sglang` router, [services](../concepts/services.md) can run either [standard SGLang workers](../examples/inference/sglang.md) or [Prefill-Decode workers](../examples/inference/sglang.md#pd-disaggregation) (aka PD disaggregation).
-
-!!! note "PD disaggregation"
-    To run services with PD disaggregation see [SGLang PD disaggregation](../examples/inference/sglang.md#pd-disaggregation).
-
-!!! note "Deprecation"
-    Configuring the SGLang router in a gateway is deprecated and will be disallowed in a future release.
-
-??? info "Policy"
-    The `policy` property allows you to configure the routing policy:
-
-    * `cache_aware` &mdash; Default policy; combines cache locality with load balancing, falling back to shortest queue. 
-    * `power_of_two` &mdash; Samples two workers and picks the lighter one.                                               
-    * `random` &mdash; Uniform random selection.                                                                    
-    * `round_robin` &mdash; Cycles through workers in order.                                                              -->
-
 ### Certificate
 
 By default, when you run a service with a gateway, `dstack` provisions an SSL certificate via Let's Encrypt for the configured domain. This automatically enables HTTPS for the service endpoint.

mkdocs/docs/concepts/services.md

@@ -340,7 +340,7 @@ Setting the minimum number of replicas to `0` allows the service to scale down t
 
 ### PD disaggregation
 
-<!-- NOTE: this section is referenced from the CLI, keep the URL unchanged -->
+<!-- NOTE: this section is referenced from pre-0.21.0 CLIs. Prefer to keep the URL unchanged -->
 
 Since 0.20.17, `dstack` supports serving a model using Prefill-Decode disaggregation. To use it, configure three replica groups: one for the router, one for prefill workers, and one for decode workers.
 

mkdocs/docs/reference/dstack.yml/gateway.md

@@ -10,16 +10,6 @@ The `gateway` configuration type allows creating and updating [gateways](../../c
       type:
         required: true
 
-### `router`
-
-=== "SGLang Model Gateway"
-
-    #SCHEMA# dstack._internal.core.models.routers.SGLangGatewayRouterConfig
-        overrides:
-          show_root_heading: false
-          type:
-            required: true
-
 ### `certificate`
 
 Set to `null` to disable certificates (e.g. for [private gateways](../../concepts/gateways.md#public-ip)).

src/dstack/_internal/cli/commands/gateway.py

@@ -19,7 +19,6 @@
 )
 from dstack._internal.core.errors import CLIError
 from dstack._internal.core.models.common import EntityReference
-from dstack._internal.core.models.gateways import GatewayStatus
 from dstack._internal.utils.logging import get_logger
 
 logger = get_logger(__name__)
@@ -110,20 +109,6 @@ def _list(self, args: argparse.Namespace):
             raise CLIError("JSON output is not supported together with --watch")
 
         gateways = self.api.client.gateways.list(self.api.project, include_imported=True)
-        deprecated_router_gateways = [
-            g.name
-            for g in gateways
-            if g.status != GatewayStatus.FAILED and g.configuration.router is not None
-        ]
-        if deprecated_router_gateways and args.format != "json":
-            logger.warning(
-                "Specifying `router` in gateway configurations is deprecated"
-                " and will be disallowed in a future release."
-                " Please migrate to replica-based routers:"
-                " https://dstack.ai/docs/concepts/services/#pd-disaggregation"
-                " (affected gateways: %s)",
-                ", ".join(deprecated_router_gateways),
-            )
         if not args.watch:
             if args.format == "json":
                 print_gateways_json(gateways, project=self.api.project)

src/dstack/_internal/cli/commands/ps.py

@@ -11,7 +11,6 @@
     console,
 )
 from dstack._internal.core.errors import CLIError
-from dstack._internal.core.models.configurations import ServiceConfiguration
 from dstack._internal.utils.logging import get_logger
 
 logger = get_logger(__name__)
@@ -69,23 +68,6 @@ def _command(self, args: argparse.Namespace):
 
         # TODO: Add a `ps --json` option to control how many job submissions are returned.
         runs = self.api.runs.list(all=args.all, limit=args.last)
-        deprecated_router_runs = [
-            run._run.run_spec.run_name
-            for run in runs
-            if not run.status.is_finished()
-            and isinstance(run._run.run_spec.configuration, ServiceConfiguration)
-            and run._run.run_spec.configuration.router is not None
-            and run._run.run_spec.run_name is not None
-        ]
-        if deprecated_router_runs and args.format != "json":
-            logger.warning(
-                "Specifying `router` in service configurations is deprecated"
-                " and will be disallowed in a future release."
-                " Please migrate to replica-based routers:"
-                " https://dstack.ai/docs/concepts/services/#pd-disaggregation"
-                " (affected runs: %s)",
-                ", ".join(deprecated_router_runs),
-            )
         if not args.watch:
             if args.format == "json":
                 run_utils.print_runs_json(self.api.project, runs)

src/dstack/_internal/cli/services/configurators/gateway.py

@@ -49,13 +49,6 @@ def apply_configuration(
             configuration=conf,
             configuration_path=configuration_path,
         )
-        if spec.configuration.router is not None:
-            logger.warning(
-                "Specifying `router` in gateway configurations is deprecated"
-                " and will be disallowed in a future release."
-                " Please migrate to replica-based routers:"
-                " https://dstack.ai/docs/concepts/services/#pd-disaggregation"
-            )
         with console.status("Getting apply plan..."):
             try:
                 plan = self.api.client.gateways.get_plan(project_name=self.api.project, spec=spec)

src/dstack/_internal/cli/services/configurators/run.py

@@ -123,14 +123,6 @@ def get_plan(
         if conf.working_dir is not None and not is_absolute_posix_path(conf.working_dir):
             raise ConfigurationError("working_dir must be absolute")
 
-        if isinstance(conf, ServiceConfiguration) and conf.router is not None:
-            logger.warning(
-                "Specifying `router` in service configurations is deprecated"
-                " and will be disallowed in a future release."
-                " Please migrate to replica-based routers:"
-                " https://dstack.ai/docs/concepts/services/#pd-disaggregation"
-            )
-
         repo = self.get_repo(conf, configuration_path, configurator_args)
         if repo is None:
             repo = init_default_virtual_repo(api=self.api)

src/dstack/_internal/core/backends/aws/compute.py

@@ -569,9 +569,7 @@ def create_gateway(
             image_id=aws_resources.get_gateway_image_id(ec2_client),
             instance_type=configuration.instance_type or DEFAULT_GATEWAY_INSTANCE_TYPE,
             iam_instance_profile=None,
-            user_data=get_gateway_user_data(
-                configuration.ssh_key_pub, router=configuration.router
-            ),
+            user_data=get_gateway_user_data(configuration.ssh_key_pub),
             tags=tags,
             security_group_id=security_group_id,
             spot=False,

src/dstack/_internal/core/backends/azure/compute.py

@@ -292,9 +292,7 @@ def create_gateway(
             image_reference=_get_gateway_image_ref(),
             vm_size=DEFAULT_GATEWAY_INSTANCE_TYPE,
             instance_name=instance_name,
-            user_data=get_gateway_user_data(
-                configuration.ssh_key_pub, router=configuration.router
-            ),
+            user_data=get_gateway_user_data(configuration.ssh_key_pub),
             ssh_pub_keys=[configuration.ssh_key_pub],
             spot=False,
             disk_size=30,

src/dstack/_internal/core/backends/base/compute.py

@@ -41,7 +41,6 @@
     SSHKey,
 )
 from dstack._internal.core.models.placement import PlacementGroup, PlacementGroupProvisioningData
-from dstack._internal.core.models.routers import AnyGatewayRouterConfig
 from dstack._internal.core.models.runs import Job, JobProvisioningData, Requirements, Run
 from dstack._internal.core.models.volumes import (
     Volume,
@@ -1089,9 +1088,7 @@ def get_run_shim_script(
     ]
 
 
-def get_gateway_user_data(
-    authorized_key: str, router: Optional[AnyGatewayRouterConfig] = None
-) -> str:
+def get_gateway_user_data(authorized_key: str) -> str:
     return get_cloud_config(
         package_update=True,
         packages=[
@@ -1107,7 +1104,7 @@ def get_gateway_user_data(
                 "s/# server_names_hash_bucket_size 64;/server_names_hash_bucket_size 128;/",
                 "/etc/nginx/nginx.conf",
             ],
-            ["su", "ubuntu", "-c", " && ".join(get_dstack_gateway_commands(router))],
+            ["su", "ubuntu", "-c", " && ".join(get_dstack_gateway_commands())],
         ],
         ssh_authorized_keys=[authorized_key],
     )
@@ -1207,22 +1204,19 @@ def get_latest_runner_build() -> Optional[str]:
     return None
 
 
-def get_dstack_gateway_wheel(build: str, router: Optional[AnyGatewayRouterConfig] = None) -> str:
+def get_dstack_gateway_wheel(build: str) -> str:
     channel = "release" if settings.DSTACK_RELEASE else "stgn"
     base_url = f"https://dstack-gateway-downloads.s3.amazonaws.com/{channel}"
     if build == "latest":
         build = _fetch_version(f"{base_url}/latest-version") or "latest"
         logger.debug("Found the latest gateway build: %s", build)
     wheel = f"{base_url}/dstack_gateway-{build}-py3-none-any.whl"
-    # Build package spec with extras if router is specified
-    if router:
-        return f"dstack-gateway[{router.type}] @ {wheel}"
     return f"dstack-gateway @ {wheel}"
 
 
-def get_dstack_gateway_commands(router: Optional[AnyGatewayRouterConfig] = None) -> List[str]:
+def get_dstack_gateway_commands() -> List[str]:
     build = get_dstack_runner_version() or "latest"
-    gateway_package = get_dstack_gateway_wheel(build, router)
+    gateway_package = get_dstack_gateway_wheel(build)
     return [
         "mkdir -p /home/ubuntu/dstack",
         "python3 -m venv /home/ubuntu/dstack/blue",

src/dstack/_internal/core/backends/gcp/compute.py

@@ -606,9 +606,7 @@ def create_gateway(
             machine_type=configuration.instance_type or DEFAULT_GATEWAY_INSTANCE_TYPE,
             accelerators=[],
             spot=False,
-            user_data=get_gateway_user_data(
-                configuration.ssh_key_pub, router=configuration.router
-            ),
+            user_data=get_gateway_user_data(configuration.ssh_key_pub),
             authorized_keys=[configuration.ssh_key_pub],
             labels=labels,
             tags=[gcp_resources.DSTACK_GATEWAY_TAG],

src/dstack/_internal/core/backends/kubernetes/compute.py

@@ -88,7 +88,6 @@
 )
 from dstack._internal.core.models.placement import PlacementGroup
 from dstack._internal.core.models.resources import GPUSpec
-from dstack._internal.core.models.routers import AnyGatewayRouterConfig
 from dstack._internal.core.models.runs import (
     Job,
     JobProvisioningData,
@@ -502,9 +501,7 @@ def create_gateway(
         )
         labels = filter_invalid_labels(labels)
 
-        commands = _get_gateway_commands(
-            authorized_keys=[configuration.ssh_key_pub], router=configuration.router
-        )
+        commands = _get_gateway_commands(authorized_keys=[configuration.ssh_key_pub])
         pod = client.V1Pod(
             metadata=client.V1ObjectMeta(
                 name=instance_name,
@@ -1375,11 +1372,9 @@ def _wait_for_load_balancer_address(
         time.sleep(1)
 
 
-def _get_gateway_commands(
-    authorized_keys: List[str], router: Optional[AnyGatewayRouterConfig] = None
-) -> List[str]:
+def _get_gateway_commands(authorized_keys: List[str]) -> List[str]:
     authorized_keys_content = "\n".join(authorized_keys).strip()
-    gateway_commands = " && ".join(get_dstack_gateway_commands(router=router))
+    gateway_commands = " && ".join(get_dstack_gateway_commands())
     quoted_gateway_commands = shlex.quote(gateway_commands)
 
     commands = [

src/dstack/_internal/core/compatibility/gateways.py

@@ -39,8 +39,6 @@ def _get_gateway_configuration_excludes(
 ) -> IncludeExcludeDictType:
     configuration_excludes: IncludeExcludeDictType = {}
 
-    if configuration.router is None:
-        configuration_excludes["router"] = True
     if configuration.replicas is None:
         configuration_excludes["replicas"] = True
 

src/dstack/_internal/core/compatibility/runs.py

@@ -9,7 +9,6 @@
 from dstack._internal.core.models.configurations import (
     ServiceConfiguration,
 )
-from dstack._internal.core.models.routers import SGLangServiceRouterConfig
 from dstack._internal.core.models.runs import (
     DEFAULT_PROBE_UNTIL_READY,
     DEFAULT_REPLICA_GROUP_NAME,
@@ -108,11 +107,6 @@ def get_run_spec_excludes(run_spec: RunSpec) -> IncludeExcludeDictType:
             # Servers prior to 0.20.8 do not support probes=None
             configuration_excludes["probes"] = True
 
-        router = run_spec.configuration.router
-        if router is None:
-            configuration_excludes["router"] = True
-        elif isinstance(router, SGLangServiceRouterConfig) and router.pd_disaggregation is False:
-            configuration_excludes["router"] = {"pd_disaggregation": True}
         if run_spec.configuration.https is None:
             configuration_excludes["https"] = True
 

src/dstack/_internal/core/models/configurations.py

@@ -39,7 +39,7 @@
     SpotPolicy,
 )
 from dstack._internal.core.models.resources import Range, ResourcesSpec
-from dstack._internal.core.models.routers import AnyServiceRouterConfig, ReplicaGroupRouterConfig
+from dstack._internal.core.models.routers import ReplicaGroupRouterConfig
 from dstack._internal.core.models.services import AnyModel, OpenAIChatModel
 from dstack._internal.core.models.unix import UnixUser
 from dstack._internal.core.models.volumes import (
@@ -1058,14 +1058,6 @@ class ServiceConfigurationParams(CoreModel):
             )
         ),
     ] = None
-    router: Annotated[
-        Optional[AnyServiceRouterConfig],
-        Field(
-            description=(
-                "Router configuration for the service. Requires a gateway with matching router enabled. "
-            ),
-        ),
-    ] = None
 
     @field_validator("port")
     @classmethod
@@ -1366,23 +1358,6 @@ def validate_at_most_one_router_replica_group(self) -> Self:
                 raise ValueError("For now replica group with `router` must have `count: 1`.")
         return self
 
-    @model_validator(mode="after")
-    def validate_replica_group_router_mutex(self) -> Self:
-        """
-        When a replica group sets `router:`, service-level `router` must be omitted.
-        (Gateway-level SGLang is rejected at service registration when a gateway is selected.)
-        """
-        replicas = self.replicas
-        if not isinstance(replicas, list):
-            return self
-        if not any(g.router is not None for g in replicas):
-            return self
-        if self.router is not None:
-            raise ValueError(
-                "Service-Level router configuration is not allowed together with replica-group `router`."
-            )
-        return self
-
 
 class ServiceConfiguration(
     ProfileParams,