[Serve] Gang scheduling + autoscaling: bounds that aren't multiples of gang_size cause DEPLOY_FAILED or endless scale flapping #67494

What happened + What you expected to happen

GangSchedulingAutoscalingPolicy rounds the autoscaling decision up to a multiple of gang_size. AutoscalingState.apply_bounds then clips it to [min_replicas, max_replicas] without regard to gang size. When a bound isn't a multiple of gang_size, the clip undoes the rounding. The scheduler can't reach that target: it rejects scale-ups that aren't whole gangs (deployment_scheduler.py, _prepare_gangs_for_deployment), and a scale-down only stops gangs that fit entirely within the replicas to stop.

The Python API accepts such bounds. The gang checks in _private/config.py and deployment.py skip validation when autoscaling_config is set. schema.py validates the bounds only for YAML configs that set num_replicas: "auto".

Reproduced with gang_size=2:

1. min=max=3 → DEPLOY_FAILED. With no replicas running, the policy holds the target at 3. It is rounded up to 4, clipped back to 3, and rejected (num_replicas_to_add 3 is not divisible by gang_size 2) on every tick. After 9 failures (min(max_constructor_retry_count=20, 3 replicas × 3)), the deployment fails with a misleading "constructor or initial health check" message.
2. min=3, max=4, no traffic → endless flapping. The deployment starts at 4 replicas. After downscale_delay_s the policy's decision of 1 is rounded to 2 and lifted by min_replicas to 3. That means stopping 1 replica, but a gang of 2 can't be split, so nothing stops. On the next tick the downscale delay restarts and holds the target at 3, which is rounded back up to 4. The cycle repeats every downscale_delay_s (1 s in the repro, 600 s by default), with 4 replicas running throughout.

Also affected (from code inspection): target_capacity. get_capacity_adjusted_num_replicas can turn aligned bounds into unaligned ones; for example max=8, gang_size=4, target_capacity=75 gives an upper bound of 6. Non-autoscaling gang deployments are affected the same way: num_replicas=8 at target_capacity=30 gives a target of 2.

Expected: a gang deployment's target is always a multiple of gang_size. Rollout and surge batch sizes in deployment_state.py already round to gang multiples.

Proposed fix:

1. Validate min_replicas, max_replicas and initial_replicas in DeploymentConfig, matching schema.py. This rejects both repro configs at deploy time.
2. Round capacity-adjusted values to gang multiples, for both autoscaling bounds and non-autoscaling num_replicas. Validation can't catch these, since they're computed at runtime. Keep 0 as 0 so scale-to-zero still works.

Open question for (2): should an unaligned capacity-adjusted value round up (slightly above the requested capacity) or down (possibly 0 gangs at low capacity)?

I'd like to work on the fix and can send a PR once there's agreement on the approach.

Versions / Dependencies

• Ray v3.0.0.dev0, on commit fb22a8a
• Python 3.11.13
• OS: Linux

Reproduction script

import sys, time
import ray
from ray import serve
from ray.serve.config import GangSchedulingConfig

ray.init(num_cpus=4)

def deployment(autoscaling_config):
    @serve.deployment(
        name="A",
        num_replicas="auto",
        ray_actor_options={"num_cpus": 0.1},
        gang_scheduling_config=GangSchedulingConfig(gang_size=2),
        autoscaling_config=autoscaling_config,
    )
    class A:
        def __call__(self):
            return "ok"
    return A

def show(app):
    s = serve.status().applications[app].deployments["A"]
    print(s.status, dict(s.replica_states), s.message[:200])

if sys.argv[1] == "1":
    try:
        serve.run(deployment({"min_replicas": 3, "max_replicas": 3}).bind(), name="app1")
    except RuntimeError as e:
        print("serve.run raised:", e)
    show("app1")
else:
    serve.run(deployment({"min_replicas": 3, "max_replicas": 4, "metrics_interval_s": 0.5,
                          "look_back_period_s": 1, "downscale_delay_s": 1}).bind(), name="app2")
    for _ in range(6):
        time.sleep(5)
        show("app2")

Case 1 output:

ERROR -- num_replicas_to_add 3 is not divisible by gang_size 2.   (x9)
serve.run raised: Deploying application app1 failed: Failed to update the deployments ['A'].
DEPLOY_FAILED {} The deployment failed to start 9 times in a row. This may be due to a problem with its constructor or initial health check failing. ...

Case 2 output, repeating about every 1.1 s:

INFO -- Downscaling Deployment(name='A', app='app2') from 4 to 3 replicas. ...
INFO -- Removing 1 replica from Deployment(name='A', app='app2').
INFO -- Upscaling Deployment(name='A', app='app2') from 3 to 4 replicas. ...

Issue Severity

Medium: It is a significant difficulty but I can work around it.