Skip to content

[Fargate] [request]: Key-driven task scheduling and hot-swap replacement for stateful Fargate workloads #2867

Description

@vishnudivakar31

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

1. Key-driven task mode — Today, Fargate services only support a flat desiredCount. We need a mode where a service accepts a dynamic map of { taskKey: workerCount } and ensures the correct number of tasks per key. The key should be available to the task at runtime (e.g., via an environment variable). This would eliminate the need to manage a separate ECS service per logical partition.

2. Hot-swap task replacement — Fargate currently launches a replacement task and waits for it to be running before retiring the old one. However, "running" does not mean the new task has fully bootstrapped its in-memory state. We need a hot-swap mechanism where the new task boots alongside the old one, fully loads its state, and explicitly signals readiness — only then should load shift to it and the old task be drained. This would provide zero downtime with no in-memory state loss.

Which service(s) is this request for?

Fargate

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

We run latency-sensitive, high-cadence workloads on Fargate where each logical entity (e.g., a facility, a site, or a tenant) requires a dedicated task. Two examples:

Real-time State Tracking

  • Continuously computes a state snapshot for a physical site every 5 seconds. Each site has a large number of tracked assets, requiring a full Fargate task per site.
  • Zero tolerance for cold starts — if state computation is interrupted for more than 20 seconds, downstream systems receive stale data and automated decisions degrade.

Periodic Job Orchestration

  • Manages job lifecycles per site on a 20-second cycle — scheduling, monitoring, and completing short-lived units of work.
  • Delays beyond 60 seconds cause job backlogs and visible service degradation.

Today, there is no way to express "run one task per site" as a single service, and no way to ensure the replacement task is fully bootstrapped before the old one is retired — both of which are critical given our tight SLAs.

Are you currently working around this issue?

Yes. We built a custom orchestration layer that stitches together multiple AWS services to achieve this:

  • EventBridge + Lambda — An EventBridge rule triggers a Lambda function every 1 minute. The Lambda reads a dynamic key set, spins up new Fargate tasks for any newly added keys, and gracefully retires tasks whose keys no longer exist.
  • DynamoDB — Stores the active key-to-task mapping and coordinates hot-swap state, ensuring the new task is fully bootstrapped before the old one is drained.
  • Fargate — Runs the actual per-site workloads.

It works, but we are essentially building a custom key-aware scheduler and hot-swap coordinator on top of Fargate — significant undifferentiated overhead that could be a native capability.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ECSAmazon Elastic Container ServiceFargateAWS FargateProposedCommunity submitted issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions