Modal Rebuilt Infrastructure for 1 Million Concurrent Sandboxes
The new architecture allows for 50,000 sandbox creations per second to better support intensive AI training and inference tasks.
Updated on Sept. 23, 2026 in Data Centers
Live Poll
Is moving away from centralized cloud orchestration the right approach for future-proofing digital infrastructure?
Modal engineers have redesigned their platform infrastructure to support 1 million concurrent sandboxes, enabling 50,000 creations per second. The system is designed to handle large-scale AI workloads with a median startup-to-code time of less than 0.5 seconds.
Why it matters
Traditional container platforms often struggle with scale due to centralized coordination and the linear scaling limits of standard Kubernetes scheduling algorithms. This architecture shift replaces that bottleneck with a decentralized model that maintains performance across massive compute clusters.
The platform supports 1 million concurrent sandboxes and 100,000 worker nodes, achieving a median startup-to-code time of under 0.5 seconds. The architecture utilizes a parallel fleet of scheduling servers to prevent the bottlenecks associated with centralized state management.
The players
Modal
A cloud infrastructure provider focused on serverless compute for AI inference, training, and heavy data processing tasks.
The details
Modal moved away from centralized scheduling to a distributed model where each worker node acts as its own source of truth. Workers accept or reject sandbox creation requests based on local resource availability and communicate with scheduling servers via RPC, or remote procedure calls. The system publishes worker state to a single Redis stream, which load testing indicates remains viable for at least 100,000 workers.
Timeline
September 23, 2026: Details regarding the infrastructure update were published.
The Tech Race
This architecture departs from traditional container platforms that rely on centralized coordination. By moving toward a decentralized worker model, Modal seeks to overcome the scaling limits that define standard Kubernetes scheduling algorithms.
Developers running AI inference and training workloads on the Modal platform will experience faster startup times and more consistent performance during high-concurrency events. The architecture supports up to 1 million concurrent sandboxes, enabling larger-scale distributed experiments without the need for manual cluster management.
The takeaway
The shift toward parallelized, decentralized scheduling demonstrates how platforms can bypass the overhead inherent in traditional centralized node management. Engineers should monitor how this design scales in production as more workloads hit the 100,000 worker node threshold.
Further reading
Explore more about how compute providers are evolving in our Data Centers section.
Source note: This article includes information reported by InfoQ.
Live Poll
Is moving away from centralized cloud orchestration the right approach for future-proofing digital infrastructure?
