dstack

dstack is an open-source, self-hostable orchestration layer for heterogeneous AI compute, licensed under the Mozilla Public License 2.0. It provides a unified control plane across clouds (AWS, GCP, Lambda, Crusoe, Nebius), Kubernetes clusters, VMs, and bare-metal — enabling developers to schedule and manage GPU/accelerator workloads without being locked into a single provider or runtime.

Key architectural capabilities include:

• Multi-accelerator support: NVIDIA, AMD, TPU, and Tenstorrent hardware targets
• Workload types: Training runs, inference serving, interactive dev environments, and fleet management — all managed through a single API
• Inference backend integrations: SGLang, Dynamo, vLLM, NIM, TensorRT-LLM
• Training framework integrations: TRL, Axolotl, Ray
• Operational primitives: Auto-scaling, port-forwarding, and ingress handling built in
• Programmatic control: Full API and CLI support for automation and CI/CD integration

With 2,270+ GitHub stars, dstack is designed for AI labs, data centers, and GPU-based training and inference workloads that require flexibility across heterogeneous infrastructure.