NVIDIA Dynamo

Open-source, low-latency inference for generative AI.

NVIDIA Dynamo is a modular inference framework for serving generative AI models in distributed environments. It scales workloads across GPU fleets with intelligent resource scheduling, request routing, optimized memory management, and data transfer.

GitHub repository · Documentation

Built for production inference

• Distributed serving: Deploy and scale inference across GPU fleets.
• Disaggregated inference: Independently optimize prefill and decode workloads.
• Open ecosystem: Works with SGLang, TensorRT-LLM, and vLLM.

Datasets and evaluation: This organization publishes Dynamo-related datasets and reproducible evaluation fixtures.

More source code, examples, and project updates: github.com/ai-dynamo