LanguageSearch

NVIDIA Dynamo 1.0

#18 in AI Inference Hardware

nvidia · v1.0 · seit 2026-03-16 · 3× · zuletzt 30. Juni 2026

20

Momentum

NVIDIA Dynamo 1.0 is an open-source inference operating system for AI factories, released on March 16, 2026 at the GTC conference. It distributes generative and agentic AI inference across large GPU clusters by disaggregating prefill and decode phases, intelligently managing KV cache across nodes, and dynamically orchestrating GPU resources. SemiAnalysis InferenceX benchmarks confirm up to 7x inference throughput improvement on NVIDIA Blackwell GPUs (GB200 NVL72, DeepSeek-R1, FP4, 1k/1k, ~50 tok/s/user) compared to an unoptimized baseline configuration. The software is freely available under the Apache 2.0 license and integrates natively with frameworks such as vLLM, SGLang, TensorRT-LLM, and LangChain.

Sources (3)