NVIDIA Releases Topograph to Optimize AI Cluster Scheduling

NVIDIA has introduced Topograph, an open-source toolkit designed to discover and normalize cluster topology for workload schedulers. By mapping physical network layouts, the tool prevents poor workload placement that fragments topology domains and forces data across shared links. This optimization is critical for distributed training and inference, where GPUs constantly exchange data. Topograph automatically detects cluster changes and updates schedulers without manual intervention, publishing data as Kubernetes node labels, Slurm configurations, or Slinky ConfigMaps.

The toolkit supports high-bandwidth environments, including fifth-generation NVIDIA NVLink for Blackwell architectures, such as the GB200 and GB300, delivering 1.8 TB/s of bidirectional bandwidth per GPU, and sixth-generation Vera Rubin systems offering 3.6 TB/s per GPU. It also integrates with Quantum InfiniBand ports reaching up to 800 Gb/s, Spectrum-X Ethernet, and Multi-Node NVLink domains. For cloud deployments, Topograph integrates with Google Cloud, Lambda, Nebius, Nscale, Crusoe, and Oracle Cloud Infrastructure.

For practitioners, Topograph bridges the gap where default schedulers fail to see physical interconnect hierarchies. It requires Kubernetes version 1.27 or later and Helm 3.10+ or 4.x, and can be deployed from the dsx-ai-factory/topograph GitHub repository. The toolkit features an API server running on port 49021 with a typical 15-second aggregation delay to handle event bursts. It integrates with the CNCF Sandbox project KAI Scheduler, Kueue TAS, and Slinky, the Kubernetes-Slurm integration from SchedMD, which NVIDIA acquired in December 2025. It also supports Slurm 25.05 per-partition configurations and Node Feature Discovery.

By utilizing Topograph alongside orchestration tools like Grove and Dynamo, operators can implement topology-aware gang scheduling. This ensures tightly coupled AI workloads remain within high-speed locality domains, reducing network congestion and maximizing tokens per watt.