Design rail-optimized topologies for high-performance workloads: Worked Example — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)

Design Rail-Optimized Topologies for High-Performance Workloads: A Worked Example

In the context of AI data center design, rail-optimized topologies are critical for maximizing throughput and minimizing latency between GPUs, which are the core compute units for AI workloads. This worked example demonstrates how to design a rail-optimized network topology tailored for a high-performance AI training cluster.

Scenario Overview

Consider an AI factory environment requiring a high-throughput, low-latency network to support distributed training across 16 GPUs. The goal is to design a rail-optimized topology that ensures efficient GPU-to-GPU communication, leveraging NVIDIA's advanced networking technologies.

Step 1: Define Performance Requirements and Constraints

• Workload type: Distributed deep learning training with frequent all-reduce operations.
• GPU count: 16 GPUs arranged in 4 nodes, each with 4 GPUs.
• Network bandwidth: Minimum 200 Gbps per GPU-to-GPU link.
• Latency: Minimize to reduce synchronization overhead.

Step 2: Select Network Components

Choose NVIDIA-certified switches and network interface cards (NICs) that support NVLink and InfiniBand HDR 200 Gbps:

• Switch: NVIDIA Quantum-2 InfiniBand switch with 200 Gbps ports.
• NIC: NVIDIA ConnectX-7 adapters supporting GPUDirect RDMA.

Step 3: Establish Rail-Optimized Topology Principles

Rail optimization involves creating multiple parallel, non-blocking data paths (rails) between GPUs to maximize bandwidth and fault tolerance. Key principles include:

• Use multiple rails per GPU to aggregate bandwidth.
• Ensure minimal hops between communicating GPUs.
• Balance traffic to avoid congestion on any single rail.

Step 4: Design the Topology

Arrange the 4 nodes in a fully connected mesh topology with dual rails per node:

• Each GPU connects via two independent rails to different switches.
• Switches are interconnected with high-speed links to form a fat-tree architecture.
• GPUs within the same node connect via NVLink for ultra-low latency.

Step 5: Map GPU-to-GPU Communication Paths

For inter-node communication:

• Assign rail 1 for primary data transfer and rail 2 as a redundant path to balance load.
• Use GPUDirect RDMA to enable direct memory access between GPUs, bypassing CPU involvement.

For intra-node communication:

• Leverage NVLink connections to maximize bandwidth and minimize latency.

Step 6: Validate and Optimize

Simulate communication patterns using NVIDIA's network simulation tools:

• Check for bottlenecks or oversubscription on rails.
• Adjust switch port assignments to balance traffic.
• Implement Quality of Service (QoS) policies to prioritize synchronization messages.

Worked Example: Calculating Aggregate Bandwidth

Problem: Calculate the aggregate bandwidth available between two GPUs located in different nodes connected via dual rails, each rail supporting 200 Gbps.

Solution:

• Each rail provides 200 Gbps.
• Two rails per GPU-to-GPU link means total bandwidth = 2 × 200 Gbps = 400 Gbps.
• This aggregation reduces communication time for large model parameter exchanges.

Summary

This step-by-step design ensures that the AI data center network topology is rail-optimized to handle high-performance workloads effectively. By carefully selecting components, structuring multi-rail connections, and validating through simulation, the network can sustain the demanding communication patterns of distributed AI training.

For further details on NVIDIA AI Networking certification topics, visit NVIDIA AI Networking.

More in this topic

Describe an AI factory networking architecture and its components: Quick Reference — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Describe an AI factory networking architecture and its components: Worked Example — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Optimize GPU-to-GPU communication patterns: Quick Reference — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Describe an AI factory networking architecture and its components — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Design rail-optimized topologies for high-performance workloads: Quick Reference — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Optimize GPU-to-GPU communication patterns: Practice Questions — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Design rail-optimized topologies for high-performance workloads — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Design rail-optimized topologies for high-performance workloads: Practice Questions — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Design rail-optimized topologies for high-performance workloads: Common Mistakes — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Optimize GPU-to-GPU communication patterns: Common Mistakes — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Describe an AI factory networking architecture and its components: Common Mistakes — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)AI Data Center Design and Optimization — NVIDIA-Certified Professional: AI NetworkingDescribe an AI factory networking architecture and its components: Practice Questions — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Optimize GPU-to-GPU communication patterns: Worked Example — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)Optimize GPU-to-GPU communication patterns — AI Data Center Design and Optimization (NVIDIA-Certified Professional: AI Networking)