Deploy the NVIDIA Network Operator for RDMA and InfiniBand: Quick Reference — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)

Quick Reference: Deploying the NVIDIA Network Operator for RDMA and InfiniBand on Kubernetes

This guide provides essential facts and steps for deploying the NVIDIA Network Operator to enable RDMA and InfiniBand support within Kubernetes clusters, a critical skill for the NVIDIA-Certified Professional: AI Networking exam.

Key Concepts

• RDMA (Remote Direct Memory Access): Enables high-throughput, low-latency networking by allowing direct memory access from one computer to another without involving the CPU.
• InfiniBand: A high-performance networking architecture commonly used in HPC and AI workloads, supporting RDMA.
• NVIDIA Network Operator: A Kubernetes operator that automates deployment and management of NVIDIA networking components, including RDMA and InfiniBand drivers and configurations.

Pre-Deployment Requirements

• Kubernetes cluster with nodes that have NVIDIA InfiniBand-capable network interfaces.
• Access to NVIDIA container registries or repositories hosting the Network Operator images.
• kubectl configured with cluster admin privileges.

Deployment Steps

1. Install the NVIDIA Network Operator Custom Resource Definitions (CRDs): Apply the CRDs to enable Kubernetes to recognize NVIDIA network resources.kubectl apply -f https://raw.githubusercontent.com/NVIDIA/network-operator/main/deploy/crds/nvidia_v1_network_crd.yaml
2. Deploy the NVIDIA Network Operator: Use the operator manifest to deploy it into the cluster.kubectl apply -f https://raw.githubusercontent.com/NVIDIA/network-operator/main/deploy/operator.yaml
3. Create a Network Custom Resource (CR): Define a Network CR to specify RDMA and InfiniBand configurations.Example snippet:apiVersion: nvidia.com/v1kind: Networkmetadata: name: nvidia-networkspec: rdma: enabled: true infiniband: enabled: true
4. Apply the Network CR:kubectl apply -f network-cr.yaml

Verifying Operator Functionality

• Check Operator Pod Status:kubectl get pods -n nvidia-network-operator — Ensure pods are Running without errors.
• Verify Network CR Status:kubectl get network nvidia-network -o yaml — Confirm status.phase is Ready.
• Inspect RDMA and InfiniBand Devices on Nodes:SSH into nodes and run:ibv_devinfo — Lists InfiniBand devices.rdma link show — Shows RDMA interfaces.
• Check DaemonSets:kubectl get daemonsets -n nvidia-network-operator — Confirm daemonsets managing drivers are active.

Common Troubleshooting Tips

• Ensure kernel modules for InfiniBand and RDMA are loaded on nodes.
• Verify network interfaces support RDMA and are properly configured.
• Check operator logs for errors:kubectl logs -n nvidia-network-operator -l app=nvidia-network-operator

Summary

Deploying the NVIDIA Network Operator for RDMA and InfiniBand involves applying CRDs, deploying the operator, creating a Network CR with RDMA and InfiniBand enabled, and verifying the deployment through pod status and device checks. This process automates the configuration of high-performance networking essential for AI workloads on Kubernetes.

More in this topic

Deploy the NVIDIA Network Operator for RDMA and InfiniBand: Worked Example — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)Deploy the NVIDIA Network Operator for RDMA and InfiniBand — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)Kubernetes Integration — NVIDIA-Certified Professional: AI NetworkingVerify operator functionality — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)Deploy the NVIDIA Network Operator for RDMA and InfiniBand: Common Mistakes — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)Deploy the NVIDIA Network Operator for RDMA and InfiniBand: Practice Questions — Kubernetes Integration (NVIDIA-Certified Professional: AI Networking)