Multi-Instance GPU (MIG) configuration for AI and HPC: Quick Reference — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)

Multi-Instance GPU (MIG) Configuration for AI and HPC — Quick Reference

The Multi-Instance GPU (MIG) technology enables partitioning of a single NVIDIA A100 or later GPU into multiple isolated GPU instances. This capability is critical for optimizing resource utilization in AI and High-Performance Computing (HPC) workloads.

Key Concepts

• MIG Instance: A partitioned GPU slice with dedicated compute, memory, and bandwidth resources.
• GPU Partitioning: Dividing a physical GPU into multiple MIG instances to run concurrent workloads.
• Use Cases: Enables workload isolation, improved utilization, and flexible resource allocation for AI training, inference, and HPC tasks.

Supported GPUs

• NVIDIA A100 and newer architectures support MIG.
• Each GPU can be partitioned into up to 7 MIG instances depending on the configuration.

MIG Profiles

MIG instances are created based on predefined profiles that specify the number of compute units, memory size, and bandwidth.

• Profiles range from small (e.g., 1g.5gb) to large (e.g., 7g.40gb) instances.
• Each profile defines the number of GPU compute slices and memory allocation.

Configuration Steps

1. Enable MIG Mode: Use nvidia-smi to enable MIG mode on the GPU:nvidia-smi -i [GPU_ID] -mig 1
2. Create MIG Instances: Define instances using profiles:nvidia-smi mig -i [GPU_ID] -cgi [Profile_ID] -C
3. Verify Instances: Check created instances:nvidia-smi mig -i [GPU_ID] -lgi
4. Assign Workloads: Use instance UUIDs to assign workloads to specific MIG instances.

Management Commands

• nvidia-smi -i [GPU_ID] -mig 1: Enable MIG mode.
• nvidia-smi mig -i [GPU_ID] -cgi [Profile_ID] -C: Create instances.
• nvidia-smi mig -i [GPU_ID] -dgi: Delete all instances.
• nvidia-smi mig -i [GPU_ID] -lgi: List instances.

Best Practices

• Plan instance sizes based on workload requirements to maximize GPU utilization.
• Use smaller MIG instances for inference and larger ones for training or HPC tasks.
• Monitor GPU and MIG instance health regularly using nvidia-smi and telemetry tools.
• Ensure driver and CUDA toolkit versions support MIG features.

Worked Example

Task: Configure a single A100 GPU into three MIG instances: one 3g.20gb and two 1g.5gb profiles.

Solution:

1. Enable MIG mode:nvidia-smi -i 0 -mig 1
2. Create the 3g.20gb instance:nvidia-smi mig -i 0 -cgi 19 -C
3. Create two 1g.5gb instances:nvidia-smi mig -i 0 -cgi 1 -C (run twice)
4. Verify instances:nvidia-smi mig -i 0 -lgi

For detailed NVIDIA documentation on MIG configuration, visit NVIDIA MIG User Guide.

More in this topic

Multi-Instance GPU (MIG) configuration for AI and HPC: Practice Questions — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC: Worked Example — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC: Common Mistakes — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Physical Layer Management — NVIDIA-Certified Professional: AI InfrastructureBlueField network platform configuration and management — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)