Multi-Instance GPU (MIG) configuration for AI and HPC: Worked Example — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)

Multi-Instance GPU (MIG) Configuration for AI and HPC: Worked Example

The Multi-Instance GPU (MIG) capability enables partitioning of a single NVIDIA A100 or later GPU into multiple isolated GPU instances. This feature is critical for optimizing resource utilization in AI and High Performance Computing (HPC) environments by allowing concurrent workloads to run independently on separate GPU instances.

Scenario Overview

Consider a data center deploying an NVIDIA A100 GPU to serve multiple AI training jobs and HPC simulations simultaneously. The goal is to configure the GPU using MIG to create isolated instances that meet the resource requirements of each workload, maximizing throughput and minimizing contention.

Step 1: Verify GPU Compatibility and Prerequisites

• Confirm the GPU model supports MIG (e.g., NVIDIA A100, A30).
• Ensure the system is running a supported NVIDIA driver version (minimum 450.xx series).
• Install NVIDIA Management Library (NVML) and NVIDIA System Management Interface (nvidia-smi) tools.

Step 2: Enable MIG Mode on the GPU

Before configuring MIG instances, the GPU must be switched to MIG mode:

• Run nvidia-smi -i 0 -mig 1 to enable MIG mode on GPU 0.
• Reboot the system or reset the GPU to apply changes.

Step 3: Determine MIG Instance Profiles

MIG supports different instance profiles, each allocating a specific fraction of GPU resources (compute units, memory, bandwidth). For example:

• 1g.5gb: 1 GPU instance with 5 GB memory.
• 2g.10gb: 2 GPU instances, each with 10 GB memory.

Assess workload requirements:

• AI training job A requires a medium instance (e.g., 2g.10gb).
• HPC job B requires a smaller instance (e.g., 1g.5gb).

Step 4: Create MIG Instances

Use nvidia-smi to create instances:

• List available profiles: nvidia-smi mig -lgip
• Create instances according to profiles:

Worked Example

Command:

• Create a 2g.10gb instance: nvidia-smi mig -i 0 -cgi 19 -C
• Create a 1g.5gb instance: nvidia-smi mig -i 0 -cgi 10 -C

Explanation: The -cgi flag specifies the GPU instance profile ID, which can be found from nvidia-smi mig -lgip. The -C flag commits the creation.

Step 5: Verify MIG Instances

Check the created instances:

• Run nvidia-smi mig -i 0 -lgi to list GPU instances.
• Confirm the instances match the desired profiles and are active.

Step 6: Assign Workloads to MIG Instances

Each MIG instance appears as a separate GPU device to the operating system and container runtimes. Assign AI and HPC workloads to the corresponding MIG device IDs to ensure isolation.

Step 7: Monitor and Manage MIG Instances

Use nvidia-smi and NVML APIs to monitor utilization, memory usage, and health of each MIG instance. Adjust instance configurations as workload demands evolve.

Summary

This worked example demonstrates the step-by-step process to configure Multi-Instance GPU (MIG) on an NVIDIA A100 GPU for concurrent AI and HPC workloads. Proper MIG configuration enables efficient resource sharing and workload isolation, essential for scalable AI infrastructure deployments.

More in this topic

Multi-Instance GPU (MIG) configuration for AI and HPC: Practice Questions — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC: Quick Reference — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC: Common Mistakes — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Physical Layer Management — NVIDIA-Certified Professional: AI InfrastructureBlueField network platform configuration and management — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)Multi-Instance GPU (MIG) configuration for AI and HPC — Physical Layer Management (NVIDIA-Certified Professional: AI Infrastructure)