NVIDIA container toolkit installation and GPU use with Docker: Worked Example — Control Plane Installation and Configuration (NVIDIA-Certified Professional: AI Infrastructure)

NVIDIA Container Toolkit Installation and GPU Use with Docker: Worked Example In the NVIDIA-Certified Professional: AI Infrastructure certification...

NVIDIA Container Toolkit Installation and GPU Use with Docker: Worked Example

In the NVIDIA-Certified Professional: AI Infrastructure certification, understanding how to install the NVIDIA Container Toolkit and enable GPU usage within Docker containers is essential. This worked example guides you through a realistic scenario where you prepare an AI infrastructure node to run GPU-accelerated Docker containers.

Scenario

You are tasked with configuring a Linux server equipped with NVIDIA GPUs to run AI workloads inside Docker containers. Your goal is to install the NVIDIA Container Toolkit, verify GPU driver compatibility, and run a test container that utilizes the GPU.

Step 1: Verify GPU Driver Installation

Before installing the container toolkit, ensure that the NVIDIA GPU drivers are correctly installed and the GPUs are recognized by the system.

Run nvidia-smi to check driver status and GPU availability.
Example output should list GPUs with their model and driver version.

Example

Command: nvidia-smi

Expected output snippet:

Step 2: Install Docker Engine

If Docker is not installed, install it following official Docker documentation. Verify installation with:

docker --version

Step 3: Add NVIDIA Package Repositories

Set up the package repositories for the NVIDIA Container Toolkit.

For Ubuntu, run:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update

Step 4: Install NVIDIA Container Toolkit

Install the toolkit package:

sudo apt-get install -y nvidia-docker2

After installation, restart the Docker daemon to apply changes:

sudo systemctl restart docker

Step 5: Configure Docker to Use NVIDIA Runtime

Verify that the Docker daemon configuration includes the NVIDIA runtime. Check /etc/docker/daemon.json for the following snippet:

{ "runtimes": { "nvidia": { "path": "nvidia-container-runtime", "runtimeArgs": [] } } }

If missing, add it and restart Docker again.

Step 6: Test GPU Access Inside a Docker Container

Run a test container to verify GPU visibility:

docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi

This command pulls the NVIDIA CUDA base image and runs nvidia-smi inside the container.

Expected Output

The output should mirror the host's nvidia-smi output, confirming GPU access:

Step 7: Run a GPU-Accelerated AI Workload Container

To further validate, run an AI framework container such as TensorFlow with GPU support:

docker run --rm --gpus all tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

The output should list available GPUs, confirming successful GPU utilization inside Docker.

Summary

Verified NVIDIA GPU driver installation with nvidia-smi.
Installed Docker and added NVIDIA package repositories.
Installed NVIDIA Container Toolkit and configured Docker to use the NVIDIA runtime.
Tested GPU access inside Docker containers using nvidia/cuda and TensorFlow images.

This step-by-step process is critical for deploying GPU-accelerated AI workloads in containerized environments, a key skill for the NVIDIA-Certified Professional: AI Infrastructure exam.

More in this topic

Ready to test your knowledge?

Put what you've learned into practice with a quick quiz and track your progress.

Test your knowledge →