NVIDIA container toolkit installation and GPU use with Docker: Quick Reference — Control Plane Installation and Configuration (NVIDIA-Certified Professional: AI Infrastructure)

NVIDIA Container Toolkit Installation and GPU Use with Docker — Quick Reference This quick reference provides essential facts and steps for...

NVIDIA Container Toolkit Installation and GPU Use with Docker — Quick Reference

This quick reference provides essential facts and steps for installing the NVIDIA Container Toolkit and enabling GPU acceleration within Docker containers, a critical skill for the NVIDIA-Certified Professional: AI Infrastructure certification.

Key Concepts

NVIDIA Container Toolkit: Enables Docker containers to access NVIDIA GPUs on the host system.
GPU Support in Docker: Requires NVIDIA drivers on the host and the container runtime to interface with GPUs.
Docker Runtime: The NVIDIA Container Toolkit configures Docker to use nvidia as a runtime for GPU-enabled containers.

Prerequisites

Host system with supported NVIDIA GPU and installed NVIDIA driver.
Docker Engine installed and running.
Root or sudo privileges for installation steps.

Installation Steps

Set up the package repository: Add the NVIDIA package repositories to your system package manager.
Install the NVIDIA Container Toolkit: Use your package manager to install nvidia-docker2 or the latest nvidia-container-toolkit package.
Configure Docker daemon: Modify /etc/docker/daemon.json to include the NVIDIA runtime:{ "runtimes": { "nvidia": { "path": "nvidia-container-runtime", "runtimeArgs": [] } } }
Restart Docker: Apply changes by restarting the Docker service.

Verifying Installation

Run docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi to check GPU visibility inside a container.
Successful output shows GPU details, driver version, and CUDA version.

Usage Rules and Tips

Use --gpus all flag in docker run to allocate all GPUs to the container.
For selective GPU allocation, specify --gpus 'device=0,1' to assign GPUs 0 and 1.
Ensure container images are built with CUDA support if GPU-accelerated workloads are required.
Keep NVIDIA drivers on the host updated for compatibility with the container toolkit.

Common Commands

docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi — Verify GPU access inside container.
docker info | grep Runtimes — Confirm NVIDIA runtime is registered.
systemctl restart docker — Restart Docker daemon after configuration changes.

Troubleshooting

If nvidia-smi fails inside container, check driver installation on host.
Verify Docker daemon configuration file syntax to avoid startup errors.
Check that the NVIDIA Container Runtime is installed and accessible in system PATH.

Note: This quick reference focuses solely on NVIDIA Container Toolkit installation and GPU usage with Docker as part of Control Plane Installation and Configuration for the NVIDIA-Certified Professional: AI Infrastructure exam.

More in this topic

Ready to test your knowledge?

Put what you've learned into practice with a quick quiz and track your progress.

Test your knowledge →