Administer Run:ai platforms: Worked Example — Administration (NVIDIA-Certified Professional: AI Operations)

{ "title": "Administer Run:ai Platforms — Worked Example for NVIDIA-Certified Professional: AI Operations", "category": "NVIDIA-Certified Professional: AI Operations", "hashtags": "Runai, AIoperations, NVIDIAcertification, clusteradministration, GPUmanagement", "content": "

Administering Run:ai Platforms — A Step-by-Step Worked Example

Run:ai is a key platform for managing AI workloads efficiently on shared GPU infrastructure, making it essential knowledge for the NVIDIA-Certified Professional: AI Operations certification. This worked example demonstrates how to administer a Run:ai platform in a realistic datacenter environment, focusing on user quota management, job scheduling, and resource optimization.

Scenario Overview

Your AI datacenter runs a Run:ai platform managing a Kubernetes cluster with multiple GPU nodes. Users submit AI training jobs that require GPU resources. The goal is to configure user quotas, submit jobs, monitor resource usage, and optimize GPU allocation to maximize cluster utilization.

Step 1: Accessing the Run:ai CLI

Begin by logging into the Run:ai CLI, which interfaces with the Run:ai platform API to manage resources and jobs.

• Open a terminal on the admin workstation.
• Authenticate with Run:ai using your credentials:runai login --username ADMIN_USER --password ADMIN_PASS

Successful login grants access to cluster administration commands.

Step 2: Define User Quotas

To ensure fair resource distribution, set GPU quotas per user or team.

• Check current quotas:runai quota list
• Set a GPU quota for user alice to 4 GPUs:runai quota set alice --gpus 4
• Verify the quota:runai quota get alice

This limits Alice’s concurrent GPU usage, preventing resource monopolization.

Step 3: Submit a Job with Resource Requests

Alice submits a training job requesting 2 GPUs.

• Create a job YAML file train-job.yaml specifying the container image, command, and GPU resources:

apiVersion: batch/v1 kind: Job metadata: name: train-model spec: template: spec: containers: - name: trainer image: ai-training-image:latest command: [\"python\", \"train.py\"] resources: limits: nvidia.com/gpu: 2 restartPolicy: Never

• Submit the job using Run:ai CLI:runai submit -f train-job.yaml --name train-model --gpus 2

Step 4: Monitor Job and Resource Usage

Track job status and GPU utilization to ensure efficient operation.

• Check job status:runai get jobs
• Monitor GPU usage:runai top

This helps detect bottlenecks or idle GPUs.

Step 5: Optimize GPU Allocation with MIG Configuration

If GPUs support Multi-Instance GPU (MIG), configure MIG profiles to partition GPUs for smaller jobs.

• List MIG-enabled GPUs:nvidia-smi -L
• Configure MIG instances for a GPU to create multiple partitions:runai mig create --gpu 0 --profile 1g.5gb
• Assign jobs to MIG instances by specifying the MIG device in job submission.

This increases GPU utilization by allowing multiple smaller jobs to run concurrently on a single physical GPU.

Step 6: Adjust Quotas and Policies as Needed

Based on monitoring, adjust quotas or scheduling policies to balance workload and maximize throughput.

• Increase Alice’s quota if underutilized:runai quota set alice --gpus 6
• Implement priority queues for critical jobs.

Summary

This example illustrated the key administrative tasks for Run:ai platforms: authenticating with the CLI, setting user quotas, submitting GPU resource requests, monitoring job and GPU status, configuring MIG for resource partitioning, and tuning quotas for optimal cluster performance. Mastery of these steps is essential for the Administration domain of the NVIDIA-Certified Professional: AI Operations exam.

For further details, refer to the official Run:ai documentation at https://www.run.ai/docs/.

" }

More in this topic

Administer Run:ai platforms: Practice Questions — Administration (NVIDIA-Certified Professional: AI Operations)Configure Multi-Instance GPU (MIG) — Administration (NVIDIA-Certified Professional: AI Operations)Describe datacenter architecture for AI workloads: Common Mistakes — Administration (NVIDIA-Certified Professional: AI Operations)Describe datacenter architecture for AI workloads — Administration (NVIDIA-Certified Professional: AI Operations)Configure Multi-Instance GPU (MIG): Practice Questions — Administration (NVIDIA-Certified Professional: AI Operations)Administer Slurm clusters: Quick Reference — Administration (NVIDIA-Certified Professional: AI Operations)Administer Run:ai platforms: Common Mistakes — Administration (NVIDIA-Certified Professional: AI Operations)Administer Run:ai platforms — Administration (NVIDIA-Certified Professional: AI Operations)Configure Multi-Instance GPU (MIG): Worked Example — Administration (NVIDIA-Certified Professional: AI Operations)Administer Slurm clusters: Practice Questions — Administration (NVIDIA-Certified Professional: AI Operations)Configure Multi-Instance GPU (MIG): Quick Reference — Administration (NVIDIA-Certified Professional: AI Operations)Administer Slurm clusters: Common Mistakes — Administration (NVIDIA-Certified Professional: AI Operations)Administer Run:ai platforms: Quick Reference — Administration (NVIDIA-Certified Professional: AI Operations)Administer Slurm clusters — Administration (NVIDIA-Certified Professional: AI Operations)Administration — NVIDIA-Certified Professional: AI OperationsDescribe datacenter architecture for AI workloads: Worked Example — Administration (NVIDIA-Certified Professional: AI Operations)Configure Multi-Instance GPU (MIG): Common Mistakes — Administration (NVIDIA-Certified Professional: AI Operations)Describe datacenter architecture for AI workloads: Quick Reference — Administration (NVIDIA-Certified Professional: AI Operations)Administer Slurm clusters: Worked Example — Administration (NVIDIA-Certified Professional: AI Operations)Describe datacenter architecture for AI workloads: Practice Questions — Administration (NVIDIA-Certified Professional: AI Operations)Administer Kubernetes environments — Administration (NVIDIA-Certified Professional: AI Operations)