Server performance optimization: Quick Reference — Troubleshoot and Optimize (NVIDIA-Certified Professional: AI Infrastructure)

Server Performance Optimization – Quick Reference This quick reference guide covers essential facts and best practices for optimizing server...

Server Performance Optimization – Quick Reference

This quick reference guide covers essential facts and best practices for optimizing server performance within NVIDIA AI Infrastructure environments, a key focus area in the NVIDIA-Certified Professional: AI Infrastructure certification.

Key Concepts

Performance Bottlenecks: Identify CPU, GPU, memory, storage, and network bottlenecks that limit throughput or increase latency.
Resource Utilization: Monitor utilization metrics to ensure balanced workloads and avoid over- or under-utilization.
Thermal and Power Management: Maintain optimal operating temperatures and power delivery to prevent throttling.

Common Performance Metrics

CPU Utilization: Percentage of CPU capacity in use; sustained high usage may indicate compute bottlenecks.
GPU Utilization: Percentage of GPU compute resources actively engaged; low utilization can signal configuration or workload issues.
Memory Bandwidth and Usage: Monitor for memory saturation or leaks impacting performance.
I/O Throughput: Disk and network throughput rates affecting data transfer speeds.
Latency: Time delays in processing or data movement affecting responsiveness.

Optimization Techniques

Load Balancing: Distribute workloads evenly across CPUs and GPUs to maximize parallelism.
Firmware and Driver Updates: Keep system firmware, NVIDIA drivers, and software stacks up to date for performance improvements and bug fixes.
BIOS and System Settings: Configure BIOS options such as power management, NUMA, and PCIe settings for optimal throughput.
GPU Affinity: Assign processes to specific GPUs to reduce contention and improve cache locality.
Memory Configuration: Use appropriate memory channels and optimize swap usage to minimize latency.
Storage Optimization: Employ fast NVMe drives and configure RAID or parallel file systems for high throughput.
Network Tuning: Optimize network interfaces and protocols to reduce bottlenecks in distributed AI workloads.

Monitoring and Tools

nvidia-smi: Command-line utility for real-time GPU monitoring and management.
DCGM (Data Center GPU Manager): NVIDIA tool for comprehensive GPU health and performance monitoring.
System Performance Tools: Use tools like top, htop, iostat, and netstat for CPU, disk, and network metrics.
Profilers: NVIDIA Nsight Systems and Nsight Compute for detailed GPU workload analysis.

Best Practices

Regularly benchmark server performance under typical AI workloads to detect regressions.
Document baseline metrics to compare after configuration changes.
Automate monitoring and alerting to quickly identify performance degradation.
Collaborate with infrastructure and application teams to align optimization efforts.

Worked Example

Scenario: A server running AI training jobs shows low GPU utilization despite high CPU usage.

Steps to Optimize:

Use nvidia-smi to confirm GPU utilization is below expected levels.
Check CPU usage with htop to identify potential CPU bottlenecks.
Review process affinity and ensure training jobs are correctly assigned to GPUs.
Update NVIDIA drivers and firmware to the latest stable versions.
Adjust BIOS settings to enable PCIe Gen4 for maximum bandwidth.
Rebalance workloads or increase batch sizes to better utilize GPU compute.
Monitor changes and verify GPU utilization improves while CPU load balances.

More in this topic

Ready to test your knowledge?

Put what you've learned into practice with a quick quiz and track your progress.

Test your knowledge →