Memory and batch optimization: Practice Questions — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)

Memory and Batch Optimization Practice Questions

These practice questions focus on memory and batch optimization techniques essential for efficient training of large language models (LLMs) using GPU acceleration. Understanding these concepts is critical for the NVIDIA-Certified Professional: Generative AI LLMs exam.

1. Which of the following batch size adjustments is most likely to reduce GPU memory usage during LLM training?

   • A. Increasing the batch size
   • B. Decreasing the batch size
   • C. Keeping the batch size constant but increasing sequence length
   • D. Increasing the number of model parameters

   Correct Answer: B

   Explanation: Decreasing the batch size reduces the number of samples processed simultaneously, directly lowering GPU memory consumption.

2. What is the primary benefit of gradient accumulation in the context of memory optimization?

   • A. It allows training with larger effective batch sizes without increasing memory usage
   • B. It reduces the number of GPUs required
   • C. It increases the learning rate automatically
   • D. It decreases model parameter size

   Correct Answer: A

   Explanation: Gradient accumulation splits a large batch into smaller micro-batches processed sequentially, enabling larger effective batch sizes without exceeding GPU memory limits.

3. Which memory optimization technique involves freeing intermediate activations during backpropagation and recomputing them as needed?

   • A. Mixed precision training
   • B. Activation checkpointing
   • C. Batch normalization
   • D. Data parallelism

   Correct Answer: B

   Explanation: Activation checkpointing saves memory by storing fewer activations and recomputing them during the backward pass, trading computation for reduced memory usage.

4. When optimizing batch size for a fixed GPU memory budget, which factor should be considered to maximize throughput without causing out-of-memory errors?

   • A. Model architecture complexity
   • B. Sequence length of input tokens
   • C. Number of GPUs in the system
   • D. Learning rate schedule

   Correct Answer: B

   Explanation: Longer sequence lengths increase memory requirements per sample, so batch size must be adjusted accordingly to fit within GPU memory.

5. Which of the following is a direct consequence of using mixed precision training for memory optimization?

   • A. Increased memory consumption due to larger data types
   • B. Reduced memory usage and faster computation
   • C. Requirement to reduce batch size
   • D. Necessity to disable gradient checkpointing

   Correct Answer: B

   Explanation: Mixed precision training uses lower-precision data types (e.g., FP16) which reduces memory usage and can accelerate computation on compatible GPUs.

6. In a multi-GPU setup, which strategy helps optimize memory usage by splitting the batch across GPUs?

   • A. Model parallelism
   • B. Data parallelism
   • C. Pipeline parallelism
   • D. Activation checkpointing

   Correct Answer: B

   Explanation: Data parallelism divides the batch across multiple GPUs, reducing per-GPU memory load and enabling larger total batch sizes.

7. Which tool or technique is most appropriate for identifying memory bottlenecks during LLM training?

   • A. Performance profiling with NVIDIA Nsight Systems
   • B. Increasing batch size without monitoring
   • C. Disabling mixed precision training
   • D. Using fixed batch sizes regardless of model size

   Correct Answer: A

   Explanation: NVIDIA Nsight Systems provides detailed profiling to identify memory usage patterns and bottlenecks, enabling targeted optimization.

More in this topic

Memory and batch optimization — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Parallelism techniques: Common Mistakes — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Parallelism techniques: Practice Questions — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Performance profiling and troubleshooting: Quick Reference — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Multi-GPU and distributed setups: Quick Reference — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Multi-GPU and distributed setups: Worked Example — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Parallelism techniques: Quick Reference — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Performance profiling and troubleshooting: Common Mistakes — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Memory and batch optimization: Quick Reference — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Multi-GPU and distributed setups — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Performance profiling and troubleshooting — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)GPU Acceleration and Optimization — NVIDIA-Certified Professional: Generative AI LLMsMulti-GPU and distributed setups: Common Mistakes — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Memory and batch optimization: Common Mistakes — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Performance profiling and troubleshooting: Worked Example — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Parallelism techniques: Worked Example — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Parallelism techniques — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Performance profiling and troubleshooting: Practice Questions — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Multi-GPU and distributed setups: Practice Questions — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)Memory and batch optimization: Worked Example — GPU Acceleration and Optimization (NVIDIA-Certified Professional: Generative AI LLMs)