Experiment design and execution: Practice Questions — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)

Practice Questions: Experiment Design and Execution for Generative AI LLMs

This set of multiple-choice questions is designed to help candidates prepare for the Experiment Design and Execution component of the NVIDIA-Certified Associate: Generative AI LLM exam. Each question focuses on key concepts such as designing experiments, executing tests, and evaluating model outputs effectively.

1. Which of the following is the most important first step when designing an experiment to evaluate a generative language model's performance on a new task?

   • A. Collecting a large dataset without labels
   • B. Defining clear evaluation metrics aligned with the task objectives
   • C. Running the model on random prompts
   • D. Using data augmentation to increase dataset size

   Correct answer: B

   Explanation: Defining clear evaluation metrics is essential to measure model performance meaningfully and guide experiment design.

2. When executing an experiment to compare two prompt engineering strategies, which practice ensures reliable results?

   • A. Testing each prompt only once on a single input
   • B. Using a fixed random seed and multiple inputs for reproducibility
   • C. Changing model parameters between prompt tests
   • D. Ignoring outlier outputs

   Correct answer: B

   Explanation: Using a fixed random seed and multiple inputs helps ensure that results are reproducible and not due to random variation.

3. Which data augmentation technique is most appropriate for improving a generative AI model's robustness in natural language tasks?

   • A. Rotating images in the dataset
   • B. Synonym replacement and paraphrasing of input text
   • C. Adding Gaussian noise to numerical features
   • D. Increasing batch size during training

   Correct answer: B

   Explanation: Synonym replacement and paraphrasing augment text data to expose the model to varied language expressions, improving robustness.

4. In an experiment comparing model outputs, what is a key reason to use both quantitative metrics and qualitative analysis?

   • A. Quantitative metrics are always sufficient alone
   • B. Qualitative analysis helps interpret nuances not captured by metrics
   • C. Qualitative analysis replaces the need for metrics
   • D. To reduce the time needed for evaluation

   Correct answer: B

   Explanation: Qualitative analysis complements quantitative metrics by providing insights into output quality, relevance, and coherence.

5. Which of the following best describes a controlled experiment in the context of testing prompt variations?

   • A. Changing multiple prompt elements simultaneously to find the best combination
   • B. Changing one prompt element at a time while keeping others constant
   • C. Randomly generating prompts without structure
   • D. Using only one prompt for all tests

   Correct answer: B

   Explanation: Changing one variable at a time allows clear attribution of performance differences to that specific change.

6. What is the primary purpose of using a validation dataset in experiment execution for generative AI models?

   • A. To train the model
   • B. To tune hyperparameters and evaluate model generalization
   • C. To test the model on unseen data after deployment
   • D. To augment the training data

   Correct answer: B

   Explanation: The validation dataset is used during experimentation to tune parameters and assess how well the model generalizes before final testing.

7. When comparing outputs from two generative AI models on the same task, which approach helps ensure a fair comparison?

   • A. Using different prompt styles for each model
   • B. Evaluating outputs with consistent prompts and metrics
   • C. Selecting outputs that favor one model
   • D. Ignoring model size and training data differences

   Correct answer: B

   Explanation: Consistent prompts and evaluation metrics provide an unbiased basis for comparing model performance.

More in this topic

Data augmentation techniques — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Data augmentation techniques: Practice Questions — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Experiment design and execution: Common Mistakes — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Comparing model outputs: Worked Example — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Experiment design and execution: Quick Reference — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Testing model performance across tasks: Common Mistakes — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Comparing model outputs: Practice Questions — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Testing model performance across tasks: Quick Reference — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Experimentation — NVIDIA-Certified Associate: Generative AI LLMPrompt engineering — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Comparing model outputs — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Testing model performance across tasks: Worked Example — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Data augmentation techniques: Worked Example — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Data augmentation techniques: Quick Reference — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Data augmentation techniques: Common Mistakes — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Comparing model outputs: Quick Reference — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Testing model performance across tasks: Practice Questions — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Experiment design and execution — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Testing model performance across tasks — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Comparing model outputs: Common Mistakes — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)Experiment design and execution: Worked Example — Experimentation (NVIDIA-Certified Associate: Generative AI LLM)