NCA-GENM NVIDIA Generative AI Multimodal Questions and Answers

GANs are purpose-built generative models: as covered in the previous question, the generator component learns the underlying distribution of a training dataset and produces new synthetic samples that resemble it — new images, audio, or other data types that did not exist in the original dataset but are statistically consistent with it. This generative capability is GAN's defining characteristic and the reason it is the correct answer among the options given, distinguishing it from the other three algorithms, all of which are fundamentally discriminative or unsupervised techniques rather than generative ones.

Decision trees (A) and support vector machines (B) are supervised discriminative algorithms — they learn a decision boundary or a set of rules to classify or predict outputs from inputs, with no mechanism for producing novel data samples resembling a training distribution. K-means clustering (C) is unsupervised but serves a partitioning function, grouping existing data points into clusters based on similarity — it identifies structure in data that already exists rather than synthesizing new data points that didn't exist before.

It's worth noting GANs are one of several generative model families (alongside variational autoencoders and diffusion models, both covered elsewhere in this set) — among the four options presented here, however, GAN is the only one designed for generation at all, making this a comparatively direct elimination once the discriminative-vs-generative distinction is applied.

Reviewer note: Marked answer (C) is factually incorrect — ReLU is well suited to deep networks and specifically helps mitigate vanishing gradients. The genuine, well-established disadvantage is the 'dying ReLU' problem (D).

I need to flag this one as well: the marked answer (C) does not hold up, and stating otherwise would misrepresent a fairly foundational deep learning fact. ReLU (Rectified Linear Unit, f(x) = max(0, x)) is, if anything, particularly well suited to deep neural networks — it was widely adopted specifically *because* it mitigates the vanishing gradient problem that plagued earlier activation functions like sigmoid and tanh in deep architectures: ReLU's gradient is a constant 1 for all positive inputs, rather than the saturating, near-zero gradients that sigmoid/tanh produce for large-magnitude inputs, which allows gradients to propagate more effectively through many layers.

The genuine, well-documented disadvantage of ReLU is option D: the "dying ReLU" problem. Because ReLU's gradient is exactly zero for any negative input, a neuron whose weighted input becomes consistently negative — often due to a large negative gradient update or an unfavorable initialization — will always output zero and will never receive a gradient large enough to recover, effectively "dying" and no longer contributing to learning. This is a real, practically significant issue that motivated variants like Leaky ReLU, Parametric ReLU (PReLU), and ELU, which allow a small non-zero gradient for negative inputs specifically to prevent neurons from dying.

Options A and B are also factually incorrect characterizations of ReLU — it is computationally cheap (a simple thresholding operation, part of its original appeal over sigmoid/tanh) and it specifically helps *avoid* vanishing gradients rather than causing them.

Zero-shot image classification, by definition, requires classifying images into categories the model was never explicitly trained to recognize, with no task-specific labeled examples. CLIP-style models enable this by encoding both images and candidate text labels (e.g., "a photo of a {class}") into a shared embedding space; classification then reduces to a similarity comparison — computing cosine similarity between the image embedding and each candidate text embedding and selecting the closest match. This is the crucial architectural step: without a shared embedding space linking visual and textual semantics, there is no mechanism to generalize to unseen classes using only their names or descriptions.

Option B directly contradicts the "zero-shot" premise — manual labeling of the target dataset is precisely what zero-shot classification is designed to avoid; if labels were being collected for the target classes, the task would be standard supervised classification, not zero-shot. Option A (image enhancement) may marginally help downstream accuracy but is not the crucial, defining step. Option D is incoherent with how CLIP-style zero-shot classification actually works — the textual description of each candidate class is the essential input that makes zero-shot generalization possible; eliminating it would remove the mechanism entirely, not improve it.

Unlike recurrent architectures, which process tokens sequentially and thereby inherently encode order through the sequence of computation, the transformer's self-attention mechanism processes all tokens in parallel and is permutation-invariant by construction — attention scores between tokens do not inherently depend on their position in the sequence. Positional encoding solves this by injecting explicit information about each token's position into its input representation, typically by adding a positional vector (computed via fixed sinusoidal functions in the original "Attention Is All You Need" formulation, or learned as trainable embeddings in many modern variants) to the token's embedding before it enters the attention layers. Without this, "the cat sat on the mat" and "the mat sat on the cat" would be indistinguishable to the self-attention mechanism, since the same set of token embeddings would be processed identically regardless of order.

Semantic meaning (option A) is the role of the token embeddings themselves, learned separately from positional information — the two are combined (typically summed) but serve distinct purposes. Positional encoding does not remove information (C); it adds it. And while attention weights do effectively encode a learned notion of token importance relative to a query (option D), that importance-weighting mechanism is a separate, downstream function of the attention layers, not the role of positional encoding itself, which only supplies order information as an input feature.

Transfer learning takes a model already trained on a large, general-purpose dataset (e.g., ImageNet for vision, or a large text corpus for language models) and adapts it to a new, typically smaller and more specific target task — either by fine-tuning some or all of the pretrained weights, or by freezing the pretrained backbone and training only new task-specific layers on top. Because the pretrained model has already learned general-purpose, reusable features (edge and texture detectors in early CNN layers, syntactic and semantic structure in language model layers), the target task requires substantially less labeled data and less compute than training a comparable model from random initialization.

Prompt engineering (C) is a related but distinct technique specific to large language and generative models: it adapts a *frozen* pretrained model's behavior through the design of the input prompt alone, without any weight updates — a lighter-weight technique than transfer learning, applicable only where a sufficiently capable pretrained model already exists. Options A and D are not standard, well-defined ML techniques matching this description; "state management and composition" and "neural network integration" are generic software-engineering-sounding terms without a specific technical meaning in this context, making them straightforward distractors to eliminate.

Word Error Rate is the standard evaluation metric for ASR systems. It measures the edit distance between the model's transcription and a human reference transcript, computed as (Substitutions + Deletions + Insertions) / Number of reference words, expressed as a percentage. Lower WER indicates better transcription accuracy. Its character-level analogue, Character Error Rate (CER), is used for languages without clear word boundaries or for morphologically complex languages.

The distractors target common confusions: CTC (Connectionist Temporal Classification) Loss (A) is a *training* objective used to align variable-length audio input with variable-length text output in ASR models like DeepSpeech — it optimizes the model but is not itself a post-hoc evaluation metric on held-out accuracy. F1 Score (B) evaluates classification tasks with defined positive/negative classes, such as keyword spotting or wake-word detection, not full transcription. Mean Opinion Score (C) is a subjective, human-rated metric used to evaluate speech *synthesis* quality (TTS) or perceived audio naturalness — the inverse task of ASR — not transcription accuracy.

On NVIDIA's Riva and NeMo ASR pipelines, WER is the benchmark reported against datasets like LibriSpeech, and it remains the figure typically referenced in the exam's Multimodal Data and Experimentation domains when discussing speech model evaluation.

Mixed-precision training performs the bulk of computation — matrix multiplications and convolutions — in a lower-precision floating-point format (typically FP16 or BF16 on NVIDIA Tensor Cores) while maintaining a master copy of weights and accumulating certain sensitive operations (like loss scaling and gradient accumulation) in FP32 to preserve numerical stability. The result is substantially faster training throughput and reduced memory footprint, since lower-precision arithmetic runs at higher effective FLOPS on hardware with dedicated Tensor Cores, without a meaningful loss of final model accuracy when combined with techniques like dynamic loss scaling to prevent gradient underflow.

Note that option A's specific mention of "double-precision" (FP64) is not how mixed precision is practiced in modern deep learning — production mixed-precision training combines FP16/BF16 with FP32, not FP64, since FP64 offers no throughput advantage on Tensor Core hardware and is rarely used in training pipelines. Despite that imprecision in the option's wording, A is still the only choice capturing the correct underlying concept: combining multiple numeric precision levels within one training run. Options B, C, and D all misdescribe mixed precision as a *data-type* or *modality* strategy, confusing numerical precision (a performance/optimization concept) with data modality (a multimodal-data concept) — a distinction the exam tests directly.

Data fusion is the process of combining information from multiple modalities into a single, unified representation that a downstream model can act on. As covered in the early-fusion and late-fusion questions elsewhere in this set, fusion can occur at different pipeline stages — raw/feature-level (early), intermediate representation level (hybrid), or decision level (late) — but in every case the defining operation is combination, not separation.

Option A describes the inverse operation and does not correspond to any standard multimodal technique under the name "fusion." Option C describes missing-data handling or imputation, a data-quality concern that is often addressed *before* fusion (a model needs some representation for each modality, even an imputed or masked one, before combining them) but is not fusion itself. Option D describes evaluation or quality assessment, a distinct concern from the mechanical act of combining modalities into one representation.

Fusion technique choice has real architectural consequences: early fusion assumes tight temporal/spatial correspondence between modalities and is sensitive to missing streams; late fusion is more robust to missing or noisy modalities since each unimodal branch can still contribute independently; hybrid/intermediate fusion, common in modern transformer-based multimodal architectures via cross-attention, aims to capture the benefits of both while mitigating each one's weaknesses.

Batch size has a direct, proportional relationship with VRAM consumption during both training and inference: each sample in a batch requires its own memory allocation for input tensors, intermediate activations at every layer, and output tensors, all of which must reside in GPU memory simultaneously while the batch is being processed. Decreasing the batch size means fewer samples occupy memory concurrently, directly reducing peak VRAM consumption — this is precisely why reducing batch size is one of the first, most common remedies when a model run fails with an out-of-memory (OOM) error on a GPU with limited VRAM.

Option C states the inverse of the correct relationship and is a genuinely important misconception to correct: increasing batch size increases VRAM consumption, not decreases it — parallelism across the batch means more simultaneous memory occupancy, not less. It's true that larger batches improve GPU compute *utilization* and *throughput* (better amortizing fixed kernel-launch overhead and better exploiting parallel hardware) up to the point VRAM allows, but that throughput benefit is a separate effect from, and does not reduce, memory consumption. Options A and B both incorrectly claim batch size is memory-neutral, when it is in fact one of the most direct, easily controlled levers for managing VRAM usage — alongside model precision (quantization, mixed precision) and activation checkpointing, covered in the Performance Optimization domain elsewhere in this set.

Hyperparameters are configuration values set *before* training begins and are not updated by the optimization process itself — learning rate, batch size, number of layers, regularization strength, and number of training epochs are canonical examples. Hyperparameter tuning is the systematic search for the combination of these values that yields the best model performance on a validation set, using strategies such as grid search, random search, or more sample-efficient approaches like Bayesian optimization and population-based training.

This is explicitly distinct from option A, which describes the *training* process itself — weights and biases are trainable parameters, updated automatically via backpropagation and gradient descent, not selected through hyperparameter search. Option B describes algorithm selection, a higher-level modeling decision that may precede hyperparameter tuning but is not what tuning itself accomplishes (you tune hyperparameters *within* a chosen algorithm/architecture). Option C describes data engineering work that happens upstream of model training entirely, unrelated to parameter search.

In practice, hyperparameter tuning requires careful experimental design to avoid overfitting to the validation set — techniques like k-fold cross-validation, held-out test sets, and tracking tools (e.g., experiment trackers logging each trial's configuration and resulting metric) are standard practice, connecting this topic directly to the Experimentation domain's broader emphasis on rigorous, reproducible model evaluation.

A histogram bins a continuous variable into contiguous intervals and plots the frequency (or density) of observations falling into each bin, making it the standard tool for visualizing the shape of a continuous distribution — skewness, modality, spread, and outliers are all immediately visible. This distinguishes it from a bar chart (B), which is designed for discrete or categorical variables where bars are separated and ordering is often arbitrary; applying a bar chart to continuous data loses the notion of a numeric scale between categories.

A line chart (C) is appropriate for showing trends of a variable across an ordered sequence, typically time, not for summarizing the overall shape of a value distribution. A pie chart (D) shows proportions of a whole across categorical segments and becomes visually unreadable and statistically meaningless for continuous data with many possible values.

In practice, histogram bin width is a critical hyperparameter: too few bins oversmooth the distribution and hide multimodality, while too many bins introduce noise. Tools like Freedman-Diaconis or Sturges' rule provide principled starting points, and kernel density estimates (KDE) are often overlaid as a smoothed alternative when bin-width sensitivity is a concern.

CNNs remain the standard architecture for image classification tasks of this kind, for the same structural reasons covered elsewhere in this set: convolutional layers exploit spatial locality and translation invariance in image data, learning hierarchical features — edges and textures in early layers, parts and objects in deeper layers — directly from labeled pixel data, without requiring hand-engineered features. With thousands of labeled cat/dog images, a CNN (trained from scratch or, more efficiently given the modest dataset size, fine-tuned from a pretrained backbone via transfer learning) is the practical, industry-standard choice.

Decision Trees (A) can technically be applied to hand-engineered image features, but they scale poorly to raw high-dimensional pixel input and cannot learn spatial hierarchies the way convolutional architectures do — they're a reasonable choice for structured/tabular data, not raw image classification. K-Means Clustering (B) is unsupervised and would group images by similarity without using the provided labels at all, making it unsuitable for a labeled classification task where you already have ground-truth cat/dog annotations to learn from directly. Linear Regression (D) predicts continuous numeric outputs and is not designed for categorical classification; even logistic regression, its classification-oriented cousin, would struggle on raw pixels without the feature-learning capacity a CNN provides.

This mirrors a nearly identical question earlier in this set (10,000 cats/dogs/birds) — expect the exam to test this CNN-for-images association repeatedly, sometimes with different distractor combinations.

A/B testing in an ML engineering context is a controlled experimental methodology where two (or more) model variants — for example, a current production model (A) and a candidate replacement (B) — are deployed simultaneously to randomly assigned, statistically comparable segments of live traffic, and their real-world performance is compared on business or task-relevant metrics (conversion rate, click-through rate, task accuracy, latency-adjusted engagement). This provides causal evidence of which model performs better under actual production conditions, which offline evaluation on static held-out datasets cannot fully capture, since production data distributions shift and downstream user behavior interacts with model outputs in ways offline metrics miss.

Option A narrows A/B testing incorrectly to UI changes alone; while A/B testing originated in and remains common for UI/UX experimentation, its application in ML engineering explicitly extends to comparing model versions, algorithms, and feature sets — not just interface elements. Option B misattributes a distinct methodology (systematic hyperparameter search, covered in the previous question) to A/B testing, which evaluates already-trained model variants rather than searching a hyperparameter space. Option C is simply incorrect — A/B testing is a cornerstone of responsible ML deployment and MLOps practice, gating rollout decisions before full production release.