NCP-AIN NVIDIA-Certified Professional AI Networking Questions and Answers

EVPN Multi-homing enables active-active redundancy without inter-switch links by using overlay routing over VXLAN and distributed control plane using BGP EVPN.

From the official NVIDIA Cumulus Linux EVPN Multihoming Documentation :

"EVPN multihoming allows multiple Top-of-Rack (ToR) switches to connect to the same server while maintaining full layer-2 redundancy without the need for inter-switch links or traditional MLAG configuration."

Key benefits:

Simplified topology (no ISL/peer-link needed)

BGP-based control plane

Fast convergence

Active-active links per host NIC

Incorrect Options:

MLAG requires ISL between switches and peer-link configuration.

VSS (Virtual Switching System) is a Cisco term, not supported in NVIDIA networking.

To upgrade the MLNX-OS on InfiniBand switches, the recommended procedure is as follows:​

Connect to the switch via SSH : Establish a secure shell connection to the switch using its management IP address.​

Fetch the MLNX-OS software image : Obtain the appropriate MLNX-OS software image from the official source or repository.​

Use the 'install' command to perform the upgrade : Execute the 'install' command on the switch to initiate the upgrade process with the fetched software image.​

This method ensures a smooth and efficient upgrade without the need for physical intervention or service disruption.​

Reference Extracts from NVIDIA Documentation:

"Click on Systems → MLNX-OS Upgrade. Select the desired upgrade method (e.g. 'Install from local file'). Select your image and click 'Install Image'."

NVIDIA NetQ is a network operations tool that provides real-time visibility and automated validation of the network fabric. It helps in identifying misconfigurations, monitoring network health, and ensuring that the fabric meets the required specifications for AI workloads.​

The correct answer is PXN (Peer eXchange Network) .

From the NVIDIA NCCL Documentation :

"PXN enables communication between GPUs connected via NVLink and NICs by treating the GPUs as a distributed switch. This architecture improves bandwidth utilization by enabling any GPU to communicate with the NIC via the shortest path available, even if it's not directly connected to the NIC."

This enhances GPU-to-NIC and NIC-to-GPU transfers, leveraging the NVLink topology. It significantly boosts performance in multi-GPU setups where not every GPU is directly connected to the NIC.

Other options:

Adaptive Routing is a fabric-level feature for dynamic path rerouting.

Graph Search Optimization is used internally for topology modeling in NCCL.

SHARP v2 is a switch-based collective acceleration method, unrelated to PXN.

To enable RDMA over Converged Ethernet (RoCE) on a Cumulus switch, the correct command is:​

nv set qos roce enable on

This command configures the Quality of Service (QoS) settings to support RoCE, ensuring that the necessary parameters for lossless Ethernet are applied.​

Within the Spectrum-X platform, the NVIDIA BlueField-3 SuperNIC is responsible for reordering out-of-order packets. When RoCE adaptive routing is employed, packets may arrive at their destination out of order due to dynamic path selection. The BlueField-3 SuperNIC handles this by reassembling the packets in the correct order at the transport layer, ensuring that the application receives data seamlessly.​

Reference Extracts from NVIDIA Documentation:

"As different packets of the same flow travel through different paths of the network, they may arrive out of order to their destination. At the RoCE transport layer, the BlueField-3 DPU takes care of the out-of-order packets and forwards the data to the application in order."​

"The BlueField-3 SuperNIC offers adaptive routing, out-of-order packet handling and optimized congestion control."​

The NVIDIA Spectrum-X networking platform is an Ethernet-based solution optimized for AI workloads, combining Spectrum-4 switches, BlueField-3 SuperNICs, and software like DOCA and NetQ to deliver high performance, low latency, and efficient data transfer. A key feature of Spectrum-X is its adaptive routing, which dynamically selects the least-congested paths for packet transmission to maximize bandwidth and minimize latency. However, this per-packet load balancing can result in packets arriving out of order at the destination, necessitating a mechanism to reorder them for seamless application performance. The question asks which Spectrum-X component is responsible for reordering these out-of-order packets.

According to NVIDIA’s official documentation, the BlueField-3 SuperNIC is the component responsible for reordering out-of-order packets in the Spectrum-X platform. The SuperNIC, a network accelerator designed for hyperscale AI workloads, handles packet reordering at the RDMA over Converged Ethernet (RoCE) transport layer. It uses its processing capabilities to transparently reorder packets and place them in the correct sequence in the host memory, ensuring that adaptive routing’s out-of-order delivery is invisible to the application. This is critical for maintaining predictable performance in AI workloads, particularly for GPU-to-GPU communication in Spectrum-X networks.

Exact Extract from NVIDIA Documentation:

“The Spectrum-4 switches are responsible for selecting the least-congested port for data transmission on a per-packet basis. As different packets of the same flow travel through different paths of the network, they may arrive out of order to their destination. The BlueField-3 SuperNIC transforms any out-of-order data at the RoCE transport layer, transparently delivering in-order data to the application.”

— NVIDIA Technical Blog: Turbocharging Generative AI Workloads with NVIDIA Spectrum-X Networking Platform

This extract confirms that option A, the SuperNIC (specifically the BlueField-3 SuperNIC), is the correct answer. The SuperNIC’s role in reordering packets ensures that the adaptive routing implemented by Spectrum-4 switches does not compromise application performance, maintaining high effective bandwidth and low tail latency for AI workloads.

NVIDIA NetQ is a comprehensive network operations tool designed to provide real-time visibility into the health and performance of NVIDIA networking environments, including Spectrum-X. It offers detailed telemetry and analytics, allowing administrators to monitor adaptive routing behaviors, detect congestion, and analyze traffic patterns. By leveraging NetQ, you can ensure that adaptive routing is functioning as intended and that the network is optimized for AI workloads.​

Reference Extracts from NVIDIA Documentation:

"The NVIDIA NetQ network validation and ASIC monitoring tool set provide visibility into the network health and behavior. The NetQ flow telemetry analysis shows the paths that data flows take as they traverse the network, providing network latency and performance insights."​

"By leveraging telemetry from Spectrum Ethernet switches and BlueField-3 SuperNICs, NVIDIA NetQ can detect network issues proactively and troubleshoot network issues faster for optimal use of network capacity."​

To identify the active Subnet Manager (SM) node in an InfiniBand fabric, the correct command sequence is:

sminfo

Displays general information about the active SM in the fabric, including its LID .

smpquery ND < LID >

Resolves the Node Description (ND) at the given LID, revealing the exact hostname or label of the SM server.

From the InfiniBand Tools Guide:

"The sminfo utility provides the LID of the master SM. Use smpquery ND < LID > to resolve the node name hosting the SM."

This two-step approach is standard for locating and validating the SM identity in fabric diagnostics.

Incorrect Options:

B (Nl) is an invalid query type.

C and D do not identify SMs.

Modern AI training (especially with LLMs) requires extremely high-speed, parallel access to large datasets. A dedicated storage fabric separates data I/O traffic from the training compute path and avoids contention.

From NVIDIA DGX Infrastructure Reference Architectures :

"Dedicated storage networks eliminate I/O bottlenecks by providing low-latency, high-bandwidth access to distributed storage for large-scale training jobs."

"Parallel access to datasets is key for performance, especially in multi-node, multi-GPU AI clusters."

Security (B) is important, but not the core reason for a storage fabric.

Cost (D) is typically increased , not reduced, with dedicated fabrics.

The NVIDIA Spectrum-X Reference Architecture (RA) 1.0.1 is designed for Ethernet AI cloud deployments and includes the SN5600 Spectrum-4 switches and BlueField-3 SuperNICs. This architecture supports adaptive routing and DOCA programmable congestion control (PCC) for lossless RoCE traffic, optimizing performance for AI workloads.

The SN5600 switch offers 64 ports of 800GbE in a dense 2U form factor, providing high throughput and low latency essential for AI applications.

Adaptive Routing in InfiniBand networks dynamically selects the optimal path for data packets based on current network conditions, such as congestion levels and link utilization. This approach ensures that traffic is evenly distributed across the network, preventing bottlenecks and maximizing overall throughput.​

By continuously monitoring the network and adjusting routes in real-time, Adaptive Routing enhances performance and reliability, making it the preferred choice for high-performance computing environments where consistent low latency and high bandwidth are critical.​

NVIDIA Air is a cloud-based network simulation tool designed to create digital twins of data center infrastructure, including Spectrum-X networks. It allows users to model switches, SuperNICs, and storage components, enabling the simulation, validation, and automation of network configurations before physical deployment. This facilitates Day 0, 1, and 2 operations, ensuring that network designs are tested and optimized for AI workloads.​

Reference Extracts from NVIDIA Documentation:

"NVIDIA Air enables cloud-scale efficiency by creating identical replicas of real-world data center infrastructure deployments."​

"NVIDIA Air allows users to model data center deployments with full software functionality, creating a digital twin. Transform and accelerate time to AI by simulating, validating, and automating changes and updates."​

"NVIDIA Air supports simulation of NVIDIA Spectrum Ethernet (Cumulus Linux and SONiC) switches and NVIDIA BlueField DPUs and SuperNICs as well as the NetQ network operations toolset."​

The mlxlink tool is used to check and debug link status and issues related to them. The tool can be used on different links and cables (passive, active, transceiver, and backplane). It is intended for advanced users with appropriate technical background.​

Comprehensive and Detailed 150 to 250 words of Explanation From NVIDIA AI Networking Topics:

The correct answer is A. Cyber-AI platform . NVIDIA’s official UFM Cyber-AI documentation states that the UFM Cyber-AI platform determines a data center’s unique “vital signs” and uses them to identify performance degradation, component failures, and abnormal usage patterns . This directly matches the requirement for proactive detection of issues in a large InfiniBand fabric. NVIDIA also documents that Cyber-AI provides Link Anomaly detection and Link Failure Prediction , where machine learning models analyze monitoring information and predict future link failures 1 to 24 hours in advance . In addition, NVIDIA’s Cyber-AI materials describe integration with AI cybersecurity capabilities to help detect and mitigate security threats , which makes Cyber-AI the only option in this list that combines predictive analytics with security-oriented intelligence. By contrast, UFM Telemetry focuses on collecting and presenting real-time network data such as bandwidth, congestion, latency, and errors, while UFM Enterprise is the broader management platform and Host Agent is not the AI analytics engine for predictive failure and threat detection. Therefore, for AI-driven analytics, predictive maintenance, anomaly detection, and security-focused analysis in InfiniBand environments, Cyber-AI platform is the verified NVIDIA answer.

To perform Flow Analysis with NetQ, the following prerequisites must be met:​

Cumulus Linux Version : NetQ Flow Analysis requires Cumulus Linux 5.x or later.​

Switch Hardware : The feature is supported on Spectrum-2 and later switch models.​

Lifecycle Management (LCM) : LCM must be enabled to utilize Flow Analysis capabilities.​

These requirements ensure compatibility and proper functioning of the Flow Analysis feature within NetQ.​

The ibstat command is utilized to display the operational status of InfiniBand Host Channel Adapters (HCAs). It provides detailed information, including the state (e.g., Active, Down), physical state (e.g., LinkUp, Polling), and link layer (e.g., InfiniBand, Ethernet) of each port on the HCA. This information is crucial for diagnosing connectivity issues and ensuring that the InfiniBand interfaces are functioning correctly.​

Reference Extracts from NVIDIA Documentation:

"The ibstat command displays the status of the host channel adapters (HCAs) in your InfiniBand fabric. The status includes the HCAs' state, physical state, and link layer."​

"For proper operation, you are looking for ‘State: Active‘ and ‘Physical State: LinkUp’."​

To enable SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) aggregation using the NCCL-SHARP plugin , the following two environment variables are required:

NCCL_COLLNET_ENABLE=1

Enables NCCL’s support for CollNet (Collective Network) operations, including SHARP.

NCCL_SHARP_AUTOINIT=1

Automatically initializes the SHARP plugin when available, activating SHARP-based collectives.

From the NVIDIA NCCL User Guide – SHARP Plugin Section :

"NCCL_COLLNET_ENABLE must be set to enable collective network acceleration features."

"NCCL_SHARP_AUTOINIT enables automatic SHARP plugin integration at NCCL runtime."

Incorrect Options:

B. NCCL_ALGO=CollNet – This variable controls the algorithm used for collectives but does not enable SHARP.

C. NCCLSPECTRUM_ENABLE – This is not a documented NCCL variable.

The NVIDIA UFM Cyber-AI Platform is specifically designed to enhance security and operational efficiency in InfiniBand data centers. It leverages AI-powered analytics to detect security threats, operational anomalies, and predict potential network failures. By analyzing real-time telemetry data, it identifies abnormal behaviors and performance degradation, enabling proactive maintenance and threat mitigation.​

This platform integrates with existing UFM Enterprise and Telemetry services to provide a comprehensive view of the network's health and security posture. It utilizes machine learning algorithms to establish baselines for normal operations and detect deviations that may indicate security breaches or hardware issues.​