NVIDIA Quantum-3 InfiniBand Switch

Overview#

Quantum-3 is the InfiniBand switch generation that arrived with Blackwell. It doubles per-port bandwidth from 400 to 800 Gb/s, extends in-network reduction with SHARPv4, and adds a high-radix Director variant aimed specifically at fabrics that need to span tens of thousands of GPUs in two or three tiers rather than four.

The platform was announced at GTC 2024 and is the reference fabric for the GB200 NVL72 and the GB300 NVL72 designs. It does not replace Quantum-2 in the field — most NDR fabrics remain in production — but is the default for new XDR-era builds.

Specifications#

SHARPv4: What Changed#

SHARPv4 raises the practical ceiling on in-network reduction in three ways: it supports lower-precision data types (BF16, FP8) common in modern training; it builds deeper reduction trees in hardware, which matters as cluster sizes pass ~4,000 endpoints; and it interleaves multiple reduction streams without blocking, so concurrent collectives from different jobs in a multi-tenant pod do not serialise.

For mixture-of-experts and other models with very large all-to-all stages, the deeper SHARPv4 tree is what keeps the collective from dominating step time.

Operational Notes#

• Director chassis can collapse a three-tier fat tree into two tiers for clusters under ~16k endpoints — fewer cables, lower latency, simpler cabling plan.
• Linear pluggable optics (LPO) are commonly used to reduce power and cost on intra-rack runs.
• UFM 6.x recommended for telemetry and topology management at XDR scale.
• Quantum-3 is interoperable with NDR endpoints but caps at NDR rates on those links — segregate where possible.