TL;DR
- 400G Ethernet was standardised by IEEE 802.3bs (2017) and 802.3cd (2018), defining 400GBASE-R as 8 lanes of 50 Gb/s or 4 lanes of 100 Gb/s PAM4.
- Dominant access-layer fabric in AI clusters from 2021-2025, and continuing as a storage and out-of-band management fabric in 800G-era builds.
- Switch ASICs include Broadcom Tomahawk 4, NVIDIA Spectrum-3/4, Cisco Silicon One Q200, Marvell Teralynx 8.
- Connectors: QSFP-DD and OSFP; transceivers include 400G-DR4, 400G-FR4, 400G-LR4, plus DACs and AOCs for intra-rack.
Overview#
400 Gigabit Ethernet was the first PAM4-era Ethernet generation to see broad enterprise and AI-cluster adoption. Standardised by IEEE 802.3bs in 2017 with additional PMDs added by 802.3cd in 2018, it forms the backbone of nearly every commodity data centre network deployed from 2020 onward and remains the dominant access-layer technology in mixed-fabric AI builds.
Within AI infrastructure, 400G commonly sits as the storage fabric (NVMe-oF, object storage) and management/access fabric, while a separate 400G/800G InfiniBand or Spectrum-X fabric carries GPU-to-GPU traffic. In smaller pods, a single 400G RoCEv2 fabric can carry both.
Specifications#
| Property | Value |
|---|---|
| Standard | IEEE 802.3bs (2017), 802.3cd (2018) |
| Per-port line rate | 400 Gb/s |
| PHY options | 8×50G PAM4 or 4×100G PAM4 |
| FEC | RS(544,514) KP4 |
| Connectors | QSFP-DD, OSFP |
| Common transceivers | 400G-DR4, FR4, LR4, ZR/ZR+ |
| Switch ASICs | Tomahawk 3/4, Spectrum-3/4, Silicon One Q200, Teralynx 8 |
Role in AI Clusters#
By mid-2026 most new training fabrics target 800G, but 400G remains pervasive: smaller training pods still ship with 400G InfiniBand (NDR) or 400G RoCEv2 because the per-port cost is meaningfully lower than 800G and the bandwidth is adequate for sub-thousand-GPU jobs.
On the storage side, 400G is more than sufficient for NVMe-oF arrays and object-store front-ends, where the bottleneck is media rather than fabric. Many production sites pair an 800G GPU fabric with a 400G storage fabric and a 100G/25G management fabric.
Operational Notes#
- DAC reach at 400G is typically 3 m or less; AOCs extend to ~30 m without optics costs.
- QSFP-DD and OSFP are mechanically incompatible — standardise per pod to avoid stock-keeping pain.
- RoCEv2 over 400G requires PFC and ECN tuning; without them, AllReduce throughput collapses under congestion.
- Breakout: a 400G port can split into 4×100G or 2×200G, simplifying mixed-speed access tiers.
References
- IEEE 802.3bs-2017 Standard · IEEE
- IEEE 802.3cd-2018 Standard · IEEE
- Ethernet Alliance Roadmap · Ethernet Alliance