TL;DR
- Rack density is the IT load drawn by a single rack, measured in kilowatts. It dictates every other engineering choice: cooling method, power feed type, floor loading, and ceiling clearance.
- Density classes broadly: low (≤ 10 kW, traditional air), medium (10-30 kW, air with containment + RDHx), high (30-60 kW, RDHx mandatory, DLC common), extreme (60 kW+, DLC required, immersion for top end).
- Average rack density has roughly doubled per decade since 2000; AI workloads accelerated the trend, with frontier training clusters now landing at 100-130 kW per rack.
- The 'right' density is a function of workload — AI training rewards extreme density (interconnect latency); general-purpose compute may be cheapest at medium density.
Density Classes in 2026#
| Class | kW per rack | Cooling method | Power topology | Typical workload |
|---|---|---|---|---|
| Low | ≤ 10 | CRAH air, basic containment | 2× 32 A 3-phase PDU | Storage, networking, light compute |
| Medium | 10-30 | CRAH air + HAC | 2× 63 A or 1× 125 A | General-purpose enterprise |
| High | 30-60 | RDHx, in-row cooling | Busbar or 2× 250 A | Inference, mixed AI/compute |
| Extreme | 60-120 | Direct-to-chip liquid | Busbar 400-1250 A | AI training, HGX-class |
| Ultra | 120-250+ | Immersion or hybrid DLC + RDHx | Multi-busbar / high-voltage DC | GB200 NVL72, frontier training |
Why Density Matters#
- Capex per kW: higher density unlocks lower capex per installed kW because fewer racks share the building, security, and connectivity overheads.
- Cooling method choice: density above ~40 kW is not practically air-coolable. The cooling method is dictated by density, not the other way around.
- Network latency: dense racks shorten cable runs, which matters for high-bandwidth GPU interconnect (NVLink switch, InfiniBand) where every metre of fibre adds latency.
- Power distribution: above ~30 kW, conventional cabling is unwieldy; busbar becomes standard.
- Structural: extreme racks (GB200 NVL72 at ~1,400 kg) require reinforced floors.
Driver: AI Workloads#
AI training is the most density-hungry workload class ever deployed at scale. The driver is not the per-GPU power (700-1200 W) but the interconnect topology: GPUs in a training job must share data at hundreds of gigabytes per second, and that bandwidth degrades exponentially with cable length and switch hops.
Packing many GPUs into one rack — connected by short NVLink cables or copper DAC — is faster than spreading them across multiple racks linked by InfiniBand. The thermal density follows from the network density. GB200 NVL72 is the most extreme commercial example: 72 GPUs in one rack, deliberately, because the topology demanded it.
Average Density Trends#
- Early 2000s: 1-3 kW per rack typical.
- 2010s: 5-10 kW per rack average; 'high-density' was 15-20 kW.
- Early 2020s: 8-15 kW per rack average; AI builds at 30-50 kW.
- Mid 2020s (current): 12-20 kW average across enterprise; AI clusters at 60-130 kW.
- Late 2020s (projected): 25-40 kW average; AI frontier at 250+ kW.
Choosing the Right Density#
- Workload-dictated: AI training picks the highest density the room can support; storage and networking are perfectly happy at low density.
- Cost-optimised: medium density (15-25 kW) is often the cheapest total cost per kW at general-purpose scale.
- Site-dictated: a chosen colo hall may cap density at 30 kW regardless of what you wanted.
- Mixed-density rooms: AI suites at 100 kW alongside general suites at 15 kW are now common; the room must support the higher number even if most racks land lower.
Pitfalls#
- Average vs peak: the design density should be the peak, not the average. A few hot racks set the cooling envelope.
- Cabling congestion: above 30 kW, cable bundles can mechanically block hot-aisle airflow if not engineered.
- Floor loading: ultra-density racks (1.5+ tonnes) exceed many existing data hall floor specs.
- Cooling commissioning: an air-cooled hall rated for 15 kW that is asked to support 25 kW does not just heat up — it develops localised hot spots that cause silent thermal throttling on individual GPUs.
- Capacity creep: a rack contracted at 25 kW that grows to 60 kW may need to physically relocate.
References
- ASHRAE TC 9.9 — Thermal Guidelines · ASHRAE
- Uptime Institute — Global Data Center Survey · Uptime Institute
- Open Compute Project — Open Rack v3 · OCP