xAI Colossus 100k H100 Cluster: Liquid Cooling Architecture, 150MW Substation Engineering, and RDMA RoCEv2 Fabric

By TechIDaily Datacenter & Supercomputing Architecture Group · Published 2026-10-21

Traditional hyperscale AI datacenters require 24 to 36 months to design, permit, construct, and commission. In mid-2024, Elon Musk’s xAI achieved the unprecedented: building and powering Colossus—the world’s largest unified AI supercomputer boasting 100,000 liquid-cooled Nvidia H100 GPUs—in just 122 days from first server rack delivery to model training in Memphis, Tennessee.

This engineering case study deconstructs the electrical substation procurement, closed-loop liquid cooling distribution, and lossless RoCEv2 (RDMA over Converged Ethernet) optical network topology.

1. Colossus Power & Cooling Infrastructure Pipeline

System Architecture
┌────────────────────────────────────────────────────────────────────────┐
│  MEMPHIS 150MW POWER DISTRIBUTION & COOLING LOOP                      │
├────────────────────────────────────────────────────────────────────────┤
│  Tennessee Valley Authority (TVA) 161 kV Grid Transmission             │
│       │                                                                │
│       ▼                                                                │
│  On-Site Substation Step-Down -> 14 Mobile Megapack Battery Buffers    │
│       │                                                                │
│       ▼                                                                │
│  415V Busway Distribution to High-Density Server Racks (40 kW / rack)  │
│       │                                                                │
│       ▼                                                                │
│  Direct-to-Chip (D2C) Liquid Cold Plates (Propylene glycol water loop) │
│       │                                                                │
│       ▼                                                                │
│  Exterior Cooling Towers & Heat Dissipation (Sub-35°C water return)    │
└────────────────────────────────────────────────────────────────────────┘

2. Infrastructure Comparison: Colossus vs. Conventional Tier-3 AI Datacenters

Engineering DimensionTraditional Hyperscale DatacenterxAI Colossus ClusterDeployment Velocity
Commissioning Timeline24 - 36 months122 days (Record)6x - 8x Faster Deployment
GPU Cluster Density16,000 - 32,000 GPUs100,000 H100s (Expanding to 200k)Largest Unified Cluster
Interconnect FabricInfiniBand Quantum-2Ethernet RoCEv2 (Custom Broadcom Tomahawk)Supply-Chain Resilient
Rack Power Density12 - 18 kW per rack35 - 45 kW per rackHigh-Density Liquid Cooled
Power Utilization (PUE)1.25 - 1.401.09 (Direct liquid cold plate)Industry Leading Efficiency

3. Network Architecture: Solving RoCEv2 Congestion at 100k Scale

While most AI clusters of this magnitude rely on Nvidia InfiniBand, xAI chose RoCEv2 over standard Ethernet:

  • Bypassing InfiniBand Supply Bottlenecks: Allowed xAI to procure high-capacity 800 Gbps optical switches and Tomahawk silicon from multiple merchant vendors.
  • Priority-Based Flow Control (PFC) & ECN: Implemented strict Explicit Congestion Notification algorithms across all spine-leaf switches to prevent buffer overflow and tail latency spikes during large gradient synchronization passes.
  • Non-Blocking Clos Topology: Configured across three tiers of switching, guaranteeing full 3.2 Tbps bisectional bandwidth per 8-GPU server node.

4. Engineering Verdict

Colossus shattered the assumption that gigawatt-scale AI infrastructure requires multi-year procurement timelines. By combining modular liquid cooling skids, mobile battery storage buffers, and standard Ethernet RDMA fabrics, xAI established a new baseline for compute velocity.