Elon Musk's xAI Builds Colossus, World's Largest AI Supercomputer with 100,000 NVIDIA GPUs, Plans to Double to 200,000

Oct 28, 2024, 06:14 PM

XAI, the artificial intelligence company founded by Elon Musk, has built the world's largest AI supercomputer, named Colossus, in Memphis, Tennessee. The multi-billion-dollar, liquid-cooled cluster comprises 100,000 NVIDIA H100 GPUs and was constructed in just 122 days. Remarkably, training began after only 19 days, a pace praised by NVIDIA CEO Jensen Huang, who noted that xAI achieved in days what usually takes others a year. Colossus delivers 95% data throughput and zero latency issues, enabled by NVIDIA's Spectrum-X Ethernet networking platform. The supercomputer includes exabytes of storage and is designed to advance AI training capabilities, including powering xAI's Grok AI model. Each compute hall contains 25,000 GPUs. XAI plans to double Colossus's size to 200,000 GPUs, further cementing its position at the forefront of AI development.