Elon Musk Activates Memphis Supercluster with 100,000 GPUs to Train Grok 3.0

Jul 22, 2024, 07:13 PM

Elon Musk has announced the activation of the Memphis Supercluster, now the world's most powerful AI training cluster. This facility, a collaboration between xAI, X, and Nvidia, features 100,000 liquid-cooled H100 GPUs on a single RDMA fabric. The cluster began training at 4:10 a.m. local time in Memphis, Tennessee, and was installed in just 19 days. The Memphis Supercluster will be used to train Grok 3.0, which is expected to be the most powerful AI model in the world upon its release in December 2024. Grok 2.0 is anticipated to launch next month.