Meta Nears Completion of AI Supercomputing Cluster with Over 100,000 Nvidia H100 GPUs for Llama 4, Costing $2 Billion

Sep 11, 2024, 04:17 PM

Meta Platforms is nearing the completion of a significant AI supercomputing cluster consisting of over 100,000 Nvidia H100 GPUs. This cluster will be used to train Meta's next-generation generative AI model, Llama 4. The project, which is being finalized in the US, represents a substantial investment, with the cost of the GPUs alone exceeding $2 billion. Notably, Meta will not use proprietary Nvidia networking gear for this cluster. This development positions Meta alongside other tech giants like Elon Musk, who also possess large-scale GPU clusters. Additionally, Meta's supercomputer has been added to The Information's AI Data Center Database.