Official MMLU Social Sciences benchmark results

Meta Releases Llama 3.1 405B, First Open-Source AI Model to Rival GPT-4o on July 23, 2024

Jul 23, 2024, 03:54 PM

Meta has officially released its highly anticipated Llama 3.1 model, including the 405B parameter version, on July 23, 2024. This model is the first open-source AI model to rival top proprietary models such as OpenAI's GPT-4o and Anthropic's Claude Sonnet 3.5 across several benchmarks. Leaked benchmarks indicate that Llama 3.1 405B outperforms GPT-4o in many areas, although it falls short in specific benchmarks like human_eval and mmlu_social_sciences. The model was trained on 15 trillion tokens and fine-tuned with public datasets and 15 million synthetic samples, requiring 30.84 million GPU hours. It supports a 128K context length and offers multilingual capabilities. Meta's CEO, Mark Zuckerberg, emphasized that the release of Llama 3.1 marks a significant moment in AI history, advocating for the benefits of open-source AI. The model is available through various platforms, including Groq, and is expected to be a game-changer in the AI landscape. Additionally, improved 70B and 8B parameter versions were also released.