Meta's Llama 3.3: 70B Model with 128K Context Window Matches 405B Performance

Dec 6, 2024, 05:22 PM

Meta Platforms Inc. has released Llama 3.3, a new 70 billion parameter language model that matches the performance of its previous 405 billion parameter model, Llama 3.1, but at a significantly lower cost. The model, which is text-only, supports eight languages and has a 128,000 token context window, is available under the Llama 3.3 Community License. This release signifies Meta's continued push towards making high-performance AI models more accessible and cost-effective for developers and users. Llama 3.3 outperforms Amazon's Nova Pro and matches the capabilities of Llama 3.1 405B, offering improved efficiency in areas like math and reasoning. It is trained on 15 trillion tokens, with a knowledge cutoff in December 2023, and is available for download on Meta and Hugging Face.