Alibaba's Qwen2 72B Model with 128K Context Window Outperforms Meta's Llama 3

Jun 6, 2024, 04:08 PM

Alibaba has unveiled its new multilingual model family, Qwen2, which outperforms Meta's Llama 3. The Qwen2 models come in five sizes: 0.5B, 1.5B, 7B, 57B-14B, and 72B, and have been trained in 29 languages, including 27 additional languages. The models excel in code and math capabilities and are available under the Apache 2.0 license for the smaller sizes. The largest model, Qwen2 72B, boasts a 128K context window and has achieved top scores on the Open LLM Leaderboard, outperforming GLM 4. The models are available on Hugging Face and come pre-quantized with MLX support.