AI Benchmark Leaderboard

Google Launches High-Performing Gemma 2 Series with 9B and 27B Parameter Models, Excelling in Benchmarks

Jun 27, 2024, 03:22 PM

Google has launched its new Gemma 2 series, featuring two lightweight models with 9 billion (9B) and 27 billion (27B) parameters. The models are designed to be efficient and high-performing, with the 27B variant performing comparably to much larger models like Llama 3 70B. Gemma 2 models are trained on 13 trillion tokens for the 27B version and 8 trillion tokens for the 9B version. They are optimized for speed and can run on a single TPU or GPU. The models are available on various platforms including Google AI Studio, Hugging Face, Kaggle, and Google Colab. Early evaluations place the 27B model ahead of other large models in benchmarks, and the 9B model is considered the best in its class. Both models support an 8K context window and are designed for responsible AI development, with ethics and safety evaluations included. The 27B model scores 75.2 on MMLU, 55.1 on AGIEval, and 51.8 on HumanEval, while the 9B model scores 71.3 on MMLU, 52.8 on AGIEval, and 40.2 on HumanEval.