Loading...
Market
Which benchmark will NVIDIA's Mistral-NeMo-Minitron 8B model achieve the highest improvement in by December 31, 2024?
2
Resolution / Starting Odds
Chatbots • 25%
Virtual assistants • 25%
Content generation • 25%
Coding • 25%
Benchmark results published by NVIDIA or independent benchmarking organizations
Story
NVIDIA's Mistral-NeMo-Minitron 8B Model Achieves High Accuracy
Aug 21, 2024, 04:08 PM
NVIDIA has announced the release of the Mistral-NeMo-Minitron 8B, a small language model derived from the Mistral NeMo 12B model through pruning and distillation techniques. This new model is designed to offer state-of-the-art accuracy while being more computationally efficient. The Mistral-NeMo-Minitron 8B achieves high accuracy in nine popular benchmarks, including those for chatbots, virtual assistants, content generation, and coding. The model was developed using 400 billion tokens for distillation, significantly reducing the computational cost compared to training from scratch. NVIDIA's innovation in pruning and distillation has enabled the creation of a commercially permissive model that maintains high performance levels. The model achieved an MMLU score of 69.5.
View original story
Similar markets
Meta's Llama 3.1-70B • 25%
OpenAI's GPT-4 • 25%
Google's Bard • 25%
Other • 25%
Yes • 50%
No • 50%
MMLU • 25%
ARC • 25%
GSM8K • 25%
None by June 30, 2024 • 25%
GPT-4o • 33%
Gemini 1.5 • 33%
Claude 3.5 Sonnet • 34%
Claude 3.5 Sonnet • 33%
GPT-4o • 33%
Google's AI Model • 33%
GPT-4 Turbo • 25%
Claude 3 Opus • 25%
Llama 3.1 405B • 25%
Other • 25%
Intel Core Ultra 200V • 33%
Qualcomm Snapdragon X Elite • 33%
AMD Strix Point • 33%
Other • 1%
Contextual Accuracy • 25%
Natural Language Processing • 25%
User Adoption • 25%
Other • 25%
Llama 3.1 405B • 25%
GPT-4o • 25%
Claude Sonnet 3.5 • 25%
Other • 25%
Yes • 50%
No • 50%
Top 1 in a benchmark • 25%
Top 5 in a benchmark • 25%
Top 10 in a benchmark • 25%
No significant milestone • 25%