Loading...
Market
Which open-source AI model will have the highest MMLU score by December 31, 2024?
2
Resolution / Starting Odds
Apple's 7B AI model • 25%
Mistral 7B • 25%
Llama 3 8B • 25%
Google's Gemma • 25%
Official AI benchmarks and performance reports
Story
Apple Releases Fully Open-Source 7B AI Model, Outperforms Mistral 7B
Jul 19, 2024, 06:23 PM
Apple has made a significant move in the AI landscape by releasing a fully open-source 7B language model, which includes the model weights, training code, and a 2.5 trillion token dataset. This model, trained primarily on English data, features a 2048 context window and combines elements of DCLM-BASELINE and StarCoder. The new model outperforms Mistral 7B and other leading open models such as Llama 3 8B and Google's Gemma. Notably, the model's MMLU is 64%. This release marks a notable achievement in transparency and openness, setting a new standard for the industry.
View original story
Similar markets
Llama 3.1 405B • 25%
GPT-4o • 25%
Claude Sonnet 3.5 • 25%
Other • 25%
Meta's Llama 3.1-70B • 25%
OpenAI's GPT-4 • 25%
Google's Bard • 25%
Other • 25%
OpenAI's O1 model • 25%
GPT-4 • 25%
Gemini • 25%
Anthropic's Claude • 25%
Google's Gemini • 25%
OpenAI's GPT • 25%
Microsoft's Azure AI • 25%
Other • 25%
Claude 3.5 Sonnet • 33%
GPT-4o • 33%
Google's AI Model • 33%
OpenAI o1-preview • 25%
Anthropic Claude 3.5 Sonnet • 25%
OpenAI o1 mini • 25%
Other • 25%
Nemotron 70B • 25%
ChatGPT4o • 25%
Sonnet 3.5 • 25%
Other • 25%
Llama 3.1 405B • 25%
GPT-4o • 25%
Claude Sonnet 3.5 • 25%
Other • 25%
Azure OpenAI • 25%
OpenAI • 25%
Meta • 25%
MistralAI • 25%
DeepSeek-R1-Lite-Preview • 25%
OpenAI's o1-preview • 25%
Google DeepMind's model • 25%
Other • 25%
Llama 3.1 405B • 25%
GPT-4o • 25%
Claude Sonnet 3.5 • 25%
Other • 25%
Claude 3.5 Sonnet • 33%
GPT-4o • 33%
Gemini • 34%