Apple Releases Fully Open-Source 7B AI Model, Outperforms Mistral 7B

Jul 19, 2024, 06:23 PM

Apple has made a significant move in the AI landscape by releasing a fully open-source 7B language model, which includes the model weights, training code, and a 2.5 trillion token dataset. This model, trained primarily on English data, features a 2048 context window and combines elements of DCLM-BASELINE and StarCoder. The new model outperforms Mistral 7B and other leading open models such as Llama 3 8B and Google's Gemma. Notably, the model's MMLU is 64%. This release marks a notable achievement in transparency and openness, setting a new standard for the industry.