Aider Polyglot Benchmark results published by reputable AI research organizations

DeepSeek Unveils 671B DeepSeek-V3 AI Model, Outperforms GPT-4o with 60 Tokens/Sec Speed

Dec 26, 2024, 05:37 PM

DeepSeek has officially released DeepSeek-V3, a new open-source AI language model with 671 billion Mixture-of-Experts (MoE) parameters and 37 billion activated parameters per token. The model reportedly outperforms leading proprietary models such as GPT-4o, Claude 3.5 Sonnet, and Llama 3.1 405b on various benchmarks, including the Aider Polyglot Benchmark, which tests language models on coding exercises across multiple programming languages. DeepSeek-V3 achieves a score of 48% on this benchmark, significantly improving from the 17% score of its predecessor, DeepSeek-V2.5. The model was trained on 14.8 trillion high-quality tokens using 2.788 million H800 GPU hours over less than two months, with a reported training cost of $5.6 million. DeepSeek-V3 also boasts a speed of 60 tokens per second, three times faster than the previous version, and supports a context length of 128,000 tokens. The model utilizes auxiliary-loss-free load balancing and FP8 mixed-precision, and it operates with high sparsity by leveraging 256 experts with only eight activated per token. The release includes fully open-source models and papers. Pricing is set at $0.27 per million input tokens and $1.10 per million output tokens.