Official announcements from AI companies or credible tech news outlets

Cerebras Achieves 969 Tokens/Second with Meta's Llama 3.1 405B, 12x Faster than GPT-4o, 240ms Time-to-First-Token

Nov 19, 2024, 06:44 AM

Cerebras Systems has achieved a significant milestone in AI performance with its deployment of Meta's Llama 3.1 405B model. The system operates at an impressive output of 969 tokens per second, making it 12 times faster than OpenAI's GPT-4o and 18 times faster than Anthropic's Claude 3.5 Sonnet. Additionally, it boasts a time-to-first-token of just 240 milliseconds and supports a context length of 128,000 tokens with 16-bit weights. This performance enhancement is attributed to Cerebras's advanced Wafer Scale Engine (WSE), which has redefined capabilities in AI and scientific computing. The company plans to introduce a public inference endpoint soon, further expanding access to this high-speed AI technology.