Cerebras Challenges Nvidia with Fastest AI Inference Service

Aug 27, 2024, 04:05 PM

Cerebras Systems has launched what it claims to be the world's fastest AI inference service, marking a significant challenge to Nvidia's dominance in the AI computing sector. The new service, powered by Cerebras' custom wafer-scale AI accelerator chips, offers AI developers access to high-speed applications at a lower cost. Cerebras has set a new record for AI inference speed, serving Llama 3.1 8B at 1,850 output tokens per second and 70B at 446 output tokens per second. This move is part of a broader trend where several chipmakers are attempting to break Nvidia's stronghold in the AI market.