SambaNova AI's Samba-1 Turbo Chips Set New Benchmark with 1,084 Tokens/s on Llama 3 Instruct (8B)

May 28, 2024, 04:23 PM

Artificial Analysis has benchmarked SambaNova AI's custom AI chips at 1,084 tokens per second on Llama 3 Instruct (8B), making it the fastest output speed recorded to date. This performance is over eight times faster than the median output speed across API providers of Meta's Llama 3. SambaNova AI's Samba-1 Turbo has set a new standard in large language model (LLM) benchmarks, surpassing other competitors in the field. The chip also processes input tokens at 5,000 tokens per second. This achievement positions SambaNova AI as a leading provider in AI inference technology. The results were highlighted at the GenAISummitSF2024, emphasizing its potential as a GPU alternative and its competitive edge over companies like Groq.