Loading...
Market
Will SambaNova Cloud achieve 1000 tokens per second for Llama 3.1 70B model by June 30, 2025?
2
Resolution / Starting Odds
Yes • 50%
No • 50%
Official performance benchmarks and announcements from SambaNova
Story
SambaNova Launches Fastest AI Inference Platform for Llama 3.1 at 570 Tokens/Second
Sep 10, 2024, 04:34 PM
SambaNova has announced the launch of its new cloud inference platform, SambaNova Cloud, which offers unprecedented speeds for AI model inference. Notably, the Llama 3.1 405B model achieves a speed of 132 tokens per second in full precision, while the Llama 3.1 70B model reaches up to 570 tokens per second. This performance is significantly faster than traditional GPUs, with claims of up to 10 times faster inference speeds. The platform operates in real-time and serves the Llama 3.1 405B model in 16-bit precision. It is available for developers starting today, with free access via API and no waitlist. The service has been independently verified and is expected to enable advanced AI applications.
View original story
Similar markets
Yes • 50%
No • 50%
Yes • 50%
No • 50%
Yes • 50%
No • 50%
Yes • 50%
No • 50%
Groq Inc. • 25%
Together Inference • 25%
NVIDIA • 25%
Other • 25%
Less than 10,000 • 25%
10,000 to 50,000 • 25%
50,001 to 100,000 • 25%
More than 100,000 • 25%
MMLU • 25%
ARC • 25%
GSM8K • 25%
None by June 30, 2024 • 25%
Yes • 50%
No • 50%
Yes • 50%
No • 50%
