Loading...
Market
What will be SambaNova Cloud's position in the AI inference market by December 31, 2024?
2
Resolution / Starting Odds
1st • 25%
2nd • 25%
3rd • 25%
4th or lower • 25%
Rankings from market research firms like Gartner or Forrester
Story
SambaNova Launches Fastest AI Inference Platform for Llama 3.1 at 570 Tokens/Second
Sep 10, 2024, 04:34 PM
SambaNova has announced the launch of its new cloud inference platform, SambaNova Cloud, which offers unprecedented speeds for AI model inference. Notably, the Llama 3.1 405B model achieves a speed of 132 tokens per second in full precision, while the Llama 3.1 70B model reaches up to 570 tokens per second. This performance is significantly faster than traditional GPUs, with claims of up to 10 times faster inference speeds. The platform operates in real-time and serves the Llama 3.1 405B model in 16-bit precision. It is available for developers starting today, with free access via API and no waitlist. The service has been independently verified and is expected to enable advanced AI applications.
View original story
Similar markets
Less than 5% • 25%
5% to 10% • 25%
10.1% to 20% • 25%
More than 20% • 25%
Leader • 25%
Top 3 • 25%
Top 5 • 25%
Top 10 • 25%
Leader • 25%
Top 3 • 25%
Top 5 • 25%
Outside Top 5 • 25%
Less than 50% • 25%
50% to 60% • 25%
60% to 70% • 25%
More than 70% • 25%
Leader • 25%
Top 3 • 25%
Top 5 • 25%
Outside Top 5 • 25%
Top 5 player • 25%
Top 10 player • 25%
Top 20 player • 25%
Outside top 20 • 25%
Leader • 25%
Top 3 • 25%
Top 5 • 25%
Outside Top 5 • 25%
Nvidia remains the leader • 25%
Nvidia faces strong competition but stays competitive • 25%
Nvidia loses significant market share • 25%
Nvidia partners or merges with another major tech company • 25%
Increases by more than 5% • 25%
Remains stable (within 5% change) • 25%
Decreases by less than 5% • 25%
Decreases by more than 5% • 25%
Leader • 25%
Strong Competitor • 25%
Moderate Competitor • 25%
Minor Player • 25%
Top 3 • 25%
Top 5 • 25%
Top 10 • 25%
Outside Top 10 • 25%