Loading...
Market
Story
Groq Inc. Achieves Breakthrough with 30,210 Tokens/Sec and MLPerf Benchmark
May 25, 2024, 01:06 AM
Groq Inc. has achieved a significant milestone in performance engineering, showcasing their technology's ability to process data at unprecedented speeds. By implementing Groq and semantic caching, users can experience a substantial increase in the speed of generating answers to queries. The company's engineers have been working diligently to improve their stack, achieving a performance rate of 30,210 tokens per second. This advancement places Groq Inc. ahead of top GPUs in terms of data processing speed. Additionally, the new MLPerf benchmark results highlight the efficiency of 8 x H100 GPUs attached to a single VM, utilizing only a small fraction of the physical host's CPU and memory. This allows for more applications to run simultaneously on the system. Notably, Groq's LPUs have shown impressive performance, and MOEs are now twice as fast in the latest MLX, running at 60 tokens per second on an M2 Ultra.
View original story
Similar markets
5%-10% • 25%
10%-15% • 25%
15%-20% • 25%
Above 20% • 25%
Less than 25% • 25%
25% to 35% • 25%
35% to 45% • 25%
More than 45% • 25%
<5% • 25%
5%-10% • 25%
10%-20% • 25%
>20% • 25%
Less than 10% • 25%
10% to 25% • 25%
25% to 50% • 25%
More than 50% • 25%
Less than 5% • 25%
5% to 10% • 25%
10% to 15% • 25%
More than 15% • 25%
Less than 10% • 33%
10% to 25% • 33%
More than 25% • 33%
Top 1 • 33%
Top 2-5 • 33%
Outside Top 5 • 34%