Surveys or usage data reports from major academic institutions

Groq's Llama3 AI Model Hits 1,000+ T/s, Outpaces GPT4

May 10, 2024, 10:48 PM

Groq Inc.'s latest AI model, Llama3, has been making significant strides in the AI industry with its advanced capabilities. The model, which operates on Groq's platform, has been noted for its exceptional speed, blazing through more than 1,000+ T/s and processing requests four times faster than the latest GPT4 model. This performance enhancement is attributed to the Llama3-70b-8192 model, which completes requests in approximately 25% of the time it takes GPT4. Additionally, Llama3 has introduced grouped query attention (GQA) across its models, improving inference efficiency and overall performance. The open-source nature of Llama3 also facilitates widespread use and customization, further democratizing AI technology.