Meta's Llama 3.1 Models Offer 4X Cheaper AI Deployments, Achieve 99% Performance

Aug 14, 2024, 09:16 AM

Meta's AI division has released the Llama 3.1 models, which have been quantized to 4 bits by Neural Magic's research team. This quantization allows for 4X cheaper deployments, reducing the need from two 8x80GB nodes to one 4x80GB node. The models, including the 405B, 70B, and 8B versions, maintain approximately 100% recovery of the original performance. The HQQ Llama-3.1-70B model, in particular, achieves 99% of the base model's performance across various benchmarks, marking a significant advancement in AI model efficiency and cost-effectiveness.