PrimeIntellect Launches INTELLECT-1: First Decentralized Training of 10B AI Model

Oct 11, 2024, 07:18 PM

PrimeIntellect has announced the launch of INTELLECT-1, the first-ever decentralized training of a 10 billion parameter AI model. This initiative scales decentralized training 10 times beyond prior efforts, allowing anyone to join and contribute to building open-source artificial general intelligence (AGI). The project boasts significant advancements, including a 25-fold improvement over the DiLoCo model introduced 11 months ago and a reduction in bandwidth usage by 1000-2000 times compared to centralized training. The training process requires only one minute of communication for every hour of training and achieves 98% compute utilization across global nodes. Prime Intellect also highlights a 400x reduction in bandwidth usage.