Prime Intellect Launches INTELLECT-1, First-Ever Decentralized Training of 10B Parameter AI Model

Oct 11, 2024, 08:07 PM

Prime Intellect has announced the launch of INTELLECT-1, the first-ever decentralized training of a 10 billion parameter model. This initiative represents a significant advancement in decentralized AI, scaling training efforts 10 times beyond previous attempts. The project is fully open-source and aims to build an open-source Artificial General Intelligence (AGI). Contributing partners include Akash, HuggingFace, SemiAnalysis, and Valory AG. The decentralized approach is expected to address major compute hurdles and reduce bandwidth requirements by 1000-2000 times compared to centralized training. INTELLECT-1 is based on the Llama architecture and improves upon DiLoCo by 25 times.