INTELLECT-1: First Decentralized 10B AI Model Released by PrimeIntellect with 100+ H100s and 30 Contributors

Nov 29, 2024, 09:22 PM

The release of INTELLECT-1 marks a significant milestone in the field of artificial intelligence as it is the first large language model trained through a fully decentralized approach. Developed by PrimeIntellect, the model features a base size of 10 billion parameters and incorporates contributions from over 30 individual compute contributors across three continents, utilizing more than 100 NVIDIA H100 GPUs. The open-source release includes the base model, intermediate checkpoints, a pre-training dataset, and post-trained instruct models developed by Arcee AI. Additionally, a comprehensive technical report detailing the training framework and methodology is available, underscoring the project's commitment to advancing open-source AI technologies.