PrimeIntellect Releases INTELLECT-1, First Decentralized 10B AI Model Trained on 1T Tokens with 30 Contributors Across 3 Continents

Nov 29, 2024, 10:14 PM

PrimeIntellect has announced the release of INTELLECT-1, the world's first fully decentralized large language model (LLM) with 10 billion parameters. This groundbreaking model was trained across three continents using over 100 NVIDIA H100 GPUs and involved 30 individual compute contributors. The release includes the base model, intermediate checkpoints, a pre-training dataset, and post-trained instruct models developed in collaboration with Arcee AI. The project is notable for its decentralized training framework, which challenges the traditional notion that AI compute needs to be centralized. The model has been evaluated on 1 trillion tokens, demonstrating promising results. The technical details and framework are available in an accompanying technical paper, and users can download the model for personal use.