NOUS Research Advances AI Training with 1.2 Billion Parameter Model

Aug 26, 2024, 05:41 PM

NOUS Research has made significant advancements in AI model training by developing a method to train large AI models using low bandwidth connections between heterogeneous hardware. This approach reduces communication requirements by 1,000-10,000 times, making decentralized training feasible. The innovation, which involves training a 1.2 billion parameter model with Nanotron technology, marks a substantial shift in AI research and development.