Japan's Fugaku-LLM AI Model: 13B Parameters, 400 Tokens

May 11, 2024, 05:24 AM

A significant advancement in artificial intelligence has been achieved with the release of Fugaku-LLM, a large language model developed by a team of Japanese researchers. The model, which consists of 13 billion parameters and handles up to 400 tokens, was trained on the Fugaku supercomputer, leveraging its CPU capabilities. This project marks one of the earliest large language model initiatives in Japan, spearheaded by the founders of Kotoba and led by notable researcher Rio Yokota. The collaborative effort involved top researchers and utilized distributed training methods.