Meta's 2024 Multi-token Prediction Makes LLMs 3x Faster

May 2, 2024, 03:23 PM

Meta has introduced a new method for training large language models (LLMs) such as GPT and Llama, titled 'Better & Faster Large Language Models via Multi-token Prediction'. This approach, detailed in a 2024 paper by F Gloeckle, B Y Idrissi, B Rozière, D Lopez-Paz, and G Synnaeve from FAIR at Meta, involves training models to predict four future tokens simultaneously. This method has shown to improve sample efficiency and increase inference speeds by up to three times.