Sepp Hochreiter Unveils xLSTM with Parallelizable LSTM, Advancing ML

May 8, 2024, 05:11 AM

AI pioneer Sepp Hochreiter has introduced a new machine learning model, the Extended Long Short-Term Memory (xLSTM), which aims to address the limitations of traditional LSTM networks. The xLSTM incorporates advanced features such as exponential gating and modified memory structures, enhancing its performance and scalability. This new architecture, which includes specialized sLSTM and mLSTM memory cells and a parallelizable LSTM, positions xLSTM as a strong competitor to state-of-the-art Transformers and State Space Models. Hochreiter, who has been working on LSTM for over 30 years, and his team at NXAI are developing European LLMs using this technology, marking a significant advancement in the field.