Sepp Hochreiter Unveils xLSTM with New Cells, Competing with Transformers

May 8, 2024, 06:15 AM

AI pioneer Sepp Hochreiter has introduced a new machine learning architecture, the Extended Long Short-Term Memory (xLSTM), which aims to address the limitations of traditional LSTMs by incorporating sLSTM and mLSTM memory cells. The xLSTM features exponential gating, modified memory structures, and a new parallelizable LSTM design, enhancing its performance and scalability. This innovation positions xLSTM as a strong competitor to state-of-the-art Transformers and State Space Models. Hochreiter's team at NXAI is also working on building European LLMs using this new architecture.