The language models of the paper "Kronecker Delta Rule for Long Context Linear Attention" pretrained on 100B tokens.
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
Kronecker Delta Rule for Long Context Linear Attention
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
Organization Card
NXAI builds efficient foundation models on xLSTM, a recurrent architecture that scales linearly in sequence length rather than quadratically like Transformer attention.
Our flagship, TiRex 🦖, is a zero-shot time series forecasting model deployed in industrial and edge settings.
⚙️ What We Build
- xLSTM: a recurrent architecture led by Sepp Hochreiter (co-inventor of LSTM, 1997). Its recurrence scales linearly in sequence length, versus the quadratic cost of self-attention, which is the basis for its efficiency claims on long-sequence and state-tracking tasks.
- Foundation models for industrial data: time series, vision and multimodal.
- Edge-ready inference: models sized and quantised for embedded and cloud-edge deployment, not data-centre-only inference.
- TiRex 🦖: a 35M-parameter zero-shot forecasting foundation model built on xLSTM, for industrial time series.
- TiRex-2 🦖: our current flagship forecasting model (38.4M active parameters, +44.1M for multivariate mode), extending TiRex with native multivariate forecasting and past/future-known covariates.
📚 Publications & Resources
All papers, code and live demos for xLSTM, TiRex, Vision-LSTM, Bio-xLSTM and LRAM in one place: nx-ai.github.io/linktree
🔬 Collaborate With Us
We welcome:
- Industrial and enterprise partners deploying efficient AI at production scale
- Researchers and developers working on xLSTM and efficient foundation models
- Open-source contributions: fork our repositories, file issues, submit pull requests
models 12
NX-AI/KronLSTM-SWA-1.5B-100B-32k
Text Generation • 1B • Updated • 8
NX-AI/KronLSTM-SWA-1.5B-100B
Text Generation • 1B • Updated • 9
NX-AI/KronLSTM-1.5B-100B-32k
Text Generation • 1B • Updated • 6
NX-AI/KronLSTM-1.5B-100B
Text Generation • 1B • Updated • 14
NX-AI/TiRex-2-fevbench
Time Series Forecasting • 82.5M • Updated • 181 • 1
NX-AI/TiRex-2-gifteval-zs
Time Series Forecasting • 82.5M • Updated • 329 • 2
NX-AI/TiRex-2-gifteval-pretrain
Time Series Forecasting • 82.5M • Updated • 90 • 2
NX-AI/TiRex-2
Time Series Forecasting • 82.5M • Updated • 6.89k • 48
NX-AI/xlstm_scaling_laws
Updated • 2
NX-AI/TiRex
Time Series Forecasting • Updated • 152k • 100
datasets 0
None public yet