KronLSTM - a NX-AI Collection

NX-AI 's Collections

KronLSTM

updated about 15 hours ago

The language models of the paper "Kronecker Delta Rule for Long Context Linear Attention" pretrained on 100B tokens.

NX-AI/KronLSTM-1.5B-100B

Text Generation • 1B • Updated about 15 hours ago • 15

Note 24 KronLSTM layers, 8K context
NX-AI/KronLSTM-1.5B-100B-32k

Text Generation • 1B • Updated about 15 hours ago • 6

Note 24 KronLSTM layers, continued at 32K context
NX-AI/KronLSTM-SWA-1.5B-100B

Text Generation • 1B • Updated about 15 hours ago • 9

Note 20 KronLSTM + 4 sliding-window attention layers, 8K context
NX-AI/KronLSTM-SWA-1.5B-100B-32k

Text Generation • 1B • Updated about 15 hours ago • 8

Note 20 KronLSTM + 4 sliding-window attention layers, continued at 32K context