What are state space models and how do they compare to transformers?

State space models like Mamba are sequence models that use linear recurrence instead of attention. They scale linearly with sequence length rather than quadratically, making them efficient for very long contexts. Mamba 2 and other variants emerged in 2023 and 2024 as competitive alternatives to transformers, though pure transformers still dominate at the largest scales as of 2026.