Hybrid linear attention versus sliding-window attention for long-context AI

An October 2026 post argues that shipping sliding-window attention (SWA) for a first release is less conservative than using hybrid linear-attention models. It cites a 2024 NVIDIA study that found an 8B Mamba-2 hybrid exceeded an 8B Transformer on all 12 standard tasks, and another study finding that larger sliding windows hurt long-context performance in the architectures tested. The post also says tools including vLLM already serve hybrid models.