Latent Sparse Attention's researchers say it matches full attention's accuracy at ultra-long contexts.
The team describes LSA as a step toward subquadratic attention. It says finding the right tokens takes 98% of the attention layer's time at 10 million tokens, and that LSA makes sparse attention up to 22.7 times faster and full attention 72.4 times faster.
37.9K
3 Sources, first seen ago
@jcrwhittingtonToday we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
@tri_daoRT @jcrwhittington: Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subq…
@j_foerst🚀@jcrwhittingtonToday we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington@tri_daoRT @jcrwhittington: Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subq…@j_foerst🚀@jcrwhittingtonToday we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington@tri_daoRT @jcrwhittington: Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subq…@j_foerst🚀@jcrwhittingtonToday we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context. Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster. With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington@tri_daoRT @jcrwhittington: Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subq…@j_foerst🚀
—
Not ranked yet
—
Not ranked yet
