Hugging Face Retweets Sliding Window Attention Resource
Retweet points readers to Papers with Code page on the attention method.
Hugging Face retweeted a post from Niels Rogge that directs people to a Papers with Code entry on Sliding Window Attention. The post responds to questions about the technique. Papers with Code is presented in the post as a site covering trending AI research papers, code, datasets, methods, and evaluation leaderboards. The retweet comes from the official Hugging Face account, which manages the Hub and Transformers library.
Combined views
1.1K
2 posts, first seen 1d ago
Reactions
2 postsNR 
Niels Rogge@NielsRoggeFor folks wondering what Sliding Window Attention is, there's a method for it on Papers with Code Sliding Window Attention (SWA): A local attention pattern that restricts each token to attending only within a fixed-size neighborhood instead of the full sequence. This reduces…
- likes: 268
- replies: 2
- bookmarks: 235
- reposts: 40
LB Lucas Beyer (bl16)@giffmana@jm_alexia @JFPuget @RheaSukthanker @CameronPashmina @Emy_Aze I'm a bit disappointed you made the title so misleading. Just add *in post training* to it, like you did here.
- likes: 721
- replies: 7
- bookmarks: 34
- reposts: 9
