Researcher Questions SWA Scaling After Convolution Result
AI researcher at Meta questions whether convolution results apply to sliding window attention.
TLDR
Lucas Beyer posted on X that a reported result applied only to the convolution operation. He added he is not confident the finding would transfer to SWA and asked whether anyone has checked yet. The post includes a photo and references a separate tweet from @msaffar3. Beyer is identified in the packet as an AI researcher at Meta with prior roles at OpenAI and DeepMind/Google Brain, known for Vision Transformer and multimodal work.
Combined views
18.7K
1 Source, first seen 30d ago