• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Researcher Questions SWA Scaling After Convolution Result

    AI researcher at Meta questions whether convolution results apply to sliding window attention.

    LB
    1 Source, 30d ago, first seen 30d ago

    TLDR

    Lucas Beyer posted on X that a reported result applied only to the convolution operation. He added he is not confident the finding would transfer to SWA and asked whether anyone has checked yet. The post includes a photo and references a separate tweet from @msaffar3. Beyer is identified in the packet as an AI researcher at Meta with prior roles at OpenAI and DeepMind/Google Brain, known for Vision Transformer and multimodal work.

    Combined views

    18.7K

    1 Source, first seen 30d ago

    Combined views

    18.7K

    1 Source, first seen 30d ago

    123 likes
    123 likes
    9 comments
    72 saves
    3 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    9 comments
    72 saves
    3 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    1 Source

    @giffmanaHowever, that result was specifically for the conv operation and I'm not confident it would transfer to SWA. Curious if anyone actually checked yet?

    1 Source

    @giffmanaHowever, that result was specifically for the conv operation and I'm not confident it would transfer to SWA. Curious if anyone actually checked yet?