Tweet notes NoPE shoutout in Stanford CS336 lectures
Xing Han Lu shares slide from lecture on attention interleaving in Command A.
TLDR
Xing Han Lu posted on X about reviewing the Stanford NLP CS336 lectures and noticing a shoutout to NoPE by a_kazemnejad and team. The attached image shows a presentation slide titled Current standard trick – interleave full and LR attention. The slide displays a diagram of the Command A model including a Tokenizer with vocabulary size 256,000 and other labeled UI boxes. Lu commented that the course is great so far.
Combined views
3.3K
1 Source, first seen 25d ago