Reactions from ranked influencers
2 posts
Another banger from Kangwook and Ying
🚀New paper on Looped Transformers! Latent reasoning is fast, but struggles to match CoT-level accuracy at scale. Can looped Transformers give us both? We find: yes! A looped padded backbone turns out to be a surprisingly simple recipe that works -- It gives latent thoughts a parallel workspace that can be supervised similarly to explicit CoT. 🧵
Wow, nice work @Kangwook_Lee!
🚀New paper on Looped Transformers! Latent reasoning is fast, but struggles to match CoT-level accuracy at scale. Can looped Transformers give us both? We find: yes! A looped padded backbone turns out to be a surprisingly simple recipe that works -- It gives latent thoughts a parallel workspace that can be supervised similarly to explicit CoT. 🧵