Reactions from ranked influencers
6 posts
Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2). Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://arxiv.org/abs/2607.13491), making the loop transformer stable and scalable!
I love the mythos around looped transformers.
Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2). Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://arxiv.org/abs/2607.13491), making the loop transformer stable and scalable!
So basically, the current frontier models are 1.5T-parameter Transformers with 48 layers, looped twice?
Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2). Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://arxiv.org/abs/2607.13491), making the loop transformer stable and scalable!
Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2). Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://arxiv.org/abs/2607.13491), making the loop transformer stable and scalable!
@willdepue lessons in this
hyperstition is real in machine learning. i really don鈥檛 think mythos is a universal transformer, but just that rumor is enough to convince people to make it work. false rumors lead to research progress
Heard that some frontier models are basically a 48-layer transformer looped twice (48L x 2). Now we are introducing DeepLoop: Depth Scaling for Looped Transformers (https://arxiv.org/abs/2607.13491), making the loop transformer stable and scalable!