Reactions from ranked influencers
3 posts
> This is based on single forward pass perf (on math)… but corresponds to pretrain quality. (Minimally, it isn't improved by post-training.) Very interesting method for measuring the model's instantaneous capacity. Ryan gets that K3 ≈ Opus 4.1. Which is more than I'd expect
Oops, I linked the wrong thing. This is a better canonical paper, but I just did quick and dirty experiments using this dataset: https://www.lesswrong.com/posts/Ty5Bmg7P6Tciy2uj2/measuring-no-cot-math-time-horizon-single-forward-pass
I suspect that Opus is significantly less sparse, both on the attention side and activation ratio side. But there's the "quality" aspect too. We're a long way off from the first Chinese model that's genuinely "Fable-class" aside from post-training.
@teortaxesTex TBC, I got results about halfway between 4 and 4.5. I think Opus 4.1 probably uses the same base model as Opus 4 (haven't checked though).