Many users dismissed ECI leaderboard claims of Kimi-K3 edging Opus 4.6 and GPT-5.3-Codex as flawed, biased, or meaningless due to frequent meaningless updates, while a few noted the value of open-weight models competing closely.
Based on 10 visible X reactions from 21 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@scaling01 this makes me much more skeptical of ECI. K3 is definitively way stronger than Opus 4.6 and 5.3-codex. it's not even close. i suspect the problem is their choice of included benchmarks rather than the ECI-style metric itself.
@scaling01 Why do you CONSISTENTLY leave out grok 4.5 from your benchmarks? Even muse spark is on here 🤣. The bias against grok is unreal in the ai community. Really a sight to behold
@scaling01 The gap is within noise, but the fact that open-weight models are competing at this level is what matters for the ecosystem.
@scaling01 Bro how can Luna be so high... It is obviously a fucked benchmark
@scaling01 Much less than I thought. Interesting!
@scaling01 No grok? Usless
The preliminary ECI of Kimi-K3 is 155.53 with 90% CI of 153.87–158.21 this is slightly ahead of Opus 4.6 (155.31) and GPT-5.3-Codex (155.36)
and I have also tested including some semi optimistic estimates of ARC-AGI-2 and WeirdML scores they don't really change the result much a final score of 156 seems likely
I computed this using the official ECI GitHub repository and existing benchmarks + benchmarks Epoch has already run themselves
Many users dismissed ECI leaderboard claims of Kimi-K3 edging Opus 4.6 and GPT-5.3-Codex as flawed, biased, or meaningless due to frequent meaningless updates, while a few noted the value of open-weight models competing closely.
Based on 10 visible X reactions from 21 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@scaling01 Yours? I was wrong
given that this is preliminary the confidence intervals should probably reflect that somehow
so on the page it would be displayed at 156