8h ago

Open Models Lag Private AI by 8-10 Months on Private Benchmarks

——0——
Original post
Lisan al Gaib#980@SCALING01OPHåvard IhleHåvard Ihle@HTIHLE

How far behind are open models? Across 17 selected benchmarks, private ones show a gap of 8-10 months today, almost 2x the gap on public ones (4-6 mo). More discussion (including limitations), code and blog in the thread.

2:48 AM · May 28, 2026 View on X
QUOTE POSTLisan al Gaib#980Lisan al Gaib@SCALING01

backward looking gap on private benchmarks is 8-10 months

I guess the forward looking one is at least 12 months (at least for Mythos and GPT-5.5)

Håvard IhleHåvard Ihle@htihle

How far behind are open models? Across 17 selected benchmarks, private ones show a gap of 8-10 months today, almost 2x the gap on public ones (4-6 mo). More discussion (including limitations), code and blog in the thread.

9:48 AM · May 28, 2026 · 24.9K Views
10:42 AM · May 28, 2026 · 8.6K Views