Reactions from ranked influencers
11 posts
@scaling01 @xeophon flops not tokens
@scaling01 @xeophon to be clear i’m not saying “small models are better”, they’re frequently not but gpt-4.5 and llama-405b were not more efficient than their smaller siblings i’m just saying that cost matters, and tokens are the wrong unit because they don’t allow comparisons across model sizes
@scaling01 @xeophon you can always spend more compute to increase intelligence, but with diminishing returns why should i care about sol-high or fable when i can do sol-xhigh-pro-max pass@1000? because cost matters the frontier is fundamentally about efficiency in intelligence-per-watt
@scaling01 @xeophon because aggregating millions of tiny models is not efficient i care about the flop curve a monkey with a typewriter solves all verifiable problems in exponential time