He compared token metrics to pre-industrial steam power usage.
token use gets too much hate as a metric - in times of technological transition peoples default will be to underuse and underestimate the new tech. “steam power used” would have been a good KPI for pre industrial civilization just as kardashev scaling remains for ours
Positive users praise token usage as a clean proxy for valuable AI work and adoption signals, while negative users call it a flawed metric that mostly tracks slop and fails to measure productivity.
No Digg Deeper questions have been answered for this story yet.
@tszzl @futurenomics Agree. People have a terrible time accepting the fuzzy correlation between inputs and outputs.
You always get the smug: “heh… inputs aren’t outputs”
But… they are strongly correlated much of the time
@tszzl Steam did one thing, so “steam used” tracked output. Tokens don’t. More tokens can mean more value or more slop, and right now it’s mostly slop dressed as productivity. Consumption isn’t a KPI
@tszzl Token usage is one of the cleanest, most grounded proxies we have right now for actual useful work being done by frontier models. It’s not perfect (quality quantity, obviously), but it cuts through the hype better than most benchmarks.
token efficiency for instance means 2 things: How good are you at getting the AI to do things and achieving your goals How good is the AI model at delivering what's wanted with minimal intermediate flux or verbosity
Don't know if there's a name for these yet, if no one has it - *speaker efficiency *model token efficiency
Right now the speaker efficiency matters alot, but longer term model token efficiency will dominate. How soon is soon? don't know but i expect rapid improvement..
Of course you have to factor in energy or compute time too because tokens are dependent on model for exact expense.
@tszzl devs stressing over token efficiency are missing the point. dumping your entire monorepo into a 1M context window feels 'wasteful' until it solves a routing bug that would've taken you 3 days.
@tszzl It's fine as a measure, it's very bad as a metric.
1. Easy to measure isn't the same as what we care about. 2. Optimizing for token use leads to obvious failure modes, not what anyone is interested in. https://www.cell.com/patterns/fulltext/S2666-3899(23)00221-0
Both metrics conflate consumption with productive use. Better metrics: output per unit of steam / tokens per outcome.
Tech can be promising but take decades to get right (fusion, carbon capture, Concorde, etc).
We’re supposed to burn tokens indefinitely until we find out? Ok bubbleboi
@_vincentpaul_ @tszzl the Chinese get it
@CurrenticAI @tszzl Differentiate companies from civilizations.
Companies should not adopt token spend as a KPI. Just like they should not adopt dollars spent.
Dollars spent however is one of, if not, the best metric we have for measuring civilization progress (GDP).
@hypersoren @tszzl Yep - I’d push it even further… intelligence is a means not an end. The end is an outcome. So we’re searching for the “telodesic” (shortest path from intent to outcome)
@tszzl The high token use is because the models are general and not specific to the task they are doing. Does a person that speaks one language need a coding model that speaks 40 languages, or generate pictures, or generates audio?
Technology likes efficiency.
@tszzl inefficient wrappers are a culprit as well in token burn. Efficient context management should keep each output to under 50k tokens, and it's highly achievable by routing messy context to light models that specifically manage context.
@tszzl remind me of how scaling laws used to get so much hate in the ML community back in 2020
@tszzl me using tokens during the technological transition
@tszzl it’s an adoption incentive, not really a business metric…
@tszzl Token use is a demand signal before it is an efficiency problem.
Every transition looks wasteful from the old regime. Then the waste becomes infrastructure and everyone pretends the curve was obvious.
@tszzl token minners have a bitter pill to swallow.
@tszzl Agree. https://open.spotify.com/episode/5qwG4uWSg8RvdubPExcEOy?si=Bf2TtEdYTcCFNVTVEbyhBw
@tszzl We need a near perpetual AI token machine to find all the new physics that we're missing. Time to get to work on that Dyson solar swarm. Then up the Kardashev ladder we go! Have to find the doomers new hobbies though so it can get done.
Exactly. People mock token usage today the same way early industrial societies could’ve mocked coal or steam consumption as “wasteful.”
When a technology becomes foundational, the metric shifts from “why are you using so much?” to “who is leveraging it most effectively?” Tokens are basically cognitive labor units for AI systems. High usage can signal experimentation, automation, research, creativity, and economic output not just spam.
