Tech journalist Timothy B. Lee says fixed-cost monthly AI plans follow all-you-can-eat subscription economics, with profits from casual users offsetting losses from heavy users
NathanpmYoung replied that compute costs could amplify losses from high-usage customers.
If I buy a gym membership and then work out there five days a week, there is a sense in which my workout as subsidized by the gym. But the gym may still be making a profit because for every customer like me there are 10 who only come in once a week or less often.
Timothy B. Lee@binarybitsThe economics of fixed-cost monthly AI plans are really breaking people's brains. All-you-can-eat plans are a common feature across the economy. You always get a few customers who you lose money on because they use the plan heavily. That's offset by lots of casual users.
So when you say that AI companies have positive margins and are hence not subsidizing users, people point to people who get $1,000/month of tokens on a $200/month plan or whatever. These people exist!
Timothy B. Lee@binarybitsIf I buy a gym membership and then work out there five days a week, there is a sense in which my workout as subsidized by the gym. But the gym may still be making a profit because for every customer like me there are 10 who only come in once a week or less often.
But their existence doesn't mean the subscription plan as a whole is losing Anthropic money. And Anthropic can and does adjust the parameters of the plan to make sure heavy users don't push the whole thing underwater.
Timothy B. Lee@binarybitsSo when you say that AI companies have positive margins and are hence not subsidizing users, people point to people who get $1,000/month of tokens on a $200/month plan or whatever. These people exist!
@NathanpmYoung You mean a heavy Claude Max user can consume more tokens (relative to average) than a heavy gym user can do workouts (relative to average)? I think this is probably true, but Anthropic can (and does) adjust usage limits to keep this under control.
Nathan is in Berkeley 🔎@NathanpmYoung@binarybits Seems like the unit economics are a bit different here. I imagine many any you can eat packages are not that loss making for the big users but AI could be, right?
@NathanpmYoung I suspect the rise of agents has shifted usage patterns in a way that made the old all-you-can-eat plans less profitable than they would have been in the past.
Timothy B. Lee@binarybits@NathanpmYoung You mean a heavy Claude Max user can consume more tokens (relative to average) than a heavy gym user can do workouts (relative to average)? I think this is probably true, but Anthropic can (and does) adjust usage limits to keep this under control.
@NathanpmYoung I dunno in the short run a tech company has a data center full of GPUs that are either being used or not, just as the gym has a set of exercise machines that are either being used or not.
Nathan is in Berkeley 🔎@NathanpmYoung@binarybits no i mean that marginal cost of gym space is usually 0 to a gym, marginal tokens aren’t
@NathanpmYoung AI companies also sell inference by the token, but most gyms will sell people day passes. These things seem more similar than different to me. What do you see as the practical takeaway of this disanalogy for AI companies?
Timothy B. Lee@binarybits@NathanpmYoung I dunno in the short run a tech company has a data center full of GPUs that are either being used or not, just as the gym has a set of exercise machines that are either being used or not.
@NathanpmYoung It is probably true that an active GPU consumes more electricity, proportionally speaking, than an active treadmill. This isn't obvious though. For example if a gym is busy they probably hire more people to wipe down the machines, answer customer questions, etc.
Nathan is in Berkeley 🔎@NathanpmYoung@binarybits The gym is heated and open anyway. l marginal costs are very low compared to price of day pass. Tokens use energy and compute. Marginal cost probably much more similar to price of token.
@NathanpmYoung Sure, I agree they are not exactly the same.
Nathan is in Berkeley 🔎@NathanpmYoung@binarybits I think that they don’t offer truly all you can eat plans suggests that the two are more different than gyms which regularly do
@binarybits Seems like the unit economics are a bit different here. I imagine many any you can eat packages are not that loss making for the big users but AI could be, right?
Timothy B. Lee@binarybitsThe economics of fixed-cost monthly AI plans are really breaking people's brains. All-you-can-eat plans are a common feature across the economy. You always get a few customers who you lose money on because they use the plan heavily. That's offset by lots of casual users.
@binarybits no i mean that marginal cost of gym space is usually 0 to a gym, marginal tokens aren’t
Timothy B. Lee@binarybits@NathanpmYoung You mean a heavy Claude Max user can consume more tokens (relative to average) than a heavy gym user can do workouts (relative to average)? I think this is probably true, but Anthropic can (and does) adjust usage limits to keep this under control.
@binarybits The gym is heated and open anyway. l marginal costs are very low compared to price of day pass.
Tokens use energy and compute. Marginal cost probably much more similar to price of token.
Timothy B. Lee@binarybits@NathanpmYoung AI companies also sell inference by the token, but most gyms will sell people day passes. These things seem more similar than different to me. What do you see as the practical takeaway of this disanalogy for AI companies?
@binarybits I think that they don’t offer truly all you can eat plans suggests that the two are more different than gyms which regularly do
Timothy B. Lee@binarybits@NathanpmYoung Remember that hyperscalers tend to do inference in large batches, so I bet the electricity consumption of an AI data center doing inference at 60% capacity isn't that different from 40% or 80%.