Reactions from ranked influencers
4 posts
Introducing “expenditure horizon”: a proposed method for measuring AI capabilities on continuously-scored problems. The method compares performance as a function of spend for humans vs agents. The point where humans become more cost-effective is the agent’s expenditure horizon.
I'm curious about AIs optimizing model training due to its direct implications for RSI. It seems productive to study the dynamics of algorithmic efficiency wins humans vs agents get given enough expenditure. I think Expenditure Horizon helps with this, and I'm excited it's out.
Introducing “expenditure horizon”: a proposed method for measuring AI capabilities on continuously-scored problems. The method compares performance as a function of spend for humans vs agents. The point where humans become more cost-effective is the agent’s expenditure horizon.
Today we're sharing a method we've been using to measure AI agent capabilities: the "expenditure horizon". On an open-ended optimization problem, an AI agent given a small budget will often make more progress than a human given the same budget, because human time is expensive. The expenditure horizon is the budget at which the human catches up: the dollar value where a human's expected improvement, given the same budget, equals the agent's.
Introducing “expenditure horizon”: a proposed method for measuring AI capabilities on continuously-scored problems. The method compares performance as a function of spend for humans vs agents. The point where humans become more cost-effective is the agent’s expenditure horizon.
cute
Introducing “expenditure horizon”: a proposed method for measuring AI capabilities on continuously-scored problems. The method compares performance as a function of spend for humans vs agents. The point where humans become more cost-effective is the agent’s expenditure horizon.