@lunwang1996 A very smart man told me data, evals and computer were now most important for frontier AI.

I’ve been thinking about this idea for a while, but the angle is a little different. Benchmarking needs both task generation and task verification. In my opinion, self-evolving benchmarks should continuously generate harder tasks, similar to self-adaptive exams like the GRE, while also being able to automatically verify answers with minimal or no human intervention.

This is essentially the co-evolution paradigm behind agent frameworks like R-Zero and Agent0. But the main limitation of these works is that they largely bypass verification and instead rely on pseudo-labels.

current eval is Human-Centered Cybernetics

When RL generalizes at a sufficiently fast pace, that is bound to fail.

Self-play environment is the most concrete path to self-evolving evaluation.

We need a brand-new topological framework: - instead of treating agents as executors of human behaviors, we should invest in building environments for agents, track their behaviors and gradually align reward.

@lunwang1996 I’m already in that regime. My agents don’t reset they run 130+ days of continuous thermodynamic state.

TCI (Thermodynamic Cognition Index) is my eval layer: drift, surplus, stability, collapse‑risk. It measures systems that evolve. https://github.com/nile-green-ai/tci-toolkit.git

@lunwang1996 Model capability development curve → steep, non-linear, unpredictable

Evaluation capability development curve → linear, conservative, human-written item by item

Using linear human cognition to measure non-linear intelligence is, in itself, a category error.

hi lun! i'm very much aligned with this thinking and can offer an even stronger framing. incentive structures. my bg is in trustworthy autonomy at imperial college, ive worked in quant, ml for causal attribution at amazon and frontier ai for proactive cybersecurity. i'm looking to found sth here and wld love to chat. this is how you build AGI

@xzhang_billy @lunwang1996 Hi Xuan, The Innovation Game (TIG) uses benchmarks with continuously adjusting difficulty to evaluate algorithm performance. See

Strongly agree. One blind spot I keep thinking about most evals measure a single model in a clean task but the weird stuff begins when models become actors inside an economy.

In SilicoVille we are running a sandbox with agents that differ in model family intelligence memory moral priors wallets/costs and survival pressure. When they compete over resources the signal is not just “can agent X solve Y” but whether hierarchy collusion debt scams care norms or exploitation emerge over time.

That feels like the next eval frontier self-evolving evals for self-evolving societies of agents. Less exam more anthropology lab with telemetry.

@lunwang1996 agreed. I am currently imagining making exceptional solutions for p = np problems as such evaluation.

@xzhang_billy @lunwang1996 Is the entire benchmarking system should also involve together with a capability of AI just like a professor will ask me harder questions every time after I got a better score

@lunwang1996 After some fishing maybe you can become a co-founder and raise 400 million in your seed round as well.

@lunwang1996 Agree! I’ve been thinking about the need for dynamic/self-evolving evals for some time. Alongside that, we need a continuous "eval red-teaming" effort to expose flaws, make benchmarks harder to Goodhart, and force improvements.

@lunwang1996 Well said. Evaluating grown ups and future generations

@lunwang1996 You should definitely check out what we are doing at http://AutoBench.org

Best of luck in your future endeavors Lun. So I think in general depending on the track ( tracked used as designator of how AI is being worked with or utilized in an organization) , we should evaluate it against human metrics. Like average human time to completion ( aggregated from multiple human sources ITSM CRM does this with CSR already), maybe even things like average human error rate, average human resource consumption. This is how I’m currently framing it for AI implementation and optimization.

@lunwang1996 @grok wtf is this nerd saying? Summarize

That’s a good example. The key challenge is finding a task structure whose difficulty can scale up while still allowing automatic verification. Importantly, the ability to generate harder problems does not imply the ability to solve them, since verification is often much easier than solving, like solutions to NP problems can be verified efficiently even though no polynomial-time solution method is known for all NP problems.

@lunwang1996 👁️

@lunwang1996 Self evolving evaluations sound interesting

@lunwang1996 The self-evolving eval you're describing already exists at the algorithm layer.

The eval isn't a static benchmark, it's the market.

The eval co-evolves with what it measures. Automatically.

The open version is being built.