EdgeBench Launches Benchmark for Long-Horizon AI Agent Research
Commentary on X
Highest ranked
Florian Brand@xeophonTECH
so giving the agent something fast to hill climb while having a judge which is more expressive and costly is really smart! the laws etc. are interesting, great to see someone do these experiments. speaking of which: cost for the paper is >1M. evals are a serious business now
Florian Brand@xeophonTECH
okay, i finally had time to look into it; edgebench is def worth the hype and well done i like the two loop design: the models need something to hill climb, but generally it is solved with some subsets (see e.g. mirrorcode) to avoid lookup tables 1/2 https://x.com/xeophon/status/2074496222912459034/photo/1 https://twitter.com/tikgiau/status/2072701593829695926