Meta Researcher Urges Paradigm Shift in Coding Benchmarks
AGI House hosts Meta researcher Kilian Lieret for talk on agentic coding benchmarks.
TLDR
Kilian Lieret, an AI research scientist at Meta with prior roles at Princeton and contributions to open-source projects including SWE-agent and mini-SWE, delivered the first AGI House Research Spotlight. The talk examined agentic coding benchmarks and argued they require a fundamental change in approach. Ofir Press amplified the event on social media, noting the presentation began with established benchmarks before exploring limitations. The session positions AGI House as a venue connecting applied AI research with community discussion on evaluation methods for autonomous coding systems.
Combined views
3.6K
2 Sources, first seen 63d ago