Loading activity…
@ofirpress
SWE-bench, ProgramBench, SWE-sweep. Postdoc @Princeton. PhD @nlpnoah @UW.
AI
The post pitches a “play it, rebuild it” approach and says users can train one agent or a whole team.
XAI
A post says Carbon scored 88.2% on SEC-bench Pro and is rolling out first to cybersecurity specialists.
AI
One poster says every model scored 0% at launch, prompting weaker ProgramBench metrics such as “almost resolved.”
AI
A Skill-Inject contributor says Meta used four Tübingen benchmarks to report the model's performance in its system card.
AI
A user predicts a 20-year wait for AI to autonomously handle maintenance, SRE, security and PM-defined features.
AI
Kimi claims a 1-million-token context window for Pro and two to three times Pro's speed for Highspeed.
XAI
A post says OpenAI and Anthropic models doubled the number of bugs found while halving costs on SWE-sweep.
AI
Wenhao Chai says the team built its tasks from published papers and refined them with field experts.
AI
The presenter says it covers benchmark results, harness design and his experiments with economic swarms.
AI
A post asks whether the method could identify the best coding-agent checkpoint without post-training every base model.
AI
A post argues that AI agents make research and testing easier, enabling more projects that eventually hit human bottlenecks.
AI
The benchmark’s creators say it evaluates that ability through 4D inverse graphics.
AI
A post describes the benchmark as focusing on visual bugs, including misrendered maps and misplaced buttons.
AI
A user says the model writes board-analysis code with access to 200,000 real-game board states and engine analysis.
XAI
A researcher says an NYC workshop talk will cover when coding-agent teams outperform single agents and where collaboration helps.
AI
A post argues that formalizing this gap is getting harder, but remains important to making AI more useful.
AI
A post argues that human tasks are now too easy to use as AI benchmarks in many domains.
AI
The anniversary post predicts engineers will write less code, listen more to users and direct AI agents.
AI
A poster jokes about “it” getting as smart as a housecat while quoting an OpenAI post and tagging @ylecun.
