Commentary on X
@FeiziSoheil Can't wait to share such exciting insights to everyone!
9:12 AM · Jul 15, 2026@Wenxiao__Wang Thanks for leading these experiments!
9:14 AM · Jul 15, 2026Current benchmarks miss the central challenge in agent optimization. Most evaluations test one-shot improvement, while real-world use requires recursive optimization as new tasks continue to arrive. To study this, we built a two-phase continual-learning evaluation on hard Terminal-Bench 2.0 tasks, comparing GEPA, Meta Harness, and RELAI. The methods behaved very differently: - GEPA improved initially but transferred negatively to unseen tasks, indicating overfitting. - Meta Harness transferred well but could not continue improving. - RELAI was the only method that both generalized to unseen tasks and improved again without regressing prior performance. Full results are in the article below. You can access RELAI’s continual-learning engine to run similar optimization studies and apply it to your own agents at http://relai.ai
9:08 AM · Jul 15, 2026Combined views
3.1K
2 posts, first seen 2h ago