Coding agents reportedly beat GEPA at prompt optimization by analyzing full logs
DAIR.AI describes a Microsoft paper’s CASD method as using an off-the-shelf coding agent to analyze full logs and write prompt rules, without needing environment access or validation data.
TLDR
According to DAIR.AI’s summary of a Microsoft paper, CASD analyzes the full set of agent logs rather than running a search loop over small batches of recorded runs. The coding agent identifies recurring failures and turns its findings into rules in one prompt. Across ALFWorld, tau2-bench retail and telecom, and Spreadsheet Bench-Verified, the reported average improvement over the unoptimized baseline is 16.6 points in one pass, compared with 10.9 for GEPA and 5.3 for SkillOpt. DAIR.AI puts the cost at about $1.60 per optimized prompt, described as more than 22 times cheaper than validation-gated search.
