The system successfully executes tasks on unseen scenes.
yet another win for neurosymbolic systems!
Really cool research using AI-driven search (a GEPA-like method) to beat the state of the art in robotics. I think these type of neurosymbolic systems where AI generates an AI + code hybrid will be very effective. Seeing this in other applications too.
Positive users appreciate GEPA's potential for evolving better robotics policies using cheaper models along with upcoming fixes, while negative users criticize the stock proposer for overfitting.
No Digg Deeper questions have been answered for this story yet.
@aleksizy We find that running GEPA on agents with cheaper models can still lead to prompts that transfer to the bigger ones. Could be worth a try. Also, the proposer in GEPA does not need to be a big model. In this robotics work we used Haiku for a bunch of that.
Really cool research using AI-driven search (a GEPA-like method) to beat the state of the art in robotics. I think these type of neurosymbolic systems where AI generates an AI + code hybrid will be very effective. Seeing this in other applications too.
What if a robot policy weren't a neural net or a test-time chat loop, but a multi-file code repo selected from a Pareto frontier of genetically evolved candidates? RHO moves all its LLM exploration to training time, then runs that repo on scenes it was never trained on. 🧵👇🏽
@matei_zaharia my main challenge in using gepa in practical agentic problems in my field (investment research and forecasting) has been cost -- a single agentic run could cost ~$40 each; so I have found costs add up very quick
@matei_zaharia @LakshyAAAgrawal @aleksizy The stock proposer in GEPA overfits and is not very good.
@GaryMarcus Details of my analysis 👇Next steps
@matei_zaharia That's helpful; we should try using much smaller models and benchmark against non-optimized large models.
@matei_zaharia We have lots to talk about
@matei_zaharia ZI’d mention the other major blocker beyond cost is speed. I was very excited when cerebras announced Kimi k2.6 at 1K tokens/second but doesn’t seem like it’s widely available. I suppose a small model might solve that too to a degree
@matei_zaharia Neurosymbolic approaches keep coming back because pure neural nets are opaque in ways that matter for robotics. The moment something breaks in the field you want to know why, not just that it did.
@GaryMarcus ai search generating code hybrids actually makes sense for robotics
what field surprised u the most seeing it in so far?
Thanks for the feedback @madmaxbr5! We are soon going to have a large upgrade that fixes this, and have also supported easily integrating custom proposers (https://gepa-ai.github.io/gepa/guides/claude-cli-as-proposer/) for a long time now.
The stock proposer is built in a way that it can work with the smallest models (initially GEPA was launched with tests on Qwen3-8B, and maintains backward compatibility so far), so we have avoided including detailed instructions there.
Dropbox discussed how they include some very lightweight instructions in the feedback returned to GEPA that prevented overfitting, and instead got good generalization (https://dropbox.tech/machine-learning/optimizing-dropbox-dash-relevance-judge-with-dspy).
@matei_zaharia Training-time search over code policies should leave replay artifacts. I would want each candidate repo to expose preconditions, tool-to-world effects, safety checks, and failure traces so the winning policy is debuggable when a scene breaks.
@GaryMarcus
@GaryMarcus the pattern holding up: pure scaling wins until someone tries a hybrid approach on the same benchmark and beats it. that's not a one-off anymore
@GaryMarcus @matei_zaharia Gary has been consistent on this for years and the evidence keeps accumulating in specific domains. The question is whether neurosymbolic architectures can generalize without the same brute-force scale that pure neural approaches require.
