LangChain launches LangSmith Engine for AI agent fixes
LangChain launched LangSmith Engine to monitor traces from AI agents and automatically generate ready-to-merge code fixes along with new evaluators. The system analyzes runs such as docs_agent, which included eight retriever calls, and surfaces recommendations for code changes, evaluator creation, and offline examples. It eliminates manual trace review by producing a fix, an evaluation, and an updated agent from every processed trace. The feature is available now at smith.langchain.com.
#702Harrison Chase@HWCHASE17“Dependably for LLM agent failures”
Saurabh@sauvast@hwchase17 Started on this and finding it awesome; also LangSmith engine sparked an idea. The "Dependabot like for LLM agent failures". LangSmith Engine gives you the smoke detector. The natural next layer is a sprinkler system; an auto-remediation with a human approval gate. A four-stage pipeline comes to mind: Classify → Patch → Eval → Shadow Trying it and will share trace results. This is a real gap in the LLMOps ecosystem; glad to see it being closed. 🔥 Will keep updated on the progress @LangChain_OSS
@hwchase17 Looks cool will try it out!
Harrison Chase@hwchase17🚀Launching: LangSmith Engine LangSmith Engine is an agent that sits on top of your traces It runs in the background and automatically identifies issues It then proactively suggests action items (code changes, evaluators to add) Try it today: http://smith.langchain.com