AISWE-Serve benchmark tests whether AI agents can build inference engines
An Nvidia team says it built 53 tasks from real SGLang engineering work to test whether AI agents can develop inference engines that serve real models.
TLDR
An Nvidia team announced SWE-Serve, a benchmark built from 53 tasks drawn from real SGLang engineering work. The team wants to test whether AI agents can develop an inference engine and make it serve real models, emphasizing the importance of live-serving tests.
SWE-Serve benchmark tests whether AI agents can build inference engines
An Nvidia team says it built 53 tasks from real SGLang engineering work to test whether AI agents can develop inference engines that serve real models.
