Researcher Reports Bugs Found in PyTorch and vLLM
AI researcher Jack Morris posted about using coding agents to find bugs in several frameworks.
Jack Morris posted that a combination of paranoia and coding agents helped him find bugs in PyTorch, PyTorch/ao quantization, slime, Tinker, vLLM, and FlashQLA GDN kernels during July. He noted he does not know how to write kernels. John Thickstun replied that agents should be used to poke holes in research code and improve personal understanding. Charlie Marsh stated agents can find real bugs in software almost indefinitely. Morris replied by asking unfortunately for whom.
it's crazy to say, but through a combination of excessive paranoia and fairly standard use of coding agents, i have found bugs in - PyTorch - PyTorch/ao quantization - slime - Tinker - vLLM - FlashQLA GDN kernels ...all just in July. I don't really even know how to write…
Combined views
Reactions
3 postsJT 
John Thickstun@jwthickstunThis the way: using agents to write research code you don't understand is a recipe for disaster. Use agents to obsessively poke holes in your research code and improve your personal understanding.
- likes: 9
- replies: 1
- bookmarks: 4
- reposts: 2
CM Charlie Marsh@charliermarsh@jxmnop Unfortunately I now believe that agents can find real bugs in software ~ad infinitum
- likes: 89
- replies: 6
- bookmarks: 3
- reposts: 0