AI Judge Discovers FlashInfer Edge Cases Affecting Inference
Josh Tobin reports using their reward hacking judge to identify issues in FlashInfer.
Josh Tobin, co-founder of Recursive Superintelligence, posted that an AI judge developed for performance optimization tasks found edge cases affecting inference in vLLM and SGLang. The post links an arXiv paper on Flash Attention stability and a GitHub pull request in FlashInfer that handles extreme negative logits in masked softmax by replacing a finite sentinel value. Tobin described the discovery as a small win for automated research during new autoresearch work. The tweet attributes the fix to application of the judge rather than manual review.
Combined views
80.2K
1 post, first seen 6d ago
AI Judge Discovers FlashInfer Edge Cases Affecting Inference
Josh Tobin reports using their reward hacking judge to identify issues in FlashInfer.
Josh Tobin, co-founder of Recursive Superintelligence, posted that an AI judge developed for performance optimization tasks found edge cases affecting inference in vLLM and SGLang. The post links an arXiv paper on Flash Attention stability and a GitHub pull request in FlashInfer that handles extreme negative logits in masked softmax by replacing a finite sentinel value. Tobin described the discovery as a small win for automated research during new autoresearch work. The tweet attributes the fix to application of the judge rather than manual review.
Reactions
1 postJT 
Josh Tobin@josh_tobin_A fun small win for automated research: We found and helped fix some edge cases that could affect inference performance in vLLM and SGLang. In our last blog post, we described a reward hacking judge we developed for performance optimization tasks. While applying the judge to…
- likes: 400
- replies: 12
- bookmarks: 328
- reposts: 42