Threadweaver Open-Sources Parallel LLM Reasoning With 3x Speedup
Related Stories
2 linkedThreadWeaver: Adaptive Threading for Efficient Parallel ReasoningScaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but inherently sequential decoding leads to substantial latency. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that achieves accuracy on par with popular sequential reasoning models while significantly reducing inference latency. ThreadWeaver utilizes a two-stage parallel trajectory generator for high-quality data creation, a trie-based training-inference co-design for compatibility with off-the-shelf engines, and a parallelization-aware reinforcement learning framework (P-GRPO). Across six mathematical reasoning benchmarks, ThreadWeaver achieves accuracy comparable to cutting-edge sequential models (e.g., 79.9% on AIME24) while delivering up to 1.53x average speedup in token latency.3h agothreadweaver-parallel.github.io
GitHub - facebookresearch/threadweaver: The implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language ModelsThe implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language Models - facebookresearch/threadweaver6h agoGitHub
GitHub - facebookresearch/threadweaver: The implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language ModelsThe implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language Models - facebookresearch/threadweaver6h agoGitHub