Shared-prefix trees reportedly sped up RL forward and backward passes by over 26x in experiments

A poster says asynchronous reinforcement learning works best with large batch sizes, but forward and backward passes become a major bottleneck, especially when agentic rollouts exceed 100,000 tokens. In their experiments, a tree structure that shares prefixes across trajectories sped those passes up by more than 26 times. They credit Amine with the improvement to TRL.