On Oct. 30, 2026, @yus167 introduced unified training as a single algorithm for pretraining, midtraining and post-training of foundation models. @yus167 expects the approach to simplify infrastructure and provide a framework for better science.
@yus167 says the proposed change makes data design, including what data to use, when and how much, the remaining decision. @yus167 says static data and reinforcement-learning data could be mixed at any point in training, with staged training remaining one possible arrangement. The account starts with pretraining, midtraining and supervised fine-tuning, which @yus167 says already share a maximum-likelihood objective. To bring reinforcement learning into that framework, the researchers draw on earlier Maximum Likelihood RL work and propose extending it from the current policy to an arbitrary generator policy. @Zanette_ai says the approach also covers reinforcement-learning objectives including REINFORCE, shifting attention toward what data a model sees, in what proportions and in what order.
@yus167 described the team's studies in controlled and practical settings as preliminary. In one post comparing approaches, @yus167 reported better in-distribution and out-of-distribution performance when mixing pretraining, midtraining and reinforcement-learning data than with staged training. The best-performing configurations used up to four epochs of midtraining data, the post noted. @Zanette_ai also reported that the team's experiments can improve accuracy, coverage and compute efficiency over conventional staged training. Those comparisons remain preliminary by @yus167's account.
Comments