Learning Fast and Slow triples LLM sample efficiency

Thanks for sharing, I agree with the motivations and ideas you mentioned, for better understanding it can be seen as FST instantiation where: *slow weights* update rule = self distillation *fast weights* update rule = GEPA

we did try one experiment in the same spirit in which we distilled FST fast-weights (gepa style prompt) back to the model using on-policy reverse KL (similar to SDFT paper) and leads to some learning but performs worse than FST w/ GEPA+RL (@LakshyAAAgrawal explained this result in more detail here: https://x.com/LakshyAAAgrawal/status/2055164968748859403).

The idea of combining RLVR signal with self distillation signal is also very interesting and we did try that as well some time back in a related project, we are planning to release that as well soon.

Figure 14 (in appendix) provides a breakdown of various ablations and comparative experiments.

First, we see that FST-trained model, even without the co-optimized prompt (“Slow only (FST w/o prompt)”, green) gains +4.2pp over RL-only model.

Next, FST with the co-optimized prompt (“Slow + Fast (FST)”, green) outperforms RL-only model with GEPA-optimized prompt (“Slow + Fast (RL + GEPA)”) by +3.8pp, which increases further if we run GEPA again on FST-optimized model instead of just taking the co-optimized prompt.

Further, we see this on other datasets as well: ⦁ On HoVer: FST 30.8% vs RL+GEPA 24.9% (+5.9pp), even using 3x fewer data points ⦁ In easy-to-hard generalization, training on Polaris and testing on HMMT25, we see: FST 39.7% vs RL+GEPA 37.1% (+2.6pp), using 1.4x fewer datapoints. This is with the co-optimized prompt. Running GEPA on top of the FST model achieves 41.4% (+4.3pp).

We tested this. We call it FST-distill in the paper (Appendix H, Figure 13). The GEPA-evolved prompt over the same model with frozen weights forms the teacher, while the student sees just the problem and is trained via on-policy reverse-KL distillation against the teacher.

On HoVer: ⦁ GEPA-only: 24.8% ⦁ FST-distill: 31% ⦁ RL alone: 45% ⦁ FST: 48%

FST-distill rises 6.2pp above GEPA alone but plateaus 14pp below plain RL and 17pp below FST.

The teacher is upper-bounded by what GEPA can elicit from a frozen base, so the student's reward signal is also indirect and never climbs through the higher-capacity slow-weight channel directly. The fast channel alone is not enough to saturate the joint ceiling, and direct policy-gradient updates on θ against reward, run alongside the fast channel, account for the remaining gain.

In addition to higher rewards, FST maintains high actor entropy throughout training due to the Pareto-frontier prompt-population design.

Having said that, distillation is an exciting slow-weight update mechanism, and we'd be excited to see richer combinations of FST with distillation-based approaches.