Ellen Su and colleagues report turning epiplexity—a measure of the structural information a learner can extract with limited computing resources—into a guide for selecting and generating AI training data. Their abstract says higher epiplexity predicts better performance on later tasks in both settings. Source
Su highlighted a synthetic-data result in a November 19, 2026, thread. The paper, Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization, first appeared on arXiv on August 12 and was revised November 4. SourceSource
The target is out-of-distribution generalization: transferring what a model learns to settings beyond its training. The authors’ hypothesis is that data rich in structural information may provide reusable building blocks for unfamiliar tasks. Epiplexity itself was previously proposed; this work aims to turn it into a signal that guides training as it happens. Source
Two ways to steer the training data
For data selection, the researchers model how training loss, a measure of prediction error, changes as a learner trains on different types of natural data. They use those relationships to predict epiplexity gains from further training, then adjust how often each type of data is sampled. The reading list can change while the learner works through it. Source
A generator model produces synthetic data for a separate learner. Its reward comes from the learner’s step-by-step change in epiplexity, calculated from the learner’s loss. That reward steers generation toward higher-epiplexity data. The generator has a pupil to answer to. SourceSource
The authors report this predictive relationship for tasks attempted without task-specific training (zero-shot) and tasks a model is further trained to perform (fine-tuning). Source
The 2.8-point claim—and its limits
Su reports that training on epiplexity-guided synthetic data yielded a 2.8-point gain on the OOD GLUE benchmark over pretrained GPT-2, without seeing any additional tokens of real data. “Additional” matters here: the comparison is against an already pretrained model, not a demonstration of learning from scratch without real data. Source
The arXiv page also lists code for EpiSelect and EpiGen, although those repositories were not inspected for this article. Source
The full experimental tables and evaluation protocol weren’t available for this article, so the conditions behind the 2.8-point result and its variability could not be assessed. No independent replication was supplied.
Comments