Announcement
A single test-set exposure may yield loss below an uncontaminated pretraining corpus's irreducible error
A co-author says overtraining with fresh data mitigates contamination's effects in generative evaluations.
TLDR
A co-author says the paper was rejected from NeurIPS 2026 and disputes the reviewers' judgment. They report that even one exposure to a test set can yield loss below the irreducible error of an uncontaminated pretraining corpus. They also say overtraining with fresh data mitigates contamination's effects, while supervised fine-tuning on training data can have opposing effects on test results depending on pretraining contamination.
Combined views
591
6 Sources, first seen ago
