Neural Thickets' reported gains may not need weight perturbations
Random biases to output logits reportedly matched or beat full-weight perturbations on most reasoning tasks tested.
TLDR
A new preprint challenges the idea that Neural Thickets' reported gains require random changes to model weights. Its authors report that adding random biases to output logits while leaving weights untouched matched or beat full-weight perturbations on most reasoning tasks tested. They propose that varied predictions, validation-based selection and voting may explain the gains, but the abstract gives no task-by-task breakdown.
Combined views
44
9 Sources, first seen ago
