Reaction
Reinforcement learning's rough experiments versus pretraining scaling 'laws'
A user calls pretraining scaling 'laws' pretentious and praises reinforcement learning for trying crude ideas that could improve gradients.
TLDR
One user says they love reinforcement learning algorithms because they find them less pretentious than pretraining scaling 'laws.' They describe the work as trying ideas that sound foolish but could yield a better gradient from the goal.
Combined views
1.7K
1 Source, first seen ago
