@industriaalist
cofounder Q Labs (http://qlabs.sh) solving data efficiency and generalization
AI
Its team claims activation perturbations can approach backprop, but an outside researcher questions the evidence.
AI
A poster says they've figured out how to use zeroth-order optimization for transformer pretraining and argues that many core assumptions in optimization research are wrong.
AI
For multi-epoch training, a project participant says the optimal number of loops increases with the compute budget and looping has a useful regularizing effect.
