Loading activity…
@industriaalist
cofounder Q Labs (http://qlabs.sh) solving data efficiency and generalization
AI
A user says ideas from Nick Cammarata’s interpretability essay inspired the work, which is still in progress.
AI
Its team claims activation perturbations can approach backprop, but an outside researcher questions the evidence.
AI
A poster says they've figured out how to use zeroth-order optimization for transformer pretraining and argues that many core assumptions in optimization research are wrong.
AI
For multi-epoch training, a project participant says the optimal number of loops increases with the compute budget and looping has a useful regularizing effect.
