Researchers Clarify Chinchilla Paper's Linear Scaling Law Over Kaplan Findings

Most people seem to have interpreted this as “how do I find the optimal scaling regime.” That’s not what I’m asking. Let’s say that I’ve determined the compute-optimal frontier and have decided what P and D I want to use for my large training. Now, after doing that, I want to…