Many users express optimism that LLMs can scale much further because the compute ceiling is still far off, while others argue architecture matters more than parameter counts or dismiss scaling timelines as perpetually delayed.
Based on 6 visible X reactions from 21 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@scaling01 The difference is the brain isn’t growing exponentially. 3 years from now it’s going to be the opposite. Then what? That’s when the really awesome stuff starts happening
@scaling01 '1-3 OOMs ahead' is the scalability equivalent of 'next quarter.' Always just over the horizon.
@scaling01 Numbers are fine but system awareness isn't measured by parameter count
@scaling01 architecture matters way more than raw parameter counts
@scaling01 '1-3 OOMs ahead' is the scalability equivalent of 'next quarter.' Always just over the horizon.
@scaling01 real, we havent even hit the compute ceiling yet
@scaling01 1Q params would be wild
we still don't have 10T models that are publicly available / broadly deployed even Fable is likely no more than 4T params (although distilled from a 10T model), most other models are below 2T the neocortex alone has ~164 trillion synapses we also know young children have much more synapses, with estimates for the whole brain going up into the low quadrillions, but these are super uncertain for some reason no one has cut up a baby brains it's probably closer to 1.5 to 2.5 OOMs (starting from ~4T Fable), with 10T Mythos it's 1-2 OOMs
LLMs are still so much smaller than the brain like we still have 1-3 OOMs of scaling ahead of us
Many users express optimism that LLMs can scale much further because the compute ceiling is still far off, while others argue architecture matters more than parameter counts or dismiss scaling timelines as perpetually delayed.
Based on 6 visible X reactions from 21 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@scaling01 1Q params would be wild
we still don't have 10T models that are publicly available / broadly deployed even Fable is likely no more than 4T params (although distilled from a 10T model), most other models are below 2T the neocortex alone has ~164 trillion synapses we also know young children have much more synapses, with estimates for the whole brain going up into the low quadrillions, but these are super uncertain for some reason no one has cut up a baby brains it's probably closer to 1.5 to 2.5 OOMs (starting from ~4T Fable), with 10T Mythos it's 1-2 OOMs