Susan Zhang responded that simplicity is in the eye of the beholder.
Reality is "Simplicity is the king" is such normie thing to say. Frontier systems are rarely ever "simple".
@OfirPress This is mostly a matter of poor tooling and bad hyperparamter setups. A complex model can be as much as 10x more effective size if done without confounders. Era of dumb scaling is more over by the day imo
Negative users dismiss the simplicity mantra for complex frontier AI systems as a crutch for limited understanding and accuse advocates of talking about it without delivering.
No Digg Deeper questions have been answered for this story yet.
@cloneofsimo simplicity simply in the eye of the beholder
Reality is "Simplicity is the king" is such normie thing to say. Frontier systems are rarely ever "simple".
that’s custom solution for a very custom (and important) problem. complexity is an opinion in a search space
models and harnesses are/should be generalized solutions for most people because the search space varies person to person
there’s a reason Pi wins on a lot of benchmarks compared to popular complex harnesses
@cloneofsimo It eventually simplifies as the technology advances
See SpaceX Raptor 2 vs Raptor 3
@cloneofsimo Things should be as simple as they need to be, and no further.
@cloneofsimo Yes but it doesn't mean that you shouldn't try to streamline what's possible
@cloneofsimo
@cloneofsimo simplicity is the king only after you've waded through the mess
frontier systems start complex and stay that way until forced into simplicity
@cloneofsimo That is the most complicated machine humanity has ever built
@cloneofsimo Nothing is ever simple until it’s solved, then you can improve on the solution until simplicity is achieved. However long that takes
@cloneofsimo Simplicity is a crutch for monkey brains to understand things.
@cloneofsimo You mostly don't need frontier, though; you need maintainable
@cloneofsimo Sophisticated != complex
@cloneofsimo Simplicity is in the eye of beholder
@cloneofsimo *as they can be
@cloneofsimo Like would argue that picrel machine can't be made with 2x fewer components?
@cloneofsimo What you’re missing is that everyone talks about simplicity but few love and deliver it