AICan language models invent solutions to new problems?
A commentator challenges the claim that large language models are limited to memory and retrieval, arguing that their learned representations allow them to find new solutions.
TLDR
A commentator disputes a quoted claim that language models cannot invent solutions to new problems, pointing to von Neumann as an example of strong memory coexisting with creativity. They argue that training shapes a model’s representations rather than simply storing facts. In a follow-up reply, they say the network’s information lies in relationships between numerical weights, while describing the limits of those relationships’ expressivity as unclear.
Can language models invent solutions to new problems?
A commentator challenges the claim that large language models are limited to memory and retrieval, arguing that their learned representations allow them to find new solutions.