LLM representations are claimed to have implicit symbolic structure

A researcher says the finding follows an eight-year project into how LLMs handle language, code and math.

1 Source, 20d ago, first seen 20d ago

Overview of the paper.
Title: The Emergent Symbolic Structure of Artificial Neural Networks
Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it?
Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds

TLDR

A researcher announcing a new paper says an eight-year project found implicit symbolic structure in LLM representations. The finding is presented as an answer to how the models excel in symbolic domains such as language, code and math. The researcher later shared a Yale News writeup on the work and likened vectors inside LLMs to pointillist paintings.

Today's Rank

—

Not ranked yet

Combined views

163

1 Source, first seen 20d ago