ARC-AGI-3 study compares how humans and AI agents solve unfamiliar games

Researchers synchronized players’ spoken thoughts with their actions, noting when they showed an understanding of game mechanics. In their comparison, humans expressed uncertainty in 37% of trajectories, versus 18% for language-model agents. Humans gave qualitative accounts of mechanisms in 50% of trajectories, versus 14% for agents. The researchers say the findings suggest people and language models may develop working understandings of unfamiliar environments differently.