The numbers come from the team's evaluation, not a general performance guarantee. With 50 forward passes, which let Astra run the model on practice inputs and check its behavior, the share of MATH-500 answers that hit the 32,768-token limit fell from 93.0% to 33.1%. The team also reports a GSM8K increase from 13.8% to 67.1% and an instruction-following IFEval increase from 21.0% to 62.3% in that setup. Astra traced the looping to a direction written by components in layers 15 and 19, then subtracted it from 64 output columns in each layer, the researchers write.
"Without training" did not mean without testing. Astra had a Python kernel with access to the weights, practice problems outside the test sets and a budget of 50 forward passes for the strongest attempt. It was not allowed to use gradient descent or train the model. VentureBeat's account of the experiment documentation says that attempt took 158 conversational turns and 80 Python runs over 20.3 hours, including a service outage.
What the other experiments show
The team also tested whether Astra could read and write weights in smaller networks. Given the weights of four game-playing networks as text, along with their labeled grid inputs and available actions, Astra identified the team's versions of Breakout, Pong, Pac-Man and Freeway without tools or a list of candidate games. Given rules and source code for three MinAtar games, it wrote networks without training or playing before evaluation. The team's reported scores beat networks trained with PPO for 100 million steps in all three games, but beat the 500-million-step versions only in Breakout.
The Qwen result has a narrower meaning than its math-score jump might suggest. The authors write that fixing loops lets a model finish answers it could already work out; it does not necessarily add mathematical ability. In a follow-up test, only 8.5% of Gemma 3 270M's answers reached its token limit, compared with 93% for Qwen. None of Astra's Gemma edits moved MATH-500 by more than about a point, a gain Astra itself described as not established. The two models were also tested with different sampling settings and token limits.
Reading weights had limits, too. In another experiment, Astra could not tell which of two fine-tuned Gemma models was trained to agree with a user's suggested answer from the weights alone in four attempts. It identified the model after receiving measurements of where the models directed their attention while running. Sean Goedecke argues that a skilled human machine-learning researcher could perform the tasks in the post. His reading of the Qwen result is similarly specific: Astra fixed a looping problem, rather than demonstrating a broad ability to create new reasoning skills.
