Looped mdlARC’s developer, posting as @evilmathkid, reported on November 20 that an 18.9-million-parameter model achieved a 70.25% top-two score on ARC-AGI-1’s public evaluation. Training included demonstrations from the evaluated tasks, but not their test answers. Source Source
The accompanying writeup puts compute at 3.7 H200 GPU-hours, roughly $13, for one training-and-evaluation run—not the entire research project. Source
The developer also launched ARC Data Squeeze, a benchmark built around a deliberately restricted collection of 1,307 public tasks. Source Source
ARC tasks present example input/output grids that follow a hidden rule, then ask for the exact output grid for a new input. The reported percentages are top-two scores, meaning either of the system’s two final attempts can be correct. Source
The writeup also reports 49% on ARC-AGI-1 with a 1.05-million-parameter model. On ARC-AGI-2’s harder public evaluation, it reports 15.14% using an 18.9-million-parameter model. The two benchmarks use separately trained models with the same method, not one identical checkpoint producing both scores. Source
The evaluation demos are part of training
The public-evaluation experiments train from scratch on public ARC tasks and ConceptARC, including demonstrations belonging to the tasks being evaluated. Their test answers remain withheld; the benchmark also permits using their test inputs. This is transductive learning, not a test on wholly unseen tasks. Source
According to the writeup, these models use no generated puzzles, text training or pretrained language model, and receive no separate test-time fine-tuning. They do, however, generate answers across 80 rotated, reflected or recolored views of each task. The reported 70.25% run also re-ranks its top 20 voted candidates using the model’s own likelihood before choosing its final attempts. Small model, yes. Single guess, no. Source
More passes through the same layers
Looped mdlARC writes its output one grid cell at a time. Its defining move is to run the middle four of eight transformer layers six times, producing 28 block passes while reusing the same weights. It doesn’t repeatedly rewrite a completed answer; once a cell is written, it stays written. Source
The developer’s experiments suggest that looping helps this setup, but the writeup acknowledges that comparison models weren’t individually tuned and that a tuned non-looped model might close some of the gap. Its explanation for why looping helps remains a hypothesis. Source
ARC Data Squeeze allows different architectures and compute budgets while fixing the training data. Its main rules prohibit outside data, pretrained models, program search and rule-based answer filters. LLM coding assistants are allowed, so “no LLMs” is a restriction on the learning pipeline—not a declaration that nobody asked one for help writing code. Source
Keep the Kaggle result in its own box
These figures are separate from the developer’s earlier 100-million-parameter Kaggle model. That model used roughly 162,000 puzzles, including data produced by LLM-written generators, and about $600 in offline pretraining compute. The writeup reports a 37.08% Kaggle score that finished scoring after the deadline; it says the official leaderboard retained the previous model’s 35.69%. None of that belongs on the smaller experiment’s $13 tab. Source Source
The scores and compute estimate are the developer’s own reports; independent reproduction and cost verification are not included in the announcement or writeup. ARC Data Squeeze’s rules require public code at a specific commit and model weights for record entries. Source Source
Comments