AI Agent Autonomously Trains 10M-Parameter Transformer in Free Colab

Couple days back @swyx posted a challenge: code a ~10M transformer in JAX/Flax/Optax, run it in free Colab, and train it on addition w/ your agent!

I gave Codex the screenshot + /goal.

It controlled Colab through Chrome, used my signed-in session, handled runtime/editing weirdness, ran the T4 job, then used subagents to audit the result.

End state: 10,652,557 params, ~19 min train, 99/100 exact random checks 🤯

Still needs cleaner evals, but autonomously babysiting the training run over chrome is pretty wild!