Claude Opus Generates Three.js Code for Lord of the Rings Scene

@karpathyWe're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It's kind of janky but fun. But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all. I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free". There might be a lot more. But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand. Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

@mckaywrigley@karpathy someone *please* fund doing the entire trilogy like this as a benchmark.

@saranormous@karpathy it’s time for The Mind Game

@elonmusk@karpathy Yah

@AndrewCurran_@mckaywrigley @karpathy OpenAI will have Astra do the balrog scene in retaliation.

@kimmonismusAs models improve, the benchmarks we use to evaluate them have to evolve too. “Create an SVG of a pelican riding a bicycle” was once surprisingly helpful. Today, frontier models routinely produce convincing results, so the test tells us increasingly little about where the actual frontier is. Karpathys experiment is a fascinating attempt to push the benchmark forward: give Opus 5 the opening of The Lord of the Rings, a massive token budget, and two hours to turn it into an interactive Three.js world. And voila! This tests far more than one-shot generation: The model has to maintain coherence over thousands of lines of code, translate prose into a spatial system, coordinate objects and animations, and continuously inspect its own work. Really love where the testing / benchmarks are moving to! Love to see GPT-5.6 and the new DeepSeek flash on this one.

@PimDeWitte@karpathy :)

@PimDeWitteState exploration is a different set of skills than state generation. LLMs are excellent state generators. Not so much state explorers, especially for modalities outside of text space. About time somebody fixed that… :)

@Teknium@karpathy @lu_sichu I doubt opus would let a regular user generate anything lord of the rings..

@karpathyMore on the pelican on the bicycle test from @simonw: https://simonwillison.net/2025/Jun/6/six-months-in-llms/ I uploaded the source here so it's playable in the browser, forkable etc. https://karpathy.ai/lotr-movie/ Look out for GTA Hobbiton dropping before GTA VI :)

@bilawalsidhu@PimDeWitte Finna explore all the states

@xf1280Testing LLM by asking it to create your favorite game that you feel nostalgic about. Everything here is procedurally generated; not one shot since I am obsessed with making it real.

@lishali88Is it on the roadmap to improve this weakness? We definitely noticed this shortcoming at @Rosebud_AI when trying to do long horizon work for making games. One example of how poorly the vision aspect works when just asking Claude to look at screenshots of a game is it barely recognizes the difference between a car in a scene that is upside down versus right side up.

@huyberyCool!! building games is a great way to test LLM. It covers reasoning, planning, coding, tool (lib) use, multimdal understanding, and more. But it seems like the community still lacks a standardized benchmark for building games?

Claude Opus Generates Three.js Code for Lord of the Rings Scene

TLDR

14 Sources

Combined views

Claude Opus Generates Three.js Code for Lord of the Rings Scene

TLDR

Combined views

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

14 Sources