Tiny transformer reportedly draws a picture in 30 seconds
A builder says a 308,224-parameter drawing model fits within a budget of 128K words of ROM on two 1989 EPROM chips and 48K of RAM.
TLDR
A builder says roughly 80 training runs produced a transformer small enough to fit the memory limits and still draw. It uses 4-bit weights, and the builder credits new programming-language operations for streaming dot products with cutting drawing time from three minutes to 30 seconds. As of the September 25, 2026 update, the project was not released. Plans included putting it on an FPGA—a reconfigurable chip—releasing a video of it drawing off a coin cell, and open-sourcing it on Hackaday.
