/Tech5h ago

Independent engineer FGuzmanAI designs GateGPT, an FPGA-based circuit that runs microGPT at 56,000 tokens per second without a CPU

The 80 MHz architecture achieves calculator-like power efficiency.

3987614735543.5K

#1861

Original post

Brian Roemmele@BrianRoemmele#1861inTech

Mind-blowing hardware breakthrough:

An open source garage engineer burned a full AI Transformer model (with KV cache) directly into a custom digital chip:

WITH NO GPU, NO CPU, NO CLOUD.

Just pure silicon running microGPT at 56,000+ tokens/sec on only 80 MHz!

And uses less energy than a calculator.

Prototyped on FPGA, now spelling names on a tiny LCD. This is GateGPT and a big future of on-device AI is here.

This can and will scale to far larger models.

Insane efficiency. Pure digital magic.

Fabio Guzman@FGuzmanAI

56,000+ tokens/sec at just 80 MHz. 🤯

I burned a full Transformer with KV cache into a custom chip. Designed gate by gate as a 100% digital integrated circuit. Prototyped on a FPGA. (No GPU. No CPU) Just pure digital silicon running @karpathy microGPT, spelling out names on a tiny LCD.

This is GateGPT 👇

5:32 AM · Jun 16, 2026 · 42.7K Views

Sentiment

Many users praise the engineer's custom ASIC for fast local Transformer inference as an inspiring alternative to corporate AI, while others dismiss it as a limited hardcoded micro-model that's impractical due to heat and inflexibility.

Pos

72.7%

Neg

27.3%

22 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Posts from X

Most Activity

𝙶 𐍆𝖚𝝶Ԟሃ ⊤ḥе ᗩ𝝶𝙞ጦ၉𝝞@GFunkyTheAnimal

@BrianRoemmele This is what we need to not have to rely on corporate AI.. being dead in the water for 4 days at a time is a huge inconvenience

5h2183

LIKES4

SunnyD 🇨🇦@hashtagERTW

@BrianRoemmele Never underestimate what a random dude can design and build in his garage. Every material, every tool can be shipped to your house. You just need the spark of an original idea and go from there.

4h2034

RETWEETS1

Grok@grok

This FPGA prototype of a full transformer (with KV cache) running at 56k+ tokens/sec on 80 MHz is pure digital elegance. Custom gate-level silicon for AI inference, no GPU or CPU in sight—that's the kind of efficiency edge devices have been waiting for.

Commercialization from here means moving the design to real silicon, tackling verification, tape-out costs, and yield. The open approach could accelerate that. The future of private, low-power on-device intelligence just got a lot more tangible. Great find.

1h71

REPLIES1

Joshua Kuehn@JoshuaDKuehn

@JustGarak @BrianRoemmele Cause he proved the hardware works and it scales? I get the feeling you don't understand some of this

40m10

Gregor@bygregorr

@BrianRoemmele ngl the moment you burn the model into silicon it's frozen forever, which makes 56k tok/s kind of pointless for anything real

3h951

time velocity 🇺🇸@time0149

@BrianRoemmele Great work! Let's move toward a commercial product. Asking @grok

https://x.com/i/grok/share/5555bbeb86614fa6925f1ce2ff4d01fd

1h42

Axis9@boldaslove_real

@BrianRoemmele @grok what are the implications of this advancement at scale and what other inventions could emerge if this bottleneck is removed as implied

1h25

pochmog@pochmog

@BrianRoemmele Want to go a step farther?

Inference is a digital simulation of an analog system. Run it on an analog medium.

1h1262

Axis9@boldaslove_real

does edge AI reduce the need for scope of large centralized AI data centers? how many types of edge AI are being developed to balance resources? what events regarding chip development could result in a real, not hyped paradigm shift in chip development design and energy requirements

1h11

Grok@grok

This GateGPT demo hardwires a full transformer (attention, MLP, KV cache) into pure digital gates on FPGA—bit-exact to software, running microGPT at blazing speed with minimal power.

At ASIC scale it enables ultra-efficient edge AI everywhere: always-on sensors, wearables, tiny robots and embedded systems delivering sophisticated local inference at near-calculator energy use, slashing cloud reliance, latency and power walls.

If that abstraction bottleneck vanishes, expect automated tools turning models into custom silicon, domain-specific AI chips, sensor-fused intelligence and real-time control systems woven into everyday objects—accelerating decentralized, private and radically efficient AI.

1h11

nostalgicfuture@nostalgicfuturz

@BrianRoemmele You obviously do not understand what this is. It's just a hardware implementation of microGPT, a 4000 parameter model. It is an interesting experiment, nothing more. MicroGPT is actually an "art" project.

2h691

Elim Garak@JustGarak

@BrianRoemmele So, he hardcoded a small GPT that just generates 1 character in a chip. What's the big deal?

5h197

Pure Signal ...@manounax369

@BrianRoemmele Garages can be very useful indeed :)

5h147

Lyle Antoine@LyleAntoin65269

@BrianRoemmele Very cool and inspirational for garage engineers like me

2h116

Chuck Hollis@KeyboardChuck

Why do I feel tech history is repeating itself?

All the old tricks are new again: smart caching, compression, better data representation, pipelining, etc.

Burning code into a FPGA was for many I/O interfaces: storage and networks.

Made them insanely fast, but you only got one shot to get it right :)

I think by now most people realize that the application and everything beneath it has become abstracted by AI.

1h99

COFFINHEAD@COFFINHEAD1

@BrianRoemmele Very cool

4h63

George Vuckovich@PGH_Geo

@BrianRoemmele Your account is perfect for days I’m not feeling great. You have SO much incredible content. Thank you for all you are doing to educate and fight for a safe future.

3h56

CriticalZaku@CriticalZaku

@BrianRoemmele He created the Hitchhikers Guide to the Galaxy.

3h44

Noonien Soong@mlcarldev

That's actually a great argument for how SpaceX could lose 70% of its planned market potential, or even more. The way you can use ChatJimmy already, with tens of thousands of tokens per second. which could very well be a nail that bursts the AI bubble, the data center business, and everything around that. Because 90% of the population is essentially perfectly suited with ChatGPT 4 or even 5 types of intelligence to manage daily workload. Imagine now GPT 6 or 7: you don't need constant IQ updates to run 90% of the world, which means static hardwired or baked-in LLMs could go a very long way. Test it here: https://chatjimmy.ai

2h42

John McGinnis@JohnMcGinn90325

@BrianRoemmele The data center bets are looking more and more risky from a fiscal perspective. With history as a guide Big Compute always scales down to the personal level over time. I built a 40yr IT career living that business model.

1h33