FPGA Board Runs Tiny LLM At 12,000 Tokens Per Second · Digg

/AI

FPGA Board Runs Tiny LLM At 12,000 Tokens Per Second

Combined views

60.1K

1 post, first seen 11d ago

311 likes comments

FPGA Board Runs Tiny LLM At 12,000 Tokens Per Second

Reactions

1 post

LL
Lamb Labs (YC S26)@LambLabs 11 days ago
12,000 tok/s on a 250$ FPGA board. It's a small model, and doesn't answer coherently to any question or prompt really. try it out on but don't expect a good chat. at least its VERY fast
trylittlelamb.com
little lamb — Lamb Labs
This is a proof-of-concept chip design, not a model intended to produce good results. It’s fast, though — and it might give you a laugh. Little lamb will get better as it
- likes: 311
- replies: 33
- bookmarks: 164
- reposts: 14

Combined views

60.1K

1 post, first seen 11d ago

311 likes

33

164 saves

14 reposts

33 comments

164 saves

14 reposts

Reactions

1 post

LL
Lamb Labs (YC S26)@LambLabs 11 days ago
12,000 tok/s on a 250$ FPGA board. It's a small model, and doesn't answer coherently to any question or prompt really. try it out on but don't expect a good chat. at least its VERY fast
trylittlelamb.com
little lamb — Lamb Labs
This is a proof-of-concept chip design, not a model intended to produce good results. It’s fast, though — and it might give you a laugh. Little lamb will get better as it
- likes: 311
- replies: 33
- bookmarks: 164
- reposts: 14