Bonsai 2 27B build claimed to run on an 8GB GPU
Its builder reports 20,964 downloads in the first week and says prebuilt bundles for RTX 30-, 40- and 50-series cards need no compiling.
TLDR
The builder says their version of PrismML’s ternary Bonsai 2 has a Qwen 3.8 MTP head added back and runs locally on an 8GB GPU. They report that a contributor’s 8GB 3060 Ti ran it at 42.8 tokens per second with 98k context; they also claim a 16GB 5060 Ti reached 67 tokens per second with vision loaded and the full 262k context window.
Combined views
7.8K
2 Sources, first seen 5h ago
