Commentary on X
@Fr0oZi this trend will become so much more important over the coming year, as top tier closed source model become too expensive.
4:12 PM · Jul 14, 2026@rohanpaul_ai offline open source chatgpt is a big win for privacy minded users
4:07 PM · Jul 14, 2026@rohanpaul_ai local llms getting this fast is honestly wild to see
4:07 PM · Jul 14, 2026This is so interesting for local LLM. Atomic Chat compressed the 295B-parameter Hy3 (to an extreme 1-bit ) model into a 92GB file and ran it 2.2x faster than its cloud API. They installed the model locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API. The big deal is that the extreme compression kept useful quality after this extreme compression. Both versions built working Flappy Bird, Arkanoid, and Snake games from one-shot prompts, with no crashes or obvious visual gap. Outputs: Hy3 1-bit local: 76.9K tokens, 15.5 min Hy3 cloud API: 75.1K tokens, 34.3 min Hy3 is Tencent's open-sourced Apache 2.0 licensed model, i.e. commercially usable by enterprises. A massive 295B MoE (21B active parameters).
1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compared one-shot outputs. 1-bit Hy3 295B GGUF (92GB) ran locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API Tasks: - Flappy Bird - Arkanoid - Snake Outputs: Hy3 1-bit local: 76.9K tokens, 15.5 min Hy3 cloud API: 75.1K tokens, 34.3 min The 1-bit games look the same as the API ones. Birds fly through the pipes, bricks break, the snake eats and grows. Nothing froze or crashed. Both models even made the same slip: the snake can cross itself and the game does not end. Getting this quality from 1 bit running locally is wild! Run Hy3 GGUF yourself in Atomic Chat in 2 clicks
Combined views
5.9K
2 posts, first seen 3h ago