Many users praise Tencent's 1-bit 295B Hy3 model for delivering strong benchmarks with minimal degradation on a single GPU, while some object to high memory demands and call the release unoriginal.
Based on 27 visible X reactions from 88 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@TencentHunyuan Love this. The 1-bit/4-bit release makes Hy3 incredibly accessible for local runs. For teams who'd rather not manage the GPU, we put Hy3 on a hosted API at Merius — free all of July. Same model, zero setup: http://merius.ai/models/hy3-tencent
via X@TencentHunyuan Amazing work. A VQ-based ~0.5-bit version for the routed experts would be really interesting. It might make Hy3 runnable on consumer PCs with 64GB RAM.
via X@TencentHunyuan 🔥🔥🔥 Very nice work making this even more accessible 🙏
via X@TencentHunyuan Great to see powerful AI on smaller hardware.
via X@Anon_not_a_non @TencentHunyuan get some work lowlife ped0!
via X@teortaxesTex 这显存配置简直是炼丹师的噩梦
via X75.4% SWE Bench Verified / 53.9% SWE Bench Pro on 1 bit quantisation is 🤪 This is in line with my expectations & you can expect even lower drop off with NVP4 base trained models - why not run everything binary? 88 Gb so works on a Macbook Max
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
This is v bearish for RAM companies. We saw strong ternary numbers from @PrismML today (27b dense on a mobile!) but the tiny degradation (~5%) from 16 bit to 1 bit precision by Tencent on a near frontier 300b model is the biggest news of the day. Star https://github.com/tencent/AngelSlim
75.4% SWE Bench Verified / 53.9% SWE Bench Pro on 1 bit quantisation is 🤪 This is in line with my expectations & you can expect even lower drop off with NVP4 base trained models - why not run everything binary? 88 Gb so works on a Macbook Max https://twitter.com/TencentHunyuan/status/2076953120765280284
The 1-bit variant retains performance within 6% of baseline.
@TencentHunyuan Great to see powerful AI on smaller hardware.
via X@Anon_not_a_non @TencentHunyuan get some work lowlife ped0!
via X@teortaxesTex 这显存配置简直是炼丹师的噩梦
via X75.4% SWE Bench Verified / 53.9% SWE Bench Pro on 1 bit quantisation is 🤪 This is in line with my expectations & you can expect even lower drop off with NVP4 base trained models - why not run everything binary? 88 Gb so works on a Macbook Max
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
This is v bearish for RAM companies. We saw strong ternary numbers from @PrismML today (27b dense on a mobile!) but the tiny degradation (~5%) from 16 bit to 1 bit precision by Tencent on a near frontier 300b model is the biggest news of the day. Star https://github.com/tencent/AngelSlim
75.4% SWE Bench Verified / 53.9% SWE Bench Pro on 1 bit quantisation is 🤪 This is in line with my expectations & you can expect even lower drop off with NVP4 base trained models - why not run everything binary? 88 Gb so works on a Macbook Max https://twitter.com/TencentHunyuan/status/2076953120765280284
91 GB for a 300B model
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
even less actually
via XMany users praise Tencent's 1-bit 295B Hy3 model for delivering strong benchmarks with minimal degradation on a single GPU, while some object to high memory demands and call the release unoriginal.
Based on 27 visible X reactions from 88 accounts; directional sample.
Ask a question below.
Published answers will appear here.
91 GB for a 300B model
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
even less actually
via X