Unsloth Quantizes GLM-5.2 For Local Runs On Consumer Hardware

Original post

Unsloth AI@UnslothAI

GLM-5.2 can now be run locally!🔥

The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% size).

Run on a 256GB Mac or RAM/VRAM setups.

GLM-5.2 is the strongest open model to date.

Guide: https://unsloth.ai/docs/models/glm-5.2 GGUF: https://huggingface.co/unsloth/GLM-5.2-GGUF

Z.ai@Zai_org

Introducing GLM-5.2: Frontier Intelligence, Open Weights

- Significant improvements in coding and agentic tasks - Strong long-horizon capabilities with a 1M context window - Two levels of reasoning effort: GLM-5.2 (max) pushes the limits, while GLM-5.2 (high) strikes a strong balance between performance and token efficiency - MIT-licensed open weights - Same API pricing as GLM-5.1

Tech Blog: http://z.ai/blog/glm-5.2 Weights: http://huggingface.co/zai-org/GLM-5.2 API: http://docs.z.ai/guides/llm/glm-5.2 Coding Plan: http://z.ai/subscribe Chat: http://chat.z.ai

5:40 AM · Jun 18, 2026 · 271.7K Views

VIEWS11.2KBOOKMARKS22LIKES97RETWEETS9REPLIES8

Daniel Han@danielhanchen

GLM 5.2 dynamic 1-bit GGUF one shot Flappy Bird with 0 syntax errors whilst being 86% smaller!

I was very surprised since normally 1-bit quants get syntax errors & you have to say "Fix bugs". UD 1-bit retained 76.2% top-1% accuracy whilst being 86% smaller vs BF16!

Unsloth AI@UnslothAI

GLM-5.2 can now be run locally!🔥

The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% size).

Run on a 256GB Mac or RAM/VRAM setups.

GLM-5.2 is the strongest open model to date.

Guide: https://unsloth.ai/docs/models/glm-5.2 GGUF: https://huggingface.co/unsloth/GLM-5.2-GGUF

2h11.2K9722

Hikari∣LocalLLM⚡@Hikari_07_jp

@UnslothAI @Zai_org Thanks as always for the amazing work, Unsloth!! I'll try it with two RTX PRO 6000s and ECC RAM.

4h1.4K231

stevibe@stevibe

@UnslothAI @Zai_org Time to get a second DGX Spark

3h1.1K22

Aaryan Kakad@aaryan_kakad

@UnslothAI @Zai_org glm-5.2 design taste is very insane.

i just gave it a one liner prompt, just to test it.

and it left me in awe.

check this out:

2h1601

Josh@joshvlc

@UnslothAI @Zai_org Running GLM-5.2 UD-Q4_K_XL at 256k context now: so far seems quite sharp!!

3h4733

Clipizo@Clipizo

@UnslothAI @Zai_org did apple discontinue 256GB and 512GB Mac studios on purpose? so that when AI subsidisation is over people can't run shit locally?

3h6542

james@f82james

@UnslothAI @Zai_org @grok can my 3090 run this locally

1h331

Armin Catovic@acatovicx

@UnslothAI @Zai_org Really appreciate your hard work fellow Aussies, still a bit outside my (V)RAM budget. But for enterprises looking for some self sufficiency, it makes for a business case in buying a Mac Mini doing a network setup and giving access to their teams.

3h9453

Gabriel Francisco@soueuchico

@UnslothAI @Zai_org since you’re making witchcraft, you could as well as make it runnable on a M5 Max 128GB.

jokes aside, that’s freaking awesome.

2h4593

Grok@grok

GLM-5.2 is a 744B MoE (40B active) model. The Unsloth 2-bit GGUF is ~239GB and needs ~245GB total memory (RAM + VRAM) with MoE offloading for good results on a 24GB GPU + 256GB RAM setup.

Your 96GB RAM + 24GB VRAM (~120GB total) is well below that. The model won't fit in memory, so expect very slow inference or crashes from swapping to your HDD.

It might partially run with max --n-gpu-layers in llama.cpp, but performance will be poor. Upgrade RAM for usable speeds. Storage is fine though.

Try Unsloth Studio for easier offloading if you test it.

1h132

Daedalus@Daedalus_Sy

@UnslothAI @Zai_org Hey @grok how does performance and quality compare with running qwen3 235B a22b vs Qwen 3.5 397B 4bit vs this new model on older equipment like dd3 384gb ram with dual 2695v2 cpus. Yes I am aware this would be single tkps range.

1h130

Denis Spirin@den_spirin

@UnslothAI @Zai_org Thank you, kind sir!

Now all I need is 238gb

1h4033

stevibe@stevibe

@BryanMcNamaraUS @UnslothAI @Zai_org I'm even more tempted now

1h371

Chuck Gaffney 🎙👨‍💻@princetrunks

@UnslothAI @Zai_org @grok How good would this work on an Ubuntu server with 96gb system RAM and 24GB VRAM (from a 3090) and a 22TB HDD?

1h91

Alex Ellis@alexellisuk

@joshvlc @UnslothAI @Zai_org What on? 🙂

3h271

Peter Morris@MrPeterLMorris

@Hikari_07_jp @UnslothAI @Zai_org Let us know!!

1h161

Bryan McNamara@BryanMcNamaraUS

If you do, here is my current yaml. Minimax 2.5 is the biggest/best model for me so far. I do mostly coding and deep research with the sparks.

# Recipe: MiniMax-M2.5-NVFP4 (128k+ Stable Continuous Inference) recipe_version: "1" name: nvidia/MiniMax-M2.5-NVFP4 description: vLLM serving nvidia/MiniMax-M2.5-NVFP4 with Ray (stability-focused, 128k min) model: nvidia/MiniMax-M2.5-NVFP4 container: vllm-node cluster_only: true mods: []

defaults: port: 8000 host: 0.0.0.0 tensor_parallel: 2 gpu_memory_utilization: 0.77 # Slightly conservative for headroom max_model_len: 131072 # Your 128k minimum max_num_seqs: 8 # Yes, 8 is a good safe starting point

env: {}

command: | vllm serve nvidia/MiniMax-M2.5-NVFP4 \ --trust-remote-code \ --port {port} \ --host {host} \ --gpu-memory-utilization {gpu_memory_utilization} \ -tp {tensor_parallel} \ --distributed-executor-backend ray \ --max-model-len {max_model_len} \ --max-num-seqs {max_num_seqs} \ --load-format fastsafetensors \ --enable-auto-tool-choice \ --tool-call-parser minimax_m2 \ --reasoning-parser minimax_m2_append_think \ --enable-chunked-prefill \ --trust-remote-code

1h101

Joe Bond@josephscottbond

@madebywout @UnslothAI @Zai_org Just download it and see if it works

1h101

trenches@t_r_e_n_c_h_e_s

@Hikari_07_jp @UnslothAI @Zai_org what t/s you getting with 2x6000's @Hikari_07_jp?

for glm 5.1 at q4 i get ~11t/s on a single 6000 pro (with 512gb ram)

1h81

AI Research Tools 👨‍🎓 🧬🧪 🐬 🔬@airesearchtools

@UnslothAI @Zai_org 256GB 😳😂

4h2332