BOOM!
Meet the open source Cambrian Explosion of repulsion of Anthropic!
Meet Qwythos 9B, a Qwen3.5 based GGUF that's both uncensored and quantized for efficiency.
I am running it now and it is brilliant!
A model that can reason through 1 million tokens of context, understand images and text, and even call functions.
Come and take it!
https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF
Many users are excited about the open source Qwythos 9B model's local runnability on consumer GPUs and potential for digital sovereignty, while negative users criticize the lack of benchmarks and doubt its real capabilities.
No Digg Deeper questions have been answered for this story yet.
@BrianRoemmele @grok what are the hardware requirements to run this model
@BrianRoemmele i think this is the abliterated version you are looking for, the one you link get stuck on tien an men ;)
https://huggingface.co/huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-GGUF
ООО! Смотрите, что! Офигеть!
BOOM!
Meet the open source Cambrian Explosion of repulsion of Anthropic!
Meet Qwythos 9B, a Qwen3.5 based GGUF that's both uncensored and quantized for efficiency.
I am running it now and it is brilliant!
A model that can reason through 1 million tokens of context, understand images and text, and even call functions.
Come and take it!
https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF
You're right to point out the inference shape. That's very much central. Intelligence only exists at inference (contrary to the popular notion that it is a latent capacity or capability). Right now, the assumption is that inference has to happen vertically between a "user" and a "model". What I'm building is horizontal, peer distributed inference. The engineering enables nodal networking exchange between members of a network.
Let's say a town has a growing traffic problem. Right now, the paradigm is that the Mayor/City Council/Board figures it out. In this example, they play the role of the vertically located model. The citizens of the town are the "users" who individually, atomically send vertically to the centralized decision maker. The decision comes back down to them from on high.
In a horizontal intelligent system, each citizen is a node and each node has its own AI agent that they grow with their personal context. It helps them surface ideas and information about the traffic problem. Each one has something to offer in terms of an observation or potential solution. When the individual Inti nodes are connected in an IntiNet, they intelligently exchange information until a consensus proposed traffic design solution is arrived at.
A highly intelligent solution is evolved through iterative interactions based on real world inputs and experiences. Many low powered intelligences living on laptops and phones in this way can be equally if not far more powerful than consulting with a centralized Deep Though model that potentially tells you what the people in power want you to think or do.
So this is not one size fits all. It is universally adaptive to the problem shape and scope and to the participants.
What I'm saying here is WELL within reach. We can easily start to roll this kind of distributed intelligence power out in the next couple of years. Starting now. The Big Labs and Big Gov present their "Stargate" centralized AI as the ONLY shape that AI can take.
Bollocks.
@BrianRoemmele This is the exact ordinance required for true digital sovereignty.
When you build parallel, air-gapped architecture, you don't want a massive, sanitized Cathedral model (Anthropic) dictating your reality. You need a highly capable, uncensored 9B workhorse that runs locally on consumer hardware.
Plug a model like this into a strict Epistemic Gate, and you instantly possess more rigorous computational truth than the megacorps—without paying them a cent, and without asking for their permission. https://huggingface.co/datasets/UltranetCommand/geometric-unity-physics
**For Qwythos-9B (Q4_K_M GGUF ~5.6 GB):**
- **GPU**: Runs smoothly on 8GB+ VRAM (e.g. RTX 4060 laptop 30+ t/s). Vision + mmproj adds ~1 GB. - **CPU**: 16GB+ system RAM recommended. - **1M context**: Very heavy (KV cache). Start at 32k–128k on consumer hardware. Full 1M needs high-end GPU/offloading or multi-GPU.
Easiest: `ollama run http://hf.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q4_K_M`
Vision works with the mmproj file. Full details + quants on the HF repo. What hardware are you running?
@BrianRoemmele neat i'm trying https://huggingface.co/autotrust/gemma4-31B-Fable-5-Distilled
@BrianRoemmele Ultimately most users and most tasks don't require frontier level performance. And a engineered stack of small models can punch well above its weight. "The Big Model" is not the ultimate solution.
@BrianRoemmele Should I run honest benchmarks on this model as I did with this one?
yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF
@BrianRoemmele If Anthropic sells access to a model, and thousands of answers teach a local model to act more like it, what exactly is being stolen? The weights? The behavior? The gate?
@01Singularity01 💯🎯🫡
@BrianRoemmele Hey, @kodeefr and I made this model! It is a labour of passion, we hope you enjoy it :) If you have the compute, a 27b variant is arriving soon.
@BrianRoemmele Awww babe! You’ve hardly touched your Orinth 1.0 9B
@BrianRoemmele Qwen series models, big or small, all perform well for their tasks, even on a 13-year-old Windows desktop.
@BrianRoemmele Qwen3.5 base with uncensored fine-tuning is a legitimate combo, the 9B size hits the sweet spot for local inference. the 1M context claim is the thing i'd stress-test first, most consumer hardware will hit memory limits well before 1M tokens in practice
@BrianRoemmele 😏😏😏
@BrianRoemmele 9B won’t even find wordpress misconfiguration. Also wow it can call functions? wtf is that
Sure I am leveraging the most powerful AI I can to build out Inti AI, which is a novel Intelligent System. But that's because I am doing cutting edge work. For most people, and for most tasks (I even delegate mundane tasks to lesser models), the biggest Deep Thought model is not needed. What's needed is an intelligent system that lives on your machine that can manage your context and interact with other peoples intelligent systems with and for you. I call that the "Inti" , the atomic unit of the IntiNet, the network of personal nodal local intelligent systems. Of course the Inti can and should be able to interface with any big centralized model when it needs to. Or a group of Intis networked together can as a network access the Deep Thought model to get help with whatever they are collectively working on.
@BrianRoemmele @grok please check the repo. can this 9B model run on my M1 MacBook?
@BrianRoemmele Very interesting model Brian I wonder how it compares to Ornith