Hugging Face co-founder Julien Chaumond launches hardware filters to find models compatible with the M4 Max, RTX 4090, and Intel CPUs

NEW on Hugging Face: Hardware filters 🖥️

A new Hardware filter on the Models page results to models that fit a specific GPU, CPU, or Apple Silicon chip, so you only see what will actually run on your machine. Set the hardware you want from your Hardware settings.

It stacks with the page's other filters and is shareable via URL, so a link like llama.cpp&hardware=apple-m4-max points anyone, even logged-out visitors, at the llama.cpp models that fit an Apple M4 Max.

A study from @Stanford showed that 71.3% of chatgpt queries could be accurately answered by a local model. I suspect a major part of enterprise AI workloads could be run locally too for free (compared to the massive costs of frontier API cost).

Also, it reduces the risk of these workloads being taken away from you because you own the models instead of renting them - which sounds like a good idea these days haha.

That's why we're introducing the ability for everyone to filter AI models on @huggingface based on your local hardware.

For me, there are 800k+ public models that fit on my M5 24GB and that I can use easily thanks to llamacpp.

Let's go local AI!

waited for this feature so long

you can now filter models for your hardware 🔥 and it works not only for GGUF/MLX compatible models but vanilla weights too 😍

@mervenoyann OH YES!!! I was waiting for this one too! Hugging Face adding pieces to the Local AI puzzle week after week 💪

you can now filter huggingface models capable to run on your hardware

@ClementDelangue @Stanford So happy to see this feature. Thank you @ClementDelangue !! 🚀

@victormustar i tried it. it told me i'm poor and go play video games

thanks i guess

@petrroyce what do you have?

get started at http://hf.co/models

@ClementDelangue @Stanford I predict that 99.99999% of current and future AI use cases can be handled by a <32b model. Smaller when we make the models specific rather than generalized.

@yacineMTB @Stanford #gpupoor

@ClementDelangue @Stanford Not any of my own queries lol

@mervenoyann Woohoo. I appreciate this must have taken a fair bit of work behind the scenes to get right.

@ClementDelangue @Stanford Sweet! An API to query compatible models based on hardware config would be super valuable.

on top of this, you don't need to put your own hardware in settings to filter for hardware

more info here 🫶🏻 https://huggingface.co/changelog/filter-models-by-hardware

@victormustar 🔥🔥

@briancaffey @Stanford lots of users requested it over the years!

@victormustar YES 🙌

@mervenoyann this and that HF cli command that pops up when you open the .safetensors file 👌

@victormustar M4 and M3 Max

on the bright side, I still can do models, GPU poor cannot stop me