Based on 10 visible X reactions from 12 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@ggerganov gemma4 crashes with tensor parallelism after what, 3 months still? qwen35 122b with TP and -ncmoe, prefill collapses to 40t/s, from 900 with single card. smarter hot/cold expert vram placement repeatedly shot down. issues and PRs ignored, tests that would catch this, non-existent
@ggerganov ten thousand releases and it's still the only inference stack i can hand to someone with no gpu and have it just work. half the local ai boom people credit to model labs actually rests on this repo. thanks for the decade of unglamorous commits.
@ggerganov Happy 10k 🥳 I absolutely love using "llama.cpp". So far the most transparent inference server 🚀
@ggerganov not enough to make gemma and qwen work properly still
@ggerganov gemma4 crashes with tensor parallelism after what, 3 months still? qwen35 122b with TP and -ncmoe, prefill collapses to 40t/s, from 900 with single card. smarter hot/cold expert vram placement repeatedly shot down. issues and PRs ignored, tests that would catch this, non-existent
@ggerganov ten thousand releases and it's still the only inference stack i can hand to someone with no gpu and have it just work. half the local ai boom people credit to model labs actually rests on this repo. thanks for the decade of unglamorous commits.
@ggerganov Happy 10k 🥳 I absolutely love using "llama.cpp". So far the most transparent inference server 🚀
@ggerganov not enough to make gemma and qwen work properly still
@ggerganov Thank you for all the work! 🙇🏻
@ggerganov Congratulations!
the 10000th release of llama.cpp 🥳 https://x.com/ggerganov/status/2077003281344135260/photo/1
https://github.com/ggml-org/llama.cpp/releases/tag/b10000
Based on 10 visible X reactions from 12 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@ggerganov Congratulations!