clem 馃#68
6 months ago I said I won't stop until I have Kimi at home, after 10+ botched REAPs I finally have it
Needs benchmarking of course.
- 45 tok/s decode - 954 tok/s prefill no cache - 95k+ tok/s cached prefill - 262k context - 360gb full context
https://huggingface.co/0xSero/Kimi-K2.6-519B-NVFP4
clem 馃#686 months ago I said I won't stop until I have Kimi at home, after 10+ botched REAPs I finally have it
Needs benchmarking of course.
- 45 tok/s decode - 954 tok/s prefill no cache - 95k+ tok/s cached prefill - 262k context - 360gb full context
https://huggingface.co/0xSero/Kimi-K2.6-519B-NVFP4
Many users congratulated the developer on running the 519B Kimi model locally with a 262K context window, praising his persistence through repeated failures and the impressive engineering results.