Gruber
gruber
AI & ML interests
None yet
Recent Activity
liked
a model
about 1 month ago
distil-whisper/distil-large-v3.5
reacted
to
danielhanchen's
post
with 🚀
about 2 months ago
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
reacted
to
danielhanchen's
post
with 🔥
about 2 months ago
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
Organizations
None yet