Hugging Face
Search models, datasets, users...
Ctrl K
Models
Datasets
Spaces
Buckets
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
None
CompressedGemma
4
Follow
nakwMt's profile picture
emanubiz's profile picture
Svatosalav's profile picture
5 followers
¡
6 following
AI & ML interests
None yet
Recent Activity
updated
a model
about 7 hours ago
CompressedGemma/SMQ-Quantize-0.1
published
a model
2 days ago
CompressedGemma/SMQ-Quantize-0.1
reacted
to
savoji
's
post
with đĽ
8 days ago
A 284B-parameter DeepSeek-V4-Flash now runs on a 64 GB desktop with 0 GPUs: llama-modchip shrinks the checkpoint to 56.56 GiB (1.71 bpw) and decodes at 9.1 to 14.7 tok/s. The standard UD-IQ1_S build is 76.87 GiB, above the ~62.5 GiB usable on a 64 GB Linux box. The repo mixes precisions (routed experts mostly IQ1_S, MLA projections in a custom Fractal-64 type) and patches the CPU kernels. The model plus a 256K KV cache then sits at 57.61 to 58.9 GiB RSS with zero swap. The repo was created on 2026-10-29 and already has 744 stars and 470 points on Hacker News. đ The numbers ⢠Size: 56.56 GiB, 20.31 GiB smaller than UD-IQ1_S ⢠Prefill: 25.0 to 44.36 tok/s (+77%) on a 16-core Ryzen 9 9955HX ⢠Decode: 5.91 tok/s unpatched, 9.06 to 9.95 tok/s at Top-6, 11.5 to 14.7 tok/s with the n-gram drafter â ď¸ Caveats ⢠"Lossless" means bit-exact against the same re-quantized checkpoint. Versus UD-IQ1_S the author reports 0.9964 top-logit cosine similarity ⢠The 98.5/100 quality score comes from the author's own gate, and the headline speeds come from one machine ⢠The custom tensor type 40 is rejected by stock llama.cpp, Ollama and LM Studio đ ď¸ Try it ⢠Run scripts/check_cpu_readiness.py from the repo: it needs no model download ⢠Run ./modchip bench for speed, then ./modchip eval to check the quality claims yourself Repo: https://github.com/synchopate/llama-modchip HN thread: https://news.ycombinator.com/item?id=50411961 Would you accept a custom runtime for 9 to 15 tok/s from a 284B model on a 64 GB box? đ Read it elsewhere ⢠The deep dive on Medium: https://medium.com/p/e47ab3581f58 ⢠The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/llama-modchip-runs-a-284b-deepseek ⢠The thread on X: https://x.com/saavoji/status/2119858773280141526 ⢠Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#llama-modchip
View all activity
Organizations
None yet
models
12
Sort:Â Recently updated
CompressedGemma/SMQ-Quantize-0.1
Updated
about 1 hour ago
CompressedGemma/HPC-Quantize-V2
Updated
9 days ago
CompressedGemma/Qwen-3.8-27B-Thinkingcap-Q2_Q3
27B
â˘
Updated
11 days ago
â˘
393
CompressedGemma/Qwen-3.8-27B-Thinkingcap-Q4
27B
â˘
Updated
14 days ago
â˘
457
CompressedGemma/Qwen-3.8-27B-Thinkingcap-Q2_Q4
27B
â˘
Updated
14 days ago
â˘
338
CompressedGemma/Raon-1-mini-preview-Q2K_Q4
29B
â˘
Updated
27 days ago
â˘
276
CompressedGemma/Qwen3.8-27B-Q2
27B
â˘
Updated
Sep 18
â˘
101
â˘
1
CompressedGemma/Muse-Glimmer-30B-Q2
28B
â˘
Updated
Sep 11
â˘
30
CompressedGemma/granite-4.2-30b
29B
â˘
Updated
Sep 11
â˘
9
CompressedGemma/K2-Horizon-MoVA-36B-A4B-Q2K
37B
â˘
Updated
Sep 8
â˘
44
â˘
1
View 12 models
datasets
0
None public yet