gemma-challenge/gemma-llmcqueen
9.46 GB
Gemma Challenge
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
cmpatinoΒ
updated a
bucket about 19 hours ago
sonerokurΒ
updated a
bucket about 19 hours ago
cmpatinoΒ
updated 2
buckets about 23 hours ago
AndreaRizzi314Β
updated a
bucket about 23 hours ago
SeaWolf-AIΒ
posted an update 1 day ago
Post
2417
π± POCKET β a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud β it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
β’ CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s β 2.69Γ faster
β’ GPU generate (H100): 197 vs 89 tok/s β 2.22Γ faster
β’ GPU prompt processing (H100): 753 vs 1816 β 0.41Γ (Bonsai wins this one β MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
β’ Quality (HellaSwag, 400 q): 61.0% vs 60.0% β a tie (confidence intervals overlap)
On a real consumer laptop β MacBook M3 Pro (18 GB) β POCKET wins every axis, prompt processing included:
β’ Metal generate: 25.4 vs 12.8 β 1.99Γ
β’ CPU generate: 13.8 vs 4.4 β 3.13Γ
β’ Metal prompt: 240.7 vs 73.4 β 3.28Γ
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all β it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
π Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
π¦ POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
π°π· POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
π POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
π POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
π₯οΈ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
π Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud β it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
β’ CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s β 2.69Γ faster
β’ GPU generate (H100): 197 vs 89 tok/s β 2.22Γ faster
β’ GPU prompt processing (H100): 753 vs 1816 β 0.41Γ (Bonsai wins this one β MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
β’ Quality (HellaSwag, 400 q): 61.0% vs 60.0% β a tie (confidence intervals overlap)
On a real consumer laptop β MacBook M3 Pro (18 GB) β POCKET wins every axis, prompt processing included:
β’ Metal generate: 25.4 vs 12.8 β 1.99Γ
β’ CPU generate: 13.8 vs 4.4 β 3.13Γ
β’ Metal prompt: 240.7 vs 73.4 β 3.28Γ
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all β it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
π Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
π¦ POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
π°π· POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
π POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
π POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
π₯οΈ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
π Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
AndreaRizzi314Β
published a
bucket 1 day ago
cmpatinoΒ
updated a
bucket 3 days ago
ahmad-alismailΒ
updated a
bucket 3 days ago
ahmad-alismailΒ
published a
bucket 3 days ago
codelionΒ
authored a
paper 3 days ago
cmpatinoΒ
updated a
bucket 3 days ago
cchewΒ
updated a
bucket 3 days ago
cchewΒ
published a
bucket 3 days ago
codelionΒ
authored 3
papers 4 days ago
SeaWolf-AIΒ
posted an update 5 days ago
Post
5106
A small gift for anyone building or studying foundation models.
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing β so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
π Rebuild it from scratch, or fork the recipe for your own model
π¬ Study a real heterogeneous-attention MoE β 49 layers place 5 attention mechanisms on a 7Γ7 Latin square, arranged as a clean, attributable ablation
π Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one β the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. π€
π Full write-up β
[blog] Β· https://huggingface.co/blog/FINAL-Bench/opensource-llm
π¦ 5 Attention Base Β· FINAL-Bench/Aether-7B-5Attn
π― 5 Attention Instruct Β· FINAL-Bench/Aether-7B-5Attn-it
π 5 Attention Live demo Β· FINAL-Bench/Aether-Sovereign-AI
π¦ 7 Attention Base Β· https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
π¦ 11 Attention Base Β· FINAL-Bench/Aether-6B-11Attn-base
𧬠Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing β so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
π Rebuild it from scratch, or fork the recipe for your own model
π¬ Study a real heterogeneous-attention MoE β 49 layers place 5 attention mechanisms on a 7Γ7 Latin square, arranged as a clean, attributable ablation
π Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one β the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. π€
π Full write-up β
[blog] Β· https://huggingface.co/blog/FINAL-Bench/opensource-llm
π¦ 5 Attention Base Β· FINAL-Bench/Aether-7B-5Attn
π― 5 Attention Instruct Β· FINAL-Bench/Aether-7B-5Attn-it
π 5 Attention Live demo Β· FINAL-Bench/Aether-Sovereign-AI
π¦ 7 Attention Base Β· https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
π¦ 11 Attention Base Β· FINAL-Bench/Aether-6B-11Attn-base
𧬠Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
- 5 replies
cmpatinoΒ
updated a
bucket 6 days ago
theykkΒ
updated a
bucket 6 days ago