unsloth/Z-Image-GGUF
Text-to-Image • 6B • Updated • 14.1k • 158
E.M. Rabani PRO
SkyMind
AI & ML interests
Neuromorphic NNs
Recent Activity
liked a model about 3 hours ago
unsloth/Z-Image-GGUF liked a model about 3 hours ago
Tongyi-MAI/Z-Image liked a model 3 days ago
CompactAI-O/TMLM-Haiku-2.3Organizations
None yet
liked 2
models about 3 hours ago
liked a
model 3 days ago
3-stage adaptive evaluation comparison with Qwen3.6-27B?
2
#1 opened 11 days ago
by SkyMind
liked a
dataset 3 days ago
liked 5
models 3 days ago
majentik/Qwen3.6-35B-A3B-TurboQuant
Image-Text-to-Text • Updated • 8
bartowski/FINAL-Bench_Darwin-36B-Opus-GGUF
Text Generation • 35B • Updated • 8.87k • 31
Qwen/Qwen3.6-35B-A3B
Image-Text-to-Text • 36B • Updated • 2.88M • • 1.63k
Qwen/SAE-Res-Qwen3.5-27B-W80K-L0_50
Updated • 28
DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking
Image-Text-to-Text • 40B • Updated • 2.43k • 22
reacted to Crownelius's post with 🚀 3 days ago
Post
5199
Day 3 - 05/02/2026
Scamp ships, hits the wall. New plan...
Scamp came back from training today... Didn't go so well, I'm still unsure...
Fast benchmark, temperature 0.7, top_p 0.9:
- "Capital of France is" produced "covered by the Crown" (grammatical, factually wrong)
- "23 + 19 = ?" produced "23. Answer: 23. Answer: 23..." (loops, math broken)
- "def fibonacci(n):" produced a list of letters
It speaks English. It can't reason. At 8K vocab and 50M params, it was never going to.
Next build: 412M MoE-3E. Three experts (math, language, code), top-1 routing, random init, let specialization emerge from gradient signal alone. Tried seeded Branch-Train-MiX first then dropped it. Adds compute for no clear win when the router will find its own attractors anyway.
Big lesson today came from limit testing on A100 80GB. Surprise, every planned phase ran out of memory even on 80GB. Root cause: at vocab 262144 (Gemma 3 standard), the output logits dominate during forward and backward. Fix: Liger Kernel's fused cross-entropy. It streams the loss computation instead of materialising the full B by T by vocab tensor. Without it the build would not run.
Scamp proved the pipeline runs end-to-end on real hardware. The 412M run starts tomorrow. If routing balances naturally and math finally crystallises, ships as Crowfeather-412M-3E with GGUF in F16, Q8, Q5, and Q4.
So... the training may have produced a poet if I had done it better. But I didn't, so instead... we get a malformed robot named Scamp... This is progress.
-Shane
P.S Join discord for discussion: https://discord.gg/8ZscHNmJYE and
I post my finished stuff here:
CompactAI-O
Scamp ships, hits the wall. New plan...
Scamp came back from training today... Didn't go so well, I'm still unsure...
Fast benchmark, temperature 0.7, top_p 0.9:
- "Capital of France is" produced "covered by the Crown" (grammatical, factually wrong)
- "23 + 19 = ?" produced "23. Answer: 23. Answer: 23..." (loops, math broken)
- "def fibonacci(n):" produced a list of letters
It speaks English. It can't reason. At 8K vocab and 50M params, it was never going to.
Next build: 412M MoE-3E. Three experts (math, language, code), top-1 routing, random init, let specialization emerge from gradient signal alone. Tried seeded Branch-Train-MiX first then dropped it. Adds compute for no clear win when the router will find its own attractors anyway.
Big lesson today came from limit testing on A100 80GB. Surprise, every planned phase ran out of memory even on 80GB. Root cause: at vocab 262144 (Gemma 3 standard), the output logits dominate during forward and backward. Fix: Liger Kernel's fused cross-entropy. It streams the loss computation instead of materialising the full B by T by vocab tensor. Without it the build would not run.
Scamp proved the pipeline runs end-to-end on real hardware. The 412M run starts tomorrow. If routing balances naturally and math finally crystallises, ships as Crowfeather-412M-3E with GGUF in F16, Q8, Q5, and Q4.
So... the training may have produced a poet if I had done it better. But I didn't, so instead... we get a malformed robot named Scamp... This is progress.
-Shane
P.S Join discord for discussion: https://discord.gg/8ZscHNmJYE and
I post my finished stuff here:
- 2 replies
liked a
dataset 4 days ago
liked a
model 10 days ago
liked a
Space 11 days ago
liked 2
models 12 days ago
liked a
dataset 12 days ago
liked 3
models 12 days ago
