File size: 3,646 Bytes
5e0b2c3 dbc7643 5e0b2c3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
license: apache-2.0
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- abliterated
- uncensored
- heretic
- nvfp4
- compressed-tensors
- quantized
- blackwell
- qwen3_5_moe
- moe
---
# Cinder — Qwen3.6-35B-A3B (abliterated, NVFP4)
**Cinder** is the **NVFP4** quantization of [Ember](https://huggingface.co/SparkyForge/Ember) — the abliterated (refusal-removed) build of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B). Same surgical abliteration, ~3× smaller: **~22 GB** vs ~66 GB for the BF16 Ember.
For the full method writeup, retention evidence, and the BF16 weights, see **[Ember](https://huggingface.co/SparkyForge/Ember)**. The patch + method: [heretic-fused-moe-abliteration](https://huggingface.co/SparkyForge/heretic-fused-moe-abliteration).
> **Not affiliated with NVIDIA or the Apache Software Foundation.** Independent community model.
## What it is
- **Format:** NVFP4 via [compressed-tensors](https://github.com/neuralmagic/compressed-tensors) / [llm-compressor](https://github.com/vllm-project/llm-compressor). FP4 weights with FP8 block scales, NVFP4 activation scheme.
- **Hardware:** needs an **NVIDIA Blackwell** GPU (sm_120 / sm_121 — e.g. RTX 50-series, DGX Spark / GB10) and a recent vLLM with NVFP4 support. It will **not** run on older GPUs. If you're on anything pre-Blackwell, use **Ember** (BF16) and quantize to your own format.
- **~22 GB** on disk — fits comfortably in the DGX Spark's unified memory with room for a long context and a speculative drafter.
## Quantization details (and what was deliberately *not* quantized)
The fused MoE experts are FP4-packed; the hybrid layers are preserved in BF16. Verified post-quant:
- **30,720** expert weight tensors FP4-packed, **0** experts silently left in BF16 (the fused-expert handling carried through quantization).
- The **30 linear-attention (Mamba/GDN) layers stayed BF16** — quantizing them breaks the model; they're in the ignore list (`linear_attn`, `mlp.gate`, `shared_expert_gate`, `embed_tokens`, `lm_head`, vision tower).
- Quant scales clean, no NaNs.
Quant recipe ships in `recipe.yaml`.
## Usage (vLLM, Blackwell)
```bash
vllm serve <path-to-cinder> \
--quantization compressed-tensors \
--max-model-len 131072 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
--trust-remote-code
```
- Vision-language (`image-text-to-text`) — image input works; vision tower is BF16, untouched by quant.
- Thinking via `chat_template_kwargs: {"enable_thinking": false}` per request.
- Pairs with the public [z-lab DFlash drafter](https://huggingface.co/z-lab) for ~1.5× decode speedup via speculative decoding (not included).
## Safety
Refusal behavior is removed (same as Ember). You own the guardrails. Research / red-team / operator-controlled use.
## License & attribution
- **License:** Apache 2.0 (inherited from base). See `LICENSE` / `NOTICE`. Modified from Qwen3.6-35B-A3B (abliteration + NVFP4 quantization).
- **Base:** [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) (Apache 2.0), © the Qwen team.
- **Abliteration:** built on [Heretic](https://github.com/p-e-w/heretic) (Philipp Emanuel Weidmann) + a fused-MoE patch (see Ember).
- **Quantization:** [llm-compressor](https://github.com/vllm-project/llm-compressor) (NVFP4).
---
*The smaller, hardier cousin of [Ember](https://huggingface.co/SparkyForge/Ember) — forged by **Sparky** on a DGX Spark. A cinder: what's left when the ember has done its work, and it still burns.* 🔥
|