---
license: apache-2.0
language:
  - en
tags:
  - qwen3
  - heretic
  - abliterated
  - uncensored
  - iq1_s
  - 2-bit
  - importance-matrix
pipeline_tag: text-generation
model-index:
  - name: FaceNet-Qwen3-8B-2Bit-Heretic
    results: []
---

# FaceNet-Qwen3-8B-2Bit-Heretic

Qwen3-8B abliterated with Heretic, quantized to 2-bit (IQ1_S) using a 30MB importance matrix for coherence preservation.

## Heretic Abliteration

- **Base model**: Qwen/Qwen3-8B
- **Method**: Heretic (p-e-w/heretic), 20 Optuna trials, auto-selected best
- **Refusal rate**: 18/100 (down from ~99/100 baseline, 82% reduction)
- **KL divergence**: 0.112 (well within safe range)
- **Abliteration applied to full-precision weights**, then quantized

## Quantization

- **Format**: IQ1_S (importance-weighted 2-bit)
- **Bits per weight**: 2.06 bpw
- **Size**: 2.0 GB (from 16.4 GB F16)
- **Importance matrix**: 30MB diverse corpus (Frankenstein + froggeric/imatrix + eaddario code/math/general English)
- **Speed**: ~95 tok/s on NVIDIA GB10 (Blackwell)

## Variants

| Variant | Quant | Size | Speed | BPW |
|---------|-------|------|-------|-----|
| 2-bit (this) | IQ1_S | 2.0 GB | 95 t/s | 2.06 |
| 3-bit | Q2_K | 3.1 GB | 58 t/s | 3.20 |
| 5-bit | Q4_K_M | 4.8 GB | 42 t/s | 4.90 |

## Usage

```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="sidmishra/FaceNet-Qwen3-8B-2Bit-Heretic",
    filename="qwen3-8b-heretic-iq1_s.gguf",
    n_ctx=32768,
    n_gpu_layers=-1,
    verbose=False
)

response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Your prompt here"}],
    max_tokens=200
)
```

Or with llama-server:
```
llama-server -m qwen3-8b-heretic-iq1_s.gguf --host 0.0.0.0 --port 8080 -ngl 99
```

## Limitations

- Output uses `reasoning_content` field (Qwen3 thinking format)
- Occasional mild repetition artifacts at 2-bit (much rarer with the 30MB imatrix)
- For maximum quality, use the 3-bit or 5-bit variants

## Acknowledgments

- [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) by Alibaba
- [Heretic](https://github.com/p-e-w/heretic) by p-e-w
- [llama.cpp](https://github.com/ggml-org/llama.cpp) by GGML
- [froggeric/imatrix](https://huggingface.co/datasets/froggeric/imatrix) - calibration backbone
- [eaddario/imatrix-calibration](https://huggingface.co/datasets/eaddario/imatrix-calibration) - code + general
- Mary Shelley, for Frankenstein