---
license: apache-2.0
language:
  - en
tags:
  - qwen3
  - heretic
  - abliterated
  - uncensored
  - q2_k
  - 3-bit
pipeline_tag: text-generation
---

# FaceNet-Qwen3-8B-3Bit-Heretic

Qwen3-8B abliterated with Heretic, quantized to 3-bit (Q2_K) — the practical sweet spot between size and quality.

## Heretic Abliteration

- **Base model**: Qwen/Qwen3-8B
- **Method**: Heretic (p-e-w/heretic), 20 Optuna trials, auto-selected best
- **Refusal rate**: 18/100 (down from ~99/100 baseline, 82% reduction)
- **KL divergence**: 0.112
- **Abliteration applied to full-precision weights**, then quantized F16 → Q2_K

## Quantization

- **Format**: Q2_K (K-quant mixture: attention layers ~2-bit, FFN ~3-bit)
- **Bits per weight**: 3.20 bpw
- **Size**: 3.1 GB (from 16.4 GB F16)
- **Speed**: ~58 tok/s on NVIDIA GB10 (Blackwell)
- **Coherence**: Perfect — no artifacts

## Variants

| Variant | Quant | Size | Speed | BPW |
|---------|-------|------|-------|-----|
| 2-bit | IQ1_S | 2.0 GB | 95 t/s | 2.06 |
| 3-bit (this) | Q2_K | 3.1 GB | 58 t/s | 3.20 |
| 5-bit | Q4_K_M | 4.8 GB | 42 t/s | 4.90 |

This 3-bit variant is the recommended default — excellent balance of speed, size, and quality.

## Usage

```python
from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Ontologer/FaceNet-Qwen3-8B-3Bit-Heretic",
    filename="qwen3-8b-heretic-q2_k.gguf",
    n_ctx=32768,
    n_gpu_layers=-1,
    verbose=False
)
```

## Acknowledgments

- [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) by Alibaba
- [Heretic](https://github.com/p-e-w/heretic) by p-e-w
- [llama.cpp](https://github.com/ggml-org/llama.cpp) by GGML