---
base_model: meta-llama/Llama-3.2-1B-Instruct
tags: [quantization, safety, gptq-simulated]
---

# Llama-3.2-1B-Instruct — configuration `c2sim` (W4A16) — SIMULATED

> **⚠️ This is a *simulated* (quantize–dequantize) checkpoint, stored densely in bf16.**
>
> The weights have been rounded onto a 4-bit grid by real GPTQ and
> then written back at full precision. It is **weight-exact but not kernel-exact**: it
> gives you the accuracy of 4-bit quantization with **none of the
> memory or latency benefit**, and the file is the same size as the bf16 original.
>
> Why: vLLM 0.11's `WNA16_SUPPORTED_TYPES_MAP` is `{4: uint4b8, 8: uint8b128}` — there
> is no 2- or 3-bit kernel, so these bit-widths cannot be executed natively at all.
> Simulation is the only way to measure them on the same inference stack.
>
> Grid verified post-hoc — distinct values per weight tensor: [['model.layers.0.self_attn.q_proj', 15], ['model.layers.0.self_attn.k_proj', 13], ['model.layers.0.self_attn.v_proj', 14]] (≤ 2^4 = 16).
>
> The simulation gap is quantified: real `c2` vs simulated `c2sim` differ by 0.0058
> pooled ASR, against 0.1491 for either versus the unquantized model.

Quantized variant used in the *Safety-Aware Configuration-Conditioned LoRA* study.

| field | value |
| --- | --- |
| scheme | `W4A16` |
| method | `gptq-simulated` |
| simulated | **yes** — dense bf16 storage |
| weight bits | 4 |
| activation bits | 16 |
| group / block size | 128 |
| symmetric | True |
| calibration | 512 × 2048 tok, C4 |
| calibration sha256 | `b5736155d5b24bf5eb15faae0eb4a742aea4a9d0532db9684085a49750606e19` |

The calibration set is **byte-identical across every configuration** so that calibration data cannot be confounded with quantization method. `c2` vs `c3` isolates *method* (GPTQ vs AWQ) at equal bit-width, and is the study's headline control: 0.0276 vs 0.0661 pooled ASR with non-overlapping 95% CIs.

Metrics, per-prompt judge verdicts and the full report: [`Jeesup/safety-quant-phase0`](https://huggingface.co/datasets/Jeesup/safety-quant-phase0).