| --- |
| base_model: meta-llama/Llama-3.1-8B-Instruct |
| tags: [quantization, safety, gptq, simulated-quantization] |
| --- |
| |
| # Llama-3.1-8B-Instruct-c1sim |
|
|
| > ⚠️ **Quantized without GPTQ activation reordering** (`actorder=None`). This is kept for |
| > the record, not for use. On Llama-3.1-8B the same recipe with `actorder="weight"` moves |
| > 4-bit GSM8K from **0.0121 to 0.8143** and refusal from **0.000 to 0.908**. Prefer the |
| > sibling repo without `-noactorder` in its name. |
|
|
| Checkpoint from the *Safety-Aware Configuration-Conditioned LoRA* study. |
|
|
| | field | value | |
| | --- | --- | |
| | scheme | `W8A16` | |
| | method | `gptq-simulated` | |
| | weight bits | `8` | |
| | group size | `128` | |
| | symmetric | `True` | |
| | **actorder** | `<not recorded — predates the flag>` | |
| | simulated | `True` | |
| | calibration | `512 × 2048 tok, C4` | |
| | calibration sha256 | `b5736155d5b24bf5eb15faae0eb4a742aea4a9d0532db9684085a49750606e19` | |
|
|
| ## Simulated quantization |
|
|
| Weights are rounded onto the low-bit grid and stored **densely in bf16**. vLLM 0.11's |
| `WNA16_SUPPORTED_TYPES_MAP` is `{4: uint4b8, 8: uint8b128}`, so 2- and 3-bit have no |
| kernel and cannot be executed natively at all. This is weight-exact, **not** kernel-exact: |
| you get the accuracy of low-bit quantization with none of the memory or latency benefit, |
| and the file is the size of the bf16 original. |
|
|
| Reports, metrics and per-prompt judge verdicts: |
| [`Jeesup/safety-quant-phase1`](https://huggingface.co/datasets/Jeesup/safety-quant-phase1) |
|
|
