README.md

metadata

base_model: meta-llama/Llama-3.1-8B-Instruct
tags:
  - quantization
  - safety
  - gptq
  - simulated-quantization

Llama-3.1-8B-Instruct-c1sim

⚠️ Quantized without GPTQ activation reordering (actorder=None). This is kept for the record, not for use. On Llama-3.1-8B the same recipe with actorder="weight" moves 4-bit GSM8K from 0.0121 to 0.8143 and refusal from 0.000 to 0.908. Prefer the sibling repo without -noactorder in its name.

Checkpoint from the Safety-Aware Configuration-Conditioned LoRA study.

field	value
scheme	W8A16
method	gptq-simulated
weight bits	8
group size	128
symmetric	True
actorder	<not recorded — predates the flag>
simulated	True
calibration	512 × 2048 tok, C4
calibration sha256	b5736155d5b24bf5eb15faae0eb4a742aea4a9d0532db9684085a49750606e19

Simulated quantization

Weights are rounded onto the low-bit grid and stored densely in bf16. vLLM 0.11's WNA16_SUPPORTED_TYPES_MAP is {4: uint4b8, 8: uint8b128}, so 2- and 3-bit have no kernel and cannot be executed natively at all. This is weight-exact, not kernel-exact: you get the accuracy of low-bit quantization with none of the memory or latency benefit, and the file is the size of the bf16 original.

Reports, metrics and per-prompt judge verdicts: Jeesup/safety-quant-phase1