macadeliccc/airoboros-34b-3.2-hqq-8bit

[jondurbin/airoboros-34b-3.2] [HQQ] [8bit]

This repository contains an HQQ (Half-Quadratic Quantization) model trained on [describe the task or dataset].

Original Repo

Model Details

• Model ID: jondurbin/airoboros-34b-3.2

• Model Architecture: yi-34b-200k

• Training Dataset: Read through original repo

• Performance: In progress

• nbits: The number of bits used for quantization: 8.

• group_size: The size of the quantization group: 128.

• quant_zero: Whether to quantize the zero values: True.

• quant_scale: Whether to quantize the scale values: True.