---
license: apache-2.0
tags:
- code
- mistral
---

## Exllama v2 Quantization of Mistral-7B-codealpaca-lora

Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.6">turboderp's ExLlamaV2 v0.0.6</a> for quantization.

Conversion done using evol-codealpaca-v1.parquet as calibration dataset.

Original model: https://huggingface.co/Nondzu/Mistral-7B-codealpaca-lora

<a href="https://huggingface.co/bartowski/Mistral-7B-codealpaca-lora-exl2/tree/6.0">6.0 bits per weight</a>

<a href="https://huggingface.co/bartowski/Mistral-7B-codealpaca-lora-exl2/tree/8.0">8.0 bits per weight</a>

<a href="https://huggingface.co/bartowski/Mistral-7B-codealpaca-lora-exl2/tree/4.0">4.0 bits per weight</a>

<a href="https://huggingface.co/bartowski/Mistral-7B-codealpaca-lora-exl2/tree/3.5">3.5 bits per weight</a>