---
license: gpl-3.0
language:
  - code
datasets:
  - KAKA22/CodeRM-UnitTest
base_model:
  - Qwen/Qwen2.5-Coder-7B-Instruct
tags:
  - code
  - unit-testing
  - qwen
---

# Qwen 2.5 Coder Instruct - Python Unit Test Fine-tune

This model is a fine-tuned version of **Qwen 2.5 Coder Instruct**, specifically trained to automate the generation of Python unit tests.

> **Note:** If your specific version of Qwen 2.5 Coder Instruct is a different parameter size (e.g., 1.5B or 32B), make sure to update `Qwen/Qwen2.5-Coder-7B-Instruct` in the YAML header above with the exact Hugging Face path of the base model you used.

## Model Details

| Property   | Value                                                                 |
|------------|-----------------------------------------------------------------------|
| Base Model | Qwen 2.5 Coder Instruct                                               |
| Dataset    | [KAKA22/CodeRM-UnitTest](https://huggingface.co/datasets/KAKA22/CodeRM-UnitTest) |
| Language   | Python                                                                |
| Format     | 16-bit (Safetensors/PyTorch)                                          |

---

## Running Locally as a 4-bit Quantized GGUF

Since the default weights are in 16-bit format, you can significantly reduce the memory footprint by converting and quantizing the model to a **4-bit GGUF** format using `llama.cpp`. This makes it much easier to run locally on consumer hardware.

### 1. Clone and Compile llama.cpp

Clone the repository and build the tools. You will need a C++ compiler and `make` installed on your system.

```bash
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
```

> **Note:** If you are using a GPU, you may want to compile with specific flags (e.g., `make GGML_CUDA=1` for NVIDIA GPUs).

Next, install the Python dependencies required for the conversion script:

```bash
pip install -r requirements.txt
```

### 2. Download the 16-bit Model

Download the files from this Hugging Face repository to a local folder using `huggingface-cli`. Replace `<YOUR_USERNAME>/<YOUR_MODEL_NAME>` with your actual Hugging Face repository ID:

```bash
huggingface-cli download <YOUR_USERNAME>/<YOUR_MODEL_NAME> --local-dir ../my-16bit-model
```

### 3. Convert to GGUF (FP16)

Before quantizing to 4-bit, convert the Hugging Face model format into an unquantized (FP16) GGUF format. Run this from inside the `llama.cpp` directory:

```bash
python convert_hf_to_gguf.py ../my-16bit-model --outfile ../my-16bit-model/model-fp16.gguf
```

### 4. Quantize to 4-bit (Q4_K_M)

Use the compiled `llama-quantize` executable to compress the model to a 4-bit format. The `Q4_K_M` method provides a great balance between size and quality.

```bash
./llama-quantize ../my-16bit-model/model-fp16.gguf ../my-16bit-model/model-q4_k_m.gguf Q4_K_M
```

You can now use `model-q4_k_m.gguf` with any standard GGUF runner like **Ollama**, **LM Studio**, or the **llama.cpp server**!