README.md

---
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
license: apache-2.0
language:
- en
tags:
- code
- coding-assistant
- lora
- fine-tuned
- gguf
- ollama
pipeline_tag: text-generation
---

# Alpha-Coder-14B

Alpha-Coder-14B is a fine-tuned version of [Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct), adapted via LoRA to produce typed, tested Python code. This repo contains both the fused fp16 weights and a Q4_K_M GGUF quant for local inference (e.g. with Ollama or llama.cpp).

## Base model attribution

This model is a derivative of **Qwen/Qwen2.5-Coder-14B-Instruct**, released by the Qwen team under the **Apache 2.0 license**. Alpha-Coder-14B is redistributed under the same license, as permitted by Apache 2.0 for derivative/renamed works, with attribution to the original model and authors.

- Base model: [Qwen/Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct)
- License: Apache 2.0

## Training details

- **Method:** LoRA fine-tuning
- **Hardware:** Apple Silicon M5, 24GB unified memory
- **Framework:** MLX (4-bit base model during training)
- **LoRA config:** rank = 64, alpha = 128, learning rate = 2e-6
- **Steps:** 6,160
- **Validation loss:** 0.383 → 0.252
- **Post-training:** LoRA adapter fused into the base model, dequantized to fp16 HF safetensors, then converted and quantized to GGUF (Q4_K_M, 8.4GB) via llama.cpp

## Benchmarks

| Benchmark | Base (Qwen2.5-Coder-14B-Instruct) | Alpha-Coder-14B |
|---|---|---|
| MMLU | 72% | 77% |
| GSM8K | ~93% (no regression) | 93% |

No measurable forgetting was observed on GSM8K after fine-tuning, while MMLU improved by 5 points.

## Files in this repo

| File | Description |
|---|---|
| `*.safetensors` | Fused fp16 weights (LoRA merged into base), full precision |
| `tokenizer*` / `*.json` | Tokenizer and config files |
| `alpha-14b-Q4_K_M.gguf` | Q4_K_M quantized GGUF, ~8.4GB, for llama.cpp / Ollama |

## Usage with Ollama

1. Download `alpha-14b-Q4_K_M.gguf` from this repo.
2. Create a `Modelfile` in the same directory (use your actual system prompt from your local Modelfile).
3. Build and run:

```bash
ollama create alpha-coder -f Modelfile
ollama run alpha-coder
```

## Usage with llama.cpp

```bash
./llama-cli -m alpha-14b-Q4_K_M.gguf -p "Write a Python function that ..."
```

## Usage with transformers (fp16 safetensors)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Jay2003Bhatt/alpha-coder-14b", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Jay2003Bhatt/alpha-coder-14b")
```

## Intended use

Alpha-Coder-14B is intended as a coding assistant producing typed, tested Python code. As with any fine-tuned model, evaluate outputs before relying on them in production, particularly for correctness and security-sensitive code.

## License

Apache 2.0, inherited from the base model. See the [Qwen2.5-Coder-14B-Instruct license](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/blob/main/LICENSE) for details.