---
license: llama2
base_model: codellama/CodeLlama-7b-hf
tags:
- code
- llama
- gguf
- merged
- python
---

# CodeLlama 7B Python AI Assistant (Merged GGUF)

This is a merged version of the QLoRA fine-tuned CodeLlama-7B model. The LoRA weights have been merged with the base model and converted to GGUF format for easy deployment.

## Model Details

- **Base Model**: CodeLlama-7b-hf
- **Original LoRA Adapter**: pranav-pvnn/codellama-7b-python-ai-assistant
- **Fine-tuning Method**: QLoRA (4-bit quantization with LoRA)
- **Format**: GGUF (self-contained, no separate adapter needed)
- **Training Framework**: Unsloth

## Available Quantizations

- `codellama-7b-merged-f16.gguf` - Full precision (FP16) - ~13 GB
- `codellama-7b-merged-Q4_K_M.gguf` - 4-bit quantization (recommended) - ~4 GB
- `codellama-7b-merged-Q5_K_M.gguf` - 5-bit quantization (higher quality) - ~5 GB
- `codellama-7b-merged-Q8_0.gguf` - 8-bit quantization (highest quality) - ~7 GB

## Usage

### With llama.cpp:
```bash
./llama-cli -m codellama-7b-merged-Q4_K_M.gguf -p "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"
```

### With Python (llama-cpp-python):
```python
from llama_cpp import Llama

llm = Llama(model_path="codellama-7b-merged-Q4_K_M.gguf")
prompt = "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"
output = llm(prompt, max_tokens=256)
print(output['choices'][0]['text'])
```

### With Ollama:
1. Create a Modelfile:
```
FROM ./codellama-7b-merged-Q4_K_M.gguf
```

2. Create the model:
```bash
ollama create my-codellama -f Modelfile
ollama run my-codellama "Write a Python function to sort a list"
```

## Training Details

- **Quantization**: 4-bit QLoRA
- **LoRA Rank**: 64
- **Learning Rate**: 2e-4
- **Epochs**: 4
- **Max Seq Length**: 2048
- **Training Data**: Custom Python programming examples (~2,000 examples)
- **GPU**: NVIDIA Tesla T4

## Prompt Format

```
### Instruction:
[Your instruction here]
### Response:
```

## License

Same as base model (Llama 2 license)

## Acknowledgements

- Base Model: [Meta's CodeLlama](https://huggingface.co/codellama/CodeLlama-7b-hf)
- Training Framework: [Unsloth](https://github.com/unslothai/unsloth)