File size: 2,218 Bytes
c0f7284 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ---
license: llama2
base_model: codellama/CodeLlama-7b-hf
tags:
- code
- llama
- gguf
- merged
- python
---
# CodeLlama 7B Python AI Assistant (Merged GGUF)
This is a merged version of the QLoRA fine-tuned CodeLlama-7B model. The LoRA weights have been merged with the base model and converted to GGUF format for easy deployment.
## Model Details
- **Base Model**: CodeLlama-7b-hf
- **Original LoRA Adapter**: pranav-pvnn/codellama-7b-python-ai-assistant
- **Fine-tuning Method**: QLoRA (4-bit quantization with LoRA)
- **Format**: GGUF (self-contained, no separate adapter needed)
- **Training Framework**: Unsloth
## Available Quantizations
- `codellama-7b-merged-f16.gguf` - Full precision (FP16) - ~13 GB
- `codellama-7b-merged-Q4_K_M.gguf` - 4-bit quantization (recommended) - ~4 GB
- `codellama-7b-merged-Q5_K_M.gguf` - 5-bit quantization (higher quality) - ~5 GB
- `codellama-7b-merged-Q8_0.gguf` - 8-bit quantization (highest quality) - ~7 GB
## Usage
### With llama.cpp:
```bash
./llama-cli -m codellama-7b-merged-Q4_K_M.gguf -p "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"
```
### With Python (llama-cpp-python):
```python
from llama_cpp import Llama
llm = Llama(model_path="codellama-7b-merged-Q4_K_M.gguf")
prompt = "### Instruction:\nWrite a Python function to calculate factorial.\n### Response:\n"
output = llm(prompt, max_tokens=256)
print(output['choices'][0]['text'])
```
### With Ollama:
1. Create a Modelfile:
```
FROM ./codellama-7b-merged-Q4_K_M.gguf
```
2. Create the model:
```bash
ollama create my-codellama -f Modelfile
ollama run my-codellama "Write a Python function to sort a list"
```
## Training Details
- **Quantization**: 4-bit QLoRA
- **LoRA Rank**: 64
- **Learning Rate**: 2e-4
- **Epochs**: 4
- **Max Seq Length**: 2048
- **Training Data**: Custom Python programming examples (~2,000 examples)
- **GPU**: NVIDIA Tesla T4
## Prompt Format
```
### Instruction:
[Your instruction here]
### Response:
```
## License
Same as base model (Llama 2 license)
## Acknowledgements
- Base Model: [Meta's CodeLlama](https://huggingface.co/codellama/CodeLlama-7b-hf)
- Training Framework: [Unsloth](https://github.com/unslothai/unsloth)
|