---
tags:
- gguf
- llama.cpp
- unsloth
- mistral
- python
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
---

# mistral-7b-python-gguf

Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference.

- Base model: Mistral 7B
- Fine-tuning framework: Unsloth
- Format: GGUF
- Author: AntoineChatry

---

# ⚠️ Disclaimer

This is an **early experimental fine-tune**.

It is **not production-ready**, not fully aligned, and not optimized for reliability or long-form reasoning.  
This project was created primarily for learning and experimentation.

Please do not expect state-of-the-art coding performance.

---

# Model Overview

This model is a conversational fine-tune of Mistral 7B trained primarily on:

- ShareGPT-style conversations
- Python-focused discussions
- Coding Q&A format

The objective was to:

- Experiment with fine-tuning
- Build a conversational Python model
- Export to GGUF for llama.cpp compatibility
- Test local inference workflows

No RLHF or advanced alignment was applied beyond the base model.

---

# Known Limitations

## Repetition Issues

- Frequently repeats phrases like:
  > "Here's the code:"
- Can loop or restate similar sentences
- Overuses patterns learned from dataset formatting

## Weak Long-Form Explanations

- Struggles with multi-paragraph structured reasoning
- May repeat itself when asked for detailed explanations
- Limited depth on conceptual explanations

## Instruction Following

- Not fully aligned
- May ignore strict formatting constraints
- Tends to prioritize generating code over detailed explanations

## Dataset Bias

- Strong ShareGPT conversational tone
- Python-heavy bias
- Some templated response structure

---

# What Works Reasonably Well

- Short Python snippets
- Basic debugging help
- Simple function generation
- Conversational coding prompts

Best performance is observed when:

- Prompts are clear and direct
- Expected output is short
- Tasks are code-focused

---

# Training Details

- Base: Mistral 7B
- Dataset format: ShareGPT-style conversational dataset (Python-oriented)
- Fine-tuned using Unsloth notebooks
- Converted to GGUF for llama.cpp compatibility
- Quantized version included (Q4_K_M)

No additional safety tuning or post-training optimization was applied.

---

# Example Usage

This model was finetuned and converted to GGUF format using Unsloth.

## llama.cpp

For text-only LLMs:

```bash
llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
```

For multimodal models:

```bash
llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
```

---
## Available Model files:
- `mistral-7b-instruct-v0.3.Q4_K_M.gguf`
---
# Ollama

An Ollama Modelfile is included for easy deployment.

Example:

```bash
ollama create mistral-python -f Modelfile
ollama run mistral-python
```

---

# Why This Model Is Public

This model represents a learning milestone.

Sharing imperfect models helps:

- Document fine-tuning progress
- Enable experimentation
- Collect feedback
- Iterate toward better versions

This is not a finished product.

---

# Unsloth

This model was trained 2x faster using Unsloth.

https://github.com/unslothai/unsloth

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

---

# License

Please refer to the original Mistral 7B license from Mistral AI.