mistral-7b-python-gguf

Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference.

• Base model: Mistral 7B
• Fine-tuning framework: Unsloth
• Format: GGUF
• Author: AntoineChatry

---

⚠️ Disclaimer

This is an early experimental fine-tune.

It is not production-ready, not fully aligned, and not optimized for reliability or long-form reasoning.
This project was created primarily for learning and experimentation.

Please do not expect state-of-the-art coding performance.

---

Model Overview

This model is a conversational fine-tune of Mistral 7B trained primarily on:

• ShareGPT-style conversations
• Python-focused discussions
• Coding Q&A format

The objective was to:

• Experiment with fine-tuning
• Build a conversational Python model
• Export to GGUF for llama.cpp compatibility
• Test local inference workflows

No RLHF or advanced alignment was applied beyond the base model.

---

Known Limitations

Repetition Issues

• Frequently repeats phrases like:

  "Here's the code:"

• Can loop or restate similar sentences
• Overuses patterns learned from dataset formatting

Weak Long-Form Explanations

• Struggles with multi-paragraph structured reasoning
• May repeat itself when asked for detailed explanations
• Limited depth on conceptual explanations

Instruction Following

• Not fully aligned
• May ignore strict formatting constraints
• Tends to prioritize generating code over detailed explanations

Dataset Bias

• Strong ShareGPT conversational tone
• Python-heavy bias
• Some templated response structure

---

What Works Reasonably Well

• Short Python snippets
• Basic debugging help
• Simple function generation
• Conversational coding prompts

Best performance is observed when:

• Prompts are clear and direct
• Expected output is short
• Tasks are code-focused

---

Training Details

• Base: Mistral 7B
• Dataset format: ShareGPT-style conversational dataset (Python-oriented)
• Fine-tuned using Unsloth notebooks
• Converted to GGUF for llama.cpp compatibility
• Quantized version included (Q4_K_M)

No additional safety tuning or post-training optimization was applied.

---

Example Usage

This model was finetuned and converted to GGUF format using Unsloth.

llama.cpp

For text-only LLMs:

llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja

For multimodal models:

llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja

---

Available Model files:

• mistral-7b-instruct-v0.3.Q4_K_M.gguf

---

Ollama

An Ollama Modelfile is included for easy deployment.

Example:

ollama create mistral-python -f Modelfile
ollama run mistral-python

---

Why This Model Is Public

This model represents a learning milestone.

Sharing imperfect models helps:

• Document fine-tuning progress
• Enable experimentation
• Collect feedback
• Iterate toward better versions

This is not a finished product.

---

Unsloth

This model was trained 2x faster using Unsloth.

https://github.com/unslothai/unsloth

---

License

Please refer to the original Mistral 7B license from Mistral AI.