| | --- |
| | tags: |
| | - gguf |
| | - llama.cpp |
| | - unsloth |
| | - mistral |
| | - python |
| | base_model: |
| | - mistralai/Mistral-7B-Instruct-v0.3 |
| | --- |
| | |
| | # mistral-7b-python-gguf |
| |
|
| | Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference. |
| |
|
| | - Base model: Mistral 7B |
| | - Fine-tuning framework: Unsloth |
| | - Format: GGUF |
| | - Author: AntoineChatry |
| |
|
| | --- |
| |
|
| | # ⚠️ Disclaimer |
| |
|
| | This is an **early experimental fine-tune**. |
| |
|
| | It is **not production-ready**, not fully aligned, and not optimized for reliability or long-form reasoning. |
| | This project was created primarily for learning and experimentation. |
| |
|
| | Please do not expect state-of-the-art coding performance. |
| |
|
| | --- |
| |
|
| | # Model Overview |
| |
|
| | This model is a conversational fine-tune of Mistral 7B trained primarily on: |
| |
|
| | - ShareGPT-style conversations |
| | - Python-focused discussions |
| | - Coding Q&A format |
| |
|
| | The objective was to: |
| |
|
| | - Experiment with fine-tuning |
| | - Build a conversational Python model |
| | - Export to GGUF for llama.cpp compatibility |
| | - Test local inference workflows |
| |
|
| | No RLHF or advanced alignment was applied beyond the base model. |
| |
|
| | --- |
| |
|
| | # Known Limitations |
| |
|
| | ## Repetition Issues |
| |
|
| | - Frequently repeats phrases like: |
| | > "Here's the code:" |
| | - Can loop or restate similar sentences |
| | - Overuses patterns learned from dataset formatting |
| |
|
| | ## Weak Long-Form Explanations |
| |
|
| | - Struggles with multi-paragraph structured reasoning |
| | - May repeat itself when asked for detailed explanations |
| | - Limited depth on conceptual explanations |
| |
|
| | ## Instruction Following |
| |
|
| | - Not fully aligned |
| | - May ignore strict formatting constraints |
| | - Tends to prioritize generating code over detailed explanations |
| |
|
| | ## Dataset Bias |
| |
|
| | - Strong ShareGPT conversational tone |
| | - Python-heavy bias |
| | - Some templated response structure |
| |
|
| | --- |
| |
|
| | # What Works Reasonably Well |
| |
|
| | - Short Python snippets |
| | - Basic debugging help |
| | - Simple function generation |
| | - Conversational coding prompts |
| |
|
| | Best performance is observed when: |
| |
|
| | - Prompts are clear and direct |
| | - Expected output is short |
| | - Tasks are code-focused |
| |
|
| | --- |
| |
|
| | # Training Details |
| |
|
| | - Base: Mistral 7B |
| | - Dataset format: ShareGPT-style conversational dataset (Python-oriented) |
| | - Fine-tuned using Unsloth notebooks |
| | - Converted to GGUF for llama.cpp compatibility |
| | - Quantized version included (Q4_K_M) |
| |
|
| | No additional safety tuning or post-training optimization was applied. |
| |
|
| | --- |
| |
|
| | # Example Usage |
| |
|
| | This model was finetuned and converted to GGUF format using Unsloth. |
| |
|
| | ## llama.cpp |
| |
|
| | For text-only LLMs: |
| |
|
| | ```bash |
| | llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja |
| | ``` |
| |
|
| | For multimodal models: |
| |
|
| | ```bash |
| | llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja |
| | ``` |
| |
|
| | --- |
| | ## Available Model files: |
| | - `mistral-7b-instruct-v0.3.Q4_K_M.gguf` |
| | --- |
| | # Ollama |
| |
|
| | An Ollama Modelfile is included for easy deployment. |
| |
|
| | Example: |
| |
|
| | ```bash |
| | ollama create mistral-python -f Modelfile |
| | ollama run mistral-python |
| | ``` |
| |
|
| | --- |
| |
|
| | # Why This Model Is Public |
| |
|
| | This model represents a learning milestone. |
| |
|
| | Sharing imperfect models helps: |
| |
|
| | - Document fine-tuning progress |
| | - Enable experimentation |
| | - Collect feedback |
| | - Iterate toward better versions |
| |
|
| | This is not a finished product. |
| |
|
| | --- |
| |
|
| | # Unsloth |
| |
|
| | This model was trained 2x faster using Unsloth. |
| |
|
| | https://github.com/unslothai/unsloth |
| |
|
| | <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/> |
| |
|
| | --- |
| |
|
| | # License |
| |
|
| | Please refer to the original Mistral 7B license from Mistral AI. |
