AntoineChatry commited on
Commit
450486b
·
verified ·
1 Parent(s): ed7ae7a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +156 -8
README.md CHANGED
@@ -3,21 +3,169 @@ tags:
3
  - gguf
4
  - llama.cpp
5
  - unsloth
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  ---
8
 
9
- # mistralv1-gguf : GGUF
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
- This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
- **Example usage**:
14
- - For text only LLMs: `llama-cli -hf AntoineChatry/mistralv1-gguf --jinja`
15
- - For multimodal models: `llama-mtmd-cli -hf AntoineChatry/mistralv1-gguf --jinja`
16
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ## Available Model files:
18
  - `mistral-7b-instruct-v0.3.Q4_K_M.gguf`
 
 
19
 
20
- ## Ollama
21
  An Ollama Modelfile is included for easy deployment.
22
- This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
23
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  - gguf
4
  - llama.cpp
5
  - unsloth
6
+ - mistral
7
+ - python
8
+ base_model:
9
+ - mistralai/Mistral-7B-Instruct-v0.3
10
+ ---
11
+
12
+ # mistral-7b-python-gguf
13
+
14
+ Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference.
15
+
16
+ - Base model: Mistral 7B
17
+ - Fine-tuning framework: Unsloth
18
+ - Format: GGUF
19
+ - Author: AntoineChatry
20
 
21
  ---
22
 
23
+ # ⚠️ Disclaimer
24
+
25
+ This is an **early experimental fine-tune**.
26
+
27
+ It is **not production-ready**, not fully aligned, and not optimized for reliability or long-form reasoning.
28
+ This project was created primarily for learning and experimentation.
29
+
30
+ Please do not expect state-of-the-art coding performance.
31
+
32
+ ---
33
+
34
+ # Model Overview
35
+
36
+ This model is a conversational fine-tune of Mistral 7B trained primarily on:
37
+
38
+ - ShareGPT-style conversations
39
+ - Python-focused discussions
40
+ - Coding Q&A format
41
+
42
+ The objective was to:
43
+
44
+ - Experiment with fine-tuning
45
+ - Build a conversational Python model
46
+ - Export to GGUF for llama.cpp compatibility
47
+ - Test local inference workflows
48
+
49
+ No RLHF or advanced alignment was applied beyond the base model.
50
+
51
+ ---
52
 
53
+ # Known Limitations
54
+
55
+ ## Repetition Issues
56
+
57
+ - Frequently repeats phrases like:
58
+ > "Here's the code:"
59
+ - Can loop or restate similar sentences
60
+ - Overuses patterns learned from dataset formatting
61
+
62
+ ## Weak Long-Form Explanations
63
+
64
+ - Struggles with multi-paragraph structured reasoning
65
+ - May repeat itself when asked for detailed explanations
66
+ - Limited depth on conceptual explanations
67
+
68
+ ## Instruction Following
69
+
70
+ - Not fully aligned
71
+ - May ignore strict formatting constraints
72
+ - Tends to prioritize generating code over detailed explanations
73
+
74
+ ## Dataset Bias
75
+
76
+ - Strong ShareGPT conversational tone
77
+ - Python-heavy bias
78
+ - Some templated response structure
79
+
80
+ ---
81
+
82
+ # What Works Reasonably Well
83
+
84
+ - Short Python snippets
85
+ - Basic debugging help
86
+ - Simple function generation
87
+ - Conversational coding prompts
88
+
89
+ Best performance is observed when:
90
+
91
+ - Prompts are clear and direct
92
+ - Expected output is short
93
+ - Tasks are code-focused
94
+
95
+ ---
96
 
97
+ # Training Details
 
 
98
 
99
+ - Base: Mistral 7B
100
+ - Dataset format: ShareGPT-style conversational dataset (Python-oriented)
101
+ - Fine-tuned using Unsloth notebooks
102
+ - Converted to GGUF for llama.cpp compatibility
103
+ - Quantized version included (Q4_K_M)
104
+
105
+ No additional safety tuning or post-training optimization was applied.
106
+
107
+ ---
108
+
109
+ # Example Usage
110
+
111
+ This model was finetuned and converted to GGUF format using Unsloth.
112
+
113
+ ## llama.cpp
114
+
115
+ For text-only LLMs:
116
+
117
+ ```bash
118
+ llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
119
+ ```
120
+
121
+ For multimodal models:
122
+
123
+ ```bash
124
+ llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja
125
+ ```
126
+
127
+ ---
128
  ## Available Model files:
129
  - `mistral-7b-instruct-v0.3.Q4_K_M.gguf`
130
+ ---
131
+ # Ollama
132
 
 
133
  An Ollama Modelfile is included for easy deployment.
134
+
135
+ Example:
136
+
137
+ ```bash
138
+ ollama create mistral-python -f Modelfile
139
+ ollama run mistral-python
140
+ ```
141
+
142
+ ---
143
+
144
+ # Why This Model Is Public
145
+
146
+ This model represents a learning milestone.
147
+
148
+ Sharing imperfect models helps:
149
+
150
+ - Document fine-tuning progress
151
+ - Enable experimentation
152
+ - Collect feedback
153
+ - Iterate toward better versions
154
+
155
+ This is not a finished product.
156
+
157
+ ---
158
+
159
+ # Unsloth
160
+
161
+ This model was trained 2x faster using Unsloth.
162
+
163
+ https://github.com/unslothai/unsloth
164
+
165
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>
166
+
167
+ ---
168
+
169
+ # License
170
+
171
+ Please refer to the original Mistral 7B license from Mistral AI.