Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,68 @@
+---
+license: apache-2.0
+language:
+  - en
+tags:
+  - mistral
+  - alpaca
+  - fine-tuning
+  - code
+  - crud
+  - sft
+  - vllm
+datasets:
+  - kramster/crud-code-tests
+base_model: mistralai/Mistral-7B-Instruct-v0.2
+---
+
+# 🧠 Evolve Mistral: Fine-Tuned Mistral-7B-Instruct on CRUD Coding Tasks
+
+This model is a fine-tuned version of [`mistralai/Mistral-7B-Instruct-v0.2`](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), adapted for reasoning about structured CRUD-based code inputs and instruction-following tasks. It was trained on a dataset in [Alpaca](https://github.com/tatsu-lab/stanford_alpaca)-style format, using supervised fine-tuning (SFT).
+
+---
+
+## 📂 Dataset
+
+The model was trained on:
+
+**[`kramster/crud-code-tests`](https://huggingface.co/datasets/kramster/crud-code-tests)**  
+A dataset of instruction-based code snippets focusing on Create, Read, Update, and Delete operations in various programming contexts. It uses the Alpaca-style JSON format with fields: `instruction`, `input`, and `output`.
+
+---
+
+## 🏗️ Training Setup
+
+| Detail              | Value |
+|---------------------|-------|
+| Base model          | `mistralai/Mistral-7B-Instruct-v0.2` |
+| LoRA Config         | r=32, alpha=16 |
+| Framework           | Axolotl + DeepSpeed + LoRA |
+| Training Steps      | 51 |
+| Epochs              | ~3.94 |
+| Mixed Precision     | bfloat16 |
+| GPU                 | NVIDIA H100 80GB |
+| Training Duration   | 10m 26s |
+| Final Train Loss    | 0.0909 |
+| Final Eval Loss     | 0.1012 |
+| FLOPs used          | 347.6 trillion |
+
+
+---
+
+## 🧪 Evaluation Summary
+
+- **Eval runtime:** 2.84s  
+- **Eval samples/sec:** 2.11  
+- **Eval steps/sec:** 1.05  
+- **Gradient norm (final):** 0.064  
+- **Final LR:** 2.93e-7
+
+---
+
+## 🧠 Example Usage
+
+vllm-api-server \
+  --model kramster/evolve-mistral \
+  --max-model-len 64000 \
+  --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' \
+  --no-enable-prefix-caching