Add model card README

README.md

@@ -0,0 +1,51 @@
+---
+license: apache-2.0
+base_model:
+  - mistralai/Devstral-Small-2507
+tags:
+  - mistral
+  - code
+  - merge
+  - lora
+  - sft
+language:
+  - en
+library_name: transformers
+---
+
+# RenCoder-Devstral-Small-2507
+
+This model is a merged version of [mistralai/Devstral-Small-2507](https://huggingface.co/mistralai/Devstral-Small-2507) with a LoRA adapter trained on [pankajmathur/OpenThoughts-Agent-v1-SFT](https://huggingface.co/datasets/pankajmathur/OpenThoughts-Agent-v1-SFT) dataset.
+
+## Model Details
+
+- **Base Model:** [mistralai/Devstral-Small-2507](https://huggingface.co/mistralai/Devstral-Small-2507)
+- **LoRA Adapter:** [pankajmathur/Devstral-Small-2507-sft-v1-adapter](https://huggingface.co/pankajmathur/Devstral-Small-2507-sft-v1-adapter)
+- **Training Dataset:** [pankajmathur/OpenThoughts-Agent-v1-SFT](https://huggingface.co/datasets/pankajmathur/OpenThoughts-Agent-v1-SFT)
+- **Parameters:** ~24B
+- **Precision:** bfloat16
+
+## Training Configuration
+
+The LoRA adapter was trained with the following configuration:
+- **LoRA Rank (r):** 32
+- **LoRA Alpha:** 16
+- **LoRA Dropout:** 0.05
+- **Target Modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
+- **Sequence Length:** 8192
+- **Learning Rate:** 0.0001
+- **Optimizer:** AdamW 8-bit
+- **Epochs:** 3
+
+## Usage
+
+
+
+## License
+
+This model inherits the Apache 2.0 license from the base Devstral-Small-2507 model.
+
+## Acknowledgements
+
+- [Mistral AI](https://mistral.ai/) for the Devstral-Small-2507 base model
+- [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) for training infrastructure