| library_name: peft | |
| license: apache-2.0 | |
| base_model: Qwen/Qwen2.5-7B-Instruct | |
| tags: | |
| - axolotl | |
| - lora | |
| - transformers | |
| - qwen | |
| - fine-tuning | |
| - text-generation | |
| datasets: | |
| - private/local # تم استخدام بيانات تدريب محلية غير منشورة | |
| pipeline_tag: text-generation | |
| model-index: | |
| - name: thoth_text_v3 | |
| results: [] | |
| [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200"/>](https://github.com/axolotl-ai-cloud/axolotl) | |
| # 🧠 Thoth Text v3 | |
| نموذج **Thoth Text v3** هو إصدار مطوّر من **Qwen/Qwen2.5-7B-Instruct**، تم تدريبه باستخدام أسلوب **LoRA fine-tuning** عبر مكتبة **Axolotl**، على بيانات نصية عربية تم إعدادها محليًا (غير منشورة). | |
| --- | |
| ## 📋 معلومات أساسية | |
| - **Base model:** [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) | |
| - **Previous adapters:** | |
| - [injazsmart/thoth_text](https://huggingface.co/injazsmart/thoth_text) | |
| - [injazsmart/thoth_text_v2](https://huggingface.co/injazsmart/thoth_text_v2) | |
| - **Current version:** `thoth_text_v3` | |
| - **Framework:** Axolotl + PEFT + Transformers | |
| - **Hardware:** 8-bit training على GPU واحد | |
| --- | |
| ## ⚙️ إعدادات التدريب (config.yaml) | |
| ```yaml | |
| adapter: lora | |
| base_model: Qwen/Qwen2.5-7B-Instruct | |
| bf16: auto | |
| lora_model_dir: injazsmart/thoth_text_v2 | |
| datasets: | |
| - path: ./data/injaz.json | |
| type: alpaca | |
| sequence_len: 4096 | |
| micro_batch_size: 16 | |
| gradient_accumulation_steps: 1 | |
| num_epochs: 2 | |
| learning_rate: 0.0001 | |
| optimizer: adamw_bnb_8bit | |
| load_in_8bit: true | |
| lora_r: 8 | |
| lora_alpha: 16 | |
| lora_dropout: 0.05 | |
| lora_target_modules: | |
| - q_proj | |
| - v_proj | |
| - k_proj | |
| - o_proj | |
| - gate_proj | |
| - down_proj | |
| - up_proj | |
| train_on_inputs: false | |
| output_dir: ./outputs/thoth_text_v3 | |