Create README.md

README.md

@@ -0,0 +1,82 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-7B-Instruct
+tags:
+- qwen
+- qwen2.5
+- mathematics
+- vietnamese
+- sft
+- flash-attention-2
+datasets:
+- 5CD-AI/Vietnamese-395k-meta-math-MetaMathQA-gg-translated
+language:
+- vi
+pipeline_tag: text-generation
+library_name: trl
+---
+
+# Qwen2.5-7B-ViMetaMathQA-Mini
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) optimized for solving mathematical problems in **Vietnamese**. 
+
+It was trained on a 100,000-sample subset of the translated MetaMathQA dataset, utilizing high-performance computing techniques including **Flash Attention 2** and **BFloat16** precision on NVIDIA H100 hardware.
+
+## Model Details
+
+- **Developed by:** PeterPaker123
+- **Language:** Vietnamese
+- **Base Model:** Qwen/Qwen2.5-7B-Instruct
+- **Fine-tuning Dataset:** 5CD-AI/Vietnamese-395k-meta-math-MetaMathQA-gg-translated (100k subset)
+- **Task:** Mathematical Reasoning and Problem Solving
+
+## Training Configuration
+
+The model was trained with the following settings to ensure high efficiency and reasoning quality:
+
+- **Hardware:** NVIDIA H100 80GB HBM3
+- **Optimization:** Flash Attention 2, TF32 enabled
+- **Precision:** BFloat16 (Mixed Precision)
+- **Optimizer:** AdamW (8-bit)
+- **Learning Rate:** 1e-5
+- **Batch Size:** 4 (Per device)
+- **Gradient Accumulation:** 4 (Effective Batch Size: 16)
+- **Max Sequence Length:** 2048 tokens (with Sequence Packing)
+- **Epochs:** 1
+
+## Intended Use
+
+This model is designed to act as a mathematical assistant for Vietnamese speakers. It is particularly effective at:
+- Solving simple algebra problems.
+- Following Vietnamese instructional prompts for mathematical logic.
+
+### System Prompt
+For best results, use the system prompt used during training:
+> `Bạn là một chuyên gia toán học. Hãy giải bài toán sau bằng tiếng Việt.`
+
+## Usage Example
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_name = "PeterPaker123/Qwen2.5-7B-ViMetaMathQA-Mini"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    attn_implementation="flash_attention_2" # Recommended for H100/A100/L4
+)
+
+messages = [
+    {"role": "system", "content": "Bạn là một chuyên gia toán học. Hãy giải bài toán sau bằng tiếng Việt."},
+    {"role": "user", "content": "Tìm x, biết 2x + 5 = 15."}
+]
+
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt").to("cuda")
+
+outputs = model.generate(**inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))