| license: apache-2.0 | |
| base_model: Qwen/Qwen2.5-72B-Instruct | |
| tags: | |
| - math | |
| - reasoning | |
| - qwen2 | |
| - lora | |
| - peft | |
| - numina | |
| library_name: peft | |
| pipeline_tag: text-generation | |
| datasets: | |
| - AI-MO/NuminaMath-CoT | |
| # Elle-72B-Math-v2 | |
| ## Model Description | |
| Elle-72B-Math-v2 is a LoRA adapter fine-tuned on [Qwen/Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) for mathematical reasoning using the NuminaMath-CoT dataset. | |
| ## Model Details | |
| - **Base Model**: Qwen/Qwen2.5-72B-Instruct | |
| - **Adapter Type**: LoRA | |
| - **LoRA Rank**: 64 | |
| - **LoRA Alpha**: 128 | |
| - **Target Modules**: All linear layers | |
| - **Training Data**: NuminaMath-CoT | |
| ## Training | |
| Fine-tuned using: | |
| - Chain-of-thought mathematical reasoning examples | |
| - Step-by-step problem decomposition | |
| - Multiple solution strategies (algebraic, numerical, symbolic) | |
| ## Usage | |
| ```python | |
| from peft import PeftModel | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| base_model = AutoModelForCausalLM.from_pretrained( | |
| "Qwen/Qwen2.5-72B-Instruct", | |
| torch_dtype="auto", | |
| device_map="auto", | |
| trust_remote_code=True | |
| ) | |
| model = PeftModel.from_pretrained(base_model, "aphoticshaman/elle-72b-math-v2") | |
| tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-72B-Instruct") | |
| ``` | |
| ## License | |
| Apache 2.0 | |