plasterlabs/Qwen2-5-Finetuned-math

Qwen2.5-1.5B-GSM8K-zh-Adapter

This model is a fine-tuned version of Qwen2.5-1.5B specialized for solving Chinese mathematical word problems. It was trained using Low-Rank Adaptation (LoRA) on the meta-math/GSM8K_zh dataset.

Model Description

• Developed by: [Your Name/Organization]
• Model Type: Causal Language Model (Fine-tuned)
• Base Model: Qwen/Qwen2.5-1.5B
• Language(s): Chinese (Primary), English
• License: Apache 2.0
• Fine-tuning Method: LoRA (PEFT)

Training Details

The model was optimized for reasoning and step-by-step mathematical solutions in a Chinese context.

• Training Hardware: Kaggle (2x T4 GPUs)
• Quantization: 4-bit (bitsandbytes)
• Epochs/Steps: 500 Steps
• Batch Size: 64 (effective)
• Learning Rate: 2e-4
• Optimizer: Paged AdamW 8-bit

How to Use

To use this adapter, you must load the base Qwen2.5-1.5B model first and then attach the adapter.

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

model_id = "Qwen/Qwen2.5-1.5B"
adapter_id = "your-username/qwen2.5-gsm8k-zh-lora" # Replace with your repo

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "<|im_start|>user\n小明有5个苹果，小红给了他3个，他又吃了2个，现在他还有几个？<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))---
license: apache-2.0
base_model: qwen/Qwen2.5-1.5B
tags:
- math
- fine-tuned
- gsm8k
- qwen
- qwen2.5
- logic
- zh
- chinese
language:
- zh
- en
datasets:
- meta-math/GSM8K_zh
metrics:
- accuracy
library_name: peft
pipeline_tag: text-generation
---

# Qwen2.5-1.5B-GSM8K-zh-Adapter

This model is a fine-tuned version of **Qwen2.5-1.5B** specialized for solving Chinese mathematical word problems. It was trained using Low-Rank Adaptation (LoRA) on the `meta-math/GSM8K_zh` dataset.

## Model Description
- **Developed by:** [Your Name/Organization]
- **Model Type:** Causal Language Model (Fine-tuned)
- **Base Model:** `Qwen/Qwen2.5-1.5B`
- **Language(s):** Chinese (Primary), English
- **License:** Apache 2.0
- **Fine-tuning Method:** LoRA (PEFT)

## Training Details
The model was optimized for reasoning and step-by-step mathematical solutions in a Chinese context.

- **Training Hardware:** Kaggle (2x T4 GPUs)
- **Quantization:** 4-bit (bitsandbytes)
- **Epochs/Steps:** 500 Steps
- **Batch Size:** 64 (effective)
- **Learning Rate:** 2e-4
- **Optimizer:** Paged AdamW 8-bit

## How to Use

To use this adapter, you must load the base Qwen2.5-1.5B model first and then attach the adapter.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

model_id = "Qwen/Qwen2.5-1.5B"
adapter_id = "your-username/qwen2.5-gsm8k-zh-lora" # Replace with your repo

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "<|im_start|>user\n小明有5个苹果，小红给了他3个，他又吃了2个，现在他还有几个？<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))