Qwen2.5-3B-Instruct-math-extra-heavy

This model is a extra-heavy pruned version of Qwen/Qwen2.5-3B-Instruct, specialized for MATH tasks.

Pruning Details

• Base Model: Qwen/Qwen2.5-3B-Instruct
• Specialization: Math
• Prune Mode: Extra-heavy
• Method: Activation-based weight pruning

Performance Comparison

Category	Original	Pruned
Python	100.0%	0.0%
HTML	6.7%	N/A
Trivia	100.0%	N/A
Math	60.0%	N/A
Reasoning	N/A	N/A
Medical	100.0%	N/A
Linux	100.0%	N/A
Writing	73.3%	N/A

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("CompactAI/Qwen2.5-3B-Instruct-math-extra-heavy")
tokenizer = AutoTokenizer.from_pretrained("CompactAI/Qwen2.5-3B-Instruct-math-extra-heavy")

License

This model inherits the license from the base model.