AnKhanh/Deepseek-Distill-Qwen-prune

DeepSeek-R1-Distill-Qwen — DAOC-calibrated SparseGPT Pruning Sweep

Unstructured sparsity sweep of deepseek-ai/DeepSeek-R1-Distill-Qwen-{1.5B,7B} pruned with SparseGPT, calibrated on DAOC-style near-miss rollout pairs (trace dataset: open-r1/OpenR1-Math-220k).

Layout

Each subfolder is a standalone HF model checkpoint (config.json + safetensors + tokenizer files):

1.5B/sparse10/   # 10% sparsity
1.5B/sparse20/   # 20% sparsity
1.5B/sparse30/   # 30% sparsity
1.5B/sparse60/   # 60% sparsity
1.5B/sparse70/   # 70% sparsity
1.5B/sparse80/   # 80% sparsity
1.5B/sparse90/   # 90% sparsity
7B/sparse10/
7B/sparse20/
7B/sparse30/
7B/sparse60/
7B/sparse70/
7B/sparse80/
7B/sparse90/

Load a specific variant with:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "AnKhanh/Deepseek-Distill-Qwen-prune",
    subfolder="7B/sparse30",
)
tokenizer = AutoTokenizer.from_pretrained(
    "AnKhanh/Deepseek-Distill-Qwen-prune",
    subfolder="7B/sparse30",
)