llaa33219/Solar-Open-100B-to-4B-test-2

Solar-Open-100B-pruned-5pct

This model is a pruned version of upstage/Solar-Open-100B.

Pruning Details

Property	Value
Original Model	upstage/Solar-Open-100B
Original Parameters	26.87B
Pruned Parameters	4.40B
Compression Ratio	0.1639 (6.1x smaller)
Strategy	Layer + Head + FFN (Recommended)
Importance Metric	magnitude
Layers Removed	36
FFN Reduction	0.00%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")

Notes

This model was created using structured pruning techniques including:

• Layer pruning (removing entire transformer layers)
• Attention head pruning (reducing number of attention heads)
• FFN dimension pruning (reducing intermediate layer sizes)

The pruning was based on magnitude importance scoring to preserve the most important weights.