llaa33219/Solar-Open-100B-pruned-5pct-test2

Solar-Open-100B-pruned-5pct

This model is a pruned version of upstage/Solar-Open-100B.

Pruning Details

Property	Value
Original Model	upstage/Solar-Open-100B
Original Parameters	9.46B
Pruned Parameters	3.09B
Compression Ratio	0.3261 (3.1x smaller)
Strategy	Layer + Head + FFN (Recommended)
Importance Metric	magnitude
Layers Removed	31
FFN Reduction	60.00%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")

Notes

This model was created using structured pruning techniques including:

• Layer pruning (removing entire transformer layers)
• Attention head pruning (reducing number of attention heads)
• FFN dimension pruning (reducing intermediate layer sizes)

The pruning was based on magnitude importance scoring to preserve the most important weights.