pruned_olmo3_4096_32_32

WARNING: This model is PRUNED ONLY, NOT retrained or distilled!

Performance will be degraded compared to the original model. This is a structural pruning checkpoint intended as a starting point for knowledge distillation or fine-tuning.

Description

Structurally pruned version of allenai/OLMo-3-7B-Instruct.

Pruning Configuration

Parameter	Original	Pruned
Intermediate size (MLP)	11008	4096
Attention heads	32	32
Layers	32	32
Hidden size	4096	4096 (unchanged)

Important Notes

1. This model has NOT been retrained after pruning
2. Performance will be significantly degraded compared to the original
3. Intended use: Starting checkpoint for distillation/fine-tuning
4. For the distillation training data, see hbfreed/dolci-distill-packed