Echoes123-3/qwen2.5-0.5b-coding-pruned

Qwen 2.5 0.5B – Coding Expert (FFN-Pruned)

This model is a coding-specialized expert derived from Qwen 2.5 0.5B using activation-based FFN neuron pruning on the HumanEval dataset.

Key features

• ~25% FFN neuron pruning
• No retraining
• Stable Python code generation
• Reduced compute per token

Method

1. Collected per-neuron FFN activations during inference
2. Pruned lowest-activation neurons per layer
3. Preserved embeddings, attention, and early representations

Intended use

• Research
• Model compression experiments
• Expert routing / MoE research

Limitations

• Specialized for coding
• General language performance may degrade

This is a research artifact, not a production model.