README.md

metadata

license: apache-2.0
base_model: Qwen/Qwen2.5-0.5B
tags:
  - coding
  - pruning
  - ffn-pruning
  - specialization

Qwen 2.5 0.5B – Coding Expert (FFN-Pruned)

This model is a coding-specialized expert derived from Qwen 2.5 0.5B using activation-based FFN neuron pruning on the HumanEval dataset.

Key features

• ~25% FFN neuron pruning
• No retraining
• Stable Python code generation
• Reduced compute per token

Method

1. Collected per-neuron FFN activations during inference
2. Pruned lowest-activation neurons per layer
3. Preserved embeddings, attention, and early representations

Intended use

• Research
• Model compression experiments
• Expert routing / MoE research

Limitations

• Specialized for coding
• General language performance may degrade

This is a research artifact, not a production model.