Drawing inspiration from how human programmers “selectively skim” source code during development and debugging, SWE-Pruner performs task-aware adaptive pruning for long contexts.
reacted to di-zhang-fdu's
post with 👀about 6 hours ago
🏙️ Hugging Face Community Post Title: 🧬 Experimenting with "Dynamic Chaos" in Tamil SLMs
Hi everyone! I just published a new experimental study on Small Language Model (SLM) resilience.
I took the Qwen2.5-0.5B model and put it through a "Chaos Phase" to see how much weight data a tiny model can lose before its understanding of classical Tamil grammar breaks.
Key highlights of the study:
Target Data: Fine-tuned on the Thirukkural (1,330 couplets + modern explanations). The Chaos Step: Applied 20% random weight pruning but implemented "Layer Protection" for the Token Embeddings and LM Head to keep the characters readable. Compression: 4-bit (Q4_K_M) quantization for extreme efficiency. Result: A surrealist classical Tamil model that is ultra-light (~300MB) and ultra-fast!