alperiox/Qwen2.5-0.5B-Instruct-arithmetic-abliterated

Qwen2.5-0.5B-Instruct - Arithmetic Abliteration

Base model: Qwen/Qwen2.5-0.5B-Instruct

Modified layer: 6/24 (25.0% depth) Method: Difference-in-means + Weight Orthogonalization Reference: Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024)

What is this?

This is a permanently abliterated version. The arithmetic direction was projected out of the model weights using:

W_new = W - r * (r^T * W)

No hooks needed. Works with any inference pipeline or quantization.

Behavior

• Arithmetic accuracy: ~0% (suppressed)
• General language: preserved

Note

100% arith suppression, 100% neutral coherence. Layer 6/24 (25% depth).

Original experiment

https://github.com/alperiox/abliteration-example-mlintern