Ling-3.0-flash-heretic

Abliterated (refusal-removed) version of inclusionAI/Ling-3.0-flash (124B MoE, KDA + Gated-MLA hybrid attention).

Method

Directional ablation (Arditi et al., Refusal in Language Models Is Mediated by a Single Direction, 2024) via Heretic (https://github.com/p-e-w/heretic), patched for Ling's hybrid attention (KDA attention.o_proj, Gated MLA attention.dense, MoE down-projections incl. shared experts). MTP head preserved (all 63,783 tensors).

Ablation parameters (per-layer weight kernel)

• attn.o_proj: max_weight 1.474 @ layer 31.34, min_weight 1.014 @ distance 11.19
• mlp.down_proj: max_weight 1.175 @ layer 35.97, min_weight 0.002 @ distance 1.82

Evaluation

Metric	Value
Refusals (harmful_behaviors test[:100])	1/100 (base: 39/100)
KL divergence vs base	0.0038
Capability spot-checks	coherent, no collapse observed

Notes

• Full BF16, 24 shards; requires trust_remote_code; modeling file patched for transformers 5.x (rope_scaling factor default).
• GGUF quantizations: bloomer010/Ling-3.0-flash-heretic-GGUF