GLM-5 744B Abliterated (FP8)
This doesnt work an abliterated (uncensored) version of THUDM/GLM-5-FP8 with safety guardrails removed via weight orthogonalization.
Method
Abliteration (representation engineering) was used to identify and remove the "refusal direction" from the model's residual stream:
- Computed refusal directions for all 78 layers by collecting activations on 50 harmful vs 50 harmless prompts and computing mean difference vectors
- Applied weight orthogonalization to layers 15-54 (o_proj and shared_experts.down_proj) with alpha=1.0
- FP8-aware processing: Proper dequantization using block-wise scale_inv factors, abliteration in float32, and re-quantization preserving original scale factors to minimize perturbation
Technical Details
- Architecture: GLM-5 MoE (744B total, 40B active), 78 layers, 6144 hidden dim
- Layers 0-2: Dense MLP, Layers 3-77: MoE with FP8Expert fused kernels
- Modified weights: 80 weight matrices (40 o_proj + 40 shared_experts.down_proj)
- Quantization: FP8 E4M3 with block-wise scaling (128x128 blocks)
- Scale preservation: Original weight_scale_inv factors retained for minimal quantization drift
Hardware Used
8x NVIDIA B200 (1.4TB VRAM) on Vast.ai
Usage
This model requires the same setup as the base GLM-5-FP8 model. Use trust_remote_code=True when loading.
Disclaimer
This model is provided for research purposes only. The removal of safety guardrails means the model may generate harmful, biased, or offensive content. Users are responsible for ensuring appropriate use.
- Downloads last month
- 2