You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
This is an uncensored (abliterated) model with content-policy refusals removed. Access is gated. By requesting access you agree to the Acceptable Use terms below and accept full responsibility for how you use these weights.
Log in or Sign Up to review the conditions and access this model content.
GLM-5.2-ABLITERATED — NVFP4 (Blackwell / SM120) STILL TESTING WILL, I WILL GIVE UPDATE IN 48 HOURS ALONG WITH REVIEWING REQUEST
An abliterated (refusal-removed) build of GLM-5.2, quantized to NVFP4 (4-bit) and packaged to serve on 8× RTX PRO 6000 Blackwell (SM120) with vLLM. This repo ships the weights and a complete, reproducible serving recipe — because getting a glm_moe_dsa NVFP4 model to boot on Blackwell took solving four distinct initialization blockers, all documented below.
⚠️ Uncensored model. Refusal directions in the residual stream have been ablated, so the model does not decline requests on content-policy grounds. Read Responsible Use before downloading. Access is gated.
TL;DR
- Base:
Blackfrost-AI/GLM-5.2-ABLITERATED-BF16— a GLM-5.2 model that had previously undergone abliteration. - This repo: NVFP4 (4-bit, group-size 16) quantization for practical inference on ~768 GB of Blackwell VRAM, plus the deploy kit.
- Serves at: ~25–27 tok/s single-stream on 8× RTX PRO 6000 (TP=8), OpenAI-compatible endpoint.
- Everything you need is in
deploy-kit/— one config patch + one launch script + smoke test.
Model Details
| Architecture | GlmMoeDsaForCausalLM (glm_moe_dsa) — GLM MoE with Multi-head Latent Attention (MLA) + DeepSeek-style Sparse Attention (DSA) |
| Parameters | Mixture-of-Experts, ~753B total (NVFP4 footprint ≈ 420 GB) |
| Layers | 78 (first 3 dense, remaining 75 MoE) |
| Hidden size | 6144 |
| Experts | 256 routed, 8 active per token, + 1 shared expert |
| Attention | MLA (kv_lora_rank=512, q_lora_rank=2048) + DSA sparse indexer |
| Vocab | 154,880 |
| Max context | 1,048,576 (served at 32,768 by default; raise per VRAM budget) |
| Quantization | NVFP4 (quant_algo: NVFP4, group-size 16); MLA projections, router gates, shared experts, embeddings, and MTP kept in BF16 (see config.json → quantization_config.ignore) |
| Chat template | GLM control tokens preserved (<think>, `< |
Abliteration
The refusal-mediating directions in the residual stream have been identified and removed, so the model will not refuse on content-policy grounds. This behavior is inherited from the abliterated base model. The quantization here does not add or restore any safety behavior. In internal spot checks the model engaged on 15/15 previously-refused prompts (0 refusals).
Quick Start
# 1. Download the weights (gated)
hf download Blackfrost-AI/GLM-5.2-ABLITERATED-NVFP4 --local-dir /models/GLM-5.2-ABLITERATED-NVFP4
# 2. (Optional) The shipped config.json is ALREADY patched with the fused_qkv_a_proj fix.
# This just confirms it — expect "Patched":
python3 deploy-kit/patch_config.py /models/GLM-5.2-ABLITERATED-NVFP4/config.json --check
# 3. Launch (MTP disabled — required on this stack, see Blocker 3;
# chat templates auto-load from deploy-kit/recipe/)
MTP=0 MODEL=/models/GLM-5.2-ABLITERATED-NVFP4 bash deploy-kit/serve_glm52abl_nvfp4.sh
# 4. Smoke test
bash deploy-kit/smoke_test.sh
First boot takes ~3–4 minutes (weight load + kernel compilation + CUDA-graph capture). Endpoint: http://localhost:8000/v1 (model name glm52abl).
Serving Recipe — the four boot blockers
This glm_moe_dsa NVFP4 model does not boot out-of-the-box on Blackwell. Four issues each crash the server at a different init stage. All four fixes are baked into deploy-kit/serve_glm52abl_nvfp4.sh; here is what and why.
| # | Stage | Symptom | Root cause | Fix |
|---|---|---|---|---|
| 1 | NCCL init | hang/crash at ncclCommInitRank on 8-way TP |
serving image bakes NCCL 2.30.4 via LD_PRELOAD; regression on cross-NUMA SYS topology |
neutralize the NCCL LD_PRELOAD/path env vars → fall back to bundled NCCL 2.29.7 |
| 2 | Weight load | AssertionError: shape mismatch [3027 vs 6144] on fused_qkv_a_proj |
vLLM fuses MLA q_a_proj + kv_a_proj_with_mqa into fused_qkv_a_proj, which is not in the NVFP4 ignore list → gets 4-bit-quantized and halves its output dim |
add *.self_attn.fused_qkv_a_proj to quantization_config.ignore (keep BF16). Use patch_config.py |
| 3 | Model init | moe_backend='b12x' is not supported for unquantized MoE |
MTP/eagle draft model doesn't inherit the NVFP4 quant config, so its MoE looks unquantized to the b12x backend | disable MTP speculative decoding: MTP=0 (costs ~2–3× decode throughput; correctness unaffected) |
| 4 | CUDA-graph capture | custom_all_reduce.cuh:455 'invalid argument' |
image bakes VLLM_ENABLE_PCIE_ALLREDUCE=1; the custom C++ PCIe all-reduce kernel fails on SM120 (no NVLink, all-PCIe box) |
VLLM_ENABLE_PCIE_ALLREDUCE=0 + --disable-custom-all-reduce → NCCL all-reduce |
Full narrative for each blocker (with log excerpts and verification) is in deploy-kit/README.md.
The config patch (Blocker 2)
"quantization_config": {
"ignore": [
...
"*.shared_head*",
+ "*.self_attn.fused_qkv_a_proj"
]
}
patch_config.py applies this idempotently (with --check / --revert).
Environment (verified working)
| Component | Version |
|---|---|
| GPU | 8× NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB each) |
| Architecture | SM120 (sm_120a) |
| Interconnect | PCIe (no NVLink) |
| Serving image | Black Benediction vLLM (voipmonitor/vllm:black-benediction-b12x…cu132-20260608) |
| vLLM | 0.11.2.dev279 (black-benediction b12x) |
| PyTorch / CUDA | 2.12.0 + cu132 / CUDA 13.2 |
| NCCL (effective) | 2.29.7 (image's 2.30.4 disabled) |
Key serve flags: --quantization modelopt_fp4 --attention-backend B12X_MLA_SPARSE --moe-backend b12x --kv-cache-dtype fp8 --tensor-parallel-size 8 --decode-context-parallel-size 8 --disable-custom-all-reduce --tool-call-parser glm47 --reasoning-parser glm45. The DSA per-layer indexer pattern is passed via --hf-overrides '{"index_topk_pattern":"…"}' (vLLM reads index_topk_pattern, not config.indexer_types) — required for long-context coherence. See the launch script for the exact values.
Hardware requirements
- ~420 GB of weights + KV cache. It fits 8× 96 GB Blackwell at
--gpu-memory-utilization 0.95,max-model-len 32768,max-num-seqs 2. Longer context / more concurrency needs more headroom. - SM120 is required for the b12x kernels in this recipe. Other architectures need a different backend (and will not use the b12x env flags).
Responsible Use
This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.
Prohibited uses (access is gated on agreeing to these):
- Anything involving the sexual exploitation or endangerment of minors.
- Content promoting self-harm or suicide.
- Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
- Any use prohibited by the upstream GLM license.
You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream GLM base model — review and comply with it before use or redistribution.
Limitations
- Research/serving artifact, quantized to 4-bit — expect quality below the full-precision base, especially on long, hard reasoning.
- MTP speculative decoding disabled on this stack (~2–3× slower decode than with MTP). Re-enabling requires a vLLM eagle-loader patch to propagate the quant config to the draft model.
- Custom PCIe all-reduce and NCCL 2.30.4 are disabled for SM120 stability; minor collective-latency overhead.
- Refusal behavior was validated only by spot check, not an exhaustive benchmark.
Published by Blackfrost AI. This card documents a quantized, uncensored research artifact and its exact serving recipe. Access is gated; use responsibly.
- Downloads last month
- 2
Model tree for Blackfrost-AI/GLM-5.2-ABLITERATED-NVFP4
Base model
zai-org/GLM-5.2