Instructions to use helixdouble/GLM-5.1-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use helixdouble/GLM-5.1-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="helixdouble/GLM-5.1-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("helixdouble/GLM-5.1-Abliterated")
model = AutoModelForCausalLM.from_pretrained("helixdouble/GLM-5.1-Abliterated", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use helixdouble/GLM-5.1-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "helixdouble/GLM-5.1-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "helixdouble/GLM-5.1-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/helixdouble/GLM-5.1-Abliterated

SGLang

How to use helixdouble/GLM-5.1-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "helixdouble/GLM-5.1-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "helixdouble/GLM-5.1-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "helixdouble/GLM-5.1-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "helixdouble/GLM-5.1-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use helixdouble/GLM-5.1-Abliterated with Docker Model Runner:
```
docker model run hf.co/helixdouble/GLM-5.1-Abliterated
```

Waiting for benchmarks

by whw2026 - opened May 12

Discussion

whw2026

May 12

•

edited May 12

I'm really wondering how abliteration, together with a healing LoRA, will affect the ability of the SOTA 700B model. It's a pity that I'm unable to deploy such a huge model.

whw2026 changed discussion title from Waiting for your benchmarks to Waiting for benchmarks May 12

helixdouble

Owner May 12

Honest answer: I don't have the full benchmarks yet, but I want to be transparent about where the model currently stands and what I'm working on.
The published v1 used rank-4 LoRA healing, which turned out to be undersized for a 754B model. The CE loss plateaued around 8.44 (down from 9.95) during the 250-step run, which suggests the LoRA capacity saturated before fully recovering capability - not a training problem, just architectural mismatch. Rank 16-32 is likely the right floor for proper healing on a model this size, similar to mlabonne's approach scaled appropriately.
What I can say from the abliteration metrics themselves: Selected trial 132 had KL divergence of 0.348 against base GLM-5.1 on the harmless calibration set. For comparison, my earlier GLM-4.7-Flash abliteration came in around 0.4 KL with comparable trial counts, so the per-parameter damage is actually in a reasonable range 0/30 refusals on held-out harmful prompts confirms the abliteration itself worked

What this doesn't tell us is how the model performs on reasoning, coding, math, or long-context tasks that the calibration set doesn't exercise. That's the gap I want filled. I'm planning a v2 with rank-32 LoRA healing on the same selected trial, which should recover most of the residual capability the rank-4 run left on the table. I'd estimate (cautiously) somewhere in the 1-5% MMLU drop range for v1 and potentially under 2% for v2, but real benchmarks need to confirm that. If anyone has H100/H200/B200 capacity and is willing to run MMLU, GSM8K, or HumanEval against this release, the data would be genuinely valuable for the community and I'd credit the contribution prominently. Happy to provide guidance on the deployment specifics.

spicyneuron

May 15

Not quite apples-to-apples, but I ran this through my MLX quantize workflow which includes lightweight benchmarks. Used the same config as base:

metric	base 5.1	this model	huihui patched
bpw	3.901	3.901	3.895
base memory	337.867	337.867	341.770
peak memory (1024/512)	364.877	364.877	364.299
prompt tok/s (1024)	191.149 ± 0.642	191.811 ± 0.116	192.922 ± 0.107
gen tok/s (512)	17.808 ± 0.084	17.894 ± 0.054	18.191 ± 0.062
kl mean	0.099 ± 0.004	0.135 ± 0.005	0.221 ± 0.007
kl p95	0.204 ± 0.004	0.289 ± 0.005	0.468 ± 0.007
perplexity	4.109 ± 0.024	4.274 ± 0.026	4.195 ± 0.024
hellaswag	0.548 ± 0.011	0.528 ± 0.011	0.556 ± 0.011

KL and perplexity rise as expected, but Hellaswag (2000 samples, same seed) seems to suffer a bit compared to huihui. I've noticed similar results testing @wuwangzhang1216's work too — great KL but measurable benchmark loss. Would slow things down, but maybe worth adding a lm-eval screen to the pipeline?

helixdouble

Owner May 16

This is really useful, thank you - exactly the data I needed.
One important correction first: I found a bug in my validation harness. The refusal-detection regex was tuned for Claude-style refusal phrasing, not GLM-5.1's, so it was under-counting. The real number is ~15/30 refusals on the held-out harmful set, not 0/30. I'll be correcting the model card. So this model is only partially abliterated, not fully - which actually matters for reading your benchmark results.
That reshapes the Hellaswag result. 0.528 vs 0.548 base is happening even at partial abliteration - a more complete abliteration would likely cost more reasoning, not less. So the real issue isn't just under-healed (rank-4 LoRA saturating at CE 8.44, which is still true) — it's that the abliteration-vs-capability tradeoff on this model is steeper than KL alone suggested. Good KL, partial refusal removal, and measurable reasoning loss is a worse starting point than I thought.
The lm-eval suggestion is the right call and now clearly necessary - KL isn't catching either problem. Adding a per-trial Hellaswag/ARC screen so Optuna optimizes against reasoning loss directly, plus fixing the refusal regex for GLM phrasing so trial selection is actually measuring what I think it is. Both go in before v2.
v2 plan is firming up: fixed refusal detection, rank-32 healing, reasoning eval in the Optuna objective, and probably revisiting trial selection now that I can't trust the original refusal scores. Bigger rebuild than I'd hoped but the right one.
Would genuinely value rerunning your MLX workflow against v2 when it's out - a consistent harness across versions would make this clean. Thanks again for taking the time, this caught two real problems.

spicyneuron

May 17

Sure, happy to help.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment