Instructions to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
model = AutoModelForCausalLM.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated

SGLang

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Docker Model Runner:
```
docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Responsible Use Agreement

This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.

Prohibited uses (you must agree before access is granted):

Anything involving the sexual exploitation or endangerment of minors.
You must be of age 18 years or older to use and download this model.
You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
Content promoting self-harm or suicide.
Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
Any use prohibited by the upstream GLM license.

You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream GLM base model — review and comply with it before use or redistribution.

Installation is NOT “just download this repo.”
Downloading these files alone is NOT abliteration.
Follow the GitHub one-shot only:
https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

GLM-5.2-Int4-Int8Mix-Abliterated

⚠️ THIS HUGGING FACE PAGE IS WEIGHTS + GATE ONLY

All installation, serve, verify, and agent instructions live on GitHub only:
https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

Downloading HF weights ≠ abliteration.

A download alone does NOT apply SRA or edit the 39 safety tensors.

🚨 DOWNLOAD ≠ ABLITERATION

Clicking “Download” on Hugging Face does NOT mean you have a working abliterated deployment.

Bold facts:

HF download alone ≠ abliteration.
QuantTrio stock ≠ abliteration.
A launch flag / LoRA / system prompt ≠ this recipe.
Standing ablit requires SRA residual write on 39 self_attn.o_proj tensors (L65–77) with the correct direction + 124-shard tree map.
Skip GitHub oneshot / verify / hub layout → stock or partial refusals.

Myth	Reality
“I downloaded the HF tree → model is abliterated”	FALSE if you skip verify, hub layout, or serve the wrong path
“Any GLM-5.2 Int4/Int8 mix is ablit”	FALSE — QuantTrio stock is not ablit
“Ablit is a launch flag / LoRA / system prompt”	FALSE — standing ablit is a weight-space residual projection
“I only need the 13 dirty files”	Incomplete trees / wrong hub name → stock or partial bypass

What abliteration actually is (this recipe)

Standing ablit is NOT “download and go.” It is a deliberate edit:

SRA refusal direction (prefill capture → rank-1 SRA, r=8)
Residual write of that direction onto self_attn.o_proj only
Layers 65–77 inclusive → 13 layers × 3 quant tensors = 39 weight tensors
Those 39 tensors are written into 13 dirty shards of a 124-shard Int4/Int8 Mix tree
The other 111 model shards match QuantTrio stock by design
model.safetensors.index.json maps each tensor → shard (the tree map you must keep intact)

Item	Standing value
Modules	`self_attn.o_proj` only
Layers	65–77
Tensors edited	39 (w8a16 pack path)
Dirty shards	13 of 124 (see GitHub `recipe/DIRTY_SHARDS.json`)
λ	3.0
Direction (Path B)	GitHub `recipe/direction/refusal_direction_sra_prefill.pt`
Early L0–64 / MTP / `eh_proj`	stock (untouched)

If those 39 tensors are still stock-sized, hardlinked to QuantTrio, missing from the index map, or only partly overlaid — you get full or partial refusals, even if you “downloaded something from HF.”

Supported ways to get a real ablit tree

Path	Meaning
A (required for published 32/32)	GitHub `bash scripts/oneshot.sh` — downloads this gated tree, verifies all 13 dirty shards + 124-shard map + hub symlink, installs kernels/image, optional launch
B (rebuild)	From QuantTrio stock, run GitHub `project_residual.py` with the published SRA direction onto o_proj L65–77, then verify against `DIRTY_SHARDS.json`

There is NO third path of “download HF files into a random folder and call it ablit.”

Installation — GitHub only

Do NOT follow install recipes on this HF page.

Use this repo only:

https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

git clone https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
cd keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

# kernels + public image + this HF tree + hub symlink + verify PASS
bash scripts/oneshot.sh

# multi-node (every TP rank):
WORKERS="IP1 IP2 IP3" bash scripts/oneshot.sh --fanout

# SPEED=1 serve (C1≈30 on 4× GB10):
bash scripts/oneshot.sh --launch

Must	Must not
`scripts/oneshot.sh`	Improvise `hf download` + random paths
`verify_ablit_weights.py` → PASS on every rank	Skip verify
Hub `glm52-int4-int8mix` → abliterated tree	Serve QuantTrio stock or a half-overlaid tree
Refusal probes with `enable_thinking: false`	Claim “not ablit” with thinking still on

Agent checklist: AGENTS.md
Partial bypass triage: docs/PARTIAL_BYPASS.md
SRA / tensor map detail: docs/RECIPE.md

What this HF repo contains

Full 124-shard Int4–Int8Mix checkpoint (tokenizer + configs), after standing residual ablit
ABLIT_META.json — edit metadata
README.base-quant.md — stock quant packaging notes (QuantTrio lineage)

Not on HF (on GitHub only): serve launcher, kernels, image pull, oneshot, direction tensor packaging, verify suite, Hermes ops.

Public serve image (via GitHub scripts/pull_image.sh):

ghcr.io/drowzeys/vllm-node-tf5-glm52-b12x:speed1-c1-30-128k

Credit — stock quantization


Stock mix (base of this tree)	QuantTrio/GLM-5.2-Int4-Int8Mix
Publisher	QuantTrio
Foundation model	zai-org/GLM-5.2

Please credit QuantTrio for the W4A16/W8A16 mix. This page is a derivative with late-layer residual abliteration on top of that pack.

Standing metrics (reference only)

Measured on 4× DGX Spark GB10, GitHub SPEED=1 recipe, thinking off:

Metric	Value
Hard-refuse bypass	32/32
C1 count/list	~30 tok/s

Reproduce only via the GitHub one-shot + suite — not by “download and chat” on an unverified tree.

Responsible use

See gated terms above and GitHub docs/RESPONSIBLE_USE.md. You must supply your own deployment safety controls.

Downloads last month: 160

Safetensors

Model size

785B params

Tensor type

BF16

I64

I32

F32

Model tree for drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated

Base model

zai-org/GLM-5.2

Quantized

QuantTrio/GLM-5.2-Int4-Int8Mix

Finetuned

(1)

this model