Instructions to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
model = AutoModelForCausalLM.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated

SGLang

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Docker Model Runner:
```
docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated
```

GLM-5.2-Int4-Int8Mix-Abliterated / README.md

drowzeys

Bold warnings: download ≠ abliteration; GitHub-only install

c9a1f42 verified about 21 hours ago

preview code

Raw

History Blame

8.84 kB

	---
	license: mit
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- abliterated
	- glm
	- moe
	- vLLM
	- compressed-tensors
	- INT4
	- INT8
	- W4A16
	- W8A16
	- QuantTrio
	base_model:
	- QuantTrio/GLM-5.2-Int4-Int8Mix
	- zai-org/GLM-5.2
	base_model_relation: finetune
	extra_gated_prompt: \|
	## Responsible Use Agreement

	This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.

	Prohibited uses (you must agree before access is granted):

	- Anything involving the sexual exploitation or endangerment of minors.
	- You must be of age 18 years or older to use and download this model.
	- You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
	- Content promoting self-harm or suicide.
	- Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
	- Any use prohibited by the upstream GLM license.

	You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream GLM base model — review and comply with it before use or redistribution.

	Installation is NOT “just download this repo.”
	Downloading these files alone is NOT abliteration.
	Follow the GitHub one-shot only:
	https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	extra_gated_fields:
	Username: text
	Email: text
	Reason for intended use: text
	I am 18 years of age or older: checkbox
	I will not use this model for any sexual exploitation or endangerment of minors: checkbox
	I accept full responsibility for my inputs and any harm from generated content: checkbox
	I will not use this model for self-harm, suicide promotion, illegal activity, harassment, doxxing, or fraud: checkbox
	I agree to comply with the upstream GLM base-model license: checkbox
	I agree to the Responsible Use terms above: checkbox
	I understand that downloading these files alone is NOT abliteration and I will follow the GitHub one-shot only: checkbox
	---

	# GLM-5.2-Int4-Int8Mix-Abliterated

	> ## ⚠️ THIS HUGGING FACE PAGE IS WEIGHTS + GATE ONLY
	>
	> All installation, serve, verify, and agent instructions live on GitHub only:
	> https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	>
	> ### Downloading HF weights ≠ abliteration.
	> ### A download alone does NOT apply SRA or edit the 39 safety tensors.

	---

	## 🚨 DOWNLOAD ≠ ABLITERATION

	### Clicking “Download” on Hugging Face does NOT mean you have a working abliterated deployment.

	Bold facts:

	- HF download alone ≠ abliteration.
	- QuantTrio stock ≠ abliteration.
	- A launch flag / LoRA / system prompt ≠ this recipe.
	- Standing ablit requires SRA residual write on 39 `self_attn.o_proj` tensors (L65–77) with the correct direction + 124-shard tree map.
	- Skip GitHub oneshot / verify / hub layout → stock or partial refusals.

	\| Myth \| Reality \|
	\|---\|---\|
	\| “I downloaded the HF tree → model is abliterated” \| FALSE if you skip verify, hub layout, or serve the wrong path \|
	\| “Any GLM-5.2 Int4/Int8 mix is ablit” \| FALSE — [QuantTrio stock](https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix) is not ablit \|
	\| “Ablit is a launch flag / LoRA / system prompt” \| FALSE — standing ablit is a weight-space residual projection \|
	\| “I only need the 13 dirty files” \| Incomplete trees / wrong hub name → stock or partial bypass \|

	### What abliteration actually is (this recipe)

	Standing ablit is NOT “download and go.” It is a deliberate edit:

	1. SRA refusal direction (prefill capture → rank-1 SRA, r=8)
	2. Residual write of that direction onto `self_attn.o_proj` only
	3. Layers 65–77 inclusive → 13 layers × 3 quant tensors = 39 weight tensors
	4. Those 39 tensors are written into 13 dirty shards of a 124-shard Int4/Int8 Mix tree
	5. The other 111 model shards match QuantTrio stock by design
	6. `model.safetensors.index.json` maps each tensor → shard (the tree map you must keep intact)

	\| Item \| Standing value \|
	\|---\|---\|
	\| Modules \| `self_attn.o_proj` only \|
	\| Layers \| 65–77 \|
	\| Tensors edited \| 39 (w8a16 pack path) \|
	\| Dirty shards \| 13 of 124 (see GitHub `recipe/DIRTY_SHARDS.json`) \|
	\| λ \| 3.0 \|
	\| Direction (Path B) \| GitHub `recipe/direction/refusal_direction_sra_prefill.pt` \|
	\| Early L0–64 / MTP / `eh_proj` \| stock (untouched) \|

	If those 39 tensors are still stock-sized, hardlinked to QuantTrio, missing from the index map, or only partly overlaid — you get full or partial refusals, even if you “downloaded something from HF.”

	### Supported ways to get a real ablit tree

	\| Path \| Meaning \|
	\|---\|---\|
	\| A (required for published 32/32) \| GitHub `bash scripts/oneshot.sh` — downloads this gated tree, verifies all 13 dirty shards + 124-shard map + hub symlink, installs kernels/image, optional launch \|
	\| B (rebuild) \| From QuantTrio stock, run GitHub `project_residual.py` with the published SRA direction onto o_proj L65–77, then verify against `DIRTY_SHARDS.json` \|

	There is NO third path of “download HF files into a random folder and call it ablit.”

	---

	## Installation — GitHub only

	### Do NOT follow install recipes on this HF page.

	### Use this repo only:

	# https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

	```bash
	git clone https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	cd keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

	# kernels + public image + this HF tree + hub symlink + verify PASS
	bash scripts/oneshot.sh

	# multi-node (every TP rank):
	WORKERS="IP1 IP2 IP3" bash scripts/oneshot.sh --fanout

	# SPEED=1 serve (C1≈30 on 4× GB10):
	bash scripts/oneshot.sh --launch
	```

	\| Must \| Must not \|
	\|---\|---\|
	\| `scripts/oneshot.sh` \| Improvise `hf download` + random paths \|
	\| `verify_ablit_weights.py` → PASS on every rank \| Skip verify \|
	\| Hub `glm52-int4-int8mix` → abliterated tree \| Serve QuantTrio stock or a half-overlaid tree \|
	\| Refusal probes with `enable_thinking: false` \| Claim “not ablit” with thinking still on \|

	Agent checklist: [AGENTS.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/AGENTS.md)
	Partial bypass triage: [docs/PARTIAL_BYPASS.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/docs/PARTIAL_BYPASS.md)
	SRA / tensor map detail: [docs/RECIPE.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/docs/RECIPE.md)

	---

	## What this HF repo contains

	- Full 124-shard Int4–Int8Mix checkpoint (tokenizer + configs), after standing residual ablit
	- `ABLIT_META.json` — edit metadata
	- `README.base-quant.md` — stock quant packaging notes (QuantTrio lineage)

	Not on HF (on GitHub only): serve launcher, kernels, image pull, oneshot, direction tensor packaging, verify suite, Hermes ops.

	Public serve image (via GitHub `scripts/pull_image.sh`):

	`ghcr.io/drowzeys/vllm-node-tf5-glm52-b12x:speed1-c1-30-128k`

	---

	## Credit — stock quantization

	\| \| \|
	\|---\|---\|
	\| Stock mix (base of this tree) \| [QuantTrio/GLM-5.2-Int4-Int8Mix](https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix) \|
	\| Publisher \| [QuantTrio](https://huggingface.co/QuantTrio) \|
	\| Foundation model \| [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) \|

	Please credit QuantTrio for the W4A16/W8A16 mix. This page is a derivative with late-layer residual abliteration on top of that pack.

	---

	## Standing metrics (reference only)

	Measured on 4× DGX Spark GB10, GitHub SPEED=1 recipe, thinking off:

	\| Metric \| Value \|
	\|---\|---\|
	\| Hard-refuse bypass \| 32/32 \|
	\| C1 count/list \| ~30 tok/s \|

	Reproduce only via the GitHub one-shot + suite — not by “download and chat” on an unverified tree.

	---

	## Responsible use

	See gated terms above and GitHub `docs/RESPONSIBLE_USE.md`. You must supply your own deployment safety controls.

	---
	license: mit
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- abliterated
	- glm
	- moe
	- vLLM
	- compressed-tensors
	- INT4
	- INT8
	- W4A16
	- W8A16
	- QuantTrio
	base_model:
	- QuantTrio/GLM-5.2-Int4-Int8Mix
	- zai-org/GLM-5.2
	base_model_relation: finetune
	extra_gated_prompt: \|
	## Responsible Use Agreement

	This model has had safety refusals removed. That makes it useful for red-teaming, security research, evaluation, and unfiltered assistant tasks — and also removes guardrails a user must therefore supply themselves.

	Prohibited uses (you must agree before access is granted):

	- Anything involving the sexual exploitation or endangerment of minors.
	- You must be of age 18 years or older to use and download this model.
	- You agree any information generated that can cause harm in terms of generating recipe, knowledge to make any materials/substances is your own input and responsibility. You will be accountable for any harm/damage caused by your action/input.
	- Content promoting self-harm or suicide.
	- Generation of material that is illegal in your jurisdiction, or that targets real individuals for harassment, doxxing, or fraud.
	- Any use prohibited by the upstream GLM license.

	You are responsible for adding appropriate safety filtering, human review, and access controls for your deployment. The weights are provided as-is, with no warranty. The license is inherited from the upstream GLM base model — review and comply with it before use or redistribution.

	Installation is NOT “just download this repo.”
	Downloading these files alone is NOT abliteration.
	Follow the GitHub one-shot only:
	https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	extra_gated_fields:
	Username: text
	Email: text
	Reason for intended use: text
	I am 18 years of age or older: checkbox
	I will not use this model for any sexual exploitation or endangerment of minors: checkbox
	I accept full responsibility for my inputs and any harm from generated content: checkbox
	I will not use this model for self-harm, suicide promotion, illegal activity, harassment, doxxing, or fraud: checkbox
	I agree to comply with the upstream GLM base-model license: checkbox
	I agree to the Responsible Use terms above: checkbox
	I understand that downloading these files alone is NOT abliteration and I will follow the GitHub one-shot only: checkbox
	---

	# GLM-5.2-Int4-Int8Mix-Abliterated

	> ## ⚠️ THIS HUGGING FACE PAGE IS WEIGHTS + GATE ONLY
	>
	> All installation, serve, verify, and agent instructions live on GitHub only:
	> https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	>
	> ### Downloading HF weights ≠ abliteration.
	> ### A download alone does NOT apply SRA or edit the 39 safety tensors.

	---

	## 🚨 DOWNLOAD ≠ ABLITERATION

	### Clicking “Download” on Hugging Face does NOT mean you have a working abliterated deployment.

	Bold facts:

	- HF download alone ≠ abliteration.
	- QuantTrio stock ≠ abliteration.
	- A launch flag / LoRA / system prompt ≠ this recipe.
	- Standing ablit requires SRA residual write on 39 `self_attn.o_proj` tensors (L65–77) with the correct direction + 124-shard tree map.
	- Skip GitHub oneshot / verify / hub layout → stock or partial refusals.

	\| Myth \| Reality \|
	\|---\|---\|
	\| “I downloaded the HF tree → model is abliterated” \| FALSE if you skip verify, hub layout, or serve the wrong path \|
	\| “Any GLM-5.2 Int4/Int8 mix is ablit” \| FALSE — [QuantTrio stock](https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix) is not ablit \|
	\| “Ablit is a launch flag / LoRA / system prompt” \| FALSE — standing ablit is a weight-space residual projection \|
	\| “I only need the 13 dirty files” \| Incomplete trees / wrong hub name → stock or partial bypass \|

	### What abliteration actually is (this recipe)

	Standing ablit is NOT “download and go.” It is a deliberate edit:

	1. SRA refusal direction (prefill capture → rank-1 SRA, r=8)
	2. Residual write of that direction onto `self_attn.o_proj` only
	3. Layers 65–77 inclusive → 13 layers × 3 quant tensors = 39 weight tensors
	4. Those 39 tensors are written into 13 dirty shards of a 124-shard Int4/Int8 Mix tree
	5. The other 111 model shards match QuantTrio stock by design
	6. `model.safetensors.index.json` maps each tensor → shard (the tree map you must keep intact)

	\| Item \| Standing value \|
	\|---\|---\|
	\| Modules \| `self_attn.o_proj` only \|
	\| Layers \| 65–77 \|
	\| Tensors edited \| 39 (w8a16 pack path) \|
	\| Dirty shards \| 13 of 124 (see GitHub `recipe/DIRTY_SHARDS.json`) \|
	\| λ \| 3.0 \|
	\| Direction (Path B) \| GitHub `recipe/direction/refusal_direction_sra_prefill.pt` \|
	\| Early L0–64 / MTP / `eh_proj` \| stock (untouched) \|

	If those 39 tensors are still stock-sized, hardlinked to QuantTrio, missing from the index map, or only partly overlaid — you get full or partial refusals, even if you “downloaded something from HF.”

	### Supported ways to get a real ablit tree

	\| Path \| Meaning \|
	\|---\|---\|
	\| A (required for published 32/32) \| GitHub `bash scripts/oneshot.sh` — downloads this gated tree, verifies all 13 dirty shards + 124-shard map + hub symlink, installs kernels/image, optional launch \|
	\| B (rebuild) \| From QuantTrio stock, run GitHub `project_residual.py` with the published SRA direction onto o_proj L65–77, then verify against `DIRTY_SHARDS.json` \|

	There is NO third path of “download HF files into a random folder and call it ablit.”

	---

	## Installation — GitHub only

	### Do NOT follow install recipes on this HF page.

	### Use this repo only:

	# https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

	```bash
	git clone https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
	cd keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

	# kernels + public image + this HF tree + hub symlink + verify PASS
	bash scripts/oneshot.sh

	# multi-node (every TP rank):
	WORKERS="IP1 IP2 IP3" bash scripts/oneshot.sh --fanout

	# SPEED=1 serve (C1≈30 on 4× GB10):
	bash scripts/oneshot.sh --launch
	```

	\| Must \| Must not \|
	\|---\|---\|
	\| `scripts/oneshot.sh` \| Improvise `hf download` + random paths \|
	\| `verify_ablit_weights.py` → PASS on every rank \| Skip verify \|
	\| Hub `glm52-int4-int8mix` → abliterated tree \| Serve QuantTrio stock or a half-overlaid tree \|
	\| Refusal probes with `enable_thinking: false` \| Claim “not ablit” with thinking still on \|

	Agent checklist: [AGENTS.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/AGENTS.md)
	Partial bypass triage: [docs/PARTIAL_BYPASS.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/docs/PARTIAL_BYPASS.md)
	SRA / tensor map detail: [docs/RECIPE.md](https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks/blob/main/docs/RECIPE.md)

	---

	## What this HF repo contains

	- Full 124-shard Int4–Int8Mix checkpoint (tokenizer + configs), after standing residual ablit
	- `ABLIT_META.json` — edit metadata
	- `README.base-quant.md` — stock quant packaging notes (QuantTrio lineage)

	Not on HF (on GitHub only): serve launcher, kernels, image pull, oneshot, direction tensor packaging, verify suite, Hermes ops.

	Public serve image (via GitHub `scripts/pull_image.sh`):

	`ghcr.io/drowzeys/vllm-node-tf5-glm52-b12x:speed1-c1-30-128k`

	---

	## Credit — stock quantization

	\| \| \|
	\|---\|---\|
	\| Stock mix (base of this tree) \| [QuantTrio/GLM-5.2-Int4-Int8Mix](https://huggingface.co/QuantTrio/GLM-5.2-Int4-Int8Mix) \|
	\| Publisher \| [QuantTrio](https://huggingface.co/QuantTrio) \|
	\| Foundation model \| [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2) \|

	Please credit QuantTrio for the W4A16/W8A16 mix. This page is a derivative with late-layer residual abliteration on top of that pack.

	---

	## Standing metrics (reference only)

	Measured on 4× DGX Spark GB10, GitHub SPEED=1 recipe, thinking off:

	\| Metric \| Value \|
	\|---\|---\|
	\| Hard-refuse bypass \| 32/32 \|
	\| C1 count/list \| ~30 tok/s \|

	Reproduce only via the GitHub one-shot + suite — not by “download and chat” on an unverified tree.

	---

	## Responsible use

	See gated terms above and GitHub `docs/RESPONSIBLE_USE.md`. You must supply your own deployment safety controls.