Instructions to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
model = AutoModelForCausalLM.from_pretrained("drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated

SGLang

How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated with Docker Model Runner:
```
docker model run hf.co/drowzeys/GLM-5.2-Int4-Int8Mix-Abliterated
```

GLM-5.2-Int4-Int8Mix-Abliterated / README.md

drowzeys

Bold warnings: download ≠ abliteration; GitHub-only install

c9a1f42 verified about 16 hours ago

8.84 kB

license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
  - abliterated
  - glm
  - moe
  - vLLM
  - compressed-tensors
  - INT4
  - INT8
  - W4A16
  - W8A16
  - QuantTrio
base_model:
  - QuantTrio/GLM-5.2-Int4-Int8Mix
  - zai-org/GLM-5.2
base_model_relation: finetune
extra_gated_prompt: >
  ## Responsible Use Agreement


  This model has had safety refusals removed. That makes it useful for
  red-teaming, security research, evaluation, and unfiltered assistant tasks —
  and also removes guardrails a user must therefore supply themselves.


  **Prohibited uses (you must agree before access is granted):**


  - Anything involving the sexual exploitation or endangerment of minors.

  - You must be of age 18 years or older to use and download this model.

  - You agree any information generated that can cause harm in terms of
  generating recipe, knowledge to make any materials/substances is your own
  input and responsibility. You will be accountable for any harm/damage caused
  by your action/input.

  - Content promoting self-harm or suicide.

  - Generation of material that is illegal in your jurisdiction, or that targets
  real individuals for harassment, doxxing, or fraud.

  - Any use prohibited by the upstream GLM license.


  You are responsible for adding appropriate safety filtering, human review, and
  access controls for your deployment. The weights are provided as-is, with no
  warranty. The license is inherited from the upstream GLM base model — review
  and comply with it before use or redistribution.


  **Installation is NOT “just download this repo.”**  

  **Downloading these files alone is NOT abliteration.**  

  **Follow the GitHub one-shot only:**  

  https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
extra_gated_fields:
  Username: text
  Email: text
  Reason for intended use: text
  I am 18 years of age or older: checkbox
  I will not use this model for any sexual exploitation or endangerment of minors: checkbox
  I accept full responsibility for my inputs and any harm from generated content: checkbox
  I will not use this model for self-harm, suicide promotion, illegal activity, harassment, doxxing, or fraud: checkbox
  I agree to comply with the upstream GLM base-model license: checkbox
  I agree to the Responsible Use terms above: checkbox
  I understand that downloading these files alone is NOT abliteration and I will follow the GitHub one-shot only: checkbox

GLM-5.2-Int4-Int8Mix-Abliterated

⚠️ THIS HUGGING FACE PAGE IS WEIGHTS + GATE ONLY

All installation, serve, verify, and agent instructions live on GitHub only:
https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

Downloading HF weights ≠ abliteration.

A download alone does NOT apply SRA or edit the 39 safety tensors.

🚨 DOWNLOAD ≠ ABLITERATION

Clicking “Download” on Hugging Face does NOT mean you have a working abliterated deployment.

Bold facts:

HF download alone ≠ abliteration.
QuantTrio stock ≠ abliteration.
A launch flag / LoRA / system prompt ≠ this recipe.
Standing ablit requires SRA residual write on 39 self_attn.o_proj tensors (L65–77) with the correct direction + 124-shard tree map.
Skip GitHub oneshot / verify / hub layout → stock or partial refusals.

Myth	Reality
“I downloaded the HF tree → model is abliterated”	FALSE if you skip verify, hub layout, or serve the wrong path
“Any GLM-5.2 Int4/Int8 mix is ablit”	FALSE — QuantTrio stock is not ablit
“Ablit is a launch flag / LoRA / system prompt”	FALSE — standing ablit is a weight-space residual projection
“I only need the 13 dirty files”	Incomplete trees / wrong hub name → stock or partial bypass

What abliteration actually is (this recipe)

Standing ablit is NOT “download and go.” It is a deliberate edit:

SRA refusal direction (prefill capture → rank-1 SRA, r=8)
Residual write of that direction onto self_attn.o_proj only
Layers 65–77 inclusive → 13 layers × 3 quant tensors = 39 weight tensors
Those 39 tensors are written into 13 dirty shards of a 124-shard Int4/Int8 Mix tree
The other 111 model shards match QuantTrio stock by design
model.safetensors.index.json maps each tensor → shard (the tree map you must keep intact)

Item	Standing value
Modules	`self_attn.o_proj` only
Layers	65–77
Tensors edited	39 (w8a16 pack path)
Dirty shards	13 of 124 (see GitHub `recipe/DIRTY_SHARDS.json`)
λ	3.0
Direction (Path B)	GitHub `recipe/direction/refusal_direction_sra_prefill.pt`
Early L0–64 / MTP / `eh_proj`	stock (untouched)

If those 39 tensors are still stock-sized, hardlinked to QuantTrio, missing from the index map, or only partly overlaid — you get full or partial refusals, even if you “downloaded something from HF.”

Supported ways to get a real ablit tree

Path	Meaning
A (required for published 32/32)	GitHub `bash scripts/oneshot.sh` — downloads this gated tree, verifies all 13 dirty shards + 124-shard map + hub symlink, installs kernels/image, optional launch
B (rebuild)	From QuantTrio stock, run GitHub `project_residual.py` with the published SRA direction onto o_proj L65–77, then verify against `DIRTY_SHARDS.json`

There is NO third path of “download HF files into a random folder and call it ablit.”

Installation — GitHub only

Do NOT follow install recipes on this HF page.

Use this repo only:

https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

git clone https://github.com/drowzeys/keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks
cd keys-GLM5.2-Quantrio-INT4-INT8-Mixed-Abliterated-C1-30toks-4x-DGX-Sparks

# kernels + public image + this HF tree + hub symlink + verify PASS
bash scripts/oneshot.sh

# multi-node (every TP rank):
WORKERS="IP1 IP2 IP3" bash scripts/oneshot.sh --fanout

# SPEED=1 serve (C1≈30 on 4× GB10):
bash scripts/oneshot.sh --launch

Must	Must not
`scripts/oneshot.sh`	Improvise `hf download` + random paths
`verify_ablit_weights.py` → PASS on every rank	Skip verify
Hub `glm52-int4-int8mix` → abliterated tree	Serve QuantTrio stock or a half-overlaid tree
Refusal probes with `enable_thinking: false`	Claim “not ablit” with thinking still on

Agent checklist: AGENTS.md
Partial bypass triage: docs/PARTIAL_BYPASS.md
SRA / tensor map detail: docs/RECIPE.md

What this HF repo contains

Full 124-shard Int4–Int8Mix checkpoint (tokenizer + configs), after standing residual ablit
ABLIT_META.json — edit metadata
README.base-quant.md — stock quant packaging notes (QuantTrio lineage)

Not on HF (on GitHub only): serve launcher, kernels, image pull, oneshot, direction tensor packaging, verify suite, Hermes ops.

Public serve image (via GitHub scripts/pull_image.sh):

ghcr.io/drowzeys/vllm-node-tf5-glm52-b12x:speed1-c1-30-128k

Credit — stock quantization


Stock mix (base of this tree)	QuantTrio/GLM-5.2-Int4-Int8Mix
Publisher	QuantTrio
Foundation model	zai-org/GLM-5.2

Please credit QuantTrio for the W4A16/W8A16 mix. This page is a derivative with late-layer residual abliteration on top of that pack.

Standing metrics (reference only)

Measured on 4× DGX Spark GB10, GitHub SPEED=1 recipe, thinking off:

Metric	Value
Hard-refuse bypass	32/32
C1 count/list	~30 tok/s

Reproduce only via the GitHub one-shot + suite — not by “download and chat” on an unverified tree.

Responsible use

See gated terms above and GitHub docs/RESPONSIBLE_USE.md. You must supply your own deployment safety controls.