Instructions to use pipenetwork/GLM-5.3-Flash-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("pipenetwork/GLM-5.3-Flash-MLX-8bit")
config = load_config("pipenetwork/GLM-5.3-Flash-MLX-8bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pipenetwork/GLM-5.3-Flash-MLX-8bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pipenetwork/GLM-5.3-Flash-MLX-8bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "pipenetwork/GLM-5.3-Flash-MLX-8bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash-MLX-8bit / README.md

pipenetwork

Upload folder using huggingface_hub

1db76a2 verified about 22 hours ago

preview code

Raw

History Blame Contribute Delete

4.5 kB

	---
	license: mit
	base_model: zai-org/GLM-5.3-Flash
	base_model_relation: quantized
	tags:
	- mlx
	- apple-silicon
	- glm5_next
	- mixture-of-experts
	- 8-bit
	pipeline_tag: image-text-to-text
	library_name: mlx
	---

	# GLM-5.3-Flash-MLX-8bit

	MLX (Apple Silicon) build of [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) — 320B-A18B
	hybrid of 34 Kimi-Delta linear-attention layers and 11 DeepSeek-sparse-attention (NoPE MLA +
	lightning indexer) layers with manifold-constrained hyper-connections — quantized to 8-bit.

	These files are modified: converted from the upstream bfloat16 release
	([GLM-5.3-Flash-BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16)) to MLX and quantized;
	the architecture is unchanged. The multi-token-prediction layer (layer 45) is not included. The
	vision tower is carried in bfloat16.

	## Runtime

	`glm5_next` landed in [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) `main` on 2026-08-26 (no
	release carries it yet). Validating that port against `transformers` 5.16 at tiny scale found two
	numerical bugs and two epsilon mismatches, which the runtime in
	[https://github.com/PipeNetwork/glm53-flash-mlx](https://github.com/PipeNetwork/glm53-flash-mlx) fixes; parity is 1e-6 end to end, exact on cached decode.

	\| what \| reference \| mlx-vlm `main` \| effect \|
	\|---\|---\|---\|---\|
	\| `swiglu_limit` \| gate clamped at 10, up at ±10, in every text MLP \| no clamp anywhere in the text stack \| formula mismatch on all 45 FFN blocks \|
	\| mHC `base`/`scale` dtype \| float32 \| converter casts to bf16; the Metal kernel then reads `base` as float4 \| `comb` mixing matrix off by ~0.5 on every layer of a converted checkpoint \|
	\| MLA low-rank norm eps \| `rms_norm_eps` = 1e-5 \| 1e-6 \| small \|
	\| indexer LayerNorm eps \| 1e-6 \| 1e-5 \| small \|

	This checkpoint keeps the mHC arrays and KDA decay parameters in float32 as stored, so it is
	safe in either runtime; the clamp is a compute-path fix and needs the patched runtime:

	```bash
	git clone https://github.com/PipeNetwork/glm53-flash-mlx && cd glm53-flash-mlx && pip install -r requirements.txt
	python scripts/smoke_generate.py /path/to/GLM-5.3-Flash-MLX-8bit
	```
	```python
	from glm53_flash_mlx.load import load
	model, processor = load("/path/to/GLM-5.3-Flash-MLX-8bit")
	```

	## Size and what is quantized

	334.1 GB on disk (bfloat16 upstream: 642.7 GB).

	\| group \| share of parameters \| this build \|
	\|---\|---:\|---\|
	\| routed experts (`switch_mlp`, 42 layers × 288) \| 304B (97%) \| 8-bit, group 64 \|
	\| KDA and MLA projections, shared experts, dense MLPs, embeddings, `lm_head` \| ~9B \| 8-bit, group 64 \|
	\| lightning-indexer projections \| 0.06B \| 8-bit, group 64 \|
	\| MoE router + correction bias, mHC arrays (fp32), KDA `A_log`/`dt_bias` (fp32), convolutions, norms \| — \| as stored \|
	\| vision tower \| 0.56B \| bfloat16 \|

	## Quality

	Perplexity on wikitext-2 (test), 288,627 tokens in 141 windows of 2048, every build scored
	on identical windows through this runtime. The 643 GB bfloat16 model does not fit a 512 GB
	machine, so the 8-bit build is the anchor (on every model we have measured, 8-bit has been
	statistically indistinguishable from bfloat16). Per-window NLL differences against 8-bit,
	bootstrapped over one shared index set (20,000 resamples):

	\| build \| size \| perplexity \| ΔNLL/token vs 8-bit [95% CI] \| windows worse \|
	\|---\|---:\|---:\|---\|---:\|
	\| [8bit](https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-8bit) \| 334.1 GB \| 3.4607 \| — \| — \|
	\| [6bit](https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-6bit) \| 255.9 GB \| 3.4646 \| +0.0011 [−0.0017, +0.0038] \| 89/141 \|
	\| [mixed-4_8bit](https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-mixed-4_8bit) \| 181.9 GB \| 3.5705 \| +0.0312 [+0.0271, +0.0355] \| 131/141 \|
	\| [4bit](https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-4bit) \| 177.6 GB \| 3.7549 \| +0.0816 [+0.0755, +0.0879] \| 140/141 \|

	Read the interval, not the point estimate; "windows worse" counts how many of the 141
	windows the build lost outright.

	Against the 8-bit anchor: 6-bit +0.1%, mixed 4/8-bit +3.2%, uniform 4-bit +8.5%. Routed experts are 97% of the parameters; the mixed build keeps the other ~9B (KDA and MLA projections, shared experts, dense layers, embeddings) at 8-bit for 4.4 GB more than uniform 4-bit.

	Greedy generation (a collapse detector, not a ranking) is coherent on every published build.

	## License

	MIT, as the upstream model. Port code: [https://github.com/PipeNetwork/glm53-flash-mlx](https://github.com/PipeNetwork/glm53-flash-mlx).