Instructions to use pipenetwork/GLM-5.3-Flash-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("pipenetwork/GLM-5.3-Flash-MLX-8bit")
config = load_config("pipenetwork/GLM-5.3-Flash-MLX-8bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "pipenetwork/GLM-5.3-Flash-MLX-8bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default pipenetwork/GLM-5.3-Flash-MLX-8bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use pipenetwork/GLM-5.3-Flash-MLX-8bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "pipenetwork/GLM-5.3-Flash-MLX-8bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "pipenetwork/GLM-5.3-Flash-MLX-8bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash-MLX-8bit / README.md

pipenetwork

Upload folder using huggingface_hub

1db76a2 verified about 17 hours ago

preview code

Raw

History Blame Contribute Delete

4.5 kB

metadata

license: mit
base_model: zai-org/GLM-5.3-Flash
base_model_relation: quantized
tags:
  - mlx
  - apple-silicon
  - glm5_next
  - mixture-of-experts
  - 8-bit
pipeline_tag: image-text-to-text
library_name: mlx

GLM-5.3-Flash-MLX-8bit

MLX (Apple Silicon) build of GLM-5.3-Flash — 320B-A18B hybrid of 34 Kimi-Delta linear-attention layers and 11 DeepSeek-sparse-attention (NoPE MLA + lightning indexer) layers with manifold-constrained hyper-connections — quantized to 8-bit.

These files are modified: converted from the upstream bfloat16 release (GLM-5.3-Flash-BF16) to MLX and quantized; the architecture is unchanged. The multi-token-prediction layer (layer 45) is not included. The vision tower is carried in bfloat16.

Runtime

glm5_next landed in mlx-vlm main on 2026-08-26 (no release carries it yet). Validating that port against transformers 5.16 at tiny scale found two numerical bugs and two epsilon mismatches, which the runtime in https://github.com/PipeNetwork/glm53-flash-mlx fixes; parity is 1e-6 end to end, exact on cached decode.

what	reference	mlx-vlm `main`	effect
`swiglu_limit`	gate clamped at 10, up at ±10, in every text MLP	no clamp anywhere in the text stack	formula mismatch on all 45 FFN blocks
mHC `base`/`scale` dtype	float32	converter casts to bf16; the Metal kernel then reads `base` as float4	`comb` mixing matrix off by ~0.5 on every layer of a converted checkpoint
MLA low-rank norm eps	`rms_norm_eps` = 1e-5	1e-6	small
indexer LayerNorm eps	1e-6	1e-5	small

This checkpoint keeps the mHC arrays and KDA decay parameters in float32 as stored, so it is safe in either runtime; the clamp is a compute-path fix and needs the patched runtime:

git clone https://github.com/PipeNetwork/glm53-flash-mlx && cd glm53-flash-mlx && pip install -r requirements.txt
python scripts/smoke_generate.py /path/to/GLM-5.3-Flash-MLX-8bit

from glm53_flash_mlx.load import load
model, processor = load("/path/to/GLM-5.3-Flash-MLX-8bit")

Size and what is quantized

334.1 GB on disk (bfloat16 upstream: 642.7 GB).

group	share of parameters	this build
routed experts (`switch_mlp`, 42 layers × 288)	304B (97%)	8-bit, group 64
KDA and MLA projections, shared experts, dense MLPs, embeddings, `lm_head`	~9B	8-bit, group 64
lightning-indexer projections	0.06B	8-bit, group 64
MoE router + correction bias, mHC arrays (fp32), KDA `A_log`/`dt_bias` (fp32), convolutions, norms	—	as stored
vision tower	0.56B	bfloat16

Quality

Perplexity on wikitext-2 (test), 288,627 tokens in 141 windows of 2048, every build scored on identical windows through this runtime. The 643 GB bfloat16 model does not fit a 512 GB machine, so the 8-bit build is the anchor (on every model we have measured, 8-bit has been statistically indistinguishable from bfloat16). Per-window NLL differences against 8-bit, bootstrapped over one shared index set (20,000 resamples):

build	size	perplexity	ΔNLL/token vs 8-bit [95% CI]	windows worse
8bit	334.1 GB	3.4607	—	—
6bit	255.9 GB	3.4646	+0.0011 [−0.0017, +0.0038]	89/141
mixed-4_8bit	181.9 GB	3.5705	+0.0312 [+0.0271, +0.0355]	131/141
4bit	177.6 GB	3.7549	+0.0816 [+0.0755, +0.0879]	140/141

Read the interval, not the point estimate; "windows worse" counts how many of the 141 windows the build lost outright.

Against the 8-bit anchor: 6-bit +0.1%, mixed 4/8-bit +3.2%, uniform 4-bit +8.5%. Routed experts are 97% of the parameters; the mixed build keeps the other ~9B (KDA and MLA projections, shared experts, dense layers, embeddings) at 8-bit for 4.4 GB more than uniform 4-bit.

Greedy generation (a collapse detector, not a ranking) is coherent on every published build.

License

MIT, as the upstream model. Port code: https://github.com/PipeNetwork/glm53-flash-mlx.