Instructions to use cowboycoderhq/GLM-5.3-Flash-2xM5Max-Pipeline with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use cowboycoderhq/GLM-5.3-Flash-2xM5Max-Pipeline with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download cowboycoderhq/GLM-5.3-Flash-2xM5Max-Pipeline --local-dir GLM-5.3-Flash-2xM5Max-Pipeline
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
GLM-5.3-Flash on two M5 Max MacBooks (Cerebros)
This card describes a recipe. It holds no model weights. The weights are the upstream GLM-5.3-Flash weights, converted for MLX as described in the repository.
What it does. Two 128 GB M5 Max MacBooks run GLM-5.3-Flash as a two-stage pipeline over one Thunderbolt 5 cable (mx.distributed, jaccl RDMA). Each Mac holds about half of the model. There is no tensor parallelism.
Cerebros. The name nods to Cerebras. The streams run as two groups. While the first Mac runs a step for one group, the second Mac runs the step for the other group, and each hands its group over at the layer boundary. Neither Mac waits.
Results (4K to 256K context, greedy, 128 generated tokens, code prompts)
Prefill, one stream
| Prompt tokens | tok/s | TTFT (first token) |
|---|---|---|
| 4K | 1,875 | 2.2 s |
| 8K | 2,216 | 3.7 s |
| 16K | 2,314 | 7.1 s |
| 32K | 2,345 | 14.0 s |
| 64K | 2,232 | 29.4 s |
| 128K | 2,009 | 65.2 s |
| 256K | 1,702 | 154.0 s |
Decode, 4K context (all streams together)
| Streams | Layout | Total tok/s | tok/s for each stream |
|---|---|---|---|
| 1 | MTP depth 1 | 34.7 | 34.7 |
| 2 | MTP depth 1, 2 groups | 67.2 | 33.6 |
| 4 | 2 groups of 2 | 70.1 | 17.5 |
| 8 | 2 groups of 4 | 93.9 | 11.7 |
| 16 | 2 groups of 8 | 119.0 | 7.4 |
One stream of 256K tokens peaks at 107.0 GB on the first Mac. The model allows 1M tokens of context. The longest run here is 256K. Every row is in the repository under docs/results/.
Limits
- Tested on one pair of Macs.
- Greedy decoding only.
- One stream at 4K prefills at 1,875 tok/s, below the 95% line of the M5 Ultra reference row (2,204). The M5 Ultra rows are the oMLX reference rows for an 80-core M5 Ultra.
- MTP pays at 1 and 2 streams only. Above 2 streams the waves run without it.
- The prefill stack changes rounding (INT8 projections, a bf16 indexer, six accurate layers). On one 32K text the KL against the reference is 1.397 nats against 1.274 for chunking alone. Decode runs the exact kernels.
- The RDMA library failed twice with a bus error. The scripts retry and restart, but a crash costs minutes.
- The project is independent. It is not affiliated with or endorsed by Apple or the authors of GLM-5.3-Flash.
Code
The code, the wave server, the sweep tools and all results are in the GitHub repository GLM-5.3-Flash-2xM5Max-Pipeline. The code is MIT licensed.