GLM-5.3-Flash on two M5 Max MacBooks (Cerebros)

This card describes a recipe. It holds no model weights. The weights are the upstream GLM-5.3-Flash weights, converted for MLX as described in the repository.

What it does. Two 128 GB M5 Max MacBooks run GLM-5.3-Flash as a two-stage pipeline over one Thunderbolt 5 cable (mx.distributed, jaccl RDMA). Each Mac holds about half of the model. There is no tensor parallelism.

Cerebros. The name nods to Cerebras. The streams run as two groups. While the first Mac runs a step for one group, the second Mac runs the step for the other group, and each hands its group over at the layer boundary. Neither Mac waits.

Results (4K to 256K context, greedy, 128 generated tokens, code prompts)

Prefill, one stream

Prompt tokens tok/s TTFT (first token)
4K 1,875 2.2 s
8K 2,216 3.7 s
16K 2,314 7.1 s
32K 2,345 14.0 s
64K 2,232 29.4 s
128K 2,009 65.2 s
256K 1,702 154.0 s

Decode, 4K context (all streams together)

Streams Layout Total tok/s tok/s for each stream
1 MTP depth 1 34.7 34.7
2 MTP depth 1, 2 groups 67.2 33.6
4 2 groups of 2 70.1 17.5
8 2 groups of 4 93.9 11.7
16 2 groups of 8 119.0 7.4

One stream of 256K tokens peaks at 107.0 GB on the first Mac. The model allows 1M tokens of context. The longest run here is 256K. Every row is in the repository under docs/results/.

Limits

  • Tested on one pair of Macs.
  • Greedy decoding only.
  • One stream at 4K prefills at 1,875 tok/s, below the 95% line of the M5 Ultra reference row (2,204). The M5 Ultra rows are the oMLX reference rows for an 80-core M5 Ultra.
  • MTP pays at 1 and 2 streams only. Above 2 streams the waves run without it.
  • The prefill stack changes rounding (INT8 projections, a bf16 indexer, six accurate layers). On one 32K text the KL against the reference is 1.397 nats against 1.274 for chunking alone. Decode runs the exact kernels.
  • The RDMA library failed twice with a bus error. The scripts retry and restart, but a crash costs minutes.
  • The project is independent. It is not affiliated with or endorsed by Apple or the authors of GLM-5.3-Flash.

Code

The code, the wave server, the sweep tools and all results are in the GitHub repository GLM-5.3-Flash-2xM5Max-Pipeline. The code is MIT licensed.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support