gitcommit90 commited on
Commit
bbbadb0
·
verified ·
1 Parent(s): bb4ca6b

Publish GLM-5.3 One-Spark discovery card

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: vllm
4
+ tags:
5
+ - glm-5.3-flash
6
+ - dgx-spark
7
+ - gb10
8
+ - exl3
9
+ - vllm
10
+ - dflash2
11
+ - speculative-decoding
12
+ - arm64
13
+ - deployment
14
+ base_model:
15
+ - zai-org/GLM-5.3-Flash
16
+ - turboderp/GLM-5.3-Flash-exl3
17
+ ---
18
+
19
+ # GLM-5.3-Flash EXL3 2.05 — One DGX Spark
20
+
21
+ A reproducible, production-capable deployment of **GLM-5.3-Flash on one NVIDIA DGX Spark**, using:
22
+
23
+ - [Turboderp's 2.05-bpw EXL3 checkpoint](https://huggingface.co/turboderp/GLM-5.3-Flash-exl3/tree/2.05bpw)
24
+ - [Inco AI's DFlash2 K7 drafter](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2)
25
+ - A TP1 ARM64/SM121 vLLM runtime derived from [MiaAI's two-Spark recipe](https://github.com/MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks)
26
+
27
+ > **64.1 tok/s structured C1 · 25.1 tok/s prose · 181.9 tok/s C4 active-stream aggregate · 262K context**
28
+
29
+ ## This repository does not contain model weights
30
+
31
+ This is a **deployment/runtime discovery page**, not a new model or quant. The software, Docker recipe, benchmark harnesses, and raw evidence live in the linked GitHub repository. Target and draft weights download directly from their original publishers.
32
+
33
+ ## Measured results
34
+
35
+ One DGX Spark, TP1, EXL3 2.05 bpw, DFlash2 K7, FP8 KV, thinking disabled:
36
+
37
+ | Benchmark | Result |
38
+ |---|---:|
39
+ | Structured C1, five-run median, temperature 0 | **64.053 tok/s** |
40
+ | Structured C1, five-run median, temperature 1.0/top-p 0.95 | **62.637 tok/s** |
41
+ | Open-ended prose, five-run median | **25.059 tok/s** |
42
+ | C4 median active stream | **40.958 tok/s/stream** |
43
+ | C4 summed active-stream convention | **181.944 tok/s** |
44
+ | C4 strict submission-to-completion wall | **91.475 tok/s** |
45
+
46
+ **C4 disclosure:** 181.944 tok/s is the sum of each stream's active decode rate, matching MiaAI's reporting convention. Because the current scheduler stages admission, strict full-batch wall throughput is 91.475 tok/s. Both numbers are published intentionally.
47
+
48
+ ## Long-context results
49
+
50
+ | Prompt | Cold prefill | Cold TTFT | Warm TTFT |
51
+ |---:|---:|---:|---:|
52
+ | 8K | **786.4 tok/s** | 10.175 s | 1.775 s |
53
+ | 16K | **822.1 tok/s** | 19.464 s | 2.786 s |
54
+ | 100K | **845.7 tok/s** | 118.250 s | 8.842 s |
55
+
56
+ Every cold request had zero prefix-cache hits; every warm checksum response was correct.
57
+
58
+ ## Links
59
+
60
+ - **GitHub source and instructions:** `https://github.com/gitcommit90/glm-5.3-one-spark`
61
+ - **Prebuilt ARM64 runtime image:** `ghcr.io/gitcommit90/glm-5.3-one-spark:general23`
62
+ - **Target checkpoint:** https://huggingface.co/turboderp/GLM-5.3-Flash-exl3/tree/2.05bpw
63
+ - **DFlash2 checkpoint:** https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2
64
+ - **Z.ai base model:** https://huggingface.co/zai-org/GLM-5.3-Flash
65
+
66
+ ## Credits and licensing
67
+
68
+ The runtime is derived from Mia's AI Lab's MIT-licensed two-Spark recipe and substantially adapted for TP1, full-model 2.05-bpw `mul1` EXL3, ARM64/SM121, and one-Spark memory limits.
69
+
70
+ Credits: Z.ai / GLM-5 Team, Turboderp and ExLlamaV3, Inco AI, Mia's AI Lab, and the vLLM contributors.
71
+
72
+ **DFlash2 is CC BY-NC-ND 4.0 for research/evaluation.** It is not bundled here. Commercial users must obtain appropriate licensing from Inco AI.