HelloSun commited on
Commit
f0053c9
·
verified ·
1 Parent(s): f0079cc

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +28 -0
README.md CHANGED
@@ -119,6 +119,34 @@ The first digit after the decimal is 1 for 9.11 and 9 for 9.90. Since 9 > 1,
119
  So, 9.9 is bigger than 9.11.
120
  ```
121
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122
  ## What the patch does
123
 
124
  1. **`src/llama-arch.cpp`** — mark `LLM_ARCH_PANGU_EMBED` as a diffusion architecture
 
119
  So, 9.9 is bigger than 9.11.
120
  ```
121
 
122
+ ## Performance (measured)
123
+
124
+ Test environment — the resources **actually usable inside the container (docker)**:
125
+
126
+ | Resource | Value |
127
+ |---|---|
128
+ | CPU | Intel Xeon Platinum 8559C |
129
+ | Effective cores | **16 vCPU** (cgroup `cpu.max` = `1600000 100000`) |
130
+ | Memory (cgroup limit) | **≈ 97 GiB** |
131
+ | GPU | **none usable** — `/dev/nvidia*` nodes exist but `cuInit()` returns `CUDA_ERROR_NO_DEVICE` and NVML init fails |
132
+ | Build / runtime | CPU-only, `-t 16` |
133
+
134
+ Measured on `pangu-diffusion-Q4_K_M.gguf` (4.59 GiB, 8.03 B params):
135
+
136
+ | Mode | Benchmark | Speed |
137
+ |---|---|---|
138
+ | **Input** — prompt processing (causal prefill) | `llama-bench` pp512 | **131.8 t/s** (131.77 ± 3.34) |
139
+ | **Output** — autoregressive decoding | `llama-bench` tg128 | **11.1 t/s** (11.11 ± 0.35) |
140
+ | **Output** — block diffusion (block 32 / small-block 4, `alg=entropy`, `temp=0`) | 224 tokens | **≈ 4.4 t/s** (≈ 55 s end-to-end) |
141
+
142
+ Notes:
143
+ - The input/prompt-processing number is the same causal prefill used by both the AR and the
144
+ diffusion path.
145
+ - Block diffusion reveals one token per denoising step, i.e. it runs up to `block_length`
146
+ (= 32) extra forward passes per block, so its tokens/s is inherently lower than AR.
147
+ - The AR figures (`pp512` / `tg128`) are the best reference for raw model throughput on
148
+ this hardware.
149
+
150
  ## What the patch does
151
 
152
  1. **`src/llama-arch.cpp`** — mark `LLM_ARCH_PANGU_EMBED` as a diffusion architecture