HelloSun commited on
Commit
ee6912b
·
verified ·
1 Parent(s): f0053c9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +16 -12
README.md CHANGED
@@ -119,7 +119,7 @@ The first digit after the decimal is 1 for 9.11 and 9 for 9.90. Since 9 > 1,
119
  So, 9.9 is bigger than 9.11.
120
  ```
121
 
122
- ## Performance (measured)
123
 
124
  Test environment — the resources **actually usable inside the container (docker)**:
125
 
@@ -131,21 +131,25 @@ Test environment — the resources **actually usable inside the container (docke
131
  | GPU | **none usable** — `/dev/nvidia*` nodes exist but `cuInit()` returns `CUDA_ERROR_NO_DEVICE` and NVML init fails |
132
  | Build / runtime | CPU-only, `-t 16` |
133
 
134
- Measured on `pangu-diffusion-Q4_K_M.gguf` (4.59 GiB, 8.03 B params):
 
 
135
 
136
- | Mode | Benchmark | Speed |
137
  |---|---|---|
138
- | **Input** — prompt processing (causal prefill) | `llama-bench` pp512 | **131.8 t/s** (131.77 ± 3.34) |
139
- | **Output** — autoregressive decoding | `llama-bench` tg128 | **11.1 t/s** (11.11 ± 0.35) |
140
- | **Output** — block diffusion (block 32 / small-block 4, `alg=entropy`, `temp=0`) | 224 tokens | **≈ 4.4 t/s** (≈ 55 s end-to-end) |
141
 
142
  Notes:
143
- - The input/prompt-processing number is the same causal prefill used by both the AR and the
144
- diffusion path.
145
- - Block diffusion reveals one token per denoising step, i.e. it runs up to `block_length`
146
- (= 32) extra forward passes per block, so its tokens/s is inherently lower than AR.
147
- - The AR figures (`pp512` / `tg128`) are the best reference for raw model throughput on
148
- this hardware.
 
 
149
 
150
  ## What the patch does
151
 
 
119
  So, 9.9 is bigger than 9.11.
120
  ```
121
 
122
+ ## Speed when you actually use it (measured)
123
 
124
  Test environment — the resources **actually usable inside the container (docker)**:
125
 
 
131
  | GPU | **none usable** — `/dev/nvidia*` nodes exist but `cuInit()` returns `CUDA_ERROR_NO_DEVICE` and NVML init fails |
132
  | Build / runtime | CPU-only, `-t 16` |
133
 
134
+ These are **end-user speeds** measured from real runs of the CLI on
135
+ `pangu-diffusion-Q4_K_M.gguf` (4.59 GiB, 8.03 B params), i.e. what you actually
136
+ experience while using it:
137
 
138
+ | What the user experiences | Mode | Speed |
139
  |---|---|---|
140
+ | **Input** — waiting for the prompt to be read before output starts | both | **≈ 78 t/s** short prompt; **≈ 132 t/s** for longer prompts (`llama-bench` pp512) |
141
+ | **Output** — text generated (autoregressive, streamed live) | `llama-cli` | **11.9 t/s** (llama-bench tg128: 11.1 t/s) |
142
+ | **Output** — text generated (block diffusion, block 32 / small-block 4, `alg=entropy`, `temp=0`) | `llama-diffusion-cli` | **≈ 4.4 t/s** |
143
 
144
  Notes:
145
+ - Input (prompt processing) is the same causal prefill for both paths; it is what you wait
146
+ for before the answer appears.
147
+ - The AR output speed (≈ 12 t/s) is what `llama-cli` prints as `Generation: X t/s`; the
148
+ diffused text is only flushed at the end of a run (unless `--diffusion-visual`).
149
+ - Block diffusion reveals one token per denoising step, i.e. up to `block_length` (= 32)
150
+ forward passes per block, so its tokens/s is inherently lower. The authors' faster
151
+ `alg=confidence_threshold` mode is not implemented here (fixed 1-token steps).
152
+ - The AR figures are the best reference for raw model throughput on this hardware.
153
 
154
  ## What the patch does
155