Update for S=256 bundle (no truncation)
Browse files
README.md
CHANGED
|
@@ -46,16 +46,17 @@ no KV cache).
|
|
| 46 |
|
| 47 |
- `macos/` β Core AI `.aimodel` (GPU) + `tokenizer/` + `metadata.json`
|
| 48 |
- `metadata.json` exposes the diffusion knobs (no recompile to retune):
|
| 49 |
-
- `seq` β canvas length (
|
|
|
|
| 50 |
- `block_size` `32`, `threshold` `1.0` β the entropy threshold trades steps for speed; lower = more gradual, higher = fewer forwards (faster), too high degrades quality
|
| 51 |
|
| 52 |
## Performance (M4 Max, GPU)
|
| 53 |
|
| 54 |
| metric | value |
|
| 55 |
|---|---|
|
| 56 |
-
| throughput | **β
|
| 57 |
-
| TTFT | ~0.
|
| 58 |
-
| forwards (NFE) | ~
|
| 59 |
| size | 4.9 GB (int4 + int8 head) |
|
| 60 |
|
| 61 |
The per-step cost is a full bidirectional forward over the whole canvas (no KV cache) β the
|
|
|
|
| 46 |
|
| 47 |
- `macos/` β Core AI `.aimodel` (GPU) + `tokenizer/` + `metadata.json`
|
| 48 |
- `metadata.json` exposes the diffusion knobs (no recompile to retune):
|
| 49 |
+
- `seq` β canvas length (`256` β answers up to ~210 tokens). No KV cache β the whole prompt+answer
|
| 50 |
+
lives in `S`; per-step cost is ~linear in `S` (a larger canvas = fuller answers but slower steps).
|
| 51 |
- `block_size` `32`, `threshold` `1.0` β the entropy threshold trades steps for speed; lower = more gradual, higher = fewer forwards (faster), too high degrades quality
|
| 52 |
|
| 53 |
## Performance (M4 Max, GPU)
|
| 54 |
|
| 55 |
| metric | value |
|
| 56 |
|---|---|
|
| 57 |
+
| throughput | **β40 tok/s** (threshold 1.0, S=256) |
|
| 58 |
+
| TTFT | ~0.3 s |
|
| 59 |
+
| forwards (NFE) | ~22 for a full 210-token answer |
|
| 60 |
| size | 4.9 GB (int4 + int8 head) |
|
| 61 |
|
| 62 |
The per-step cost is a full bidirectional forward over the whole canvas (no KV cache) β the
|