mlboydaisuke commited on
Commit
2b7f2d4
Β·
verified Β·
1 Parent(s): 680ce68

Update for S=256 bundle (no truncation)

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -46,16 +46,17 @@ no KV cache).
46
 
47
  - `macos/` β€” Core AI `.aimodel` (GPU) + `tokenizer/` + `metadata.json`
48
  - `metadata.json` exposes the diffusion knobs (no recompile to retune):
49
- - `seq` β€” canvas length (this **v1 preview** is `128` β‡’ answers up to ~80 tokens; a longer-context build is planned)
 
50
  - `block_size` `32`, `threshold` `1.0` β€” the entropy threshold trades steps for speed; lower = more gradual, higher = fewer forwards (faster), too high degrades quality
51
 
52
  ## Performance (M4 Max, GPU)
53
 
54
  | metric | value |
55
  |---|---|
56
- | throughput | **β‰ˆ38 tok/s** (threshold 1.0) |
57
- | TTFT | ~0.2 s |
58
- | forwards (NFE) | ~11 for an 83-token answer |
59
  | size | 4.9 GB (int4 + int8 head) |
60
 
61
  The per-step cost is a full bidirectional forward over the whole canvas (no KV cache) β€” the
 
46
 
47
  - `macos/` β€” Core AI `.aimodel` (GPU) + `tokenizer/` + `metadata.json`
48
  - `metadata.json` exposes the diffusion knobs (no recompile to retune):
49
+ - `seq` β€” canvas length (`256` β‡’ answers up to ~210 tokens). No KV cache β‡’ the whole prompt+answer
50
+ lives in `S`; per-step cost is ~linear in `S` (a larger canvas = fuller answers but slower steps).
51
  - `block_size` `32`, `threshold` `1.0` β€” the entropy threshold trades steps for speed; lower = more gradual, higher = fewer forwards (faster), too high degrades quality
52
 
53
  ## Performance (M4 Max, GPU)
54
 
55
  | metric | value |
56
  |---|---|
57
+ | throughput | **β‰ˆ40 tok/s** (threshold 1.0, S=256) |
58
+ | TTFT | ~0.3 s |
59
+ | forwards (NFE) | ~22 for a full 210-token answer |
60
  | size | 4.9 GB (int4 + int8 head) |
61
 
62
  The per-step cost is a full bidirectional forward over the whole canvas (no KV cache) β€” the