Harsha001 commited on
Commit
f309917
·
verified ·
1 Parent(s): c970e7a

Model card

Browse files
Files changed (1) hide show
  1. README.md +9 -5
README.md CHANGED
@@ -21,19 +21,23 @@ attention every fourth layer, image input). The multi-token prediction layer of
21
 
22
  | File | Size | Weights | Fits |
23
  |---|---:|---|---|
24
- | `DaisyRose-27B-Q8_0_ROCMFPX.gguf` | (pending) | 8-bit ROCmFPX, token embedding and output Q8_0 | Strix Halo, 32 GB AMD GPUs |
25
- | `DaisyRose-27B-Q6_0_ROCMFPX.gguf` | (pending) | 6-bit ROCmFPX, token embedding and output Q6_K | 24 GB AMD GPUs, Strix Halo |
26
- | `DaisyRose-27B-Q4_0_ROCMFP4_STRIX_O8.gguf` | (pending) | ROCmFP4, attention, delta net output and output layer 8-bit ROCmFPX | 16 to 24 GB AMD GPUs |
27
  | `mmproj-DaisyRose-27B-F16.gguf` | 0.9 GB | vision encoder, F16 | add for image input |
28
 
29
  ## Quality
30
 
31
- KL divergence against the BF16 weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), CPU.
 
32
  Lower is better. "Same top token" is how often the file's most likely next token matches BF16.
33
 
34
  | Build | Size | Perplexity | KL to BF16 | Same top token |
35
  |---|---:|---:|---:|---:|
36
- | BF16 (reference, not published) | 54.6 GB | (pending) | | |
 
 
 
37
 
38
  Every file also passed a greedy chat check before upload ("What is the capital of France?" gives "Paris").
39
 
 
21
 
22
  | File | Size | Weights | Fits |
23
  |---|---:|---|---|
24
+ | `DaisyRose-27B-Q8_0_ROCMFPX.gguf` | 28.3 GB | 8-bit ROCmFPX, token embedding and output Q8_0 | Strix Halo, 32 GB AMD GPUs |
25
+ | `DaisyRose-27B-Q6_0_ROCMFPX.gguf` | 22.5 GB | 6-bit ROCmFPX, token embedding and output Q6_K | 24 GB AMD GPUs, Strix Halo |
26
+ | `DaisyRose-27B-Q4_0_ROCMFP4_STRIX_O8.gguf` | 19.2 GB | ROCmFP4, attention, delta net output and output layer 8-bit ROCmFPX | 16 to 24 GB AMD GPUs |
27
  | `mmproj-DaisyRose-27B-F16.gguf` | 0.9 GB | vision encoder, F16 | add for image input |
28
 
29
  ## Quality
30
 
31
+ KL divergence against the BF16 weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2). Each file was
32
+ expanded to F16 and scored on the same GPU backend as the BF16 reference, so the numbers show the weight format only.
33
  Lower is better. "Same top token" is how often the file's most likely next token matches BF16.
34
 
35
  | Build | Size | Perplexity | KL to BF16 | Same top token |
36
  |---|---:|---:|---:|---:|
37
+ | BF16 (reference, not published) | 54.6 GB | 5.941 | | |
38
+ | **Q8_0_ROCMFPX** | 28.3 GB | 5.946 | 0.00025 | 99.3% |
39
+ | **Q6_0_ROCMFPX** | 22.5 GB | 5.947 | 0.0020 | 98.0% |
40
+ | **Q4_0_ROCMFP4_STRIX_O8** | 19.2 GB | 5.998 | 0.019 | 94.0% |
41
 
42
  Every file also passed a greedy chat check before upload ("What is the capital of France?" gives "Paris").
43