Hugging Face's logo

mozilla-ai
/
granite-34b-code-instruct-llamafile

Text Generation
Transformers
llamafile
code
granite
Eval Results
Model card Files
xet
Community
1
jartine commited on
Commit
7a5f8e0
·
verified ·
1 Parent(s): deada0d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -272,12 +272,14 @@ It can be changed, e.g. `--temp 0.8`.
272
 
273
  | hardware | model\_filename | size | test | t/s |
274
  | :----------------------------------------- | :--------------------------------------- | ---------: | ------------: | --------------: |
275
- | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | pp512 | 186.14 |
276
- | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | tg16 | 14.13 |
277
- | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | pp512 | 94.34 |
278
- | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | tg16 | 5.61 |
279
  | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | pp512 | 95.08 |
280
  | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | tg16 | 7.78 |
 
 
281
 
282
  ## About Quantization
283
 
 
272
 
273
  | hardware | model\_filename | size | test | t/s |
274
  | :----------------------------------------- | :--------------------------------------- | ---------: | ------------: | --------------: |
275
+ | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | pp512 | 159.02 |
276
+ | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | tg16 | 15.39 |
277
+ | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | pp512 | 186.14 |
278
+ | Apple M2 Ultra (Metal GPU) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | tg16 | 14.13 |
279
  | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | pp512 | 95.08 |
280
  | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | tg16 | 7.78 |
281
+ | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | pp512 | 94.34 |
282
+ | AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | tg16 | 5.61 |
283
 
284
  ## About Quantization
285