license: apache-2.0
base_model:
- Writer/palmyra-mini
- Writer/palmyra-mini-thinking-a
- Writer/palmyra-mini-thinking-b
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- text-generation-inference
- code
- math
- coder
palmyra-mini-thinking-AIO-GGUF
The palmyra-mini models demonstrates exceptional capabilities in complex reasoning and mathematical problem-solving domains. Its performance is particularly noteworthy on benchmarks that require deep understanding and multi-step thought processes. A key strength of the model is its proficiency in grade-school-level math problems, as evidenced by its impressive score of 0.818 on the gsm8k (strict-match) benchmark. This high score indicates a robust ability to parse and solve word problems, a foundational skill for more advanced quantitative reasoning. This aptitude for mathematics is further confirmed by its outstanding performance on the MATH500 benchmark, where it also achieved a score of 0.818. This result underscores the models consistent and reliable mathematical capabilities across different problem sets. The model also shows strong performance on the AMC23 benchmark, with a solid score of 0.6. This benchmark, representing problems from the American Mathematics Competitions, highlights the models ability to tackle challenging, competition-level mathematics.
Palmyra Mini GGUF Variants
| Model Name | Download Link |
|---|---|
| palmyra-mini-GGUF | Link |
| palmyra-mini-thinking-a-GGUF | Link |
| palmyra-mini-thinking-b-GGUF | Link |
Model Files
palmyra-mini
| File Name | Quant Type | File Size |
|---|---|---|
| palmyra-mini.BF16.gguf | BF16 | 3.56 GB |
| palmyra-mini.F16.gguf | F16 | 3.56 GB |
| palmyra-mini.F32.gguf | F32 | 7.11 GB |
| palmyra-mini.Q2_K.gguf | Q2_K | 752 MB |
| palmyra-mini.Q3_K_L.gguf | Q3_K_L | 980 MB |
| palmyra-mini.Q3_K_M.gguf | Q3_K_M | 924 MB |
| palmyra-mini.Q3_K_S.gguf | Q3_K_S | 861 MB |
| palmyra-mini.Q4_0.gguf | Q4_0 | 1.07 GB |
| palmyra-mini.Q4_1.gguf | Q4_1 | 1.16 GB |
| palmyra-mini.Q4_K.gguf | Q4_K | 1.12 GB |
| palmyra-mini.Q4_K_M.gguf | Q4_K_M | 1.12 GB |
| palmyra-mini.Q4_K_S.gguf | Q4_K_S | 1.07 GB |
| palmyra-mini.Q5_0.gguf | Q5_0 | 1.26 GB |
| palmyra-mini.Q5_1.gguf | Q5_1 | 1.35 GB |
| palmyra-mini.Q5_K.gguf | Q5_K | 1.28 GB |
| palmyra-mini.Q5_K_M.gguf | Q5_K_M | 1.28 GB |
| palmyra-mini.Q5_K_S.gguf | Q5_K_S | 1.26 GB |
| palmyra-mini.Q6_K.gguf | Q6_K | 1.46 GB |
| palmyra-mini.Q8_0.gguf | Q8_0 | 1.89 GB |
palmyra-mini-thinking-a
| File Name | Quant Type | File Size |
|---|---|---|
| palmyra-mini-thinking-a.BF16.gguf | BF16 | 3.56 GB |
| palmyra-mini-thinking-a.F16.gguf | F16 | 3.56 GB |
| palmyra-mini-thinking-a.F32.gguf | F32 | 7.11 GB |
| palmyra-mini-thinking-a.Q2_K.gguf | Q2_K | 752 MB |
| palmyra-mini-thinking-a.Q3_K_L.gguf | Q3_K_L | 980 MB |
| palmyra-mini-thinking-a.Q3_K_M.gguf | Q3_K_M | 924 MB |
| palmyra-mini-thinking-a.Q3_K_S.gguf | Q3_K_S | 861 MB |
| palmyra-mini-thinking-a.Q4_0.gguf | Q4_0 | 1.07 GB |
| palmyra-mini-thinking-a.Q4_1.gguf | Q4_1 | 1.16 GB |
| palmyra-mini-thinking-a.Q4_K.gguf | Q4_K | 1.12 GB |
| palmyra-mini-thinking-a.Q4_K_M.gguf | Q4_K_M | 1.12 GB |
| palmyra-mini-thinking-a.Q4_K_S.gguf | Q4_K_S | 1.07 GB |
| palmyra-mini-thinking-a.Q5_0.gguf | Q5_0 | 1.26 GB |
| palmyra-mini-thinking-a.Q5_1.gguf | Q5_1 | 1.35 GB |
| palmyra-mini-thinking-a.Q5_K.gguf | Q5_K | 1.28 GB |
| palmyra-mini-thinking-a.Q5_K_M.gguf | Q5_K_M | 1.28 GB |
| palmyra-mini-thinking-a.Q5_K_S.gguf | Q5_K_S | 1.26 GB |
| palmyra-mini-thinking-a.Q6_K.gguf | Q6_K | 1.46 GB |
| palmyra-mini-thinking-a.Q8_0.gguf | Q8_0 | 1.89 GB |
palmyra-mini-thinking-b
| File Name | Quant Type | File Size |
|---|---|---|
| palmyra-mini-thinking-b.BF16.gguf | BF16 | 3.09 GB |
| palmyra-mini-thinking-b.F16.gguf | F16 | 3.09 GB |
| palmyra-mini-thinking-b.F32.gguf | F32 | 6.18 GB |
| palmyra-mini-thinking-b.Q2_K.gguf | Q2_K | 676 MB |
| palmyra-mini-thinking-b.Q3_K_L.gguf | Q3_K_L | 880 MB |
| palmyra-mini-thinking-b.Q3_K_M.gguf | Q3_K_M | 824 MB |
| palmyra-mini-thinking-b.Q3_K_S.gguf | Q3_K_S | 761 MB |
| palmyra-mini-thinking-b.Q4_0.gguf | Q4_0 | 935 MB |
| palmyra-mini-thinking-b.Q4_1.gguf | Q4_1 | 1.02 GB |
| palmyra-mini-thinking-b.Q4_K.gguf | Q4_K | 986 MB |
| palmyra-mini-thinking-b.Q4_K_M.gguf | Q4_K_M | 986 MB |
| palmyra-mini-thinking-b.Q4_K_S.gguf | Q4_K_S | 940 MB |
| palmyra-mini-thinking-b.Q5_0.gguf | Q5_0 | 1.1 GB |
| palmyra-mini-thinking-b.Q5_1.gguf | Q5_1 | 1.18 GB |
| palmyra-mini-thinking-b.Q5_K.gguf | Q5_K | 1.13 GB |
| palmyra-mini-thinking-b.Q5_K_M.gguf | Q5_K_M | 1.13 GB |
| palmyra-mini-thinking-b.Q5_K_S.gguf | Q5_K_S | 1.1 GB |
| palmyra-mini-thinking-b.Q6_K.gguf | Q6_K | 1.27 GB |
| palmyra-mini-thinking-b.Q8_0.gguf | Q8_0 | 1.65 GB |
Quants Usage
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
