File size: 6,174 Bytes
1cdbf45 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 |
---
tags:
- Coder
- Math
- qwen2
- thinking
- reasoning
model-index:
- name: Palmyra-mini-thinking-a
results: []
license: apache-2.0
language:
- en
---
[](https://hf.co/QuantFactory)
# QuantFactory/palmyra-mini-thinking-a-GGUF
This is quantized version of [Writer/palmyra-mini-thinking-a](https://huggingface.co/Writer/palmyra-mini-thinking-a) created using llama.cpp
# Original Model Card
<div align="center">
<h1>Palmyra-mini-thinking-a</h1>
</div>
### Model Description
- **Language(s) (NLP):** English
- **License:** Apache-2.0
- **Finetuned from model:** Qwen/Qwen2.5-1.5B
- **Context window:** 131,072 tokens
- **Parameters:** 1.7 billion
## Model Details
The palmyra-mini-thinking-a model demonstrates exceptional performance in advanced mathematical reasoning and competitive programming. Its capabilities are highlighted by an outstanding score of 0.886 on the 'MATH500' benchmark, showcasing a robust ability to solve complex mathematical problems. The strength of the model in quantitative challenges is further confirmed by its score of 0.8287 on 'gsm8k (strict-match)', which demonstrates proficiency in multi-step arithmetic reasoning. Additionally, the model proves its aptitude for high-level problem-solving with a score of 0.8 on 'AMC23'. The model also shows strong potential in the coding domain, achieving a score of 0.5631 on 'Codeforces (pass_rate)' and 0.5481 on 'Olympiadbench (extractive_match)', indicating competence in generating correct solutions for programming challenges.
## Benchmark Performance
This section provides a detailed breakdown of the palmyra-mini-thinking-a model's performance across a standardized set of industry benchmarks. The data is presented in its original order from the source evaluation.
| Benchmark | Score |
|:-----------------------------------------------------------------|---------:|
| gsm8k (strict-match) | 0.8287 |
| minerva_math(exact_match) | 0.3842 |
| mmlu_pro(exact_match) | 0.2748 |
| hendrycks_math | 0.0054 |
| ifeval (inst_level_loose_acc) | 0.3657 |
| mathqa (acc) | 0.4171 |
| humaneval (pass@1) | 0.2378 |
| BBH (get-answer)(exact_match) | 0.462 |
| mbpp | 0.304 |
| leadboard_musr (acc_norm) | 0.3413 |
| gpqa lighteval gpqa diamond_pass@1:8_samples | 0.3826 |
| AIME24(pass@1)(avg-of-1) | 0.4333 |
| AIME25(pass@1)(avg-of-1) | 0.3667 |
| Livecodebench-codegen (livecodebench/code_generation_lite v4_v5) | 0.1784 |
| AMC23 | 0.8 |
| MATH500 | 0.886 |
| Minerva | 0.3493 |
| Olympiadbench (extractive_match) | 0.5481 |
| Codecontests (pass_rate) | 0.1778 |
| Codeforces (pass_rate) | 0.5631 |
| Taco (pass_rate) | 0.3083 |
| APPS (all_levels) | 0.0447 |
| HMMT23 (extractive_match) | 0.1 |
| Average | 0.380839 |
### Use with transformers
You can run conversational inference using the Transformers Auto classes with the `generate()` function. Here's an example:
```py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Writer/palmyra-mini-thinking-a"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
attn_implementation="flash_attention_2",
)
messages = [
{
"role": "user",
"content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
}
],
input_ids = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
)
gen_conf = {
"max_new_tokens": 256,
"eos_token_id": tokenizer.eos_token_id,
"temperature": 0.3,
"top_p": 0.9,
}
with torch.inference_mode():
output_id = model.generate(input_ids, **gen_conf)
output_text = tokenizer.decode(output_id[0][input_ids.shape[1] :])
print(output_text)
```
## Running with vLLM
```py
vllm serve Writer/palmyra-mini-thinking-a
```
```py
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Writer/palmyra-mini-thinking-a",
"messages": [
{
"role": "user",
"content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
}
],
"max_tokens": 8000,
"temperature": 0.2
}'
```
## Ethical Considerations
As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
### Citation and Related Information
To cite this model:
```
@misc{Palmyra-mini-thinking-a,
author = {Writer Engineering team},
title = {{Palmyra-mini: A powerful LLM designed for math and coding}},
howpublished = {\url{https://dev.writer.com}},
year = 2025,
month = Sep
}
```
Contact Hello@writer.com
|