File size: 7,928 Bytes
df14e83 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 |
---
tags:
- Coder
- Math
- qwen2
- thinking
- reasoning
model-index:
- name: Palmyra-mini-thinking-b
results: []
license: apache-2.0
language:
- en
---
[](https://hf.co/QuantFactory)
# QuantFactory/palmyra-mini-thinking-b-GGUF
This is quantized version of [Writer/palmyra-mini-thinking-b](https://huggingface.co/Writer/palmyra-mini-thinking-b) created using llama.cpp
# Original Model Card
<div align="center">
<h1>Palmyra-mini-thinking-b</h1>
</div>
<p align="center">
<img src="https://huggingface.co/Writer/palmyra-mini-thinking-b/resolve/main/logo-mini-b%20benchmark-performance.png?download=true" width="800"/>
</p>
### Model Description
- **Language(s) (NLP):** English
- **License:** Apache-2.0
- **Finetuned from model:** Qwen/Qwen2.5-1.5B
- **Context window:** 131,072 tokens
- **Parameters:** 1.7 billion
## Introduction
Palmyra-mini-thinking-b represents a significant step forward in generative AI, demonstrating exceptional capabilities in complex reasoning and problem-solving domains. This model excels in mathematical and programming challenges, showcasing a robust understanding of abstract concepts and logical structures. Its performance is not just a measure of its power but a testament to its specialized training, which has honed its ability to tackle tasks that demand deep, multi-step thinking.
## Mathematical Prowess
The model's mathematical abilities are particularly noteworthy. It achieves an impressive score of 0.925 on the AMC23 benchmark, indicating a strong grasp of advanced high school mathematics. This is further complemented by its performance on MATH500, where it scores 0.882, proving its proficiency across a wide range of mathematical problems. The model also shows its strength in competitive mathematics, scoring 0.6 on AIME24(pass@1)(avg-of-1) and 0.5733 on Olympiadbench (extractive_match). These scores highlight the model's capacity for sophisticated mathematical reasoning, making it a powerful tool for both educational and research applications.
## Excellence in Competitive Programming
Beyond mathematics, Palmyra-mini-thinking-b demonstrates strong performance in the competitive programming arena. Its score of 0.6343 on the Codeforces (pass_rate) benchmark underscores its ability to understand complex algorithmic problems and generate correct, efficient code. This capability suggests the model is well-suited for tasks involving code generation, debugging, and algorithmic design, making it a valuable asset for software developers and computer science researchers.
## Benchmark Scores (sampling params: temperature:0.6, top_p:0.95)
Pass@1(avg-of-64)
| Benchmark | Pass@1 (avg-of-64) | Majority@64 |
| :-------- | :------------------- | :----------- |
| AIME24 | 59.43% | 71.67% |
| AIME25 | 49.69% | 60.00% |
| GPQA | 42.01% | 47.22% |
| HMMT25 | 27.86% | 30.00% |
| HLE | 5.22% | N/A |
| MMLU-PRO | 55.49% | 60.60% |
| MATH500 | 93.80% | 95.40% |
| LCB | 34.51% | N/A |
LCB here is version v6_2408_2505
Pass@1(avg-of-1)
| Benchmark | Score (%) |
|:-----------------------------------------------------------------|------------:|
| GSM8K (strict-match) | 42.68% |
| Minerva Math (exact match) | 7.08% |
| MMLU-PRO (exact match) | 29.26% |
| MATH (Hendrycks) | 0.16% |
| IFEval (inst_level_loose_acc) | 32.97% |
| MathQA (acc) | 30.45% |
| HumanEval (pass@1) | 7.32% |
| BBH (get-answer)(exact match) | 28.80% |
| MBPP | 16.80% |
| GPQA (diamond, pass@1: 8 samples) | 39.58% |
| AIME24 (pass@1)(avg-of-1) | 60.00% |
| AIME25 (pass@1)(avg-of-1) | 50.00% |
| Livecodebench-codegen (livecodebench/code_generation_lite v4_v5) | 28.73% |
| AMC23 | 92.50% |
| MATH500 | 88.20% |
| Minerva | 29.41% |
| Olympiadbench (extractive_match) | 57.33% |
| Codecontests (pass_rate) | 20.18% |
| Codeforces (pass_rate) | 63.43% |
| Taco (pass_rate) | 34.56% |
| APPS (all_levels) | 5.84% |
| HMMT (Feb 2025) (extractive_match) | 23.33% |
| Average | 35.94% |
### Use with transformers
You can run conversational inference using the Transformers Auto classes with the `generate()` function. Here's an example:
```py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Writer/palmyra-mini-thinking-b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto",
attn_implementation="flash_attention_2",
)
messages = [
{
"role": "user",
"content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
}
],
input_ids = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
)
gen_conf = {
"max_new_tokens": 256,
"eos_token_id": tokenizer.eos_token_id,
"temperature": 0.3,
"top_p": 0.9,
}
with torch.inference_mode():
output_id = model.generate(input_ids, **gen_conf)
output_text = tokenizer.decode(output_id[0][input_ids.shape[1] :])
print(output_text)
```
## Running with vLLM
```py
vllm serve Writer/palmyra-mini-thinking-b
```
```py
curl -X POST http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Writer/palmyra-mini-thinking-b",
"messages": [
{
"role": "user",
"content": "You have a 3-liter jug and a 5-liter jug. How can you measure exactly 4 liters of water?"
}
],
"max_tokens": 8000,
"temperature": 0.2
}'
```
## Ethical Considerations
As with any language model, there is a potential for generating biased or inaccurate information. Users should be aware of these limitations and use the model responsibly.
### Footnotes
- Base model: This model builds on NVIDIA's OpenReasoning-Nemotron-1.5B (`https://huggingface.co/nvidia/OpenReasoning-Nemotron-1.5B`).
- Evaluation methodology:
- Pass@1 (avg-of-1): computed using `lm_eval` and `lighteval`.
- Pass@1 (avg-of-64) and Majority@64: computed using `nemoskills`.
### Citation and Related Information
To cite this model:
```
@misc{Palmyra-mini-thinking-b,
author = {Writer Engineering team},
title = {{Palmyra-mini: A powerful LLM designed for math and coding}},
howpublished = {\url{https://dev.writer.com}},
year = 2025,
month = Sep
}
```
Contact Hello@writer.com
|