File size: 1,253 Bytes
c73ec2c df824a0 7d01848 c73ec2c c390aac f0232f8 c73ec2c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
---
base_model:
- unsloth/Llama-3.2-1B
tags:
- text-generation-inference
- transformers
- math
- conversational
- llama
- meta
license: apache-2.0
language:
- en
library_name: transformers
---
# GsMath-Llama-1B
## Model Description:
This is a fine-tuned version of [unsloth/Llama-3.2-1B](https://huggingface.co/unsloth/Llama-3.2-1B)!
- **recommended settings for inference:** min_p = 0.1 and temperature = 1.5 , Read this [Tweet](https://x.com/menhguin/status/1826132708508213629) to understand why.
- **License :** apache-2.0
- **Finetuned from model :** unsloth/Llama-3.2-1B
## Benchmarks:
We evaluate both models on GSM8K using the standard lm-eval 5-shot exact-match protocol. Under identical decoding and extraction settings,GsMath-Llama-1B outperforms Meta’s Llama-3.2-1B by 2x,demonstrating an improvement in small-model mathematical capability.
| Model | Params | GSM8K (5-shot, EM) |
| ----------------------------- | ------ | ------------------ |
| **GsMath-Llama-1B** | 1B | **0.137** |
| Llama-3.2-1B | 1B | 0.068 |
<p align="left">
<img alt="GsMath-Llama-1B" src="https://huggingface.co/CannaeAI/GRPO-Gsmath-Llama-1B-IT/resolve/main/GSM8K-IT.png">
</p>
|