Improve language tag
#1
by lbourdois - opened
README.md
CHANGED
|
@@ -1,78 +1,89 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
-
|
| 7 |
-
|
| 8 |
-
-
|
| 9 |
-
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
##
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
1.
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
datasets:
|
| 3 |
+
- PowerInfer/QWQ-LONGCOT-500K
|
| 4 |
+
- PowerInfer/LONGCOT-Refine-500K
|
| 5 |
+
base_model:
|
| 6 |
+
- Qwen/Qwen2.5-3B-Instruct
|
| 7 |
+
language:
|
| 8 |
+
- zho
|
| 9 |
+
- eng
|
| 10 |
+
- fra
|
| 11 |
+
- spa
|
| 12 |
+
- por
|
| 13 |
+
- deu
|
| 14 |
+
- ita
|
| 15 |
+
- rus
|
| 16 |
+
- jpn
|
| 17 |
+
- kor
|
| 18 |
+
- vie
|
| 19 |
+
- tha
|
| 20 |
+
- ara
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
[](https://hf.co/QuantFactory)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
# QuantFactory/SmallThinker-3B-Preview-GGUF
|
| 27 |
+
This is quantized version of [PowerInfer/SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview) created using llama.cpp
|
| 28 |
+
|
| 29 |
+
# Original Model Card
|
| 30 |
+
|
| 31 |
+
# SmallThinker-3B-preview
|
| 32 |
+
|
| 33 |
+
We introduce **SmallThinker-3B-preview**, a new model fine-tuned from the [Qwen2.5-3b-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) model.
|
| 34 |
+
|
| 35 |
+
## Benchmark Performance
|
| 36 |
+
|
| 37 |
+
| Model | AIME24 | AMC23 | GAOKAO2024_I | GAOKAO2024_II | MMLU_STEM | AMPS_Hard | math_comp |
|
| 38 |
+
|---------|--------|-------|--------------|---------------|-----------|-----------|-----------|
|
| 39 |
+
| Qwen2.5-3B-Instruct | 6.67 | 45 | 50 | 35.8 | 59.8 | - | - |
|
| 40 |
+
| SmallThinker | 16.667 | 57.5 | 64.2 | 57.1 | 68.2 | 70 | 46.8 |
|
| 41 |
+
| GPT-4o | 9.3 | - | - | - | 64.2 | 57 | 50 |
|
| 42 |
+
|
| 43 |
+
Limitation: Due to SmallThinker's current limitations in instruction following, for math_comp we adopt a more lenient evaluation method where only correct answers are required, without constraining responses to follow the specified AAAAA format.
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
## Intended Use Cases
|
| 47 |
+
|
| 48 |
+
SmallThinker is designed for the following use cases:
|
| 49 |
+
|
| 50 |
+
1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
|
| 51 |
+
2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model. From my test, in llama.cpp we can get 70% speedup (from 40 tokens/s to 70 tokens/s).
|
| 52 |
+
|
| 53 |
+
## Training Details
|
| 54 |
+
|
| 55 |
+
The model was trained using 8 H100 GPUs with a global batch size of 16. The specific configuration is as follows:
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
neat_packing: true
|
| 59 |
+
cutoff_len: 16384
|
| 60 |
+
per_device_train_batch_size: 2
|
| 61 |
+
gradient_accumulation_steps: 1
|
| 62 |
+
learning_rate: 1.0e-5
|
| 63 |
+
num_train_epochs: 3
|
| 64 |
+
lr_scheduler_type: cosine
|
| 65 |
+
warmup_ratio: 0.02
|
| 66 |
+
bf16: true
|
| 67 |
+
ddp_timeout: 180000000
|
| 68 |
+
weight_decay: 0.0
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
The SFT (Supervised Fine-Tuning) process was conducted in two phases:
|
| 72 |
+
|
| 73 |
+
1. First Phase:
|
| 74 |
+
- Used only the PowerInfer/QWQ-LONGCOT-500K dataset
|
| 75 |
+
- Trained for 1.5 epochs
|
| 76 |
+
|
| 77 |
+
2. Second Phase:
|
| 78 |
+
- Combined training with PowerInfer/QWQ-LONGCOT-500K and PowerInfer/LONGCOT-Refine datasets
|
| 79 |
+
- Continued training for an additional 2 epochs
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
## Limitations & Disclaimer
|
| 83 |
+
|
| 84 |
+
Please be aware of the following limitations:
|
| 85 |
+
|
| 86 |
+
* **Language Limitation:** The model has only been trained on English-language datasets, hence its capabilities in other languages are still lacking.
|
| 87 |
+
* **Limited Knowledge:** Due to limited SFT data and the model's relatively small scale, its reasoning capabilities are constrained by its knowledge base.
|
| 88 |
+
* **Unpredictable Outputs:** The model may produce unexpected outputs due to its size and probabilistic generation paradigm. Users should exercise caution and validate the model's responses.
|
| 89 |
+
* **Repetition Issue:** The model tends to repeat itself when answering high-difficulty questions. Please increase the `repetition_penalty` to mitigate this issue.
|