Update README.md
Browse files
README.md
CHANGED
|
@@ -57,6 +57,25 @@ library_name: transformers
|
|
| 57 |
- **Reasoning Improvement:** In this version of Atlas, I have enhanced it's reasoning via synthetic data from models such as Gemini-2.0 Flash Thinking so that it can improve on reasoning.
|
| 58 |
---
|
| 59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
### **Intended Use Cases**
|
| 61 |
Atlas Pro works best for:
|
| 62 |
- **Technical Professionals:** Helping developers, engineers, and scientists solve complex problems.
|
|
|
|
| 57 |
- **Reasoning Improvement:** In this version of Atlas, I have enhanced it's reasoning via synthetic data from models such as Gemini-2.0 Flash Thinking so that it can improve on reasoning.
|
| 58 |
---
|
| 59 |
|
| 60 |
+
# **Evaluation**
|
| 61 |
+
Below are the evaluations of the Atlas-Pro models and Deepseek's R1 Qwen Distills (The model that started the whole Atlas family):
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
| **Metric** | **Spestly Atlas Pro (7B)** | **Spestly Atlas Pro (1.5B)** | DeepSeek-R1-Distill-Qwen (7B) | DeepSeek-R1-Distill-Qwen (1.5B) |
|
| 65 |
+
|-------------------------|---------------------------|------------------------------|-----------------------------------|-------------------------------------|
|
| 66 |
+
| **Average** | **22.65%** | 12.93% | 11.73% | 7.53% |
|
| 67 |
+
| **IFEval** | 31.54% | 24.30% | **40.38%** | 34.63% |
|
| 68 |
+
| **BBH** | **25.27%** | 9.08% | 7.88% | 4.73% |
|
| 69 |
+
| **MATH** | **38.90%** | 25.83% | 0.00% | 0.00% |
|
| 70 |
+
| **GPQA** | **11.63%** | 6.26% | 3.91% | 2.97% |
|
| 71 |
+
| **MUSR** | **6.65%** | 1.86% | 3.55% | 2.08% |
|
| 72 |
+
| **MMLU-Pro** | **21.89%** | 10.28% | 14.68% | 0.78% |
|
| 73 |
+
| **Carbon Emissions (kg)** | 0.69 kg | **0.59 kg** | 0.68 kg | 0.62 kg |
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
---
|
| 79 |
### **Intended Use Cases**
|
| 80 |
Atlas Pro works best for:
|
| 81 |
- **Technical Professionals:** Helping developers, engineers, and scientists solve complex problems.
|