Update README.md
Browse files
README.md
CHANGED
|
@@ -9,10 +9,10 @@ license: mit
|
|
| 9 |
---
|
| 10 |
|
| 11 |
## **Training & Fine-Tuning**
|
| 12 |
-
RombUltima-32B is based on a **
|
| 13 |
|
| 14 |
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
|
| 15 |
-
- **Precision:** Trained and fine-tuned in **
|
| 16 |
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
|
| 17 |
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
|
| 18 |
|
|
|
|
| 9 |
---
|
| 10 |
|
| 11 |
## **Training & Fine-Tuning**
|
| 12 |
+
RombUltima-32B is based on a **slerp merge** of its parent models using equal weighting (0.5 each), resulting in a **balanced fusion** that leverages both structured knowledge from Rombos and enhanced generalization from Ultima.
|
| 13 |
|
| 14 |
- **Tokenization Approach:** Uses a **union-based tokenizer** to maximize vocabulary coverage.
|
| 15 |
+
- **Precision:** Trained and fine-tuned in **bfloat16** for efficient inference.
|
| 16 |
- **Long-Context Support:** Supports up to **32K tokens** (based on Qwen-32B), with stable generation up to **8K tokens**, depending on hardware constraints.
|
| 17 |
- **Multilingual Strength:** Strong performance in **English, French, Chinese, and other global languages**.
|
| 18 |
|