Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,9 @@ license: apache-2.0
|
|
| 14 |
|
| 15 |
**Dolphy AI's First step into the world of Machine Learning.**
|
| 16 |
|
| 17 |
-
This is a fine tune of Qwen 3 4B 2507 Instruct, a lightweight but capable model that can outperform many larger models.
|
|
|
|
|
|
|
| 18 |
|
| 19 |
**Compatibility**
|
| 20 |
|
|
|
|
| 14 |
|
| 15 |
**Dolphy AI's First step into the world of Machine Learning.**
|
| 16 |
|
| 17 |
+
This is a fine tune of Qwen 3 4B 2507 Instruct, a lightweight but capable model that can outperform many larger models. We used Unsloth LoRA Finetuning on an extensive range of high quality diverse datasets. Dolphy 1.0 was fine tuned on 1.5M examples throughout it's fine tuning pipeline.
|
| 18 |
+
|
| 19 |
+
Dolphy 1.0 builds on the flaws of other LoRA finetuned models. While normal training pipelines train on one dataset and merge the adapters, then continue training on more datasets. We would make the model revise the previous datasets so it wouldn't forget previously learned datasets, or have one dataset dominate over the others.
|
| 20 |
|
| 21 |
**Compatibility**
|
| 22 |
|