Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ library_name: transformers
|
|
| 24 |

|
| 25 |
|
| 26 |
# What is it
|
| 27 |
-
This is a 1.0 Fijik series with **1 billion** parameters, dense 56 layer transformer LLM based on Qwen2.5, specifically, it was merged using Mergekit to be twice as large as Qwen2.5
|
| 28 |
|
| 29 |
After merging, we used a custom dataset mix meant for this model, to improve its performance even more.
|
| 30 |
- **Step 1 for fine-tuning via unsloth:** SFT on an estimated 5 million tokens. (more or less)
|
|
|
|
| 24 |

|
| 25 |
|
| 26 |
# What is it
|
| 27 |
+
This is a 1.0 Fijik series with **1 billion** parameters, dense 56 layer transformer LLM based on Qwen2.5, specifically, it was merged using Mergekit to be twice as large as Qwen2.5 0.5B.
|
| 28 |
|
| 29 |
After merging, we used a custom dataset mix meant for this model, to improve its performance even more.
|
| 30 |
- **Step 1 for fine-tuning via unsloth:** SFT on an estimated 5 million tokens. (more or less)
|