Update README.md
Browse files
README.md
CHANGED
|
@@ -17,11 +17,11 @@ This Mixture-of-Experts model is the combination of the following:
|
|
| 17 |
It is created using the following `mergekit-moe` config:
|
| 18 |
|
| 19 |
```yaml
|
| 20 |
-
base_model:
|
| 21 |
gate_mode: hidden
|
| 22 |
dtype: bfloat16
|
| 23 |
experts:
|
| 24 |
-
- source_model:
|
| 25 |
positive_prompts:
|
| 26 |
- "Solve the equation"
|
| 27 |
- "Derive the formula"
|
|
@@ -30,7 +30,7 @@ experts:
|
|
| 30 |
- "Find the integral of the function"
|
| 31 |
- "Find the first order derivative"
|
| 32 |
- "What is the answer to this math question"
|
| 33 |
-
- source_model:
|
| 34 |
positive_prompts:
|
| 35 |
- "Write a python program"
|
| 36 |
- "Write a java program"
|
|
@@ -43,7 +43,7 @@ experts:
|
|
| 43 |
- "How can I do this in Javascript"
|
| 44 |
- "Create a website with HTML"
|
| 45 |
shared_experts:
|
| 46 |
-
- source_model:
|
| 47 |
positive_prompts:
|
| 48 |
- "Hello, who are you?"
|
| 49 |
- "I need help with"
|
|
|
|
| 17 |
It is created using the following `mergekit-moe` config:
|
| 18 |
|
| 19 |
```yaml
|
| 20 |
+
base_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
|
| 21 |
gate_mode: hidden
|
| 22 |
dtype: bfloat16
|
| 23 |
experts:
|
| 24 |
+
- source_model: Qwen/Qwen2.5-Math-7B-Instruct
|
| 25 |
positive_prompts:
|
| 26 |
- "Solve the equation"
|
| 27 |
- "Derive the formula"
|
|
|
|
| 30 |
- "Find the integral of the function"
|
| 31 |
- "Find the first order derivative"
|
| 32 |
- "What is the answer to this math question"
|
| 33 |
+
- source_model: rombodawg/Rombos-Coder-V2.5-Qwen-7b
|
| 34 |
positive_prompts:
|
| 35 |
- "Write a python program"
|
| 36 |
- "Write a java program"
|
|
|
|
| 43 |
- "How can I do this in Javascript"
|
| 44 |
- "Create a website with HTML"
|
| 45 |
shared_experts:
|
| 46 |
+
- source_model: rombodawg/Rombos-LLM-V2.5-Qwen-7b
|
| 47 |
positive_prompts:
|
| 48 |
- "Hello, who are you?"
|
| 49 |
- "I need help with"
|