|
|
--- |
|
|
base_model: |
|
|
- meta-llama/Llama-3.2-1B-Instruct |
|
|
library_name: transformers |
|
|
tags: |
|
|
- mergekit |
|
|
- merge |
|
|
- NewtonBot |
|
|
- MIXdevAI |
|
|
- NewtonBotFamilyTree |
|
|
- chemistry |
|
|
- biology |
|
|
- code |
|
|
license: gpl-3.0 |
|
|
datasets: |
|
|
- fka/awesome-chatgpt-prompts |
|
|
- TeichAI/claude-4.5-opus-high-reasoning-250x |
|
|
- KingNish/reasoning-base-20k |
|
|
language: |
|
|
- ru |
|
|
- en |
|
|
- ja |
|
|
- ro |
|
|
- fr |
|
|
--- |
|
|
# MIXdevAI-llama |
|
|
|
|
|
Легковестная ИИ, на основе Llama 3 из семейства Newton bot, созданная Kolyadual |
|
|
|
|
|
Модели, которые были слиты: |
|
|
* meta-llama/Llama-3.2-1B-Instruct |
|
|
* KingNish/Reasoning-Llama-1b-v0.1 |
|
|
|
|
|
### Конфигурации |
|
|
|
|
|
Для создания данной модели использовалась следующая конфигурация YAML: |
|
|
|
|
|
```yaml |
|
|
# merge_config.yaml |
|
|
slices: |
|
|
- sources: |
|
|
- model: /home/kolyadual/newton-pocket/llama3 |
|
|
layer_range: [0, 16] |
|
|
- model: /home/kolyadual/newton-pocket/llama-reasoning |
|
|
layer_range: [0, 16] |
|
|
merge_method: slerp |
|
|
base_model: /home/kolyadual/newton-pocket/llama3 |
|
|
parameters: |
|
|
t: |
|
|
- filter: self_attn |
|
|
value: [0, 0.5, 0.3, 0.7, 1] |
|
|
- filter: mlp |
|
|
value: [1, 0.5, 0.7, 0.3, 0] |
|
|
- value: 0.5 |
|
|
dtype: bfloat16 |
|
|
|
|
|
|
|
|
``` |
