|
|
--- |
|
|
base_model: |
|
|
- Lambent/Mira-v1.21-27B-rlvr |
|
|
tags: |
|
|
- gguf |
|
|
- mergekit |
|
|
- merge |
|
|
- Lambent |
|
|
--- |
|
|
|
|
|
<div align="center"> |
|
|
<h1 style="font-size: 1.2em; font-weight: bold;"> |
|
|
All credits go to:</h1> |
|
|
<h1 style="font-size: 1.2em; font-weight: bold;"> <a href="https://huggingface.co/Lambent">Lambent</a> for the wonderful model series. They did all the work, I just ran a few commands.</h1> |
|
|
</div> |
|
|
<div align="center"> |
|
|
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/Demeter_Altemps_Inv8546.jpg/500px-Demeter_Altemps_Inv8546.jpg" width="200" alt="Aphrodite"> |
|
|
</div> |
|
|
<div align="center"> |
|
|
<h1 style="font-size: 1.23em; font-weight: bold;"> |
|
|
<a href="https://web.archive.org/web/20251008221112if_/https://uh.edu/~cldue/texts/demeter.html">Homeric Hymn to Demeter</a></h1> |
|
|
</h1> |
|
|
</div> |
|
|
### 📖 Overview [Files will be uploaded in 2 1/2 hours, give or take.] |
|
|
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/arcee-ai/mergekit. It utilizes custom merge methods. |
|
|
|
|
|
### The Recipe [Is it any good?] |
|
|
|
|
|
ϻira Dεmeter| Methods: [Karcher, MagStock, MagVar, CABS] |
|
|
|
|
|
<details> |
|
|
<summary><b>View Configuration</b></summary> |
|
|
|
|
|
```yaml |
|
|
# https://huggingface.co/Lambent |
|
|
name: Mira-Aphrodite-v19-freckles |
|
|
|
|
|
merge_method: magstock |
|
|
|
|
|
models: |
|
|
- model: ./Ingredients/Lambent_Mira-v1.8.1a-27B |
|
|
parameters: |
|
|
weight: 0.5 |
|
|
density: 0.5 |
|
|
epsilon: 0.4 |
|
|
- model: ./Ingredients/Mira-v14 # karcher |
|
|
parameters: |
|
|
weight: 0.5 |
|
|
density: 0.5 |
|
|
|
|
|
base_model: ./Ingredients/Lambent_Mira-v1.8.1a-27B |
|
|
|
|
|
dtype: bfloat16 |
|
|
|
|
|
parameters: |
|
|
rescale: true |
|
|
t_factor: 1.0 |
|
|
|
|
|
tokenizer: |
|
|
source: union |
|
|
|
|
|
--- |
|
|
name: Mira-Aphrodite-v31 |
|
|
|
|
|
merge_method: magvar |
|
|
|
|
|
base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr |
|
|
models: |
|
|
- model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr |
|
|
parameters: |
|
|
weight: 0.40 |
|
|
- model: ./Ingredients/Lambent_Mira-v1.20-27B-dpo |
|
|
parameters: |
|
|
weight: 0.20 |
|
|
- model: ./Ingredients/Lambent_Mira-v1.19.3c-27B-dpo |
|
|
parameters: |
|
|
weight: 0.20 |
|
|
- model: ./Ingredients/Lambent_Mira-v1.19-27B-dpo |
|
|
parameters: |
|
|
weight: 0.20 |
|
|
|
|
|
parameters: |
|
|
top_k: 0.45 |
|
|
eps: 1e-6 |
|
|
bond: 0.2 |
|
|
|
|
|
tokenizer: |
|
|
source: base |
|
|
|
|
|
dtype: float32 |
|
|
out_dtype: bfloat16 |
|
|
|
|
|
--- |
|
|
name: Mira-Demeter |
|
|
|
|
|
merge_method: cabs |
|
|
|
|
|
base_model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/ |
|
|
|
|
|
models: |
|
|
- model: ./Ingredients/Lambent_Mira-v1.21-27B-rlvr/ |
|
|
- model: ./Ingredients/Mira-Aphrodite-v31/ |
|
|
parameters: |
|
|
weight: 0.6 |
|
|
n_val: 16 |
|
|
m_val: 32 |
|
|
- model: ./Ingredients/Mira-Aphrodite-v19-freckles/ |
|
|
parameters: |
|
|
weight: 0.4 |
|
|
n_val: 11 |
|
|
m_val: 33 |
|
|
|
|
|
|
|
|
default_n_val: 8 |
|
|
default_m_val: 32 |
|
|
|
|
|
pruning_order: |
|
|
- ./Ingredients/Mira-Aphrodite-v31/ |
|
|
- ./Ingredients/Mira-Aphrodite-v19-freckles/ |
|
|
|
|
|
# base_model: base |
|
|
|
|
|
dtype: float32 |
|
|
out_dtype: bfloat16 |
|
|
|
|
|
tokenizer: |
|
|
source: union |
|
|
|
|
|
``` |
|
|
|
|
|
## </details> |
|
|
|
|
|
### A conversation snippet |
|
|
|
|
|
<details> |
|
|
<summary><b>View Conversation</b></summary> |
|
|
|
|
|
```yaml |
|
|
# |
|
|
``` |
|
|
``` |
|
|
</details> |
|
|
</div> |
|
|
``` |
|
|
|
