timpal0l commited on
Commit
eeef749
·
verified ·
1 Parent(s): d3ce7d2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -13
README.md CHANGED
@@ -20,20 +20,16 @@ pipeline_tag: text-generation
20
  This model is a [Slerp Merge](https://github.com/cg123/mergekit/blob/main/mergekit/merge_methods/slerp.py) of [timpal0l/Mistral-7B-v0.1-flashback-v2/](https://huggingface.co/timpal0l/Mistral-7B-v0.1-flashback-v2) and [RJuro/munin-neuralbeagle-7b](https://huggingface.co/RJuro/munin-neuralbeagle-7b).
21
  ## config.yaml
22
  ```yaml
23
- slices:
24
- - sources:
25
- - model: timpal0l/Mistral-7B-v0.1-flashback-v2
26
- layer_range: [0, 32]
27
- - model: RJuro/munin-neuralbeagle-7b
28
- layer_range: [0, 32]
29
- merge_method: slerp
 
30
  base_model: timpal0l/Mistral-7B-v0.1-flashback-v2
31
  parameters:
32
- t:
33
- - filter: self_attn
34
- value: [0, 0.5, 0.3, 0.7, 1]
35
- - filter: mlp
36
- value: [1, 0.5, 0.7, 0.3, 0]
37
- - value: 0.5
38
  dtype: bfloat16
39
  ```
 
20
  This model is a [Slerp Merge](https://github.com/cg123/mergekit/blob/main/mergekit/merge_methods/slerp.py) of [timpal0l/Mistral-7B-v0.1-flashback-v2/](https://huggingface.co/timpal0l/Mistral-7B-v0.1-flashback-v2) and [RJuro/munin-neuralbeagle-7b](https://huggingface.co/RJuro/munin-neuralbeagle-7b).
21
  ## config.yaml
22
  ```yaml
23
+ models:
24
+ - model: timpal0l/Mistral-7B-v0.1-flashback-v2
25
+ # No parameters necessary for base model
26
+ - model: RJuro/munin-neuralbeagle-7b
27
+ parameters:
28
+ density: 0.53
29
+ weight: 0.6
30
+ merge_method: dare_ties
31
  base_model: timpal0l/Mistral-7B-v0.1-flashback-v2
32
  parameters:
33
+ int8_mask: true
 
 
 
 
 
34
  dtype: bfloat16
35
  ```