Muhammad2003 commited on
Commit
5404f45
·
verified ·
1 Parent(s): 375444c

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1 -48
README.md CHANGED
@@ -1,48 +1 @@
1
- ---
2
- base_model:
3
- - aaditya/OpenBioLLM-Llama3-70B
4
- - meta-llama/Meta-Llama-3-70B-Instruct
5
- library_name: transformers
6
- tags:
7
- - mergekit
8
- - merge
9
-
10
- ---
11
- # merge
12
-
13
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
-
15
- ## Merge Details
16
- ### Merge Method
17
-
18
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [aaditya/OpenBioLLM-Llama3-70B](https://huggingface.co/aaditya/OpenBioLLM-Llama3-70B) as a base.
19
-
20
- ### Models Merged
21
-
22
- The following models were included in the merge:
23
- * [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct)
24
-
25
- ### Configuration
26
-
27
- The following YAML configuration was used to produce this model:
28
-
29
- ```yaml
30
-
31
- models:
32
- - model: aaditya/OpenBioLLM-Llama3-70B
33
- parameters:
34
- density: 0.53
35
- weight: 0.4
36
- - model: meta-llama/Meta-Llama-3-70B-Instruct
37
- parameters:
38
- density: 0.53
39
- weight: 0.3
40
-
41
- merge_method: dare_ties
42
- base_model: aaditya/OpenBioLLM-Llama3-70B
43
- parameters:
44
- int8_mask: true
45
- dtype: bfloat16
46
-
47
-
48
- ```
 
1
+ ## LlamaMed-70B