jpcorb20 commited on
Commit
25bb0fe
·
verified ·
1 Parent(s): 5808a44

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -72
README.md DELETED
@@ -1,72 +0,0 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # bc_5_checkpoints_mediphi3_5
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the breadcrumbs_ties merge method using ./Phi-3.5-mini-instruct/ as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * ./guideline_mediphi3_5/
22
- * ./clinical_mediphi3_5/
23
- * ./medcode_mediphi3_5/
24
- * ./pmc_mediphi3_5/
25
- * ./medwiki_mediphi3_5/
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- base_model: ./Phi-3.5-mini-instruct/
33
- dtype: bfloat16
34
- merge_method: breadcrumbs_ties
35
- parameters:
36
- int8_mask: 1.0
37
- normalize: 0.0
38
- slices:
39
- - sources:
40
- - layer_range: [0, 32]
41
- model: ./pmc_mediphi3_5/
42
- parameters:
43
- density: 0.493602636300503
44
- gamma: 1.6720241134317
45
- weight: 0.06352674368303199
46
- - layer_range: [0, 32]
47
- model: ./clinical_mediphi3_5/
48
- parameters:
49
- density: 0.13945100550914405
50
- gamma: 0.6330106659403112
51
- weight: 0.5538401962393489
52
- - layer_range: [0, 32]
53
- model: ./medcode_mediphi3_5/
54
- parameters:
55
- density: 0.9265424817500185
56
- gamma: 1.9263067682230453
57
- weight: 0.2591472607914765
58
- - layer_range: [0, 32]
59
- model: ./medwiki_mediphi3_5/
60
- parameters:
61
- density: 0.22471877839197088
62
- gamma: 0.8906459105787018
63
- weight: 0.017904019341096117
64
- - layer_range: [0, 32]
65
- model: ./guideline_mediphi3_5/
66
- parameters:
67
- density: 0.766362712622725
68
- gamma: 0.2721241668046967
69
- weight: 1.604598536238056
70
- - layer_range: [0, 32]
71
- model: ./Phi-3.5-mini-instruct/
72
- ```