jeffmeloy lbourdois commited on
Commit
a1b4a5f
·
verified ·
1 Parent(s): b681026

Improve language tag (#1)

Browse files

- Improve language tag (48301f8a90301625be65780dabe166826ebe2a31)


Co-authored-by: Loïck BOURDOIS <lbourdois@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +38 -26
README.md CHANGED
@@ -1,27 +1,39 @@
1
- ---
2
- license: apache-2.0
3
- base_model:
4
- - Qwen/Qwen2.5-7B
5
- pipeline_tag: text-generation
6
- language:
7
- - en
8
- library_name: transformers
9
- tags:
10
- - text-generation-inference
11
- ---
12
-
13
- ## Model Description
14
-
15
- Optimized Layer Merging (OLM)
16
- Is a transformer optimization framework implementing automated layer recombination.
17
-
18
- Olm create Frankenstein's monster out of language models by cherry-picking the best performing layers across different models to create a superior hybrid.
19
- The core mechanism:
20
-
21
- - Takes multiple language models as input
22
- - Uses a base model as the foundation
23
- - Iteratively replaces individual layers, evaluating performance on specified datasets
24
- - Keeps the best performing layer at each position based on metrics like perplexity, exact match, and a custom "quality" score
25
- - Builds a fusion model layer-by-layer while maintaining or improving performance
26
-
 
 
 
 
 
 
 
 
 
 
 
 
27
  https://github.com/jeffmeloy/olm
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen2.5-7B
5
+ pipeline_tag: text-generation
6
+ language:
7
+ - zho
8
+ - eng
9
+ - fra
10
+ - spa
11
+ - por
12
+ - deu
13
+ - ita
14
+ - rus
15
+ - jpn
16
+ - kor
17
+ - vie
18
+ - tha
19
+ - ara
20
+ library_name: transformers
21
+ tags:
22
+ - text-generation-inference
23
+ ---
24
+
25
+ ## Model Description
26
+
27
+ Optimized Layer Merging (OLM)
28
+ Is a transformer optimization framework implementing automated layer recombination.
29
+
30
+ Olm create Frankenstein's monster out of language models by cherry-picking the best performing layers across different models to create a superior hybrid.
31
+ The core mechanism:
32
+
33
+ - Takes multiple language models as input
34
+ - Uses a base model as the foundation
35
+ - Iteratively replaces individual layers, evaluating performance on specified datasets
36
+ - Keeps the best performing layer at each position based on metrics like perplexity, exact match, and a custom "quality" score
37
+ - Builds a fusion model layer-by-layer while maintaining or improving performance
38
+
39
  https://github.com/jeffmeloy/olm