Hugging Face's logo

Spaces:
FuseAI
/
README
Running

App Files Community
1
Wanfq commited on
Commit
3de0c5f
·
verified ·
1 Parent(s): 758f7bd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +94 -1
README.md CHANGED
@@ -7,4 +7,97 @@ sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ <p align="center" width="100%">
11
+ </p>
12
+
13
+ <div id="top" align="center">
14
+
15
+ **Knowledge Fusion of Large Language Models**
16
+
17
+
18
+ <h4> |<a href="https://arxiv.org/abs/2401.10491"> 📑 FuseLLM Paper @ICLR2024 </a> |
19
+ <a href="https://arxiv.org/abs/2401.10491"> 📑 FuseChat Tech Report </a> |
20
+ <a href="https://huggingface.co/FuseAI"> 🤗 Huggingface Repo </a> |
21
+ <a href="https://github.com/fanqiwan/FuseLLM"> 🐱 Github Repo </a> |
22
+ </h4>
23
+
24
+ <p align="center">
25
+ <img src="https://github.com/18907305772/FuseLLM/blob/main/assets/logo.png" width="95%"> <br>
26
+ </p>
27
+
28
+ </div>
29
+
30
+
31
+ ## News
32
+
33
+ ### FuseChat
34
+
35
+ - **Feb 26, 2024:** 🔥🔥 We release [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM), which is the fusion of three prominent chat LLMs with diverse architectures and scales, namely [NH2-Mixtral-8x7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO), [NH2-Solar-10.7B](https://huggingface.co/NousResearch/Nous-Hermes-2-SOLAR-10.7B), and [OpenChat-3.5-7B](https://huggingface.co/openchat/openchat_3.5). FuseChat-7B-VaRM achieves an average performance of **8.22** on MT-Bench, outperforming various powerful chat LLMs like [Starling-7B](https://huggingface.co/berkeley-nest/Starling-LM-7B-alpha), [Yi-34B-Chat](https://huggingface.co/01-ai/Yi-34B-Chat), and [Tulu-2-DPO-70B](https://huggingface.co/allenai/tulu-2-dpo-70b), even surpassing [GPT-3.5 (March)](https://platform.openai.com/docs/models/gpt-3-5-turbo), [Claude-2.1](https://www.anthropic.com/news/claude-2-1), and approaching [Mixtral-8x7B-Instruct](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
36
+
37
+ <p align="center">
38
+ <img src="https://github.com/18907305772/FuseLLM/blob/main/FuseChat/assets/fig_0.png" width="70%"> <br>
39
+ </p>
40
+
41
+ | Priority Models | #Params | MT-Bench | Open Source Models | #Params | MT-Bench |
42
+ |-----------------------------------------------------------------------|---------|----------|-----------------------------------------------------------------------|---------|----------|
43
+ | GPT-4-1106-preview | - | 9.32 | Qwen1.5-72B-Chat | 72B | 8.61 |
44
+ | GPT-4-0613 | - | 9.18 | Nous-Hermes-2-Mixtral-8x7B-DPO | 8x7B | 8.33 |
45
+ | GPT-4-0314 | - | 8.96 | Mixtral-8x7B-Instruct-v0.1 | 8x7B | 8.30 |
46
+ | Mistral Medium | - | 8.61 | 🤗 [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM) | 7B | 8.22 |
47
+ | GPT-3.5-Turbo-0613 | - | 8.39 | Starling-LM-7B-alpha | 7B | 8.09 |
48
+ | GPT-3.5-Turbo-1106 | - | 8.32 | Tulu-2-DPO-70B | 70B | 7.89 |
49
+ | 🤗 [FuseChat-7B-VaRM](https://huggingface.co/FuseAI/FuseChat-7B-VaRM) | 7B | 8.22 | OpenChat-3.5 | 7B | 7.81 |
50
+ | Claude-2.1 | - | 8.18 | OpenChat-3.5-0106 | 7B | 7.80 |
51
+ | Claude-2.0 | - | 8.06 | WizardLM-70B-v1.0 | 70B | 7.71 |
52
+ | GPT-3.5-Turbo-0314 | - | 7.94 | Yi-34B-Chat | 34B | 7.67 |
53
+ | Claude-1 | - | 7.90 | Nous-Hermes-2-SOLAR-10.7B | 10.7B | 7.66 |
54
+
55
+ ### FuseLLM
56
+
57
+ - **Jan 22, 2024:** 🔥 We release [FuseLLM-7B](https://huggingface.co/Wanfq/FuseLLM-7B), which is the fusion of three open-source foundation LLMs with distinct architectures, including [Llama-2-7B](https://huggingface.co/meta-llama/Llama-2-7b-hf), [OpenLLaMA-7B](https://huggingface.co/openlm-research/open_llama_7b_v2), and [MPT-7B](https://huggingface.co/mosaicml/mpt-7b).
58
+
59
+ | Model | BBH | ARC-easy | ARC-challenge | BoolQ | HellaSwag | OpenBookQA |
60
+ |----------------------------------------------------------|-------|----------|---------------|-------|-----------|------------|
61
+ | OpenLLaMA-7B | 33.87 | 69.70 | 41.38 | 72.29 | 74.53 | 41.00 |
62
+ | MPT-7B | 33.38 | 70.12 | 42.15 | 74.74 | 76.25 | 42.40 |
63
+ | Llama-2-7B | 39.70 | 74.58 | 46.33 | 77.71 | 76.00 | 44.20 |
64
+ | Llama-2-CLM-7B | 40.44 | 74.54 | 46.50 | 76.88 | 76.57 | 44.80 |
65
+ | 🤗 [FuseLLM-7B](https://huggingface.co/Wanfq/FuseLLM-7B) | 41.75 | 75.04 | 47.44 | 78.13 | 76.78 | 45.40 |
66
+
67
+
68
+ | Model | MultiPL-E | TrivialQA | DROP | LAMBADA | IWSLT2017 | SciBench |
69
+ |----------------------------------------------------------|-----------|-----------|-------|---------|-----------|----------|
70
+ | OpenLLaMA-7B | 18.11 | 39.96 | 22.31 | 70.31 | 5.51 | 0.68 |
71
+ | MPT-7B | 17.26 | 28.89 | 23.54 | 70.08 | 5.49 | 0.88 |
72
+ | Llama-2-7B | 14.63 | 52.46 | 27.25 | 73.28 | 6.48 | 0.14 |
73
+ | Llama-2-CLM-7B | 14.83 | 53.14 | 28.51 | 73.45 | 6.91 | 0.94 |
74
+ | 🤗 [FuseLLM-7B](https://huggingface.co/Wanfq/FuseLLM-7B) | 15.56 | 54.49 | 28.97 | 73.72 | 6.75 | 1.65 |
75
+
76
+
77
+ ## Citation
78
+
79
+ Please cite the paper if you refer to our model or code or data or paper from FuseLLM.
80
+ ```
81
+ @inproceedings{wan2024knowledge,
82
+ title={Knowledge Fusion of Large Language Models},
83
+ author={Fanqi Wan and Xinting Huang and Deng Cai and Xiaojun Quan and Wei Bi and Shuming Shi},
84
+ booktitle={The Twelfth International Conference on Learning Representations},
85
+ year={2024},
86
+ url={https://openreview.net/pdf?id=jiDsk12qcz}
87
+ }
88
+ ```
89
+
90
+ Please cite the paper if you refer to our model or code or data or paper from FuseChat.
91
+ ```
92
+ @article{wan2024fusechat,
93
+ title={FuseChat: Knowledge Fusion of Chat Models},
94
+ author={Fanqi Wan and Ziyi Yang and Longguang Zhong and Xiaojun Quan and Xinting Huang and Wei Bi},
95
+ journal={arXiv preprint arXiv:2401.10491},
96
+ year={2024}
97
+ }
98
+ ```
99
+
100
+ ## Star History
101
+
102
+ [![Star History Chart](https://api.star-history.com/svg?repos=fanqiwan/FuseLLM&type=Timeline)](https://star-history.com/#fanqiwan/FuseLLM&Timeline)
103
+