Text Generation
PyTorch
English
llama
t1101675 commited on
Commit
67d5182
·
verified ·
1 Parent(s): a782f64

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -3
README.md CHANGED
@@ -1,3 +1,32 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - databricks/databricks-dolly-15k
5
+ language:
6
+ - en
7
+ metrics:
8
+ - rouge
9
+ pipeline_tag: text-generation
10
+ ---
11
+ # SFT-Llama-7B
12
+
13
+ [paper](https://arxiv.org/abs/2306.08543) | [code](https://github.com/microsoft/LMOps/tree/main/minillm)
14
+
15
+ **SFT-Llama-7B** is an Llama-7B model supervised fine-tuned on [databricks-dolly-15k](https://huggingface.co/datasets/aisquared/databricks-dolly-15k).
16
+
17
+ It is used as a baseline for [MiniLLM](https://huggingface.co/MiniLLM/MiniLLM-Llama-7B).
18
+
19
+ ## Other Baselines
20
+ + [KD](https://huggingface.co/MiniLLM/KD-Llama-7B)
21
+ + [SeqKD](https://huggingface.co/MiniLLM/SeqKD-Llama-7B)
22
+
23
+
24
+ ## Citation
25
+ ```
26
+ @inproceedings{minillm,
27
+ title={MiniLLM: Knowledge Distillation of Large Language Models},
28
+ author={Gu, Yuxian and Dong, Li and Wei, Furu and Huang, Minlie},
29
+ booktitle={Proceedings of ICLR},
30
+ year={2024}
31
+ }
32
+ ```