Hwanjun commited on
Commit
3b9d639
·
verified ·
1 Parent(s): cd074d7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -2
README.md CHANGED
@@ -1,3 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
- {}
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <div align="center">
2
+ <b style="font-size: 40px;">SummLlama3-8B</b>
3
+ </div>
4
+
5
+ Are you looking for a summarizer that outperforms the nearly 10x larger Llama3-70B-Instruct while offering much faster inference speed?
6
+
7
+ Our **SummLlama3-8B** could be exactly what you need!
8
+
9
+ SummLlama3 is initialized from Llama3-8B-Instruct, with additional training using Direct Preference Optimization (DPO) based on human-like summarization feedback.
10
+
11
+ Please refer to [our paper](link) to catch up how to exploit LLM-generated feedback in the context of text summarization.
12
+
13
+ We also released a larger model, **SummLlama3-70B**. Please go to the [Huggingface link](link) for this model.
14
+
15
  ---
16
+
17
+ Here is a brief overview of our summarizer:
18
+
19
+ Rather than relying on expensive human feedback, we utilize high-quality, multi-dimensional, and fine-grained feedback generated by large language models (LLMs).
20
+
21
+ This model excels at **faithfulness**, **completeness**, and **conciseness**, which are the three human-preferred aspects to judge what is a good summarizer.
22
+
23
+ - Faithfulness: a summarizer does not manipulate the information in the input text and add any information not directly inferable from the input text.
24
+ - Completeness: a summarizer ensures the inclusion of all key information from the input text in the output summary.
25
+ - Conciseness: a summarizer refrains from incorporating information outside the key information in the output, maintaining a succinct and focused summary.
26
+
27
+ Based on our comprehensive evaluation, which included both human and automated assessments of summary quality, SummLlama3 demonstrated significant improvements over the original Llama3 series.
28
+
29
+ Here is the results:
30
+
31
+ ## Human Evaluation
32
+
33
+
34
+
35
+ ## Autoamted Evaluation using [FineSurE](link)
36
+
37
+
38
+
39
+
40
+