haining
/

scientific_abstract_simplification

text2text-generation

text2text generation

text-generation-inference

Model card Files Files and versions

haining commited on Dec 16, 2022

Commit

7203ecd

·

1 Parent(s): 3ef0559

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -121,7 +121,7 @@ We finetuned the base model (flan-t5-large) on multiple relevant tasks with stan
 | Recontextualization                | Editor Abstract                  | "contextualize: "                          | 2,200           |
 | Simplification                     | Wiki Auto                        | "simplify: "                               | 57,000          |
 | Summarization                      | CNN/DailyMail                    | "summarize: "                              | 165,000         |
-| Total                              | Challenge-proportional Mixture   | n/a                                        | 263,400         |
 - Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.

 | Recontextualization                | Editor Abstract                  | "contextualize: "                          | 2,200           |
 | Simplification                     | Wiki Auto                        | "simplify: "                               | 57,000          |
 | Summarization                      | CNN/DailyMail                    | "summarize: "                              | 165,000         |
+| Total                              | Challenge-proportional Mixing    | n/a                                        | 263,400         |
 - Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.