autoevaluator
HF Staff
Add evaluation results on the default config and test split of gigaword
fb8c7a9
| language: | |
| - en | |
| license: mit | |
| tags: | |
| - summarization | |
| model-index: | |
| - name: facebook/bart-large-xsum | |
| results: | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: cnn_dailymail | |
| type: cnn_dailymail | |
| config: 3.0.0 | |
| split: test | |
| metrics: | |
| - type: rouge | |
| value: 25.2697 | |
| name: ROUGE-1 | |
| verified: true | |
| - type: rouge | |
| value: 7.6638 | |
| name: ROUGE-2 | |
| verified: true | |
| - type: rouge | |
| value: 17.1808 | |
| name: ROUGE-L | |
| verified: true | |
| - type: rouge | |
| value: 21.7933 | |
| name: ROUGE-LSUM | |
| verified: true | |
| - type: loss | |
| value: 3.5042972564697266 | |
| name: loss | |
| verified: true | |
| - type: gen_len | |
| value: 27.4462 | |
| name: gen_len | |
| verified: true | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: xsum | |
| type: xsum | |
| config: default | |
| split: test | |
| metrics: | |
| - type: rouge | |
| value: 45.4525 | |
| name: ROUGE-1 | |
| verified: true | |
| - type: rouge | |
| value: 22.3455 | |
| name: ROUGE-2 | |
| verified: true | |
| - type: rouge | |
| value: 37.2302 | |
| name: ROUGE-L | |
| verified: true | |
| - type: rouge | |
| value: 37.2323 | |
| name: ROUGE-LSUM | |
| verified: true | |
| - type: loss | |
| value: 2.3128726482391357 | |
| name: loss | |
| verified: true | |
| - type: gen_len | |
| value: 25.5435 | |
| name: gen_len | |
| verified: true | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: samsum | |
| type: samsum | |
| config: samsum | |
| split: train | |
| metrics: | |
| - type: rouge | |
| value: 24.7852 | |
| name: ROUGE-1 | |
| verified: true | |
| - type: rouge | |
| value: 5.2533 | |
| name: ROUGE-2 | |
| verified: true | |
| - type: rouge | |
| value: 18.6792 | |
| name: ROUGE-L | |
| verified: true | |
| - type: rouge | |
| value: 20.629 | |
| name: ROUGE-LSUM | |
| verified: true | |
| - type: loss | |
| value: 3.746837854385376 | |
| name: loss | |
| verified: true | |
| - type: gen_len | |
| value: 23.1206 | |
| name: gen_len | |
| verified: true | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: samsum | |
| type: samsum | |
| config: samsum | |
| split: test | |
| metrics: | |
| - type: rouge | |
| value: 24.9158 | |
| name: ROUGE-1 | |
| verified: true | |
| - type: rouge | |
| value: 5.5837 | |
| name: ROUGE-2 | |
| verified: true | |
| - type: rouge | |
| value: 18.8935 | |
| name: ROUGE-L | |
| verified: true | |
| - type: rouge | |
| value: 20.76 | |
| name: ROUGE-LSUM | |
| verified: true | |
| - type: loss | |
| value: 3.775235891342163 | |
| name: loss | |
| verified: true | |
| - type: gen_len | |
| value: 23.0928 | |
| name: gen_len | |
| verified: true | |
| - task: | |
| type: summarization | |
| name: Summarization | |
| dataset: | |
| name: gigaword | |
| type: gigaword | |
| config: default | |
| split: test | |
| metrics: | |
| - type: rouge | |
| value: 16.579 | |
| name: ROUGE-1 | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDFkOTg5ZTM4MDk3NGY4NTcxYzgyY2IxMzlkYmQ0OGUxNzM2YzBhMTI3YjZhMjAzMzZjM2FlZTJmOGM0NmNhMCIsInZlcnNpb24iOjF9.v29vzy3WA75dWDALKoo_pnByEd1kJUQb1TICtczTMvpFmmZBUj1JMbEAX9DMwQa8gq30hbIAvnXa7B0w33oKDQ | |
| - type: rouge | |
| value: 3.7036 | |
| name: ROUGE-2 | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjMxYjM2MGUwZTI4OGJiZTMzMzY2OWNhNGIyM2NkOTk5ODI3OGRhZGRmMjIyZGNkMzg2MTI3OTg0NjgzMWRkNCIsInZlcnNpb24iOjF9.QWCiactfiS4V_rdnjpt3MJUciVpT7VlMRzGmFJ2PxioadewS2kApbYqu6CCilqC9hfVfsLg-VxrH-Os16Xp2Cg | |
| - type: rouge | |
| value: 14.3642 | |
| name: ROUGE-L | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjJjNDgxOTYwOGVmMWYxN2U5ZmM3MDg4NmYwMDllMjgzNjI2OTUxZDQyMzQ5NmY5ZWIwMjlkODdhZDcyNjIzMCIsInZlcnNpb24iOjF9.HbWvD3wc7PRAyT87I-5do9GU0cCXQuEIBDAVSDdGEKXfY-LQ_HBI87vl46z7okCPB7xqAQ-jBsLFmjekw-DSAA | |
| - type: rouge | |
| value: 14.3846 | |
| name: ROUGE-LSUM | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzBmNjNhZmMyNDY0N2Y4OTJmYTk0NWEwYjJjZDJiODMzMmUwNzhmMjVlYzkxMTg1YWYyZThlODY4MjE4NTAwYSIsInZlcnNpb24iOjF9.EZkXXT058YsPzxr-_k-asvYgBaf5IjJyuUzn_IrMEvqW6khWxq8MnnWeGkVPqhVq7hunVvrEYGo2N5ZJbCYOCQ | |
| - type: loss | |
| value: 6.708169460296631 | |
| name: loss | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGM3NmJjNmE4MWM3MGFjM2YzMDRjNGY1MzhjYWI0ZWVhNDQ2ZDA3ODVhZTVhZjRjZjFmMDMzNjZjMDRmMmQxNyIsInZlcnNpb24iOjF9.0IiFBEZWGIkz6FUNbxFwDsPivj0Vdzd0txS_5u0zraLxiML-rnitSWOip-WYbNRO9Gmllu3nRzeKBFFjkOj3DQ | |
| - type: gen_len | |
| value: 23.694 | |
| name: gen_len | |
| verified: true | |
| verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTc2MzlmNjE5ZmVkYTVmZGM5NzFhNDNkODcwOGUyNTFkOTkyOTNmMDNlNjU0YTlmYWMyNzljMDFhZTQzZmZiNSIsInZlcnNpb24iOjF9.MbmvTN8XJuMVhd4JJQdKpnijOKaS2OA2_cjO53777newl0EOHJtTF0CMS3sOjIVt7lrJUEr2JcfY7n8BPxDaAg | |
| ### Bart model finetuned on xsum | |
| docs: https://huggingface.co/transformers/model_doc/bart.html | |
| finetuning: examples/seq2seq/ (as of Aug 20, 2020) | |
| Metrics: ROUGE > 22 on xsum. | |
| variants: search for distilbart | |
| paper: https://arxiv.org/abs/1910.13461 |