Add evaluation results on the section config and train split of ccdv/pubmed-summarization

131ff77 over 2 years ago

4.24 kB

license: apache-2.0
tags:
  - summarization
  - generated_from_trainer
datasets:
  - cnn_dailymail
model-index:
  - name: bart-base-finetuned-summarization-cnn-ver3
    results:
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: ccdv/pubmed-summarization
          type: ccdv/pubmed-summarization
          config: section
          split: train
        metrics:
          - type: rouge
            value: 7.4825
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMmMyZDE4ZDFiNjA2ZTBkMTQzMDFiZWFlYjQ4NWI3MWM2MTVmNWM2MmQ2YmYxNGUyMzI1MjcxZWI1OTExYzI4YiIsInZlcnNpb24iOjF9.vehISM8OOOdyb0ZkN8udebOes-YRnUqyV6D6ctQUtCaEXxGjFQQgNLyJJmI7eU28Oum55TRpH82zA4lU9YX6Dg
          - type: rouge
            value: 2.19
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWYxY2ZiNTdjZTRlYTYwM2Q0MmFmMWZkZDJlZTgxODgxMWQ5ZGRiOThkMzNkZjJlNGYyZjMwOTBkNGRlNWYxOCIsInZlcnNpb24iOjF9.VrB5STHbL8ArFnlfloSMzY9oJLRKYmZgoMxrrujRt17hb1H_KKC0l7VG7k1Alja2N2nj8-8WcsVP49fJGS7JAQ
          - type: rouge
            value: 6.2296
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODQ4OTE5MTk0NzI0OTk3MzFhYTNjMmU5ZjA3N2YyOWQ5ODM4MmE5NzE1MzM3NTI3NGM4ODE4Mjk4OGE3YzI1NSIsInZlcnNpb24iOjF9.FJp0BMVhWPMMmME68tHTMNIisB-COWMrfrZAPj9V82lG1fFwxu69bbfqGGONCdpJOZHYazxi5X6aoL2mB3uyAw
          - type: rouge
            value: 6.915
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNjk2YTY5NTQ5M2YzMmViMDQ3MTllZjM1MjkyZTUwOGJmM2Q3ZDU2ZDU5NDZlMTJmMThjZjM0ODJlZmYzNmU1MiIsInZlcnNpb24iOjF9.CNm77jjHRr9eDgGUhUe9S-KoivV8KcVHIo_4rclEsX8JxbhwXhd06gF7gvlkUyedAqyodS0QEJzQtAiQ7txMAA
          - type: loss
            value: 6.158000469207764
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNWFlYWQ1Njk1YTdlNGY3YTgxYzY1Mzc1NzcwYjQyZjkxNTc5ZWViN2FjYjk1MDliMDgzODE1MDgyMzkzMWM3ZSIsInZlcnNpb24iOjF9.eVB-DwOcGg7-y7QG1mNOiU45b1SS39kesQPzU_rzpYknnlFK7Z_AMU9mNj87V2Z2q63VnyIH0uRCM-ijghziAw
          - type: gen_len
            value: 20
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZjMjhjOGUzMWI5N2Q3MWNlNjU2NjM1MDliYWE0YTFlMjM0YTcxYzBhNmVjNjM2YzJlZTdkYWEwMjFlYjY4OSIsInZlcnNpb24iOjF9.7ZPCbgPyUL7MamdNtxIhxditULp3ob6HgYyLMUnsd-xeZ08EPcSwVyM4wgN-REJga4nlNlc94LWzF296K6bnBA

bart-base-finetuned-summarization-cnn-ver3

This model is a fine-tuned version of facebook/bart-base on the cnn_dailymail dataset. It achieves the following results on the evaluation set:

Loss: 2.9827
Bertscore-mean-precision: 0.8811
Bertscore-mean-recall: 0.8554
Bertscore-mean-f1: 0.8679
Bertscore-median-precision: 0.8809
Bertscore-median-recall: 0.8545
Bertscore-median-f1: 0.8669

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 1
eval_batch_size: 1
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Bertscore-mean-precision	Bertscore-mean-recall	Bertscore-mean-f1	Bertscore-median-precision	Bertscore-median-recall	Bertscore-median-f1
3.632	1.0	5742	2.9827	0.8811	0.8554	0.8679	0.8809	0.8545	0.8669

Framework versions

Transformers 4.24.0
Pytorch 1.12.1+cu113
Datasets 2.7.1
Tokenizers 0.13.2