UDA-LIDI/barto_emergency_keys_summarize

bart_es_keys

This model is a fine-tuned version of vgaraujov/bart-base-spanish. It achieves the following results on the evaluation set:

• Loss: 1.3818
• Rouge1: 63.6736
• Rouge2: 37.301
• Rougel: 63.0481
• Rougelsum: 62.97
• Gen Len: 7.6

Model description

This checkpoint extracts keywords or context from emergency transcribed calls. Add the prefix "summarize: " before a test text to see the checkpoint's responses.

Intended uses & limitations

Under privacy agreement.

Training and evaluation data

Training data used has been provided by the ECU 911 service under a strict confidentiality agreement.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.4822	1.0	50	1.5713	51.0101	26.2004	50.4041	50.4791	7.33
1.5446	2.0	100	1.4584	57.2195	31.0281	56.4802	56.4801	7.75
1.2388	3.0	150	1.3971	61.0085	34.3564	60.1558	60.1153	7.84
1.0489	4.0	200	1.3611	62.0307	35.323	61.3034	61.1902	7.53
0.8992	5.0	250	1.3973	62.8046	37.3484	62.2618	62.2231	7.845
0.8357	6.0	300	1.3836	63.2165	36.9899	62.7019	62.5911	7.825
0.7731	7.0	350	1.3818	63.6736	37.301	63.0481	62.97	7.6

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0