nightner/roberta2roberta_financial_lora_v1

README.md

@@ -18,13 +18,14 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/roberta2roberta_L-24_cnn_daily_mail](https://huggingface.co/google/roberta2roberta_L-24_cnn_daily_mail) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 3.7671
-- Rouge1: 46.04
-- Rouge2: 24.94
-- Rougel: 33.78
-- Bertscore P: 87.25
-- Bertscore R: 85.03
-- Bertscore F1: 86.02
+- Loss: 2.3415
+- Rouge1: 39.24
+- Rouge2: 20.62
+- Rougel: 29.42
+- Rougelsum: 29.35
+- Bertscore P: 84.82
+- Bertscore R: 85.16
+- Bertscore F1: 84.9
 
 ## Model description
 
@@ -44,26 +45,22 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 3e-05
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 1
+- eval_batch_size: 1
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- total_train_batch_size: 4
 - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 5
+- num_epochs: 2
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Bertscore P | Bertscore R | Bertscore F1 |
-|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:-----------:|:-----------:|:------------:|
-| No log        | 0.8   | 10   | 4.1610          | 43.72  | 22.28  | 31.23  | 86.78       | 84.54       | 85.57        |
-| No log        | 1.56  | 20   | 4.0149          | 45.37  | 23.55  | 32.49  | 86.84       | 84.74       | 85.69        |
-| No log        | 2.32  | 30   | 3.8893          | 45.05  | 24.14  | 32.83  | 87.15       | 84.71       | 85.82        |
-| No log        | 3.08  | 40   | 3.8113          | 45.1   | 24.05  | 32.37  | 87.11       | 85.08       | 85.98        |
-| 18.8832       | 3.88  | 50   | 3.7732          | 45.98  | 24.85  | 33.82  | 87.13       | 85.03       | 85.96        |
-| 18.8832       | 4.64  | 60   | 3.7671          | 46.04  | 24.94  | 33.78  | 87.25       | 85.03       | 86.02        |
+| Training Loss | Epoch  | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Bertscore P | Bertscore R | Bertscore F1 |
+|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-----------:|:-----------:|:------------:|
+| 13.8236       | 0.7251 | 300  | 2.8115          | 36.62  | 19.66  | 27.32  | 27.25     | 83.21       | 84.38       | 83.62        |
+| 10.8258       | 1.4495 | 600  | 2.3415          | 39.24  | 20.62  | 29.42  | 29.35     | 84.82       | 85.16       | 84.9         |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a8d59f86d472fff550241f97e76b627a0bb06b176e8abbd4eedb26b1dbd72995
+oid sha256:76885af14102f0fcc29d0c8dbb0b27b2e1d39760a59551a69fd98a1063d65526
 size 12611432

runs/Feb17_10-46-41_0bfd00035eae/events.out.tfevents.1739789224.0bfd00035eae.2888.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f71f0768f9f17e357b1a735e453581a26d0ef718876ea28cfe6be0e15e13937e
+size 12283

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9be7a562e9528b9f6ee315e6d75d62f3ea6982b49644339a058467afe79ae474
+oid sha256:3e16beb67675cca3530aabca05d390cc84b64ffac7516c9386be2263bca9606c
 size 5432