Hugging Face's logo

GiantTreeG
/
german-jeopardy-longt5-base

Transformers
TensorBoard
Safetensors
German
longt5
text2text-generation
question-generation
german
Generated from Trainer
Eval Results (legacy)
Model card Files
xet
Metrics Community
Marvin commited on
Commit
c80d287
·
unverified ·
0 Parent(s):

Initial commit

Browse files
Files changed (15) hide show
  1. .gitattributes +4 -0
  2. README.md +130 -0
  3. added_tokens.json +3 -0
  4. all_results.json +64 -0
  5. config.json +38 -0
  6. generation_config.json +7 -0
  7. model.safetensors +3 -0
  8. runs/Oct20_15-49-15_MARVIN-PC/events.out.tfevents.1697809770.MARVIN-PC.5688.0 +3 -0
  9. runs/Oct20_15-49-15_MARVIN-PC/events.out.tfevents.1697835801.MARVIN-PC.5688.1 +3 -0
  10. special_tokens_map.json +14 -0
  11. spiece.model +3 -0
  12. tokenizer.json +3 -0
  13. tokenizer_config.json +843 -0
  14. trainer_state.json +768 -0
  15. training_args.bin +0 -0
.gitattributes ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
2
+ *.model filter=lfs diff=lfs merge=lfs -text
3
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
4
+ runs/** filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - de
4
+ tags:
5
+ - question-generation
6
+ - german
7
+ - text2text-generation
8
+ - generated_from_trainer
9
+ datasets:
10
+ - lmqg/qg_dequad
11
+ metrics:
12
+ - bleu4
13
+ - f1
14
+ - rouge
15
+ - exact_match
16
+ model-index:
17
+ - name: german-jeopardy-longt5-base
18
+ results:
19
+ - task:
20
+ name: Sequence-to-sequence Language Modeling
21
+ type: text2text-generation
22
+ dataset:
23
+ name: lmqg/qg_dequad
24
+ type: default
25
+ args: default
26
+ metrics:
27
+ - name: BLEU-4
28
+ type: bleu4
29
+ value: 10.80
30
+ - name: F1
31
+ type: f1
32
+ value: 34.41
33
+ - name: ROUGE-1
34
+ type: rouge1
35
+ value: 35.31
36
+ - name: ROUGE-2
37
+ type: rouge2
38
+ value: 16.35
39
+ - name: ROUGE-L
40
+ type: rougel
41
+ value: 33.91
42
+ - name: ROUGE-Lsum
43
+ type: rougelsum
44
+ value: 33.96
45
+ - name: Exact Match
46
+ type: exact_match
47
+ value: 1.36
48
+ ---
49
+
50
+
51
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
52
+ should probably proofread and complete it, then remove this comment. -->
53
+
54
+ # german-jeopardy-longt5-base
55
+
56
+ This model is a fine-tuned version of [google/long-t5-tglobal-base](https://huggingface.co/google/long-t5-tglobal-base) on the [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) dataset.
57
+ It achieves the following results on the evaluation set:
58
+ - Loss: 1.8533
59
+ - Brevity Penalty: 0.8910
60
+ - System Length: 18642
61
+ - Reference Length: 20793
62
+ - ROUGE-1: 35.31
63
+ - ROUGE-2: 16.35
64
+ - ROUGE-L: 33.91
65
+ - ROUGE-Lsum: 33.96
66
+ - Exact Match: 1.36
67
+ - BLEU: 10.80
68
+ - F1: 34.41
69
+
70
+ ## Model description
71
+
72
+ See [google/long-t5-tglobal-base](https://huggingface.co/google/long-t5-tglobal-base) for more information about the
73
+ model architecture.
74
+ The model was trained on a single NVIDIA RTX 3090 GPU with 24GB of VRAM.
75
+
76
+ ## Intended uses & limitations
77
+
78
+ This model can be used for question generation on German text.
79
+
80
+ ## Training and evaluation data
81
+
82
+ See [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad).
83
+
84
+ ## Training procedure
85
+
86
+ ### Training hyperparameters
87
+
88
+ The following hyperparameters were used during training:
89
+ - learning_rate: 0.0001
90
+ - train_batch_size: 8
91
+ - eval_batch_size: 4
92
+ - seed: 7
93
+ - gradient_accumulation_steps: 8
94
+ - total_train_batch_size: 64
95
+ - optimizer: Adafactor
96
+ - lr_scheduler_type: constant
97
+ - num_epochs: 20
98
+
99
+ ### Training results
100
+
101
+ | Training Loss | Epoch | Step | BLEU | Brevity Penalty | Counts 1 | Counts 2 | Counts 3 | Counts 4 | Exact Match | F1 | Gen Len | Validation Loss | Precisions 1 | Precisions 2 | Precisions 3 | Precisions 4 | Reference Length | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Lsum | System Length | Totals 1 | Totals 2 | Totals 3 | Totals 4 |
102
+ |:-------------:|:-----:|:----:|:-------:|:---------------:|:--------:|:--------:|:--------:|:--------:|:-----------:|:------:|:-------:|:---------------:|:------------:|:------------:|:------------:|:------------:|:----------------:|:-------:|:-------:|:-------:|:----------:|:-------------:|:--------:|:--------:|:--------:|:--------:|
103
+ | 3.1671 | 1.0 | 145 | 5.9441 | 0.7156 | 6177 | 1669 | 604 | 179 | 0.0023 | 0.2528 | 12.0218 | 2.1902 | 38.7954 | 12.1665 | 5.2458 | 1.9227 | 21250 | 0.2595 | 0.1035 | 0.2491 | 0.2492 | 15922 | 15922 | 13718 | 11514 | 9310 |
104
+ | 2.5597 | 2.0 | 291 | 7.7787 | 0.7556 | 6785 | 2044 | 804 | 293 | 0.0064 | 0.2864 | 12.6084 | 2.0164 | 40.876 | 14.1994 | 6.595 | 2.9338 | 21250 | 0.2931 | 0.1291 | 0.2817 | 0.2818 | 16599 | 16599 | 14395 | 12191 | 9987 |
105
+ | 2.3464 | 2.99 | 436 | 9.2407 | 0.7935 | 7251 | 2326 | 969 | 400 | 0.0073 | 0.3114 | 13.2296 | 1.9138 | 42.0129 | 15.45 | 7.5403 | 3.7569 | 21250 | 0.3162 | 0.1456 | 0.3031 | 0.3031 | 17259 | 17259 | 15055 | 12851 | 10647 |
106
+ | 2.1679 | 4.0 | 582 | 9.6363 | 0.7795 | 7382 | 2393 | 1006 | 434 | 0.0109 | 0.3226 | 13.1207 | 1.8524 | 43.3903 | 16.1591 | 7.981 | 4.1727 | 21250 | 0.3272 | 0.1504 | 0.3147 | 0.3149 | 17013 | 17013 | 14809 | 12605 | 10401 |
107
+ | 2.0454 | 5.0 | 728 | 10.3812 | 0.7665 | 7581 | 2555 | 1111 | 482 | 0.0132 | 0.3357 | 12.9782 | 1.7997 | 45.1599 | 17.5204 | 8.9749 | 4.7371 | 21250 | 0.3401 | 0.1606 | 0.3278 | 0.3279 | 16787 | 16787 | 14583 | 12379 | 10175 |
108
+ | 1.9502 | 5.99 | 873 | 10.7668 | 0.7992 | 7759 | 2618 | 1162 | 511 | 0.0127 | 0.3406 | 13.4841 | 1.7696 | 44.6973 | 17.2748 | 8.9723 | 4.7548 | 21250 | 0.3452 | 0.1631 | 0.3321 | 0.3319 | 17359 | 17359 | 15155 | 12951 | 10747 |
109
+ | 1.8414 | 7.0 | 1019 | 11.3408 | 0.7721 | 7791 | 2693 | 1236 | 570 | 0.015 | 0.347 | 13.0563 | 1.7472 | 46.147 | 18.3459 | 9.9078 | 5.5496 | 21250 | 0.3513 | 0.1679 | 0.3391 | 0.3391 | 16883 | 16883 | 14679 | 12475 | 10271 |
110
+ | 1.7614 | 8.0 | 1165 | 11.8447 | 0.8198 | 8024 | 2799 | 1296 | 610 | 0.0145 | 0.352 | 13.515 | 1.7203 | 45.2643 | 18.0313 | 9.7305 | 5.4881 | 21250 | 0.3565 | 0.1711 | 0.3422 | 0.3423 | 17727 | 17727 | 15523 | 13319 | 11115 |
111
+ | 1.6997 | 9.0 | 1310 | 11.9689 | 0.8027 | 8046 | 2835 | 1314 | 615 | 0.0168 | 0.3568 | 13.4306 | 1.7167 | 46.183 | 18.6293 | 10.0968 | 5.6892 | 21250 | 0.3613 | 0.1746 | 0.3466 | 0.3466 | 17422 | 17422 | 15218 | 13014 | 10810 |
112
+ | 1.6159 | 10.0 | 1456 | 12.5678 | 0.8182 | 8087 | 2928 | 1395 | 681 | 0.0181 | 0.3564 | 13.5268 | 1.6892 | 45.6944 | 18.8976 | 10.4966 | 6.1429 | 21250 | 0.3612 | 0.1795 | 0.3485 | 0.3482 | 17698 | 17698 | 15494 | 13290 | 11086 |
113
+ | 1.5681 | 10.99 | 1601 | 12.497 | 0.813 | 8154 | 2933 | 1383 | 664 | 0.0168 | 0.3605 | 13.6044 | 1.6923 | 46.3164 | 19.0442 | 10.4797 | 6.0402 | 21250 | 0.3654 | 0.1789 | 0.3506 | 0.3505 | 17605 | 17605 | 15401 | 13197 | 10993 |
114
+ | 1.4987 | 12.0 | 1747 | 12.8959 | 0.8169 | 8295 | 3011 | 1432 | 697 | 0.0181 | 0.3675 | 13.6134 | 1.6825 | 46.928 | 19.461 | 10.7929 | 6.2997 | 21250 | 0.3734 | 0.1846 | 0.3576 | 0.3577 | 17676 | 17676 | 15472 | 13268 | 11064 |
115
+ | 1.4461 | 13.0 | 1893 | 12.8688 | 0.8139 | 8246 | 3005 | 1424 | 700 | 0.0191 | 0.3658 | 13.5812 | 1.6784 | 46.7964 | 19.4915 | 10.7773 | 6.3584 | 21250 | 0.3725 | 0.1857 | 0.358 | 0.3576 | 17621 | 17621 | 15417 | 13213 | 11009 |
116
+ | 1.4002 | 13.99 | 2038 | 13.4526 | 0.8329 | 8457 | 3130 | 1504 | 745 | 0.02 | 0.3727 | 13.9179 | 1.6725 | 47.0749 | 19.8591 | 11.0939 | 6.5621 | 21250 | 0.3797 | 0.1915 | 0.3637 | 0.3634 | 17965 | 17965 | 15761 | 13557 | 11353 |
117
+ | 1.3391 | 15.0 | 2184 | 13.211 | 0.8283 | 8443 | 3091 | 1468 | 719 | 0.0204 | 0.3737 | 13.9133 | 1.6783 | 47.2177 | 19.7168 | 10.8959 | 6.3803 | 21250 | 0.3804 | 0.1901 | 0.3634 | 0.363 | 17881 | 17881 | 15677 | 13473 | 11269 |
118
+ | 1.2921 | 16.0 | 2330 | 13.4907 | 0.8373 | 8457 | 3147 | 1511 | 747 | 0.0195 | 0.3716 | 13.9882 | 1.6738 | 46.8662 | 19.8662 | 11.0801 | 6.5337 | 21250 | 0.3782 | 0.1902 | 0.3624 | 0.3624 | 18045 | 18045 | 15841 | 13637 | 11433 |
119
+ | 1.2572 | 17.0 | 2475 | 13.8581 | 0.8267 | 8473 | 3219 | 1561 | 783 | 0.02 | 0.3753 | 13.7618 | 1.6770 | 47.4598 | 20.57 | 11.6103 | 6.9656 | 21250 | 0.3821 | 0.1948 | 0.3669 | 0.3665 | 17853 | 17853 | 15649 | 13445 | 11241 |
120
+ | 1.199 | 18.0 | 2621 | 13.7496 | 0.8326 | 8484 | 3190 | 1551 | 771 | 0.0186 | 0.3745 | 13.8798 | 1.6934 | 47.2409 | 20.2475 | 11.4456 | 6.7947 | 21250 | 0.3812 | 0.1922 | 0.3657 | 0.3658 | 17959 | 17959 | 15755 | 13551 | 11347 |
121
+ | 1.1668 | 18.99 | 2766 | 13.7379 | 0.8395 | 8504 | 3179 | 1541 | 776 | 0.0204 | 0.376 | 13.9256 | 1.6926 | 47.0198 | 20.0164 | 11.2663 | 6.7631 | 21250 | 0.3828 | 0.1939 | 0.3665 | 0.3665 | 18086 | 18086 | 15882 | 13678 | 11474 |
122
+ | 1.1164 | 19.91 | 2900 | 14.1906 | 0.8529 | 8625 | 3250 | 1609 | 820 | 0.0204 | 0.3803 | 14.069 | 1.7026 | 47.0463 | 20.15 | 11.5548 | 6.996 | 21250 | 0.3874 | 0.1964 | 0.3716 | 0.3715 | 18333 | 18333 | 16129 | 13925 | 11721 |
123
+
124
+
125
+ ### Framework versions
126
+
127
+ - Transformers 4.34.1
128
+ - Pytorch 2.1.0
129
+ - Datasets 2.12.0
130
+ - Tokenizers 0.14.1
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "<hl>": 32100
3
+ }
all_results.json ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 19.91,
3
+ "eval_bleu": 14.1906,
4
+ "eval_bp": 0.8529,
5
+ "eval_counts_1": 8625,
6
+ "eval_counts_2": 3250,
7
+ "eval_counts_3": 1609,
8
+ "eval_counts_4": 820,
9
+ "eval_exact_match": 0.0204,
10
+ "eval_f1": 0.3803,
11
+ "eval_gen_len": 14.069,
12
+ "eval_loss": 1.7026218175888062,
13
+ "eval_precisions_1": 47.0463,
14
+ "eval_precisions_2": 20.15,
15
+ "eval_precisions_3": 11.5548,
16
+ "eval_precisions_4": 6.996,
17
+ "eval_ref_len": 21250,
18
+ "eval_rouge1": 0.3874,
19
+ "eval_rouge2": 0.1964,
20
+ "eval_rougeL": 0.3716,
21
+ "eval_rougeLsum": 0.3715,
22
+ "eval_runtime": 469.43,
23
+ "eval_samples": 2204,
24
+ "eval_samples_per_second": 4.695,
25
+ "eval_steps_per_second": 1.174,
26
+ "eval_sys_len": 18333,
27
+ "eval_totals_1": 18333,
28
+ "eval_totals_2": 16129,
29
+ "eval_totals_3": 13925,
30
+ "eval_totals_4": 11721,
31
+ "predict_bleu": 10.6077,
32
+ "predict_bp": 0.8937,
33
+ "predict_counts_1": 7767,
34
+ "predict_counts_2": 2500,
35
+ "predict_counts_3": 1096,
36
+ "predict_counts_4": 496,
37
+ "predict_exact_match": 0.0136,
38
+ "predict_f1": 0.3372,
39
+ "predict_gen_len": 14.7201,
40
+ "predict_loss": 1.9222917556762695,
41
+ "predict_precisions_1": 41.5525,
42
+ "predict_precisions_2": 15.1625,
43
+ "predict_precisions_3": 7.6729,
44
+ "predict_precisions_4": 4.106,
45
+ "predict_ref_len": 20793,
46
+ "predict_rouge1": 0.3466,
47
+ "predict_rouge2": 0.1612,
48
+ "predict_rougeL": 0.333,
49
+ "predict_rougeLsum": 0.3333,
50
+ "predict_runtime": 485.2231,
51
+ "predict_samples": 2204,
52
+ "predict_samples_per_second": 4.542,
53
+ "predict_steps_per_second": 1.136,
54
+ "predict_sys_len": 18692,
55
+ "predict_totals_1": 18692,
56
+ "predict_totals_2": 16488,
57
+ "predict_totals_3": 14284,
58
+ "predict_totals_4": 12080,
59
+ "train_loss": 0.0,
60
+ "train_runtime": 0.023,
61
+ "train_samples": 9314,
62
+ "train_samples_per_second": 8097618.841,
63
+ "train_steps_per_second": 126063.424
64
+ }
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "google/long-t5-tglobal-base",
3
+ "architectures": [
4
+ "LongT5ForConditionalGeneration"
5
+ ],
6
+ "d_ff": 2048,
7
+ "d_kv": 64,
8
+ "d_model": 768,
9
+ "decoder_start_token_id": 0,
10
+ "dense_act_fn": "gelu_new",
11
+ "dropout_rate": 0.1,
12
+ "encoder_attention_type": "transient-global",
13
+ "eos_token_id": 1,
14
+ "feed_forward_proj": "gated-gelu",
15
+ "global_block_size": 16,
16
+ "initializer_factor": 1.0,
17
+ "is_encoder_decoder": true,
18
+ "is_gated_act": true,
19
+ "layer_norm_epsilon": 1e-06,
20
+ "length_penalty": 0.0,
21
+ "local_radius": 127,
22
+ "max_length": 64,
23
+ "model_type": "longt5",
24
+ "n_positions": 4096,
25
+ "num_beams": 4,
26
+ "num_decoder_layers": 12,
27
+ "num_heads": 12,
28
+ "num_layers": 12,
29
+ "output_past": true,
30
+ "pad_token_id": 0,
31
+ "relative_attention_max_distance": 128,
32
+ "relative_attention_num_buckets": 32,
33
+ "tie_word_embeddings": false,
34
+ "torch_dtype": "float32",
35
+ "transformers_version": "4.34.1",
36
+ "use_cache": true,
37
+ "vocab_size": 32128
38
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "decoder_start_token_id": 0,
4
+ "eos_token_id": 1,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.34.1"
7
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94856673cf7d3f40d2d5b7960d3358d016dd935fb18f4ba57e728d29956fca9d
3
+ size 990386200
runs/Oct20_15-49-15_MARVIN-PC/events.out.tfevents.1697809770.MARVIN-PC.5688.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf26e1d0b3b972466213a64fcc553689f4baaa8a588e68d7896403b9126735bf
3
+ size 37412
runs/Oct20_15-49-15_MARVIN-PC/events.out.tfevents.1697835801.MARVIN-PC.5688.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73779fc3198c82d7457db101df6021d2ef7131e000614883388d3d88a467fd5e
3
+ size 1550
special_tokens_map.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<hl>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ }
10
+ ],
11
+ "eos_token": "</s>",
12
+ "pad_token": "<pad>",
13
+ "unk_token": "<unk>"
14
+ }
spiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
3
+ size 791656
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b67a2f89256b5b5d094d345e9c795a23ba89d52dd887c12777a90addc62d843c
3
+ size 2422566
tokenizer_config.json ADDED
@@ -0,0 +1,843 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<pad>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "</s>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "<unk>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "32000": {
28
+ "content": "<extra_id_99>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "32001": {
36
+ "content": "<extra_id_98>",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "32002": {
44
+ "content": "<extra_id_97>",
45
+ "lstrip": false,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ },
51
+ "32003": {
52
+ "content": "<extra_id_96>",
53
+ "lstrip": false,
54
+ "normalized": false,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": true
58
+ },
59
+ "32004": {
60
+ "content": "<extra_id_95>",
61
+ "lstrip": false,
62
+ "normalized": false,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": true
66
+ },
67
+ "32005": {
68
+ "content": "<extra_id_94>",
69
+ "lstrip": false,
70
+ "normalized": false,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": true
74
+ },
75
+ "32006": {
76
+ "content": "<extra_id_93>",
77
+ "lstrip": false,
78
+ "normalized": false,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": true
82
+ },
83
+ "32007": {
84
+ "content": "<extra_id_92>",
85
+ "lstrip": false,
86
+ "normalized": false,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": true
90
+ },
91
+ "32008": {
92
+ "content": "<extra_id_91>",
93
+ "lstrip": false,
94
+ "normalized": false,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": true
98
+ },
99
+ "32009": {
100
+ "content": "<extra_id_90>",
101
+ "lstrip": false,
102
+ "normalized": false,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": true
106
+ },
107
+ "32010": {
108
+ "content": "<extra_id_89>",
109
+ "lstrip": false,
110
+ "normalized": false,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": true
114
+ },
115
+ "32011": {
116
+ "content": "<extra_id_88>",
117
+ "lstrip": false,
118
+ "normalized": false,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": true
122
+ },
123
+ "32012": {
124
+ "content": "<extra_id_87>",
125
+ "lstrip": false,
126
+ "normalized": false,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": true
130
+ },
131
+ "32013": {
132
+ "content": "<extra_id_86>",
133
+ "lstrip": false,
134
+ "normalized": false,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": true
138
+ },
139
+ "32014": {
140
+ "content": "<extra_id_85>",
141
+ "lstrip": false,
142
+ "normalized": false,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": true
146
+ },
147
+ "32015": {
148
+ "content": "<extra_id_84>",
149
+ "lstrip": false,
150
+ "normalized": false,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": true
154
+ },
155
+ "32016": {
156
+ "content": "<extra_id_83>",
157
+ "lstrip": false,
158
+ "normalized": false,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": true
162
+ },
163
+ "32017": {
164
+ "content": "<extra_id_82>",
165
+ "lstrip": false,
166
+ "normalized": false,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": true
170
+ },
171
+ "32018": {
172
+ "content": "<extra_id_81>",
173
+ "lstrip": false,
174
+ "normalized": false,
175
+ "rstrip": false,
176
+ "single_word": false,
177
+ "special": true
178
+ },
179
+ "32019": {
180
+ "content": "<extra_id_80>",
181
+ "lstrip": false,
182
+ "normalized": false,
183
+ "rstrip": false,
184
+ "single_word": false,
185
+ "special": true
186
+ },
187
+ "32020": {
188
+ "content": "<extra_id_79>",
189
+ "lstrip": false,
190
+ "normalized": false,
191
+ "rstrip": false,
192
+ "single_word": false,
193
+ "special": true
194
+ },
195
+ "32021": {
196
+ "content": "<extra_id_78>",
197
+ "lstrip": false,
198
+ "normalized": false,
199
+ "rstrip": false,
200
+ "single_word": false,
201
+ "special": true
202
+ },
203
+ "32022": {
204
+ "content": "<extra_id_77>",
205
+ "lstrip": false,
206
+ "normalized": false,
207
+ "rstrip": false,
208
+ "single_word": false,
209
+ "special": true
210
+ },
211
+ "32023": {
212
+ "content": "<extra_id_76>",
213
+ "lstrip": false,
214
+ "normalized": false,
215
+ "rstrip": false,
216
+ "single_word": false,
217
+ "special": true
218
+ },
219
+ "32024": {
220
+ "content": "<extra_id_75>",
221
+ "lstrip": false,
222
+ "normalized": false,
223
+ "rstrip": false,
224
+ "single_word": false,
225
+ "special": true
226
+ },
227
+ "32025": {
228
+ "content": "<extra_id_74>",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false,
233
+ "special": true
234
+ },
235
+ "32026": {
236
+ "content": "<extra_id_73>",
237
+ "lstrip": false,
238
+ "normalized": false,
239
+ "rstrip": false,
240
+ "single_word": false,
241
+ "special": true
242
+ },
243
+ "32027": {
244
+ "content": "<extra_id_72>",
245
+ "lstrip": false,
246
+ "normalized": false,
247
+ "rstrip": false,
248
+ "single_word": false,
249
+ "special": true
250
+ },
251
+ "32028": {
252
+ "content": "<extra_id_71>",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "32029": {
260
+ "content": "<extra_id_70>",
261
+ "lstrip": false,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "32030": {
268
+ "content": "<extra_id_69>",
269
+ "lstrip": false,
270
+ "normalized": false,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": true
274
+ },
275
+ "32031": {
276
+ "content": "<extra_id_68>",
277
+ "lstrip": false,
278
+ "normalized": false,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": true
282
+ },
283
+ "32032": {
284
+ "content": "<extra_id_67>",
285
+ "lstrip": false,
286
+ "normalized": false,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": true
290
+ },
291
+ "32033": {
292
+ "content": "<extra_id_66>",
293
+ "lstrip": false,
294
+ "normalized": false,
295
+ "rstrip": false,
296
+ "single_word": false,
297
+ "special": true
298
+ },
299
+ "32034": {
300
+ "content": "<extra_id_65>",
301
+ "lstrip": false,
302
+ "normalized": false,
303
+ "rstrip": false,
304
+ "single_word": false,
305
+ "special": true
306
+ },
307
+ "32035": {
308
+ "content": "<extra_id_64>",
309
+ "lstrip": false,
310
+ "normalized": false,
311
+ "rstrip": false,
312
+ "single_word": false,
313
+ "special": true
314
+ },
315
+ "32036": {
316
+ "content": "<extra_id_63>",
317
+ "lstrip": false,
318
+ "normalized": false,
319
+ "rstrip": false,
320
+ "single_word": false,
321
+ "special": true
322
+ },
323
+ "32037": {
324
+ "content": "<extra_id_62>",
325
+ "lstrip": false,
326
+ "normalized": false,
327
+ "rstrip": false,
328
+ "single_word": false,
329
+ "special": true
330
+ },
331
+ "32038": {
332
+ "content": "<extra_id_61>",
333
+ "lstrip": false,
334
+ "normalized": false,
335
+ "rstrip": false,
336
+ "single_word": false,
337
+ "special": true
338
+ },
339
+ "32039": {
340
+ "content": "<extra_id_60>",
341
+ "lstrip": false,
342
+ "normalized": false,
343
+ "rstrip": false,
344
+ "single_word": false,
345
+ "special": true
346
+ },
347
+ "32040": {
348
+ "content": "<extra_id_59>",
349
+ "lstrip": false,
350
+ "normalized": false,
351
+ "rstrip": false,
352
+ "single_word": false,
353
+ "special": true
354
+ },
355
+ "32041": {
356
+ "content": "<extra_id_58>",
357
+ "lstrip": false,
358
+ "normalized": false,
359
+ "rstrip": false,
360
+ "single_word": false,
361
+ "special": true
362
+ },
363
+ "32042": {
364
+ "content": "<extra_id_57>",
365
+ "lstrip": false,
366
+ "normalized": false,
367
+ "rstrip": false,
368
+ "single_word": false,
369
+ "special": true
370
+ },
371
+ "32043": {
372
+ "content": "<extra_id_56>",
373
+ "lstrip": false,
374
+ "normalized": false,
375
+ "rstrip": false,
376
+ "single_word": false,
377
+ "special": true
378
+ },
379
+ "32044": {
380
+ "content": "<extra_id_55>",
381
+ "lstrip": false,
382
+ "normalized": false,
383
+ "rstrip": false,
384
+ "single_word": false,
385
+ "special": true
386
+ },
387
+ "32045": {
388
+ "content": "<extra_id_54>",
389
+ "lstrip": false,
390
+ "normalized": false,
391
+ "rstrip": false,
392
+ "single_word": false,
393
+ "special": true
394
+ },
395
+ "32046": {
396
+ "content": "<extra_id_53>",
397
+ "lstrip": false,
398
+ "normalized": false,
399
+ "rstrip": false,
400
+ "single_word": false,
401
+ "special": true
402
+ },
403
+ "32047": {
404
+ "content": "<extra_id_52>",
405
+ "lstrip": false,
406
+ "normalized": false,
407
+ "rstrip": false,
408
+ "single_word": false,
409
+ "special": true
410
+ },
411
+ "32048": {
412
+ "content": "<extra_id_51>",
413
+ "lstrip": false,
414
+ "normalized": false,
415
+ "rstrip": false,
416
+ "single_word": false,
417
+ "special": true
418
+ },
419
+ "32049": {
420
+ "content": "<extra_id_50>",
421
+ "lstrip": false,
422
+ "normalized": false,
423
+ "rstrip": false,
424
+ "single_word": false,
425
+ "special": true
426
+ },
427
+ "32050": {
428
+ "content": "<extra_id_49>",
429
+ "lstrip": false,
430
+ "normalized": false,
431
+ "rstrip": false,
432
+ "single_word": false,
433
+ "special": true
434
+ },
435
+ "32051": {
436
+ "content": "<extra_id_48>",
437
+ "lstrip": false,
438
+ "normalized": false,
439
+ "rstrip": false,
440
+ "single_word": false,
441
+ "special": true
442
+ },
443
+ "32052": {
444
+ "content": "<extra_id_47>",
445
+ "lstrip": false,
446
+ "normalized": false,
447
+ "rstrip": false,
448
+ "single_word": false,
449
+ "special": true
450
+ },
451
+ "32053": {
452
+ "content": "<extra_id_46>",
453
+ "lstrip": false,
454
+ "normalized": false,
455
+ "rstrip": false,
456
+ "single_word": false,
457
+ "special": true
458
+ },
459
+ "32054": {
460
+ "content": "<extra_id_45>",
461
+ "lstrip": false,
462
+ "normalized": false,
463
+ "rstrip": false,
464
+ "single_word": false,
465
+ "special": true
466
+ },
467
+ "32055": {
468
+ "content": "<extra_id_44>",
469
+ "lstrip": false,
470
+ "normalized": false,
471
+ "rstrip": false,
472
+ "single_word": false,
473
+ "special": true
474
+ },
475
+ "32056": {
476
+ "content": "<extra_id_43>",
477
+ "lstrip": false,
478
+ "normalized": false,
479
+ "rstrip": false,
480
+ "single_word": false,
481
+ "special": true
482
+ },
483
+ "32057": {
484
+ "content": "<extra_id_42>",
485
+ "lstrip": false,
486
+ "normalized": false,
487
+ "rstrip": false,
488
+ "single_word": false,
489
+ "special": true
490
+ },
491
+ "32058": {
492
+ "content": "<extra_id_41>",
493
+ "lstrip": false,
494
+ "normalized": false,
495
+ "rstrip": false,
496
+ "single_word": false,
497
+ "special": true
498
+ },
499
+ "32059": {
500
+ "content": "<extra_id_40>",
501
+ "lstrip": false,
502
+ "normalized": false,
503
+ "rstrip": false,
504
+ "single_word": false,
505
+ "special": true
506
+ },
507
+ "32060": {
508
+ "content": "<extra_id_39>",
509
+ "lstrip": false,
510
+ "normalized": false,
511
+ "rstrip": false,
512
+ "single_word": false,
513
+ "special": true
514
+ },
515
+ "32061": {
516
+ "content": "<extra_id_38>",
517
+ "lstrip": false,
518
+ "normalized": false,
519
+ "rstrip": false,
520
+ "single_word": false,
521
+ "special": true
522
+ },
523
+ "32062": {
524
+ "content": "<extra_id_37>",
525
+ "lstrip": false,
526
+ "normalized": false,
527
+ "rstrip": false,
528
+ "single_word": false,
529
+ "special": true
530
+ },
531
+ "32063": {
532
+ "content": "<extra_id_36>",
533
+ "lstrip": false,
534
+ "normalized": false,
535
+ "rstrip": false,
536
+ "single_word": false,
537
+ "special": true
538
+ },
539
+ "32064": {
540
+ "content": "<extra_id_35>",
541
+ "lstrip": false,
542
+ "normalized": false,
543
+ "rstrip": false,
544
+ "single_word": false,
545
+ "special": true
546
+ },
547
+ "32065": {
548
+ "content": "<extra_id_34>",
549
+ "lstrip": false,
550
+ "normalized": false,
551
+ "rstrip": false,
552
+ "single_word": false,
553
+ "special": true
554
+ },
555
+ "32066": {
556
+ "content": "<extra_id_33>",
557
+ "lstrip": false,
558
+ "normalized": false,
559
+ "rstrip": false,
560
+ "single_word": false,
561
+ "special": true
562
+ },
563
+ "32067": {
564
+ "content": "<extra_id_32>",
565
+ "lstrip": false,
566
+ "normalized": false,
567
+ "rstrip": false,
568
+ "single_word": false,
569
+ "special": true
570
+ },
571
+ "32068": {
572
+ "content": "<extra_id_31>",
573
+ "lstrip": false,
574
+ "normalized": false,
575
+ "rstrip": false,
576
+ "single_word": false,
577
+ "special": true
578
+ },
579
+ "32069": {
580
+ "content": "<extra_id_30>",
581
+ "lstrip": false,
582
+ "normalized": false,
583
+ "rstrip": false,
584
+ "single_word": false,
585
+ "special": true
586
+ },
587
+ "32070": {
588
+ "content": "<extra_id_29>",
589
+ "lstrip": false,
590
+ "normalized": false,
591
+ "rstrip": false,
592
+ "single_word": false,
593
+ "special": true
594
+ },
595
+ "32071": {
596
+ "content": "<extra_id_28>",
597
+ "lstrip": false,
598
+ "normalized": false,
599
+ "rstrip": false,
600
+ "single_word": false,
601
+ "special": true
602
+ },
603
+ "32072": {
604
+ "content": "<extra_id_27>",
605
+ "lstrip": false,
606
+ "normalized": false,
607
+ "rstrip": false,
608
+ "single_word": false,
609
+ "special": true
610
+ },
611
+ "32073": {
612
+ "content": "<extra_id_26>",
613
+ "lstrip": false,
614
+ "normalized": false,
615
+ "rstrip": false,
616
+ "single_word": false,
617
+ "special": true
618
+ },
619
+ "32074": {
620
+ "content": "<extra_id_25>",
621
+ "lstrip": false,
622
+ "normalized": false,
623
+ "rstrip": false,
624
+ "single_word": false,
625
+ "special": true
626
+ },
627
+ "32075": {
628
+ "content": "<extra_id_24>",
629
+ "lstrip": false,
630
+ "normalized": false,
631
+ "rstrip": false,
632
+ "single_word": false,
633
+ "special": true
634
+ },
635
+ "32076": {
636
+ "content": "<extra_id_23>",
637
+ "lstrip": false,
638
+ "normalized": false,
639
+ "rstrip": false,
640
+ "single_word": false,
641
+ "special": true
642
+ },
643
+ "32077": {
644
+ "content": "<extra_id_22>",
645
+ "lstrip": false,
646
+ "normalized": false,
647
+ "rstrip": false,
648
+ "single_word": false,
649
+ "special": true
650
+ },
651
+ "32078": {
652
+ "content": "<extra_id_21>",
653
+ "lstrip": false,
654
+ "normalized": false,
655
+ "rstrip": false,
656
+ "single_word": false,
657
+ "special": true
658
+ },
659
+ "32079": {
660
+ "content": "<extra_id_20>",
661
+ "lstrip": false,
662
+ "normalized": false,
663
+ "rstrip": false,
664
+ "single_word": false,
665
+ "special": true
666
+ },
667
+ "32080": {
668
+ "content": "<extra_id_19>",
669
+ "lstrip": false,
670
+ "normalized": false,
671
+ "rstrip": false,
672
+ "single_word": false,
673
+ "special": true
674
+ },
675
+ "32081": {
676
+ "content": "<extra_id_18>",
677
+ "lstrip": false,
678
+ "normalized": false,
679
+ "rstrip": false,
680
+ "single_word": false,
681
+ "special": true
682
+ },
683
+ "32082": {
684
+ "content": "<extra_id_17>",
685
+ "lstrip": false,
686
+ "normalized": false,
687
+ "rstrip": false,
688
+ "single_word": false,
689
+ "special": true
690
+ },
691
+ "32083": {
692
+ "content": "<extra_id_16>",
693
+ "lstrip": false,
694
+ "normalized": false,
695
+ "rstrip": false,
696
+ "single_word": false,
697
+ "special": true
698
+ },
699
+ "32084": {
700
+ "content": "<extra_id_15>",
701
+ "lstrip": false,
702
+ "normalized": false,
703
+ "rstrip": false,
704
+ "single_word": false,
705
+ "special": true
706
+ },
707
+ "32085": {
708
+ "content": "<extra_id_14>",
709
+ "lstrip": false,
710
+ "normalized": false,
711
+ "rstrip": false,
712
+ "single_word": false,
713
+ "special": true
714
+ },
715
+ "32086": {
716
+ "content": "<extra_id_13>",
717
+ "lstrip": false,
718
+ "normalized": false,
719
+ "rstrip": false,
720
+ "single_word": false,
721
+ "special": true
722
+ },
723
+ "32087": {
724
+ "content": "<extra_id_12>",
725
+ "lstrip": false,
726
+ "normalized": false,
727
+ "rstrip": false,
728
+ "single_word": false,
729
+ "special": true
730
+ },
731
+ "32088": {
732
+ "content": "<extra_id_11>",
733
+ "lstrip": false,
734
+ "normalized": false,
735
+ "rstrip": false,
736
+ "single_word": false,
737
+ "special": true
738
+ },
739
+ "32089": {
740
+ "content": "<extra_id_10>",
741
+ "lstrip": false,
742
+ "normalized": false,
743
+ "rstrip": false,
744
+ "single_word": false,
745
+ "special": true
746
+ },
747
+ "32090": {
748
+ "content": "<extra_id_9>",
749
+ "lstrip": false,
750
+ "normalized": false,
751
+ "rstrip": false,
752
+ "single_word": false,
753
+ "special": true
754
+ },
755
+ "32091": {
756
+ "content": "<extra_id_8>",
757
+ "lstrip": false,
758
+ "normalized": false,
759
+ "rstrip": false,
760
+ "single_word": false,
761
+ "special": true
762
+ },
763
+ "32092": {
764
+ "content": "<extra_id_7>",
765
+ "lstrip": false,
766
+ "normalized": false,
767
+ "rstrip": false,
768
+ "single_word": false,
769
+ "special": true
770
+ },
771
+ "32093": {
772
+ "content": "<extra_id_6>",
773
+ "lstrip": false,
774
+ "normalized": false,
775
+ "rstrip": false,
776
+ "single_word": false,
777
+ "special": true
778
+ },
779
+ "32094": {
780
+ "content": "<extra_id_5>",
781
+ "lstrip": false,
782
+ "normalized": false,
783
+ "rstrip": false,
784
+ "single_word": false,
785
+ "special": true
786
+ },
787
+ "32095": {
788
+ "content": "<extra_id_4>",
789
+ "lstrip": false,
790
+ "normalized": false,
791
+ "rstrip": false,
792
+ "single_word": false,
793
+ "special": true
794
+ },
795
+ "32096": {
796
+ "content": "<extra_id_3>",
797
+ "lstrip": false,
798
+ "normalized": false,
799
+ "rstrip": false,
800
+ "single_word": false,
801
+ "special": true
802
+ },
803
+ "32097": {
804
+ "content": "<extra_id_2>",
805
+ "lstrip": false,
806
+ "normalized": false,
807
+ "rstrip": false,
808
+ "single_word": false,
809
+ "special": true
810
+ },
811
+ "32098": {
812
+ "content": "<extra_id_1>",
813
+ "lstrip": false,
814
+ "normalized": false,
815
+ "rstrip": false,
816
+ "single_word": false,
817
+ "special": true
818
+ },
819
+ "32099": {
820
+ "content": "<extra_id_0>",
821
+ "lstrip": false,
822
+ "normalized": false,
823
+ "rstrip": false,
824
+ "single_word": false,
825
+ "special": true
826
+ },
827
+ "32100": {
828
+ "content": "<hl>",
829
+ "lstrip": false,
830
+ "normalized": false,
831
+ "rstrip": false,
832
+ "single_word": false,
833
+ "special": true
834
+ }
835
+ },
836
+ "clean_up_tokenization_spaces": true,
837
+ "eos_token": "</s>",
838
+ "extra_ids": 100,
839
+ "model_max_length": 1000000000000000019884624838656,
840
+ "pad_token": "<pad>",
841
+ "tokenizer_class": "T5Tokenizer",
842
+ "unk_token": "<unk>"
843
+ }
trainer_state.json ADDED
@@ -0,0 +1,768 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 19.914163090128756,
5
+ "eval_steps": 500,
6
+ "global_step": 2900,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 1.0,
13
+ "learning_rate": 0.0001,
14
+ "loss": 3.1671,
15
+ "step": 145
16
+ },
17
+ {
18
+ "epoch": 1.0,
19
+ "eval_bleu": 5.9441,
20
+ "eval_bp": 0.7156,
21
+ "eval_counts_1": 6177,
22
+ "eval_counts_2": 1669,
23
+ "eval_counts_3": 604,
24
+ "eval_counts_4": 179,
25
+ "eval_exact_match": 0.0023,
26
+ "eval_f1": 0.2528,
27
+ "eval_gen_len": 12.0218,
28
+ "eval_loss": 2.190216541290283,
29
+ "eval_precisions_1": 38.7954,
30
+ "eval_precisions_2": 12.1665,
31
+ "eval_precisions_3": 5.2458,
32
+ "eval_precisions_4": 1.9227,
33
+ "eval_ref_len": 21250,
34
+ "eval_rouge1": 0.2595,
35
+ "eval_rouge2": 0.1035,
36
+ "eval_rougeL": 0.2491,
37
+ "eval_rougeLsum": 0.2492,
38
+ "eval_runtime": 793.0147,
39
+ "eval_samples_per_second": 2.779,
40
+ "eval_steps_per_second": 0.695,
41
+ "eval_sys_len": 15922,
42
+ "eval_totals_1": 15922,
43
+ "eval_totals_2": 13718,
44
+ "eval_totals_3": 11514,
45
+ "eval_totals_4": 9310,
46
+ "step": 145
47
+ },
48
+ {
49
+ "epoch": 2.0,
50
+ "learning_rate": 0.0001,
51
+ "loss": 2.5597,
52
+ "step": 291
53
+ },
54
+ {
55
+ "epoch": 2.0,
56
+ "eval_bleu": 7.7787,
57
+ "eval_bp": 0.7556,
58
+ "eval_counts_1": 6785,
59
+ "eval_counts_2": 2044,
60
+ "eval_counts_3": 804,
61
+ "eval_counts_4": 293,
62
+ "eval_exact_match": 0.0064,
63
+ "eval_f1": 0.2864,
64
+ "eval_gen_len": 12.6084,
65
+ "eval_loss": 2.016404151916504,
66
+ "eval_precisions_1": 40.876,
67
+ "eval_precisions_2": 14.1994,
68
+ "eval_precisions_3": 6.595,
69
+ "eval_precisions_4": 2.9338,
70
+ "eval_ref_len": 21250,
71
+ "eval_rouge1": 0.2931,
72
+ "eval_rouge2": 0.1291,
73
+ "eval_rougeL": 0.2817,
74
+ "eval_rougeLsum": 0.2818,
75
+ "eval_runtime": 817.9822,
76
+ "eval_samples_per_second": 2.694,
77
+ "eval_steps_per_second": 0.674,
78
+ "eval_sys_len": 16599,
79
+ "eval_totals_1": 16599,
80
+ "eval_totals_2": 14395,
81
+ "eval_totals_3": 12191,
82
+ "eval_totals_4": 9987,
83
+ "step": 291
84
+ },
85
+ {
86
+ "epoch": 2.99,
87
+ "learning_rate": 0.0001,
88
+ "loss": 2.3464,
89
+ "step": 436
90
+ },
91
+ {
92
+ "epoch": 2.99,
93
+ "eval_bleu": 9.2407,
94
+ "eval_bp": 0.7935,
95
+ "eval_counts_1": 7251,
96
+ "eval_counts_2": 2326,
97
+ "eval_counts_3": 969,
98
+ "eval_counts_4": 400,
99
+ "eval_exact_match": 0.0073,
100
+ "eval_f1": 0.3114,
101
+ "eval_gen_len": 13.2296,
102
+ "eval_loss": 1.9138075113296509,
103
+ "eval_precisions_1": 42.0129,
104
+ "eval_precisions_2": 15.45,
105
+ "eval_precisions_3": 7.5403,
106
+ "eval_precisions_4": 3.7569,
107
+ "eval_ref_len": 21250,
108
+ "eval_rouge1": 0.3162,
109
+ "eval_rouge2": 0.1456,
110
+ "eval_rougeL": 0.3031,
111
+ "eval_rougeLsum": 0.3031,
112
+ "eval_runtime": 765.0466,
113
+ "eval_samples_per_second": 2.881,
114
+ "eval_steps_per_second": 0.72,
115
+ "eval_sys_len": 17259,
116
+ "eval_totals_1": 17259,
117
+ "eval_totals_2": 15055,
118
+ "eval_totals_3": 12851,
119
+ "eval_totals_4": 10647,
120
+ "step": 436
121
+ },
122
+ {
123
+ "epoch": 4.0,
124
+ "learning_rate": 0.0001,
125
+ "loss": 2.1679,
126
+ "step": 582
127
+ },
128
+ {
129
+ "epoch": 4.0,
130
+ "eval_bleu": 9.6363,
131
+ "eval_bp": 0.7795,
132
+ "eval_counts_1": 7382,
133
+ "eval_counts_2": 2393,
134
+ "eval_counts_3": 1006,
135
+ "eval_counts_4": 434,
136
+ "eval_exact_match": 0.0109,
137
+ "eval_f1": 0.3226,
138
+ "eval_gen_len": 13.1207,
139
+ "eval_loss": 1.8524010181427002,
140
+ "eval_precisions_1": 43.3903,
141
+ "eval_precisions_2": 16.1591,
142
+ "eval_precisions_3": 7.981,
143
+ "eval_precisions_4": 4.1727,
144
+ "eval_ref_len": 21250,
145
+ "eval_rouge1": 0.3272,
146
+ "eval_rouge2": 0.1504,
147
+ "eval_rougeL": 0.3147,
148
+ "eval_rougeLsum": 0.3149,
149
+ "eval_runtime": 882.4242,
150
+ "eval_samples_per_second": 2.498,
151
+ "eval_steps_per_second": 0.624,
152
+ "eval_sys_len": 17013,
153
+ "eval_totals_1": 17013,
154
+ "eval_totals_2": 14809,
155
+ "eval_totals_3": 12605,
156
+ "eval_totals_4": 10401,
157
+ "step": 582
158
+ },
159
+ {
160
+ "epoch": 5.0,
161
+ "learning_rate": 0.0001,
162
+ "loss": 2.0454,
163
+ "step": 728
164
+ },
165
+ {
166
+ "epoch": 5.0,
167
+ "eval_bleu": 10.3812,
168
+ "eval_bp": 0.7665,
169
+ "eval_counts_1": 7581,
170
+ "eval_counts_2": 2555,
171
+ "eval_counts_3": 1111,
172
+ "eval_counts_4": 482,
173
+ "eval_exact_match": 0.0132,
174
+ "eval_f1": 0.3357,
175
+ "eval_gen_len": 12.9782,
176
+ "eval_loss": 1.7996737957000732,
177
+ "eval_precisions_1": 45.1599,
178
+ "eval_precisions_2": 17.5204,
179
+ "eval_precisions_3": 8.9749,
180
+ "eval_precisions_4": 4.7371,
181
+ "eval_ref_len": 21250,
182
+ "eval_rouge1": 0.3401,
183
+ "eval_rouge2": 0.1606,
184
+ "eval_rougeL": 0.3278,
185
+ "eval_rougeLsum": 0.3279,
186
+ "eval_runtime": 519.8377,
187
+ "eval_samples_per_second": 4.24,
188
+ "eval_steps_per_second": 1.06,
189
+ "eval_sys_len": 16787,
190
+ "eval_totals_1": 16787,
191
+ "eval_totals_2": 14583,
192
+ "eval_totals_3": 12379,
193
+ "eval_totals_4": 10175,
194
+ "step": 728
195
+ },
196
+ {
197
+ "epoch": 5.99,
198
+ "learning_rate": 0.0001,
199
+ "loss": 1.9502,
200
+ "step": 873
201
+ },
202
+ {
203
+ "epoch": 5.99,
204
+ "eval_bleu": 10.7668,
205
+ "eval_bp": 0.7992,
206
+ "eval_counts_1": 7759,
207
+ "eval_counts_2": 2618,
208
+ "eval_counts_3": 1162,
209
+ "eval_counts_4": 511,
210
+ "eval_exact_match": 0.0127,
211
+ "eval_f1": 0.3406,
212
+ "eval_gen_len": 13.4841,
213
+ "eval_loss": 1.7696163654327393,
214
+ "eval_precisions_1": 44.6973,
215
+ "eval_precisions_2": 17.2748,
216
+ "eval_precisions_3": 8.9723,
217
+ "eval_precisions_4": 4.7548,
218
+ "eval_ref_len": 21250,
219
+ "eval_rouge1": 0.3452,
220
+ "eval_rouge2": 0.1631,
221
+ "eval_rougeL": 0.3321,
222
+ "eval_rougeLsum": 0.3319,
223
+ "eval_runtime": 542.6731,
224
+ "eval_samples_per_second": 4.061,
225
+ "eval_steps_per_second": 1.015,
226
+ "eval_sys_len": 17359,
227
+ "eval_totals_1": 17359,
228
+ "eval_totals_2": 15155,
229
+ "eval_totals_3": 12951,
230
+ "eval_totals_4": 10747,
231
+ "step": 873
232
+ },
233
+ {
234
+ "epoch": 7.0,
235
+ "learning_rate": 0.0001,
236
+ "loss": 1.8414,
237
+ "step": 1019
238
+ },
239
+ {
240
+ "epoch": 7.0,
241
+ "eval_bleu": 11.3408,
242
+ "eval_bp": 0.7721,
243
+ "eval_counts_1": 7791,
244
+ "eval_counts_2": 2693,
245
+ "eval_counts_3": 1236,
246
+ "eval_counts_4": 570,
247
+ "eval_exact_match": 0.015,
248
+ "eval_f1": 0.347,
249
+ "eval_gen_len": 13.0563,
250
+ "eval_loss": 1.7471755743026733,
251
+ "eval_precisions_1": 46.147,
252
+ "eval_precisions_2": 18.3459,
253
+ "eval_precisions_3": 9.9078,
254
+ "eval_precisions_4": 5.5496,
255
+ "eval_ref_len": 21250,
256
+ "eval_rouge1": 0.3513,
257
+ "eval_rouge2": 0.1679,
258
+ "eval_rougeL": 0.3391,
259
+ "eval_rougeLsum": 0.3391,
260
+ "eval_runtime": 455.2485,
261
+ "eval_samples_per_second": 4.841,
262
+ "eval_steps_per_second": 1.21,
263
+ "eval_sys_len": 16883,
264
+ "eval_totals_1": 16883,
265
+ "eval_totals_2": 14679,
266
+ "eval_totals_3": 12475,
267
+ "eval_totals_4": 10271,
268
+ "step": 1019
269
+ },
270
+ {
271
+ "epoch": 8.0,
272
+ "learning_rate": 0.0001,
273
+ "loss": 1.7614,
274
+ "step": 1165
275
+ },
276
+ {
277
+ "epoch": 8.0,
278
+ "eval_bleu": 11.8447,
279
+ "eval_bp": 0.8198,
280
+ "eval_counts_1": 8024,
281
+ "eval_counts_2": 2799,
282
+ "eval_counts_3": 1296,
283
+ "eval_counts_4": 610,
284
+ "eval_exact_match": 0.0145,
285
+ "eval_f1": 0.352,
286
+ "eval_gen_len": 13.515,
287
+ "eval_loss": 1.7203415632247925,
288
+ "eval_precisions_1": 45.2643,
289
+ "eval_precisions_2": 18.0313,
290
+ "eval_precisions_3": 9.7305,
291
+ "eval_precisions_4": 5.4881,
292
+ "eval_ref_len": 21250,
293
+ "eval_rouge1": 0.3565,
294
+ "eval_rouge2": 0.1711,
295
+ "eval_rougeL": 0.3422,
296
+ "eval_rougeLsum": 0.3423,
297
+ "eval_runtime": 457.6091,
298
+ "eval_samples_per_second": 4.816,
299
+ "eval_steps_per_second": 1.204,
300
+ "eval_sys_len": 17727,
301
+ "eval_totals_1": 17727,
302
+ "eval_totals_2": 15523,
303
+ "eval_totals_3": 13319,
304
+ "eval_totals_4": 11115,
305
+ "step": 1165
306
+ },
307
+ {
308
+ "epoch": 9.0,
309
+ "learning_rate": 0.0001,
310
+ "loss": 1.6997,
311
+ "step": 1310
312
+ },
313
+ {
314
+ "epoch": 9.0,
315
+ "eval_bleu": 11.9689,
316
+ "eval_bp": 0.8027,
317
+ "eval_counts_1": 8046,
318
+ "eval_counts_2": 2835,
319
+ "eval_counts_3": 1314,
320
+ "eval_counts_4": 615,
321
+ "eval_exact_match": 0.0168,
322
+ "eval_f1": 0.3568,
323
+ "eval_gen_len": 13.4306,
324
+ "eval_loss": 1.7166661024093628,
325
+ "eval_precisions_1": 46.183,
326
+ "eval_precisions_2": 18.6293,
327
+ "eval_precisions_3": 10.0968,
328
+ "eval_precisions_4": 5.6892,
329
+ "eval_ref_len": 21250,
330
+ "eval_rouge1": 0.3613,
331
+ "eval_rouge2": 0.1746,
332
+ "eval_rougeL": 0.3466,
333
+ "eval_rougeLsum": 0.3466,
334
+ "eval_runtime": 543.9804,
335
+ "eval_samples_per_second": 4.052,
336
+ "eval_steps_per_second": 1.013,
337
+ "eval_sys_len": 17422,
338
+ "eval_totals_1": 17422,
339
+ "eval_totals_2": 15218,
340
+ "eval_totals_3": 13014,
341
+ "eval_totals_4": 10810,
342
+ "step": 1310
343
+ },
344
+ {
345
+ "epoch": 10.0,
346
+ "learning_rate": 0.0001,
347
+ "loss": 1.6159,
348
+ "step": 1456
349
+ },
350
+ {
351
+ "epoch": 10.0,
352
+ "eval_bleu": 12.5678,
353
+ "eval_bp": 0.8182,
354
+ "eval_counts_1": 8087,
355
+ "eval_counts_2": 2928,
356
+ "eval_counts_3": 1395,
357
+ "eval_counts_4": 681,
358
+ "eval_exact_match": 0.0181,
359
+ "eval_f1": 0.3564,
360
+ "eval_gen_len": 13.5268,
361
+ "eval_loss": 1.689180612564087,
362
+ "eval_precisions_1": 45.6944,
363
+ "eval_precisions_2": 18.8976,
364
+ "eval_precisions_3": 10.4966,
365
+ "eval_precisions_4": 6.1429,
366
+ "eval_ref_len": 21250,
367
+ "eval_rouge1": 0.3612,
368
+ "eval_rouge2": 0.1795,
369
+ "eval_rougeL": 0.3485,
370
+ "eval_rougeLsum": 0.3482,
371
+ "eval_runtime": 661.754,
372
+ "eval_samples_per_second": 3.331,
373
+ "eval_steps_per_second": 0.833,
374
+ "eval_sys_len": 17698,
375
+ "eval_totals_1": 17698,
376
+ "eval_totals_2": 15494,
377
+ "eval_totals_3": 13290,
378
+ "eval_totals_4": 11086,
379
+ "step": 1456
380
+ },
381
+ {
382
+ "epoch": 10.99,
383
+ "learning_rate": 0.0001,
384
+ "loss": 1.5681,
385
+ "step": 1601
386
+ },
387
+ {
388
+ "epoch": 10.99,
389
+ "eval_bleu": 12.497,
390
+ "eval_bp": 0.813,
391
+ "eval_counts_1": 8154,
392
+ "eval_counts_2": 2933,
393
+ "eval_counts_3": 1383,
394
+ "eval_counts_4": 664,
395
+ "eval_exact_match": 0.0168,
396
+ "eval_f1": 0.3605,
397
+ "eval_gen_len": 13.6044,
398
+ "eval_loss": 1.6923038959503174,
399
+ "eval_precisions_1": 46.3164,
400
+ "eval_precisions_2": 19.0442,
401
+ "eval_precisions_3": 10.4797,
402
+ "eval_precisions_4": 6.0402,
403
+ "eval_ref_len": 21250,
404
+ "eval_rouge1": 0.3654,
405
+ "eval_rouge2": 0.1789,
406
+ "eval_rougeL": 0.3506,
407
+ "eval_rougeLsum": 0.3505,
408
+ "eval_runtime": 528.2815,
409
+ "eval_samples_per_second": 4.172,
410
+ "eval_steps_per_second": 1.043,
411
+ "eval_sys_len": 17605,
412
+ "eval_totals_1": 17605,
413
+ "eval_totals_2": 15401,
414
+ "eval_totals_3": 13197,
415
+ "eval_totals_4": 10993,
416
+ "step": 1601
417
+ },
418
+ {
419
+ "epoch": 12.0,
420
+ "learning_rate": 0.0001,
421
+ "loss": 1.4987,
422
+ "step": 1747
423
+ },
424
+ {
425
+ "epoch": 12.0,
426
+ "eval_bleu": 12.8959,
427
+ "eval_bp": 0.8169,
428
+ "eval_counts_1": 8295,
429
+ "eval_counts_2": 3011,
430
+ "eval_counts_3": 1432,
431
+ "eval_counts_4": 697,
432
+ "eval_exact_match": 0.0181,
433
+ "eval_f1": 0.3675,
434
+ "eval_gen_len": 13.6134,
435
+ "eval_loss": 1.6824951171875,
436
+ "eval_precisions_1": 46.928,
437
+ "eval_precisions_2": 19.461,
438
+ "eval_precisions_3": 10.7929,
439
+ "eval_precisions_4": 6.2997,
440
+ "eval_ref_len": 21250,
441
+ "eval_rouge1": 0.3734,
442
+ "eval_rouge2": 0.1846,
443
+ "eval_rougeL": 0.3576,
444
+ "eval_rougeLsum": 0.3577,
445
+ "eval_runtime": 636.4551,
446
+ "eval_samples_per_second": 3.463,
447
+ "eval_steps_per_second": 0.866,
448
+ "eval_sys_len": 17676,
449
+ "eval_totals_1": 17676,
450
+ "eval_totals_2": 15472,
451
+ "eval_totals_3": 13268,
452
+ "eval_totals_4": 11064,
453
+ "step": 1747
454
+ },
455
+ {
456
+ "epoch": 13.0,
457
+ "learning_rate": 0.0001,
458
+ "loss": 1.4461,
459
+ "step": 1893
460
+ },
461
+ {
462
+ "epoch": 13.0,
463
+ "eval_bleu": 12.8688,
464
+ "eval_bp": 0.8139,
465
+ "eval_counts_1": 8246,
466
+ "eval_counts_2": 3005,
467
+ "eval_counts_3": 1424,
468
+ "eval_counts_4": 700,
469
+ "eval_exact_match": 0.0191,
470
+ "eval_f1": 0.3658,
471
+ "eval_gen_len": 13.5812,
472
+ "eval_loss": 1.6783509254455566,
473
+ "eval_precisions_1": 46.7964,
474
+ "eval_precisions_2": 19.4915,
475
+ "eval_precisions_3": 10.7773,
476
+ "eval_precisions_4": 6.3584,
477
+ "eval_ref_len": 21250,
478
+ "eval_rouge1": 0.3725,
479
+ "eval_rouge2": 0.1857,
480
+ "eval_rougeL": 0.358,
481
+ "eval_rougeLsum": 0.3576,
482
+ "eval_runtime": 521.7174,
483
+ "eval_samples_per_second": 4.225,
484
+ "eval_steps_per_second": 1.056,
485
+ "eval_sys_len": 17621,
486
+ "eval_totals_1": 17621,
487
+ "eval_totals_2": 15417,
488
+ "eval_totals_3": 13213,
489
+ "eval_totals_4": 11009,
490
+ "step": 1893
491
+ },
492
+ {
493
+ "epoch": 13.99,
494
+ "learning_rate": 0.0001,
495
+ "loss": 1.4002,
496
+ "step": 2038
497
+ },
498
+ {
499
+ "epoch": 13.99,
500
+ "eval_bleu": 13.4526,
501
+ "eval_bp": 0.8329,
502
+ "eval_counts_1": 8457,
503
+ "eval_counts_2": 3130,
504
+ "eval_counts_3": 1504,
505
+ "eval_counts_4": 745,
506
+ "eval_exact_match": 0.02,
507
+ "eval_f1": 0.3727,
508
+ "eval_gen_len": 13.9179,
509
+ "eval_loss": 1.6725177764892578,
510
+ "eval_precisions_1": 47.0749,
511
+ "eval_precisions_2": 19.8591,
512
+ "eval_precisions_3": 11.0939,
513
+ "eval_precisions_4": 6.5621,
514
+ "eval_ref_len": 21250,
515
+ "eval_rouge1": 0.3797,
516
+ "eval_rouge2": 0.1915,
517
+ "eval_rougeL": 0.3637,
518
+ "eval_rougeLsum": 0.3634,
519
+ "eval_runtime": 592.5507,
520
+ "eval_samples_per_second": 3.72,
521
+ "eval_steps_per_second": 0.93,
522
+ "eval_sys_len": 17965,
523
+ "eval_totals_1": 17965,
524
+ "eval_totals_2": 15761,
525
+ "eval_totals_3": 13557,
526
+ "eval_totals_4": 11353,
527
+ "step": 2038
528
+ },
529
+ {
530
+ "epoch": 15.0,
531
+ "learning_rate": 0.0001,
532
+ "loss": 1.3391,
533
+ "step": 2184
534
+ },
535
+ {
536
+ "epoch": 15.0,
537
+ "eval_bleu": 13.211,
538
+ "eval_bp": 0.8283,
539
+ "eval_counts_1": 8443,
540
+ "eval_counts_2": 3091,
541
+ "eval_counts_3": 1468,
542
+ "eval_counts_4": 719,
543
+ "eval_exact_match": 0.0204,
544
+ "eval_f1": 0.3737,
545
+ "eval_gen_len": 13.9133,
546
+ "eval_loss": 1.6783130168914795,
547
+ "eval_precisions_1": 47.2177,
548
+ "eval_precisions_2": 19.7168,
549
+ "eval_precisions_3": 10.8959,
550
+ "eval_precisions_4": 6.3803,
551
+ "eval_ref_len": 21250,
552
+ "eval_rouge1": 0.3804,
553
+ "eval_rouge2": 0.1901,
554
+ "eval_rougeL": 0.3634,
555
+ "eval_rougeLsum": 0.363,
556
+ "eval_runtime": 547.4964,
557
+ "eval_samples_per_second": 4.026,
558
+ "eval_steps_per_second": 1.006,
559
+ "eval_sys_len": 17881,
560
+ "eval_totals_1": 17881,
561
+ "eval_totals_2": 15677,
562
+ "eval_totals_3": 13473,
563
+ "eval_totals_4": 11269,
564
+ "step": 2184
565
+ },
566
+ {
567
+ "epoch": 16.0,
568
+ "learning_rate": 0.0001,
569
+ "loss": 1.2921,
570
+ "step": 2330
571
+ },
572
+ {
573
+ "epoch": 16.0,
574
+ "eval_bleu": 13.4907,
575
+ "eval_bp": 0.8373,
576
+ "eval_counts_1": 8457,
577
+ "eval_counts_2": 3147,
578
+ "eval_counts_3": 1511,
579
+ "eval_counts_4": 747,
580
+ "eval_exact_match": 0.0195,
581
+ "eval_f1": 0.3716,
582
+ "eval_gen_len": 13.9882,
583
+ "eval_loss": 1.6737552881240845,
584
+ "eval_precisions_1": 46.8662,
585
+ "eval_precisions_2": 19.8662,
586
+ "eval_precisions_3": 11.0801,
587
+ "eval_precisions_4": 6.5337,
588
+ "eval_ref_len": 21250,
589
+ "eval_rouge1": 0.3782,
590
+ "eval_rouge2": 0.1902,
591
+ "eval_rougeL": 0.3624,
592
+ "eval_rougeLsum": 0.3624,
593
+ "eval_runtime": 652.072,
594
+ "eval_samples_per_second": 3.38,
595
+ "eval_steps_per_second": 0.845,
596
+ "eval_sys_len": 18045,
597
+ "eval_totals_1": 18045,
598
+ "eval_totals_2": 15841,
599
+ "eval_totals_3": 13637,
600
+ "eval_totals_4": 11433,
601
+ "step": 2330
602
+ },
603
+ {
604
+ "epoch": 17.0,
605
+ "learning_rate": 0.0001,
606
+ "loss": 1.2572,
607
+ "step": 2475
608
+ },
609
+ {
610
+ "epoch": 17.0,
611
+ "eval_bleu": 13.8581,
612
+ "eval_bp": 0.8267,
613
+ "eval_counts_1": 8473,
614
+ "eval_counts_2": 3219,
615
+ "eval_counts_3": 1561,
616
+ "eval_counts_4": 783,
617
+ "eval_exact_match": 0.02,
618
+ "eval_f1": 0.3753,
619
+ "eval_gen_len": 13.7618,
620
+ "eval_loss": 1.676971435546875,
621
+ "eval_precisions_1": 47.4598,
622
+ "eval_precisions_2": 20.57,
623
+ "eval_precisions_3": 11.6103,
624
+ "eval_precisions_4": 6.9656,
625
+ "eval_ref_len": 21250,
626
+ "eval_rouge1": 0.3821,
627
+ "eval_rouge2": 0.1948,
628
+ "eval_rougeL": 0.3669,
629
+ "eval_rougeLsum": 0.3665,
630
+ "eval_runtime": 452.0799,
631
+ "eval_samples_per_second": 4.875,
632
+ "eval_steps_per_second": 1.219,
633
+ "eval_sys_len": 17853,
634
+ "eval_totals_1": 17853,
635
+ "eval_totals_2": 15649,
636
+ "eval_totals_3": 13445,
637
+ "eval_totals_4": 11241,
638
+ "step": 2475
639
+ },
640
+ {
641
+ "epoch": 18.0,
642
+ "learning_rate": 0.0001,
643
+ "loss": 1.199,
644
+ "step": 2621
645
+ },
646
+ {
647
+ "epoch": 18.0,
648
+ "eval_bleu": 13.7496,
649
+ "eval_bp": 0.8326,
650
+ "eval_counts_1": 8484,
651
+ "eval_counts_2": 3190,
652
+ "eval_counts_3": 1551,
653
+ "eval_counts_4": 771,
654
+ "eval_exact_match": 0.0186,
655
+ "eval_f1": 0.3745,
656
+ "eval_gen_len": 13.8798,
657
+ "eval_loss": 1.6934301853179932,
658
+ "eval_precisions_1": 47.2409,
659
+ "eval_precisions_2": 20.2475,
660
+ "eval_precisions_3": 11.4456,
661
+ "eval_precisions_4": 6.7947,
662
+ "eval_ref_len": 21250,
663
+ "eval_rouge1": 0.3812,
664
+ "eval_rouge2": 0.1922,
665
+ "eval_rougeL": 0.3657,
666
+ "eval_rougeLsum": 0.3658,
667
+ "eval_runtime": 869.0302,
668
+ "eval_samples_per_second": 2.536,
669
+ "eval_steps_per_second": 0.634,
670
+ "eval_sys_len": 17959,
671
+ "eval_totals_1": 17959,
672
+ "eval_totals_2": 15755,
673
+ "eval_totals_3": 13551,
674
+ "eval_totals_4": 11347,
675
+ "step": 2621
676
+ },
677
+ {
678
+ "epoch": 18.99,
679
+ "learning_rate": 0.0001,
680
+ "loss": 1.1668,
681
+ "step": 2766
682
+ },
683
+ {
684
+ "epoch": 18.99,
685
+ "eval_bleu": 13.7379,
686
+ "eval_bp": 0.8395,
687
+ "eval_counts_1": 8504,
688
+ "eval_counts_2": 3179,
689
+ "eval_counts_3": 1541,
690
+ "eval_counts_4": 776,
691
+ "eval_exact_match": 0.0204,
692
+ "eval_f1": 0.376,
693
+ "eval_gen_len": 13.9256,
694
+ "eval_loss": 1.6926020383834839,
695
+ "eval_precisions_1": 47.0198,
696
+ "eval_precisions_2": 20.0164,
697
+ "eval_precisions_3": 11.2663,
698
+ "eval_precisions_4": 6.7631,
699
+ "eval_ref_len": 21250,
700
+ "eval_rouge1": 0.3828,
701
+ "eval_rouge2": 0.1939,
702
+ "eval_rougeL": 0.3665,
703
+ "eval_rougeLsum": 0.3665,
704
+ "eval_runtime": 580.7372,
705
+ "eval_samples_per_second": 3.795,
706
+ "eval_steps_per_second": 0.949,
707
+ "eval_sys_len": 18086,
708
+ "eval_totals_1": 18086,
709
+ "eval_totals_2": 15882,
710
+ "eval_totals_3": 13678,
711
+ "eval_totals_4": 11474,
712
+ "step": 2766
713
+ },
714
+ {
715
+ "epoch": 19.91,
716
+ "learning_rate": 0.0001,
717
+ "loss": 1.1164,
718
+ "step": 2900
719
+ },
720
+ {
721
+ "epoch": 19.91,
722
+ "eval_bleu": 14.1906,
723
+ "eval_bp": 0.8529,
724
+ "eval_counts_1": 8625,
725
+ "eval_counts_2": 3250,
726
+ "eval_counts_3": 1609,
727
+ "eval_counts_4": 820,
728
+ "eval_exact_match": 0.0204,
729
+ "eval_f1": 0.3803,
730
+ "eval_gen_len": 14.069,
731
+ "eval_loss": 1.7026218175888062,
732
+ "eval_precisions_1": 47.0463,
733
+ "eval_precisions_2": 20.15,
734
+ "eval_precisions_3": 11.5548,
735
+ "eval_precisions_4": 6.996,
736
+ "eval_ref_len": 21250,
737
+ "eval_rouge1": 0.3874,
738
+ "eval_rouge2": 0.1964,
739
+ "eval_rougeL": 0.3716,
740
+ "eval_rougeLsum": 0.3715,
741
+ "eval_runtime": 462.8982,
742
+ "eval_samples_per_second": 4.761,
743
+ "eval_steps_per_second": 1.19,
744
+ "eval_sys_len": 18333,
745
+ "eval_totals_1": 18333,
746
+ "eval_totals_2": 16129,
747
+ "eval_totals_3": 13925,
748
+ "eval_totals_4": 11721,
749
+ "step": 2900
750
+ },
751
+ {
752
+ "epoch": 19.91,
753
+ "step": 2900,
754
+ "total_flos": 2.54036307345408e+17,
755
+ "train_loss": 1.724,
756
+ "train_runtime": 25476.0,
757
+ "train_samples_per_second": 7.312,
758
+ "train_steps_per_second": 0.114
759
+ }
760
+ ],
761
+ "logging_steps": 500,
762
+ "max_steps": 2900,
763
+ "num_train_epochs": 20,
764
+ "save_steps": 500,
765
+ "total_flos": 2.54036307345408e+17,
766
+ "trial_name": null,
767
+ "trial_params": null
768
+ }
training_args.bin ADDED
Binary file (4.73 kB). View file