ilsilfverskiold/iptc-newscodes-multilingual-text-classification

README.md

@@ -0,0 +1,90 @@
+---
+license: mit
+base_model: roberta-base
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: news_category_classification
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# news_category_classification
+
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.8703
+- Accuracy: 0.7194
+- F1: 0.7086
+- Precision: 0.7186
+- Recall: 0.7194
+- Accuracy Label Arts, culture, entertainment and media: 0.8333
+- Accuracy Label Conflict, war and peace: 0.7287
+- Accuracy Label Crime, law and justice: 0.8497
+- Accuracy Label Disaster, accident, and emergency incident: 0.8931
+- Accuracy Label Economy, business, and finance: 0.7722
+- Accuracy Label Environment: 0.3125
+- Accuracy Label Health: 0.8
+- Accuracy Label Human interest: 0.3333
+- Accuracy Label Labour: 0.5
+- Accuracy Label Lifestyle and leisure: 0.5
+- Accuracy Label Politics: 0.6403
+- Accuracy Label Religion: 0.0
+- Accuracy Label Science and technology: 0.4167
+- Accuracy Label Society: 0.1579
+- Accuracy Label Sport: 0.9615
+- Accuracy Label Weather: 1.0
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 3
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1     | Precision | Recall | Accuracy Label Arts, culture, entertainment and media | Accuracy Label Conflict, war and peace | Accuracy Label Crime, law and justice | Accuracy Label Disaster, accident, and emergency incident | Accuracy Label Economy, business, and finance | Accuracy Label Environment | Accuracy Label Health | Accuracy Label Human interest | Accuracy Label Labour | Accuracy Label Lifestyle and leisure | Accuracy Label Politics | Accuracy Label Religion | Accuracy Label Science and technology | Accuracy Label Society | Accuracy Label Sport | Accuracy Label Weather |
+|:-------------:|:------:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|:-----------------------------------------------------:|:--------------------------------------:|:-------------------------------------:|:---------------------------------------------------------:|:---------------------------------------------:|:--------------------------:|:---------------------:|:-----------------------------:|:---------------------:|:------------------------------------:|:-----------------------:|:-----------------------:|:-------------------------------------:|:----------------------:|:--------------------:|:----------------------:|
+| 1.8543        | 0.4796 | 200  | 1.4875          | 0.5046   | 0.4359 | 0.4820    | 0.5046 | 0.0833                                                | 0.8298                                 | 0.0751                                | 0.8855                                                    | 0.7722                                        | 0.0                        | 0.4                   | 0.0                           | 0.0                   | 0.5                                  | 0.4101                  | 0.0                     | 0.0                                   | 0.0                    | 0.9615               | 0.0                    |
+| 1.2587        | 0.9592 | 400  | 1.1350          | 0.6582   | 0.6443 | 0.6778    | 0.6582 | 0.5833                                                | 0.7766                                 | 0.8844                                | 0.5573                                                    | 0.8228                                        | 0.25                       | 0.9                   | 0.25                          | 1.0                   | 0.75                                 | 0.5324                  | 0.0                     | 0.3333                                | 0.0                    | 0.9231               | 0.0                    |
+| 0.7676        | 1.4388 | 600  | 0.9734          | 0.6917   | 0.6766 | 0.7148    | 0.6917 | 0.5                                                   | 0.6489                                 | 0.9075                                | 0.7634                                                    | 0.8228                                        | 0.25                       | 0.8                   | 0.3333                        | 0.5                   | 0.875                                | 0.6763                  | 0.0                     | 0.4167                                | 0.0351                 | 0.8846               | 1.0                    |
+| 0.7325        | 1.9185 | 800  | 0.8975          | 0.7079   | 0.6902 | 0.7174    | 0.7079 | 0.8333                                                | 0.7606                                 | 0.8497                                | 0.8855                                                    | 0.7089                                        | 0.25                       | 0.8                   | 0.3333                        | 0.5                   | 0.375                                | 0.6259                  | 0.0                     | 0.5                                   | 0.0351                 | 0.9615               | 1.0                    |
+| 0.5137        | 2.3981 | 1000 | 0.8819          | 0.7148   | 0.7014 | 0.7235    | 0.7148 | 0.9167                                                | 0.7021                                 | 0.8728                                | 0.8855                                                    | 0.7215                                        | 0.375                      | 0.8                   | 0.25                          | 0.5                   | 0.625                                | 0.6475                  | 0.0                     | 0.5833                                | 0.0877                 | 1.0                  | 1.0                    |
+| 0.5674        | 2.8777 | 1200 | 0.8749          | 0.7194   | 0.7086 | 0.7207    | 0.7194 | 0.8333                                                | 0.75                                   | 0.8497                                | 0.8931                                                    | 0.7342                                        | 0.3125                     | 0.8                   | 0.25                          | 0.5                   | 0.375                                | 0.6475                  | 0.0                     | 0.5                                   | 0.1404                 | 0.9615               | 1.0                    |
+
+
+### Framework versions
+
+- Transformers 4.40.2
+- Pytorch 2.2.1+cu121
+- Datasets 2.19.1
+- Tokenizers 0.19.1

all_results.json

@@ -0,0 +1,28 @@
+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.7193995381062356,
+    "eval_accuracy_label_arts, culture, entertainment and media": 0.8333333333333334,
+    "eval_accuracy_label_conflict, war and peace": 0.7287234042553191,
+    "eval_accuracy_label_crime, law and justice": 0.8497109826589595,
+    "eval_accuracy_label_disaster, accident, and emergency incident": 0.8931297709923665,
+    "eval_accuracy_label_economy, business, and finance": 0.7721518987341772,
+    "eval_accuracy_label_environment": 0.3125,
+    "eval_accuracy_label_health": 0.8,
+    "eval_accuracy_label_human interest": 0.3333333333333333,
+    "eval_accuracy_label_labour": 0.5,
+    "eval_accuracy_label_lifestyle and leisure": 0.5,
+    "eval_accuracy_label_politics": 0.6402877697841727,
+    "eval_accuracy_label_religion": 0.0,
+    "eval_accuracy_label_science and technology": 0.4166666666666667,
+    "eval_accuracy_label_society": 0.15789473684210525,
+    "eval_accuracy_label_sport": 0.9615384615384616,
+    "eval_accuracy_label_weather": 1.0,
+    "eval_f1": 0.7086386434582244,
+    "eval_loss": 0.8703184723854065,
+    "eval_precision": 0.7185855451718517,
+    "eval_recall": 0.7193995381062356,
+    "eval_runtime": 6.4919,
+    "eval_samples_per_second": 133.397,
+    "eval_steps_per_second": 8.472,
+    "step": 1251
+}

config.json

@@ -0,0 +1,64 @@
+{
+  "_name_or_path": "roberta-base",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "arts, culture, entertainment and media",
+    "1": "conflict, war and peace",
+    "2": "crime, law and justice",
+    "3": "disaster, accident, and emergency incident",
+    "4": "economy, business, and finance",
+    "5": "environment",
+    "6": "health",
+    "7": "human interest",
+    "8": "labour",
+    "9": "lifestyle and leisure",
+    "10": "politics",
+    "11": "religion",
+    "12": "science and technology",
+    "13": "society",
+    "14": "sport",
+    "15": "weather"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "arts, culture, entertainment and media": 0,
+    "conflict, war and peace": 1,
+    "crime, law and justice": 2,
+    "disaster, accident, and emergency incident": 3,
+    "economy, business, and finance": 4,
+    "environment": 5,
+    "health": 6,
+    "human interest": 7,
+    "labour": 8,
+    "lifestyle and leisure": 9,
+    "politics": 10,
+    "religion": 11,
+    "science and technology": 12,
+    "society": 13,
+    "sport": 14,
+    "weather": 15
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

eval_results.json

@@ -0,0 +1,28 @@
+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.7193995381062356,
+    "eval_accuracy_label_arts, culture, entertainment and media": 0.8333333333333334,
+    "eval_accuracy_label_conflict, war and peace": 0.7287234042553191,
+    "eval_accuracy_label_crime, law and justice": 0.8497109826589595,
+    "eval_accuracy_label_disaster, accident, and emergency incident": 0.8931297709923665,
+    "eval_accuracy_label_economy, business, and finance": 0.7721518987341772,
+    "eval_accuracy_label_environment": 0.3125,
+    "eval_accuracy_label_health": 0.8,
+    "eval_accuracy_label_human interest": 0.3333333333333333,
+    "eval_accuracy_label_labour": 0.5,
+    "eval_accuracy_label_lifestyle and leisure": 0.5,
+    "eval_accuracy_label_politics": 0.6402877697841727,
+    "eval_accuracy_label_religion": 0.0,
+    "eval_accuracy_label_science and technology": 0.4166666666666667,
+    "eval_accuracy_label_society": 0.15789473684210525,
+    "eval_accuracy_label_sport": 0.9615384615384616,
+    "eval_accuracy_label_weather": 1.0,
+    "eval_f1": 0.7086386434582244,
+    "eval_loss": 0.8703184723854065,
+    "eval_precision": 0.7185855451718517,
+    "eval_recall": 0.7193995381062356,
+    "eval_runtime": 6.4919,
+    "eval_samples_per_second": 133.397,
+    "eval_steps_per_second": 8.472,
+    "step": 1251
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0bb16bad35a0be49bc055367eff711777e5c1233c2075a0cea4ff2bfb4e791a
+size 498655888

runs/May16_07-34-56_add8af2633c6/events.out.tfevents.1715844897.add8af2633c6.851.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bad41f09347409948834e3f0a1431f8ff16c4a20877cd95dbcdb329cc87b744
+size 42482

runs/May16_07-34-56_add8af2633c6/events.out.tfevents.1715845950.add8af2633c6.851.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a43b7db1a26a971f34df40ec9e2f166d071239177d7cb5a60226804670bc8aa7
+size 1774

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,57 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}

trainer_state.json

@@ -0,0 +1,1101 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 200,
+  "global_step": 1251,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.023980815347721823,
+      "grad_norm": 2.6759583950042725,
+      "learning_rate": 4.0000000000000003e-07,
+      "loss": 2.7491,
+      "step": 10
+    },
+    {
+      "epoch": 0.047961630695443645,
+      "grad_norm": 3.8336291313171387,
+      "learning_rate": 8.000000000000001e-07,
+      "loss": 2.7607,
+      "step": 20
+    },
+    {
+      "epoch": 0.07194244604316546,
+      "grad_norm": 2.368133068084717,
+      "learning_rate": 1.2000000000000002e-06,
+      "loss": 2.7517,
+      "step": 30
+    },
+    {
+      "epoch": 0.09592326139088729,
+      "grad_norm": 3.0174171924591064,
+      "learning_rate": 1.6000000000000001e-06,
+      "loss": 2.7446,
+      "step": 40
+    },
+    {
+      "epoch": 0.11990407673860912,
+      "grad_norm": 2.466191053390503,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 2.7385,
+      "step": 50
+    },
+    {
+      "epoch": 0.14388489208633093,
+      "grad_norm": 2.4236373901367188,
+      "learning_rate": 2.4000000000000003e-06,
+      "loss": 2.7465,
+      "step": 60
+    },
+    {
+      "epoch": 0.16786570743405277,
+      "grad_norm": 2.943648338317871,
+      "learning_rate": 2.8000000000000003e-06,
+      "loss": 2.7122,
+      "step": 70
+    },
+    {
+      "epoch": 0.19184652278177458,
+      "grad_norm": 2.8402764797210693,
+      "learning_rate": 3.2000000000000003e-06,
+      "loss": 2.6785,
+      "step": 80
+    },
+    {
+      "epoch": 0.2158273381294964,
+      "grad_norm": 3.464658498764038,
+      "learning_rate": 3.6000000000000003e-06,
+      "loss": 2.6623,
+      "step": 90
+    },
+    {
+      "epoch": 0.23980815347721823,
+      "grad_norm": 4.406158924102783,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 2.5789,
+      "step": 100
+    },
+    {
+      "epoch": 0.2637889688249401,
+      "grad_norm": 4.503725051879883,
+      "learning_rate": 4.4e-06,
+      "loss": 2.5131,
+      "step": 110
+    },
+    {
+      "epoch": 0.28776978417266186,
+      "grad_norm": 3.690744400024414,
+      "learning_rate": 4.800000000000001e-06,
+      "loss": 2.4688,
+      "step": 120
+    },
+    {
+      "epoch": 0.3117505995203837,
+      "grad_norm": 6.098635196685791,
+      "learning_rate": 5.2e-06,
+      "loss": 2.4166,
+      "step": 130
+    },
+    {
+      "epoch": 0.33573141486810554,
+      "grad_norm": 6.481673717498779,
+      "learning_rate": 5.600000000000001e-06,
+      "loss": 2.3415,
+      "step": 140
+    },
+    {
+      "epoch": 0.3597122302158273,
+      "grad_norm": 7.292446136474609,
+      "learning_rate": 6e-06,
+      "loss": 2.1885,
+      "step": 150
+    },
+    {
+      "epoch": 0.38369304556354916,
+      "grad_norm": 5.510867595672607,
+      "learning_rate": 6.4000000000000006e-06,
+      "loss": 2.1908,
+      "step": 160
+    },
+    {
+      "epoch": 0.407673860911271,
+      "grad_norm": 7.096005439758301,
+      "learning_rate": 6.800000000000001e-06,
+      "loss": 2.1191,
+      "step": 170
+    },
+    {
+      "epoch": 0.4316546762589928,
+      "grad_norm": 6.810784816741943,
+      "learning_rate": 7.2000000000000005e-06,
+      "loss": 1.9654,
+      "step": 180
+    },
+    {
+      "epoch": 0.4556354916067146,
+      "grad_norm": 10.735515594482422,
+      "learning_rate": 7.600000000000001e-06,
+      "loss": 1.9332,
+      "step": 190
+    },
+    {
+      "epoch": 0.47961630695443647,
+      "grad_norm": 8.408844947814941,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 1.8543,
+      "step": 200
+    },
+    {
+      "epoch": 0.47961630695443647,
+      "eval_accuracy": 0.5046189376443418,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.08333333333333333,
+      "eval_accuracy_label_conflict, war and peace": 0.8297872340425532,
+      "eval_accuracy_label_crime, law and justice": 0.07514450867052024,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.8854961832061069,
+      "eval_accuracy_label_economy, business, and finance": 0.7721518987341772,
+      "eval_accuracy_label_environment": 0.0,
+      "eval_accuracy_label_health": 0.4,
+      "eval_accuracy_label_human interest": 0.0,
+      "eval_accuracy_label_labour": 0.0,
+      "eval_accuracy_label_lifestyle and leisure": 0.5,
+      "eval_accuracy_label_politics": 0.41007194244604317,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.0,
+      "eval_accuracy_label_society": 0.0,
+      "eval_accuracy_label_sport": 0.9615384615384616,
+      "eval_accuracy_label_weather": 0.0,
+      "eval_f1": 0.435890569795384,
+      "eval_loss": 1.4875246286392212,
+      "eval_precision": 0.48203900386478227,
+      "eval_recall": 0.5046189376443418,
+      "eval_runtime": 6.7717,
+      "eval_samples_per_second": 127.886,
+      "eval_steps_per_second": 8.122,
+      "step": 200
+    },
+    {
+      "epoch": 0.5035971223021583,
+      "grad_norm": 7.815601348876953,
+      "learning_rate": 8.400000000000001e-06,
+      "loss": 1.8394,
+      "step": 210
+    },
+    {
+      "epoch": 0.5275779376498801,
+      "grad_norm": 7.226934432983398,
+      "learning_rate": 8.8e-06,
+      "loss": 1.7945,
+      "step": 220
+    },
+    {
+      "epoch": 0.5515587529976019,
+      "grad_norm": 9.982332229614258,
+      "learning_rate": 9.200000000000002e-06,
+      "loss": 1.6757,
+      "step": 230
+    },
+    {
+      "epoch": 0.5755395683453237,
+      "grad_norm": 11.880012512207031,
+      "learning_rate": 9.600000000000001e-06,
+      "loss": 1.6992,
+      "step": 240
+    },
+    {
+      "epoch": 0.5995203836930456,
+      "grad_norm": 13.93294620513916,
+      "learning_rate": 1e-05,
+      "loss": 1.6815,
+      "step": 250
+    },
+    {
+      "epoch": 0.6235011990407674,
+      "grad_norm": 8.252928733825684,
+      "learning_rate": 1.04e-05,
+      "loss": 1.5667,
+      "step": 260
+    },
+    {
+      "epoch": 0.6474820143884892,
+      "grad_norm": 17.642982482910156,
+      "learning_rate": 1.0800000000000002e-05,
+      "loss": 1.4947,
+      "step": 270
+    },
+    {
+      "epoch": 0.6714628297362111,
+      "grad_norm": 17.528329849243164,
+      "learning_rate": 1.1200000000000001e-05,
+      "loss": 1.5726,
+      "step": 280
+    },
+    {
+      "epoch": 0.6954436450839329,
+      "grad_norm": 13.026844024658203,
+      "learning_rate": 1.16e-05,
+      "loss": 1.48,
+      "step": 290
+    },
+    {
+      "epoch": 0.7194244604316546,
+      "grad_norm": 9.975683212280273,
+      "learning_rate": 1.2e-05,
+      "loss": 1.4571,
+      "step": 300
+    },
+    {
+      "epoch": 0.7434052757793765,
+      "grad_norm": 14.437339782714844,
+      "learning_rate": 1.2400000000000002e-05,
+      "loss": 1.4895,
+      "step": 310
+    },
+    {
+      "epoch": 0.7673860911270983,
+      "grad_norm": 15.256731986999512,
+      "learning_rate": 1.2800000000000001e-05,
+      "loss": 1.3526,
+      "step": 320
+    },
+    {
+      "epoch": 0.7913669064748201,
+      "grad_norm": 12.484004974365234,
+      "learning_rate": 1.3200000000000002e-05,
+      "loss": 1.4127,
+      "step": 330
+    },
+    {
+      "epoch": 0.815347721822542,
+      "grad_norm": 11.139123916625977,
+      "learning_rate": 1.3600000000000002e-05,
+      "loss": 1.2786,
+      "step": 340
+    },
+    {
+      "epoch": 0.8393285371702638,
+      "grad_norm": 13.130227088928223,
+      "learning_rate": 1.4e-05,
+      "loss": 1.3938,
+      "step": 350
+    },
+    {
+      "epoch": 0.8633093525179856,
+      "grad_norm": 23.369935989379883,
+      "learning_rate": 1.4400000000000001e-05,
+      "loss": 1.3072,
+      "step": 360
+    },
+    {
+      "epoch": 0.8872901678657075,
+      "grad_norm": 12.180148124694824,
+      "learning_rate": 1.48e-05,
+      "loss": 1.2709,
+      "step": 370
+    },
+    {
+      "epoch": 0.9112709832134293,
+      "grad_norm": 10.738848686218262,
+      "learning_rate": 1.5200000000000002e-05,
+      "loss": 1.1652,
+      "step": 380
+    },
+    {
+      "epoch": 0.935251798561151,
+      "grad_norm": 15.606388092041016,
+      "learning_rate": 1.5600000000000003e-05,
+      "loss": 1.1456,
+      "step": 390
+    },
+    {
+      "epoch": 0.9592326139088729,
+      "grad_norm": 12.829926490783691,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 1.2587,
+      "step": 400
+    },
+    {
+      "epoch": 0.9592326139088729,
+      "eval_accuracy": 0.6581986143187067,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.5833333333333334,
+      "eval_accuracy_label_conflict, war and peace": 0.776595744680851,
+      "eval_accuracy_label_crime, law and justice": 0.884393063583815,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.5572519083969466,
+      "eval_accuracy_label_economy, business, and finance": 0.8227848101265823,
+      "eval_accuracy_label_environment": 0.25,
+      "eval_accuracy_label_health": 0.9,
+      "eval_accuracy_label_human interest": 0.25,
+      "eval_accuracy_label_labour": 1.0,
+      "eval_accuracy_label_lifestyle and leisure": 0.75,
+      "eval_accuracy_label_politics": 0.5323741007194245,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.3333333333333333,
+      "eval_accuracy_label_society": 0.0,
+      "eval_accuracy_label_sport": 0.9230769230769231,
+      "eval_accuracy_label_weather": 0.0,
+      "eval_f1": 0.6443030541380625,
+      "eval_loss": 1.1349692344665527,
+      "eval_precision": 0.677835895160577,
+      "eval_recall": 0.6581986143187067,
+      "eval_runtime": 6.6147,
+      "eval_samples_per_second": 130.92,
+      "eval_steps_per_second": 8.315,
+      "step": 400
+    },
+    {
+      "epoch": 0.9832134292565947,
+      "grad_norm": 9.225656509399414,
+      "learning_rate": 1.64e-05,
+      "loss": 1.1672,
+      "step": 410
+    },
+    {
+      "epoch": 1.0071942446043165,
+      "grad_norm": 18.772287368774414,
+      "learning_rate": 1.6800000000000002e-05,
+      "loss": 1.0881,
+      "step": 420
+    },
+    {
+      "epoch": 1.0311750599520384,
+      "grad_norm": 9.325098991394043,
+      "learning_rate": 1.72e-05,
+      "loss": 1.092,
+      "step": 430
+    },
+    {
+      "epoch": 1.0551558752997603,
+      "grad_norm": 13.573101043701172,
+      "learning_rate": 1.76e-05,
+      "loss": 1.0611,
+      "step": 440
+    },
+    {
+      "epoch": 1.079136690647482,
+      "grad_norm": 13.348872184753418,
+      "learning_rate": 1.8e-05,
+      "loss": 1.1549,
+      "step": 450
+    },
+    {
+      "epoch": 1.1031175059952039,
+      "grad_norm": 14.022222518920898,
+      "learning_rate": 1.8400000000000003e-05,
+      "loss": 1.0195,
+      "step": 460
+    },
+    {
+      "epoch": 1.1270983213429258,
+      "grad_norm": 21.117116928100586,
+      "learning_rate": 1.88e-05,
+      "loss": 0.928,
+      "step": 470
+    },
+    {
+      "epoch": 1.1510791366906474,
+      "grad_norm": 11.059869766235352,
+      "learning_rate": 1.9200000000000003e-05,
+      "loss": 1.0565,
+      "step": 480
+    },
+    {
+      "epoch": 1.1750599520383693,
+      "grad_norm": 23.053054809570312,
+      "learning_rate": 1.9600000000000002e-05,
+      "loss": 1.0343,
+      "step": 490
+    },
+    {
+      "epoch": 1.1990407673860912,
+      "grad_norm": 21.17502212524414,
+      "learning_rate": 2e-05,
+      "loss": 1.0108,
+      "step": 500
+    },
+    {
+      "epoch": 1.223021582733813,
+      "grad_norm": 11.383284568786621,
+      "learning_rate": 1.9733688415446073e-05,
+      "loss": 0.952,
+      "step": 510
+    },
+    {
+      "epoch": 1.2470023980815348,
+      "grad_norm": 11.280231475830078,
+      "learning_rate": 1.9467376830892145e-05,
+      "loss": 0.8728,
+      "step": 520
+    },
+    {
+      "epoch": 1.2709832134292567,
+      "grad_norm": 12.41702938079834,
+      "learning_rate": 1.9201065246338217e-05,
+      "loss": 0.8844,
+      "step": 530
+    },
+    {
+      "epoch": 1.2949640287769784,
+      "grad_norm": 19.46651268005371,
+      "learning_rate": 1.893475366178429e-05,
+      "loss": 0.8724,
+      "step": 540
+    },
+    {
+      "epoch": 1.3189448441247003,
+      "grad_norm": 15.608144760131836,
+      "learning_rate": 1.866844207723036e-05,
+      "loss": 0.9734,
+      "step": 550
+    },
+    {
+      "epoch": 1.3429256594724222,
+      "grad_norm": 18.18840217590332,
+      "learning_rate": 1.8402130492676432e-05,
+      "loss": 0.9805,
+      "step": 560
+    },
+    {
+      "epoch": 1.3669064748201438,
+      "grad_norm": 11.583151817321777,
+      "learning_rate": 1.8135818908122504e-05,
+      "loss": 0.973,
+      "step": 570
+    },
+    {
+      "epoch": 1.3908872901678657,
+      "grad_norm": 17.457368850708008,
+      "learning_rate": 1.7869507323568576e-05,
+      "loss": 1.0181,
+      "step": 580
+    },
+    {
+      "epoch": 1.4148681055155876,
+      "grad_norm": 12.27700138092041,
+      "learning_rate": 1.7603195739014648e-05,
+      "loss": 0.9532,
+      "step": 590
+    },
+    {
+      "epoch": 1.4388489208633093,
+      "grad_norm": 7.117462635040283,
+      "learning_rate": 1.733688415446072e-05,
+      "loss": 0.7676,
+      "step": 600
+    },
+    {
+      "epoch": 1.4388489208633093,
+      "eval_accuracy": 0.6916859122401847,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.5,
+      "eval_accuracy_label_conflict, war and peace": 0.648936170212766,
+      "eval_accuracy_label_crime, law and justice": 0.9075144508670521,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.7633587786259542,
+      "eval_accuracy_label_economy, business, and finance": 0.8227848101265823,
+      "eval_accuracy_label_environment": 0.25,
+      "eval_accuracy_label_health": 0.8,
+      "eval_accuracy_label_human interest": 0.3333333333333333,
+      "eval_accuracy_label_labour": 0.5,
+      "eval_accuracy_label_lifestyle and leisure": 0.875,
+      "eval_accuracy_label_politics": 0.6762589928057554,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.4166666666666667,
+      "eval_accuracy_label_society": 0.03508771929824561,
+      "eval_accuracy_label_sport": 0.8846153846153846,
+      "eval_accuracy_label_weather": 1.0,
+      "eval_f1": 0.6766423167988369,
+      "eval_loss": 0.9734256267547607,
+      "eval_precision": 0.7148268355364681,
+      "eval_recall": 0.6916859122401847,
+      "eval_runtime": 6.651,
+      "eval_samples_per_second": 130.206,
+      "eval_steps_per_second": 8.269,
+      "step": 600
+    },
+    {
+      "epoch": 1.4628297362110312,
+      "grad_norm": 25.153915405273438,
+      "learning_rate": 1.707057256990679e-05,
+      "loss": 0.9907,
+      "step": 610
+    },
+    {
+      "epoch": 1.486810551558753,
+      "grad_norm": 16.892343521118164,
+      "learning_rate": 1.6804260985352863e-05,
+      "loss": 0.8732,
+      "step": 620
+    },
+    {
+      "epoch": 1.5107913669064748,
+      "grad_norm": 10.182963371276855,
+      "learning_rate": 1.6537949400798935e-05,
+      "loss": 0.885,
+      "step": 630
+    },
+    {
+      "epoch": 1.5347721822541966,
+      "grad_norm": 19.816036224365234,
+      "learning_rate": 1.6271637816245007e-05,
+      "loss": 0.853,
+      "step": 640
+    },
+    {
+      "epoch": 1.5587529976019185,
+      "grad_norm": 10.677128791809082,
+      "learning_rate": 1.6005326231691082e-05,
+      "loss": 0.8802,
+      "step": 650
+    },
+    {
+      "epoch": 1.5827338129496402,
+      "grad_norm": 17.55293846130371,
+      "learning_rate": 1.573901464713715e-05,
+      "loss": 0.9211,
+      "step": 660
+    },
+    {
+      "epoch": 1.6067146282973621,
+      "grad_norm": 16.497119903564453,
+      "learning_rate": 1.5472703062583222e-05,
+      "loss": 0.8114,
+      "step": 670
+    },
+    {
+      "epoch": 1.630695443645084,
+      "grad_norm": 12.596357345581055,
+      "learning_rate": 1.5206391478029296e-05,
+      "loss": 0.8601,
+      "step": 680
+    },
+    {
+      "epoch": 1.6546762589928057,
+      "grad_norm": 20.318208694458008,
+      "learning_rate": 1.4940079893475368e-05,
+      "loss": 0.9026,
+      "step": 690
+    },
+    {
+      "epoch": 1.6786570743405276,
+      "grad_norm": 30.881654739379883,
+      "learning_rate": 1.467376830892144e-05,
+      "loss": 0.7974,
+      "step": 700
+    },
+    {
+      "epoch": 1.7026378896882495,
+      "grad_norm": 13.157632827758789,
+      "learning_rate": 1.440745672436751e-05,
+      "loss": 0.7966,
+      "step": 710
+    },
+    {
+      "epoch": 1.7266187050359711,
+      "grad_norm": 12.032391548156738,
+      "learning_rate": 1.4141145139813583e-05,
+      "loss": 0.667,
+      "step": 720
+    },
+    {
+      "epoch": 1.750599520383693,
+      "grad_norm": 14.202383995056152,
+      "learning_rate": 1.3874833555259655e-05,
+      "loss": 0.7959,
+      "step": 730
+    },
+    {
+      "epoch": 1.774580335731415,
+      "grad_norm": 14.563888549804688,
+      "learning_rate": 1.3608521970705725e-05,
+      "loss": 0.7965,
+      "step": 740
+    },
+    {
+      "epoch": 1.7985611510791366,
+      "grad_norm": 14.846811294555664,
+      "learning_rate": 1.3342210386151799e-05,
+      "loss": 0.7133,
+      "step": 750
+    },
+    {
+      "epoch": 1.8225419664268585,
+      "grad_norm": 10.936131477355957,
+      "learning_rate": 1.307589880159787e-05,
+      "loss": 0.8209,
+      "step": 760
+    },
+    {
+      "epoch": 1.8465227817745804,
+      "grad_norm": 12.352126121520996,
+      "learning_rate": 1.2809587217043942e-05,
+      "loss": 0.7926,
+      "step": 770
+    },
+    {
+      "epoch": 1.870503597122302,
+      "grad_norm": 13.540229797363281,
+      "learning_rate": 1.2543275632490016e-05,
+      "loss": 0.7735,
+      "step": 780
+    },
+    {
+      "epoch": 1.894484412470024,
+      "grad_norm": 19.36661148071289,
+      "learning_rate": 1.2276964047936086e-05,
+      "loss": 0.6911,
+      "step": 790
+    },
+    {
+      "epoch": 1.9184652278177459,
+      "grad_norm": 18.726093292236328,
+      "learning_rate": 1.2010652463382158e-05,
+      "loss": 0.7325,
+      "step": 800
+    },
+    {
+      "epoch": 1.9184652278177459,
+      "eval_accuracy": 0.707852193995381,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.8333333333333334,
+      "eval_accuracy_label_conflict, war and peace": 0.7606382978723404,
+      "eval_accuracy_label_crime, law and justice": 0.8497109826589595,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.8854961832061069,
+      "eval_accuracy_label_economy, business, and finance": 0.7088607594936709,
+      "eval_accuracy_label_environment": 0.25,
+      "eval_accuracy_label_health": 0.8,
+      "eval_accuracy_label_human interest": 0.3333333333333333,
+      "eval_accuracy_label_labour": 0.5,
+      "eval_accuracy_label_lifestyle and leisure": 0.375,
+      "eval_accuracy_label_politics": 0.6258992805755396,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.5,
+      "eval_accuracy_label_society": 0.03508771929824561,
+      "eval_accuracy_label_sport": 0.9615384615384616,
+      "eval_accuracy_label_weather": 1.0,
+      "eval_f1": 0.690238303485375,
+      "eval_loss": 0.8974892497062683,
+      "eval_precision": 0.7174126972067543,
+      "eval_recall": 0.707852193995381,
+      "eval_runtime": 6.6296,
+      "eval_samples_per_second": 130.627,
+      "eval_steps_per_second": 8.296,
+      "step": 800
+    },
+    {
+      "epoch": 1.9424460431654675,
+      "grad_norm": 13.153111457824707,
+      "learning_rate": 1.1744340878828231e-05,
+      "loss": 0.7655,
+      "step": 810
+    },
+    {
+      "epoch": 1.9664268585131894,
+      "grad_norm": 13.287749290466309,
+      "learning_rate": 1.1478029294274303e-05,
+      "loss": 0.7605,
+      "step": 820
+    },
+    {
+      "epoch": 1.9904076738609113,
+      "grad_norm": 18.040851593017578,
+      "learning_rate": 1.1211717709720373e-05,
+      "loss": 0.9042,
+      "step": 830
+    },
+    {
+      "epoch": 2.014388489208633,
+      "grad_norm": 8.862879753112793,
+      "learning_rate": 1.0945406125166447e-05,
+      "loss": 0.6534,
+      "step": 840
+    },
+    {
+      "epoch": 2.038369304556355,
+      "grad_norm": 17.532255172729492,
+      "learning_rate": 1.0679094540612518e-05,
+      "loss": 0.6613,
+      "step": 850
+    },
+    {
+      "epoch": 2.062350119904077,
+      "grad_norm": 16.810340881347656,
+      "learning_rate": 1.0412782956058588e-05,
+      "loss": 0.6129,
+      "step": 860
+    },
+    {
+      "epoch": 2.0863309352517985,
+      "grad_norm": 24.42365264892578,
+      "learning_rate": 1.014647137150466e-05,
+      "loss": 0.6734,
+      "step": 870
+    },
+    {
+      "epoch": 2.1103117505995206,
+      "grad_norm": 6.36922025680542,
+      "learning_rate": 9.880159786950732e-06,
+      "loss": 0.6116,
+      "step": 880
+    },
+    {
+      "epoch": 2.1342925659472423,
+      "grad_norm": 11.740708351135254,
+      "learning_rate": 9.613848202396806e-06,
+      "loss": 0.6638,
+      "step": 890
+    },
+    {
+      "epoch": 2.158273381294964,
+      "grad_norm": 14.415084838867188,
+      "learning_rate": 9.347536617842877e-06,
+      "loss": 0.5951,
+      "step": 900
+    },
+    {
+      "epoch": 2.182254196642686,
+      "grad_norm": 7.7266154289245605,
+      "learning_rate": 9.08122503328895e-06,
+      "loss": 0.6171,
+      "step": 910
+    },
+    {
+      "epoch": 2.2062350119904077,
+      "grad_norm": 15.567154884338379,
+      "learning_rate": 8.814913448735021e-06,
+      "loss": 0.5291,
+      "step": 920
+    },
+    {
+      "epoch": 2.2302158273381294,
+      "grad_norm": 14.188578605651855,
+      "learning_rate": 8.548601864181093e-06,
+      "loss": 0.58,
+      "step": 930
+    },
+    {
+      "epoch": 2.2541966426858515,
+      "grad_norm": 12.079745292663574,
+      "learning_rate": 8.282290279627165e-06,
+      "loss": 0.6561,
+      "step": 940
+    },
+    {
+      "epoch": 2.278177458033573,
+      "grad_norm": 9.993086814880371,
+      "learning_rate": 8.015978695073236e-06,
+      "loss": 0.619,
+      "step": 950
+    },
+    {
+      "epoch": 2.302158273381295,
+      "grad_norm": 8.76229476928711,
+      "learning_rate": 7.749667110519308e-06,
+      "loss": 0.5619,
+      "step": 960
+    },
+    {
+      "epoch": 2.326139088729017,
+      "grad_norm": 13.1128511428833,
+      "learning_rate": 7.48335552596538e-06,
+      "loss": 0.6011,
+      "step": 970
+    },
+    {
+      "epoch": 2.3501199040767387,
+      "grad_norm": 8.369950294494629,
+      "learning_rate": 7.217043941411453e-06,
+      "loss": 0.5869,
+      "step": 980
+    },
+    {
+      "epoch": 2.3741007194244603,
+      "grad_norm": 11.256261825561523,
+      "learning_rate": 6.950732356857524e-06,
+      "loss": 0.6031,
+      "step": 990
+    },
+    {
+      "epoch": 2.3980815347721824,
+      "grad_norm": 15.561044692993164,
+      "learning_rate": 6.6844207723035955e-06,
+      "loss": 0.5137,
+      "step": 1000
+    },
+    {
+      "epoch": 2.3980815347721824,
+      "eval_accuracy": 0.7147806004618937,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.9166666666666666,
+      "eval_accuracy_label_conflict, war and peace": 0.7021276595744681,
+      "eval_accuracy_label_crime, law and justice": 0.8728323699421965,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.8854961832061069,
+      "eval_accuracy_label_economy, business, and finance": 0.7215189873417721,
+      "eval_accuracy_label_environment": 0.375,
+      "eval_accuracy_label_health": 0.8,
+      "eval_accuracy_label_human interest": 0.25,
+      "eval_accuracy_label_labour": 0.5,
+      "eval_accuracy_label_lifestyle and leisure": 0.625,
+      "eval_accuracy_label_politics": 0.6474820143884892,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.5833333333333334,
+      "eval_accuracy_label_society": 0.08771929824561403,
+      "eval_accuracy_label_sport": 1.0,
+      "eval_accuracy_label_weather": 1.0,
+      "eval_f1": 0.7013615090517632,
+      "eval_loss": 0.8819282054901123,
+      "eval_precision": 0.7235415877948681,
+      "eval_recall": 0.7147806004618937,
+      "eval_runtime": 6.6195,
+      "eval_samples_per_second": 130.826,
+      "eval_steps_per_second": 8.309,
+      "step": 1000
+    },
+    {
+      "epoch": 2.422062350119904,
+      "grad_norm": 9.799003601074219,
+      "learning_rate": 6.418109187749668e-06,
+      "loss": 0.7382,
+      "step": 1010
+    },
+    {
+      "epoch": 2.446043165467626,
+      "grad_norm": 10.710299491882324,
+      "learning_rate": 6.151797603195739e-06,
+      "loss": 0.4921,
+      "step": 1020
+    },
+    {
+      "epoch": 2.470023980815348,
+      "grad_norm": 16.74972152709961,
+      "learning_rate": 5.885486018641812e-06,
+      "loss": 0.6228,
+      "step": 1030
+    },
+    {
+      "epoch": 2.4940047961630696,
+      "grad_norm": 10.896918296813965,
+      "learning_rate": 5.619174434087883e-06,
+      "loss": 0.5489,
+      "step": 1040
+    },
+    {
+      "epoch": 2.5179856115107913,
+      "grad_norm": 10.855011940002441,
+      "learning_rate": 5.3528628495339554e-06,
+      "loss": 0.495,
+      "step": 1050
+    },
+    {
+      "epoch": 2.5419664268585134,
+      "grad_norm": 18.55707359313965,
+      "learning_rate": 5.086551264980027e-06,
+      "loss": 0.6214,
+      "step": 1060
+    },
+    {
+      "epoch": 2.565947242206235,
+      "grad_norm": 15.628390312194824,
+      "learning_rate": 4.820239680426099e-06,
+      "loss": 0.5327,
+      "step": 1070
+    },
+    {
+      "epoch": 2.5899280575539567,
+      "grad_norm": 12.142158508300781,
+      "learning_rate": 4.553928095872171e-06,
+      "loss": 0.5343,
+      "step": 1080
+    },
+    {
+      "epoch": 2.6139088729016784,
+      "grad_norm": 12.887943267822266,
+      "learning_rate": 4.287616511318243e-06,
+      "loss": 0.5576,
+      "step": 1090
+    },
+    {
+      "epoch": 2.6378896882494005,
+      "grad_norm": 12.8055419921875,
+      "learning_rate": 4.0213049267643145e-06,
+      "loss": 0.5332,
+      "step": 1100
+    },
+    {
+      "epoch": 2.661870503597122,
+      "grad_norm": 7.185519218444824,
+      "learning_rate": 3.7549933422103863e-06,
+      "loss": 0.5075,
+      "step": 1110
+    },
+    {
+      "epoch": 2.6858513189448443,
+      "grad_norm": 15.14069938659668,
+      "learning_rate": 3.4886817576564585e-06,
+      "loss": 0.5682,
+      "step": 1120
+    },
+    {
+      "epoch": 2.709832134292566,
+      "grad_norm": 13.919915199279785,
+      "learning_rate": 3.2223701731025303e-06,
+      "loss": 0.5288,
+      "step": 1130
+    },
+    {
+      "epoch": 2.7338129496402876,
+      "grad_norm": 12.382174491882324,
+      "learning_rate": 2.956058588548602e-06,
+      "loss": 0.589,
+      "step": 1140
+    },
+    {
+      "epoch": 2.7577937649880093,
+      "grad_norm": 14.584649085998535,
+      "learning_rate": 2.689747003994674e-06,
+      "loss": 0.5221,
+      "step": 1150
+    },
+    {
+      "epoch": 2.7817745803357314,
+      "grad_norm": 16.116188049316406,
+      "learning_rate": 2.4234354194407458e-06,
+      "loss": 0.5006,
+      "step": 1160
+    },
+    {
+      "epoch": 2.805755395683453,
+      "grad_norm": 15.878268241882324,
+      "learning_rate": 2.157123834886818e-06,
+      "loss": 0.5888,
+      "step": 1170
+    },
+    {
+      "epoch": 2.8297362110311752,
+      "grad_norm": 21.30999183654785,
+      "learning_rate": 1.8908122503328896e-06,
+      "loss": 0.5271,
+      "step": 1180
+    },
+    {
+      "epoch": 2.853717026378897,
+      "grad_norm": 15.067368507385254,
+      "learning_rate": 1.6245006657789616e-06,
+      "loss": 0.6186,
+      "step": 1190
+    },
+    {
+      "epoch": 2.8776978417266186,
+      "grad_norm": 10.538869857788086,
+      "learning_rate": 1.3581890812250334e-06,
+      "loss": 0.5674,
+      "step": 1200
+    },
+    {
+      "epoch": 2.8776978417266186,
+      "eval_accuracy": 0.7193995381062356,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.8333333333333334,
+      "eval_accuracy_label_conflict, war and peace": 0.75,
+      "eval_accuracy_label_crime, law and justice": 0.8497109826589595,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.8931297709923665,
+      "eval_accuracy_label_economy, business, and finance": 0.7341772151898734,
+      "eval_accuracy_label_environment": 0.3125,
+      "eval_accuracy_label_health": 0.8,
+      "eval_accuracy_label_human interest": 0.25,
+      "eval_accuracy_label_labour": 0.5,
+      "eval_accuracy_label_lifestyle and leisure": 0.375,
+      "eval_accuracy_label_politics": 0.6474820143884892,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.5,
+      "eval_accuracy_label_society": 0.14035087719298245,
+      "eval_accuracy_label_sport": 0.9615384615384616,
+      "eval_accuracy_label_weather": 1.0,
+      "eval_f1": 0.7085982937128052,
+      "eval_loss": 0.8748757243156433,
+      "eval_precision": 0.7206584762787962,
+      "eval_recall": 0.7193995381062356,
+      "eval_runtime": 6.6285,
+      "eval_samples_per_second": 130.649,
+      "eval_steps_per_second": 8.298,
+      "step": 1200
+    },
+    {
+      "epoch": 2.9016786570743403,
+      "grad_norm": 14.816320419311523,
+      "learning_rate": 1.0918774966711052e-06,
+      "loss": 0.5438,
+      "step": 1210
+    },
+    {
+      "epoch": 2.9256594724220624,
+      "grad_norm": 11.16421890258789,
+      "learning_rate": 8.255659121171772e-07,
+      "loss": 0.5369,
+      "step": 1220
+    },
+    {
+      "epoch": 2.949640287769784,
+      "grad_norm": 15.75862979888916,
+      "learning_rate": 5.592543275632491e-07,
+      "loss": 0.6039,
+      "step": 1230
+    },
+    {
+      "epoch": 2.973621103117506,
+      "grad_norm": 12.396734237670898,
+      "learning_rate": 2.9294274300932093e-07,
+      "loss": 0.519,
+      "step": 1240
+    },
+    {
+      "epoch": 2.997601918465228,
+      "grad_norm": 12.56032943725586,
+      "learning_rate": 2.6631158455392814e-08,
+      "loss": 0.4579,
+      "step": 1250
+    },
+    {
+      "epoch": 3.0,
+      "step": 1251,
+      "total_flos": 5261961565790208.0,
+      "train_loss": 1.1299877060021903,
+      "train_runtime": 920.3065,
+      "train_samples_per_second": 43.456,
+      "train_steps_per_second": 1.359
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.7193995381062356,
+      "eval_accuracy_label_arts, culture, entertainment and media": 0.8333333333333334,
+      "eval_accuracy_label_conflict, war and peace": 0.7287234042553191,
+      "eval_accuracy_label_crime, law and justice": 0.8497109826589595,
+      "eval_accuracy_label_disaster, accident, and emergency incident": 0.8931297709923665,
+      "eval_accuracy_label_economy, business, and finance": 0.7721518987341772,
+      "eval_accuracy_label_environment": 0.3125,
+      "eval_accuracy_label_health": 0.8,
+      "eval_accuracy_label_human interest": 0.3333333333333333,
+      "eval_accuracy_label_labour": 0.5,
+      "eval_accuracy_label_lifestyle and leisure": 0.5,
+      "eval_accuracy_label_politics": 0.6402877697841727,
+      "eval_accuracy_label_religion": 0.0,
+      "eval_accuracy_label_science and technology": 0.4166666666666667,
+      "eval_accuracy_label_society": 0.15789473684210525,
+      "eval_accuracy_label_sport": 0.9615384615384616,
+      "eval_accuracy_label_weather": 1.0,
+      "eval_f1": 0.7086386434582244,
+      "eval_loss": 0.8703184723854065,
+      "eval_precision": 0.7185855451718517,
+      "eval_recall": 0.7193995381062356,
+      "eval_runtime": 6.4919,
+      "eval_samples_per_second": 133.397,
+      "eval_steps_per_second": 8.472,
+      "step": 1251
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1251,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 1000,
+  "total_flos": 5261961565790208.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3345fa95975f1231ffb6009d81d7091bb402424e276265d169d2a78e172e28f3
+size 4984