multi-e5-base_lmd-comments_v1

README.md

@@ -0,0 +1,110 @@
+---
+license: mit
+base_model: intfloat/multilingual-e5-base
+tags:
+- generated_from_trainer
+metrics:
+- f1
+- accuracy
+model-index:
+- name: multi-e5-base_lmd-comments_v1
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# multi-e5-base_lmd-comments_v1
+
+This model is a fine-tuned version of [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6013
+- F1: 0.8539
+- Accuracy: 0.8505
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 2
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | F1     | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:--------:|
+| 1.0859        | 0.04  | 100  | 1.0531          | 0.6574 | 0.6716   |
+| 0.9993        | 0.08  | 200  | 0.8669          | 0.7260 | 0.7266   |
+| 0.7514        | 0.12  | 300  | 0.7077          | 0.7582 | 0.7699   |
+| 0.6928        | 0.17  | 400  | 0.8825          | 0.7521 | 0.7856   |
+| 0.6349        | 0.21  | 500  | 0.6178          | 0.7947 | 0.7778   |
+| 0.5939        | 0.25  | 600  | 0.6509          | 0.7977 | 0.7847   |
+| 0.6461        | 0.29  | 700  | 0.5820          | 0.7951 | 0.7778   |
+| 0.5346        | 0.33  | 800  | 0.7509          | 0.8022 | 0.8014   |
+| 0.622         | 0.37  | 900  | 0.8165          | 0.7902 | 0.7935   |
+| 0.6124        | 0.41  | 1000 | 0.6754          | 0.7885 | 0.7788   |
+| 0.6285        | 0.46  | 1100 | 0.5179          | 0.8284 | 0.8210   |
+| 0.5838        | 0.5   | 1200 | 0.5335          | 0.8149 | 0.8024   |
+| 0.6116        | 0.54  | 1300 | 0.5584          | 0.8301 | 0.8240   |
+| 0.5578        | 0.58  | 1400 | 0.6135          | 0.7670 | 0.7404   |
+| 0.534         | 0.62  | 1500 | 0.6407          | 0.8153 | 0.8063   |
+| 0.5343        | 0.66  | 1600 | 0.5500          | 0.8381 | 0.8279   |
+| 0.5575        | 0.7   | 1700 | 0.6355          | 0.8170 | 0.8102   |
+| 0.5961        | 0.75  | 1800 | 0.5465          | 0.8450 | 0.8446   |
+| 0.5623        | 0.79  | 1900 | 0.5787          | 0.7875 | 0.7719   |
+| 0.5436        | 0.83  | 2000 | 0.6035          | 0.8494 | 0.8525   |
+| 0.5662        | 0.87  | 2100 | 0.4979          | 0.8210 | 0.8083   |
+| 0.5227        | 0.91  | 2200 | 0.6157          | 0.8320 | 0.8328   |
+| 0.557         | 0.95  | 2300 | 0.5721          | 0.8488 | 0.8505   |
+| 0.5369        | 0.99  | 2400 | 0.5191          | 0.8222 | 0.8112   |
+| 0.4846        | 1.04  | 2500 | 0.4966          | 0.8304 | 0.8201   |
+| 0.4094        | 1.08  | 2600 | 0.6385          | 0.8368 | 0.8328   |
+| 0.4045        | 1.12  | 2700 | 0.6165          | 0.8476 | 0.8456   |
+| 0.4835        | 1.16  | 2800 | 0.6792          | 0.8536 | 0.8535   |
+| 0.3949        | 1.2   | 2900 | 0.7400          | 0.8559 | 0.8574   |
+| 0.4157        | 1.24  | 3000 | 0.6305          | 0.8319 | 0.8279   |
+| 0.4638        | 1.28  | 3100 | 0.5507          | 0.8438 | 0.8387   |
+| 0.4438        | 1.33  | 3200 | 0.5208          | 0.8426 | 0.8348   |
+| 0.4224        | 1.37  | 3300 | 0.6534          | 0.8330 | 0.8279   |
+| 0.4703        | 1.41  | 3400 | 0.6486          | 0.8456 | 0.8427   |
+| 0.4912        | 1.45  | 3500 | 0.6370          | 0.8544 | 0.8505   |
+| 0.4396        | 1.49  | 3600 | 0.6866          | 0.8520 | 0.8505   |
+| 0.437         | 1.53  | 3700 | 0.6436          | 0.8536 | 0.8505   |
+| 0.3993        | 1.57  | 3800 | 0.6910          | 0.8517 | 0.8515   |
+| 0.3712        | 1.62  | 3900 | 0.6449          | 0.8588 | 0.8564   |
+| 0.4156        | 1.66  | 4000 | 0.6478          | 0.8560 | 0.8535   |
+| 0.4564        | 1.7   | 4100 | 0.6475          | 0.8577 | 0.8555   |
+| 0.4273        | 1.74  | 4200 | 0.6044          | 0.8456 | 0.8397   |
+| 0.4284        | 1.78  | 4300 | 0.6219          | 0.8499 | 0.8466   |
+| 0.4466        | 1.82  | 4400 | 0.5871          | 0.8535 | 0.8496   |
+| 0.3932        | 1.86  | 4500 | 0.5940          | 0.8551 | 0.8515   |
+| 0.3848        | 1.91  | 4600 | 0.5995          | 0.8559 | 0.8525   |
+| 0.4337        | 1.95  | 4700 | 0.6023          | 0.8539 | 0.8505   |
+| 0.3991        | 1.99  | 4800 | 0.6013          | 0.8539 | 0.8505   |
+
+
+### Framework versions
+
+- Transformers 4.38.1
+- Pytorch 2.1.2
+- Datasets 2.1.0
+- Tokenizers 0.15.2

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "intfloat/multilingual-e5-base",
+  "architectures": [
+    "XLMRobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.38.1",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 250002
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6364629546595aa3154bf00cad69cdfd0695fe06f0f6e72c3234fa050e97e6d
+size 1112208084

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5baf6e7361b584742f64641e8a8cbf2b3e1ce1ca4c06a115bfdde6113d7e11a4
+size 4856