Training in progress epoch 0

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,60 @@
+---
+license: apache-2.0
+tags:
+- generated_from_keras_callback
+model-index:
+- name: HanSSH/mt5-small-finetuned-amazon-en-es
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information Keras had access to. You should
+probably proofread and complete it, then remove this comment. -->
+
+# HanSSH/mt5-small-finetuned-amazon-en-es
+
+This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Train Loss: 1.7231
+- Validation Loss: 1.5811
+- Epoch: 7
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5.6e-05, 'decay_steps': 9672, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
+- training_precision: mixed_float16
+
+### Training results
+
+| Train Loss | Validation Loss | Epoch |
+|:----------:|:---------------:|:-----:|
+| 12.6850    | 2.6962          | 0     |
+| 2.9141     | 2.3047          | 1     |
+| 2.3124     | 2.1101          | 2     |
+| 2.0972     | 1.9355          | 3     |
+| 1.9639     | 1.8009          | 4     |
+| 1.8591     | 1.7010          | 5     |
+| 1.7735     | 1.6233          | 6     |
+| 1.7231     | 1.5811          | 7     |
+
+
+### Framework versions
+
+- Transformers 4.21.3
+- TensorFlow 2.10.0
+- Datasets 2.4.0
+- Tokenizers 0.12.1

.ipynb_checkpoints/config-checkpoint.json

@@ -0,0 +1,30 @@
+{
+  "_name_or_path": "google/mt5-small",
+  "architectures": [
+    "MT5ForConditionalGeneration"
+  ],
+  "d_ff": 1024,
+  "d_kv": 64,
+  "d_model": 512,
+  "decoder_start_token_id": 0,
+  "dense_act_fn": "gelu_new",
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "is_gated_act": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "mt5",
+  "num_decoder_layers": 8,
+  "num_heads": 6,
+  "num_layers": 8,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "tie_word_embeddings": false,
+  "tokenizer_class": "T5Tokenizer",
+  "transformers_version": "4.21.3",
+  "use_cache": true,
+  "vocab_size": 250112
+}

README.md

@@ -14,9 +14,9 @@ probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Train Loss: 1.7231
-- Validation Loss: 1.5811
-- Epoch: 7
+- Train Loss: 10.8170
+- Validation Loss: 4.9147
+- Epoch: 0
 
 ## Model description
 
@@ -35,21 +35,14 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5.6e-05, 'decay_steps': 9672, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
+- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5.6e-05, 'decay_steps': 2418, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
 - training_precision: mixed_float16
 
 ### Training results
 
 | Train Loss | Validation Loss | Epoch |
 |:----------:|:---------------:|:-----:|
-| 12.6850    | 2.6962          | 0     |
-| 2.9141     | 2.3047          | 1     |
-| 2.3124     | 2.1101          | 2     |
-| 2.0972     | 1.9355          | 3     |
-| 1.9639     | 1.8009          | 4     |
-| 1.8591     | 1.7010          | 5     |
-| 1.7735     | 1.6233          | 6     |
-| 1.7231     | 1.5811          | 7     |
+| 10.8170    | 4.9147          | 0     |
 
 
 ### Framework versions

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:eb0203d4b6a6bf5e8b4d4099d50984df8c4f6e4d8f0bb4b0af31133d23aa35d4
+oid sha256:02662779d35424ec4df16e77d8898f4cd3651cff1010b7caf8679ede6b9b67dc
 size 1201094528

tokenizer.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b5baf8c1abc460afe5e7ca79cf84f74686db46217f874a8fb735f4981e7758ed
-size 16330621
+oid sha256:cd38144ed4e51886247b55a7fd9b6fcca7afd7355345b522149ba074b455f2cb
+size 16330466