Model save

README.md

@@ -1,6 +1,6 @@
 ---
-license: apache-2.0
-base_model: distilgpt2
+license: mit
+base_model: roberta-base
 tags:
 - generated_from_trainer
 datasets:
@@ -8,13 +8,13 @@ datasets:
 metrics:
 - accuracy
 model-index:
-- name: clm
+- name: mlm
   results:
   - task:
-      name: Causal Language Modeling
-      type: text-generation
+      name: Masked Language Modeling
+      type: fill-mask
     dataset:
-      name: wikitext wikitext-2-raw-v1
+      name: wikitext
       type: wikitext
       config: wikitext-2-raw-v1
       split: validation
@@ -22,18 +22,18 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.37187601824698596
+      value: 0.7278010101558682
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# clm
+# mlm
 
-This model is a fine-tuned version of [distilgpt2](https://huggingface.co/distilgpt2) on the wikitext wikitext-2-raw-v1 dataset.
+This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the wikitext dataset.
 It achieves the following results on the evaluation set:
-- Loss: 3.4802
-- Accuracy: 0.3719
+- Loss: 1.2607
+- Accuracy: 0.7278
 
 ## Model description
 
@@ -61,13 +61,15 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 1.0
+- num_epochs: 3.0
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|
-| 3.6158        | 0.99  | 72   | 3.4802          | 0.3719   |
+| 1.3758        | 1.0   | 150  | 1.2826          | 0.7277   |
+| 1.3763        | 2.0   | 300  | 1.2747          | 0.7272   |
+| 1.3558        | 3.0   | 450  | 1.2607          | 0.7278   |
 
 
 ### Framework versions