Model save

README.md

@@ -12,9 +12,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # gpt2-chinese-mini
 
-This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 4.7057
+This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
 
 ## Model description
 
@@ -33,41 +31,18 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0006
-- train_batch_size: 32
+- learning_rate: 0.0003
+- train_batch_size: 48
 - eval_batch_size: 16
 - seed: 42
 - gradient_accumulation_steps: 2
-- total_train_batch_size: 64
-- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- total_train_batch_size: 96
+- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 372
-- num_epochs: 10
+- lr_scheduler_warmup_steps: 2000
+- num_epochs: 2
 - mixed_precision_training: Native AMP
 
-### Training results
-
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 7.6487        | 0.5650 | 200  | 7.4767          |
-| 6.7887        | 1.1299 | 400  | 6.7074          |
-| 6.2897        | 1.6949 | 600  | 6.2049          |
-| 5.8297        | 2.2599 | 800  | 5.8749          |
-| 5.6539        | 2.8249 | 1000 | 5.6174          |
-| 5.3118        | 3.3898 | 1200 | 5.4056          |
-| 5.1755        | 3.9548 | 1400 | 5.1988          |
-| 4.873         | 4.5198 | 1600 | 5.0580          |
-| 4.686         | 5.0847 | 1800 | 4.9458          |
-| 4.5836        | 5.6497 | 2000 | 4.8757          |
-| 4.331         | 6.2147 | 2200 | 4.8240          |
-| 4.3591        | 6.7797 | 2400 | 4.7726          |
-| 4.1832        | 7.3446 | 2600 | 4.7492          |
-| 4.1718        | 7.9096 | 2800 | 4.7212          |
-| 4.0537        | 8.4746 | 3000 | 4.7159          |
-| 4.0378        | 9.0395 | 3200 | 4.7076          |
-| 4.0014        | 9.6045 | 3400 | 4.7057          |
-
-
 ### Framework versions
 
 - Transformers 4.57.1

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ab7ea49340f2a9087900548b887bb3c9df9e17480605a27cebe0230394c57107
-size 269999560
+oid sha256:1f1a168064b8d70bd0836355a9a1a1f2af91dcd3d913825d2b0259657f2eeba6
+size 270000320

tokenizer.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:10cdfc86a0f501c1847c9aab5fae94daf4c77d07fcf5e83488fd81e8d9dd6a53
-size 745163
+oid sha256:f456d338b1afd928a42792367f757e504e3dd9d65f5c13732808efe0458dbdfd
+size 734072

tokenizer_config.json

@@ -34,22 +34,6 @@
       "rstrip": false,
       "single_word": false,
       "special": true
-    },
-    "4": {
-      "content": "<mask>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
-    },
-    "5": {
-      "content": "<sep>",
-      "lstrip": false,
-      "normalized": false,
-      "rstrip": false,
-      "single_word": false,
-      "special": false
     }
   },
   "bos_token": "<s>",

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:031e21a7ecd4ded02b1e698e035d9892e6dca813d24770814dac75fc064dc6ee
+oid sha256:39e41d0dda03681524fefde738bea42051c7804f9df486a9c8ccaf982a311b53
 size 5841