eternalGenius/rubert_level2

README.md

@@ -1,6 +1,6 @@
 ---
 library_name: transformers
-base_model: DeepPavlov/rubert-base-cased
+base_model: eternalGenius/rubert_level2
 tags:
 - generated_from_trainer
 model-index:
@@ -13,12 +13,12 @@ should probably proofread and complete it, then remove this comment. -->
 
 # rubert_level2
 
-This model is a fine-tuned version of [DeepPavlov/rubert-base-cased](https://huggingface.co/DeepPavlov/rubert-base-cased) on the None dataset.
+This model is a fine-tuned version of [eternalGenius/rubert_level2](https://huggingface.co/eternalGenius/rubert_level2) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2285
-- F1 Micro: 0.5449
-- F1 Macro: 0.4966
-- F1 Weighted: 0.4948
+- Loss: 0.1661
+- F1 Micro: 0.7178
+- F1 Macro: 0.7076
+- F1 Weighted: 0.7118
 
 ## Model description
 
@@ -37,29 +37,31 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 2e-05
+- learning_rate: 5e-06
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 5
+- lr_scheduler_warmup_steps: 0.1
+- num_epochs: 15
+- mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro | F1 Weighted |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|:-----------:|
-| 0.3616        | 1.0   | 97   | 0.3379          | 0.0      | 0.0      | 0.0         |
-| 0.2963        | 2.0   | 194  | 0.2936          | 0.0759   | 0.0496   | 0.0621      |
-| 0.2489        | 3.0   | 291  | 0.2571          | 0.2576   | 0.2077   | 0.2098      |
-| 0.2138        | 4.0   | 388  | 0.2371          | 0.5032   | 0.4528   | 0.4480      |
-| 0.2011        | 5.0   | 485  | 0.2285          | 0.5449   | 0.4966   | 0.4948      |
+| 0.1082        | 1.0   | 97   | 0.1827          | 0.6862   | 0.6628   | 0.6653      |
+| 0.0939        | 2.0   | 194  | 0.1743          | 0.7165   | 0.7000   | 0.7067      |
+| 0.0861        | 3.0   | 291  | 0.1737          | 0.7198   | 0.7049   | 0.7055      |
+| 0.0796        | 4.0   | 388  | 0.1735          | 0.7160   | 0.7074   | 0.7095      |
+| 0.0771        | 5.0   | 485  | 0.1699          | 0.7089   | 0.6921   | 0.6923      |
+| 0.0668        | 6.0   | 582  | 0.1661          | 0.7178   | 0.7076   | 0.7118      |
 
 
 ### Framework versions
 
-- Transformers 4.57.1
-- Pytorch 2.8.0+cu128
+- Transformers 5.0.0
+- Pytorch 2.10.0+cu128
 - Datasets 4.0.0
 - Tokenizers 0.22.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c5803f96246c77e78c8d3fcddff4eedea298c07bef31a4b5fe925483fb9b5c79
+oid sha256:a30eaf03664ae21673c57cc697688eec0fbfafc93c991ea2503df21432bee54b
 size 711471116

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f69b2f3cc112be17d377924145a397414f4822261fc8d4e4115ca48527892265
-size 5841
+oid sha256:9771be977be3b8504664036ddfa90cf77e365fe48fa1f596388626f3a57d2cfc
+size 5201