--- library_name: transformers language: - ru license: apache-2.0 base_model: deepvk/RuModernBERT-base tags: - nli - russian - text-classification - generated_from_trainer datasets: - cointegrated/nli-rus-translated-v2021 metrics: - accuracy model-index: - name: rumodernbert-nli results: - task: name: Text Classification type: text-classification dataset: name: cointegrated/nli-rus-translated-v2021 type: cointegrated/nli-rus-translated-v2021 metrics: - name: Accuracy type: accuracy value: 0.8142784874162134 --- # rumodernbert-nli This model is a fine-tuned version of [deepvk/RuModernBERT-base](https://huggingface.co/deepvk/RuModernBERT-base) on the cointegrated/nli-rus-translated-v2021 dataset. It achieves the following results on the evaluation set: - Loss: 0.5055 - Accuracy: 0.8143 - Macro F1: 0.8045 - Mean Roc Auc: 0.9351 - Roc Auc Entailment: 0.9496 - Roc Auc Contradiction: 0.9487 - Roc Auc Neutral: 0.9071 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 16 - eval_batch_size: 32 - seed: 42 - distributed_type: multi-GPU - gradient_accumulation_steps: 2 - total_train_batch_size: 32 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 0.06 - num_epochs: 3 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | Macro F1 | Mean Roc Auc | Roc Auc Entailment | Roc Auc Contradiction | Roc Auc Neutral | |:-------------:|:------:|:-----:|:---------------:|:--------:|:--------:|:------------:|:------------------:|:---------------------:|:---------------:| | 2.2104 | 0.0123 | 500 | 1.0678 | 0.4379 | 0.3603 | 0.5997 | 0.6448 | 0.5677 | 0.5868 | | 1.5496 | 0.0245 | 1000 | 0.7990 | 0.6590 | 0.6391 | 0.8146 | 0.8588 | 0.8359 | 0.7492 | | 1.3165 | 0.0368 | 1500 | 0.7175 | 0.7100 | 0.6866 | 0.8577 | 0.8929 | 0.8815 | 0.7985 | | 1.2338 | 0.0491 | 2000 | 0.6983 | 0.7160 | 0.6927 | 0.8672 | 0.9033 | 0.8898 | 0.8084 | | 1.1754 | 0.0613 | 2500 | 0.7087 | 0.7241 | 0.7076 | 0.8708 | 0.8986 | 0.9002 | 0.8135 | | 1.1110 | 0.0736 | 3000 | 0.6735 | 0.7313 | 0.7159 | 0.8796 | 0.9114 | 0.9033 | 0.8242 | | 1.1363 | 0.0859 | 3500 | 0.6352 | 0.7442 | 0.7293 | 0.8881 | 0.9147 | 0.9095 | 0.8402 | | 1.1236 | 0.0981 | 4000 | 0.6343 | 0.7435 | 0.7261 | 0.8868 | 0.9141 | 0.9071 | 0.8393 | | 1.0685 | 0.1104 | 4500 | 0.6201 | 0.7544 | 0.7383 | 0.8960 | 0.9220 | 0.9155 | 0.8505 | | 1.1451 | 0.1227 | 5000 | 0.6254 | 0.7515 | 0.7284 | 0.8950 | 0.9249 | 0.9150 | 0.8452 | | 1.0986 | 0.1349 | 5500 | 0.5993 | 0.7610 | 0.7512 | 0.9012 | 0.9275 | 0.9163 | 0.8598 | | 1.0415 | 0.1472 | 6000 | 0.6263 | 0.7549 | 0.7313 | 0.8992 | 0.9249 | 0.9181 | 0.8548 | | 1.0306 | 0.1595 | 6500 | 0.6624 | 0.7402 | 0.7137 | 0.8984 | 0.9240 | 0.9190 | 0.8522 | | 1.0294 | 0.1717 | 7000 | 0.6017 | 0.7693 | 0.7534 | 0.9077 | 0.9289 | 0.9240 | 0.8704 | | 1.0892 | 0.1840 | 7500 | 0.6379 | 0.7562 | 0.7328 | 0.9042 | 0.9264 | 0.9213 | 0.8648 | | 1.0483 | 0.1963 | 8000 | 0.5756 | 0.7693 | 0.7520 | 0.9091 | 0.9298 | 0.9266 | 0.8709 | | 1.0151 | 0.2085 | 8500 | 0.5726 | 0.7742 | 0.7633 | 0.9107 | 0.9313 | 0.9269 | 0.8739 | | 0.9694 | 0.2208 | 9000 | 0.6317 | 0.7681 | 0.7525 | 0.9087 | 0.9312 | 0.9243 | 0.8706 | | 1.0105 | 0.2331 | 9500 | 0.5735 | 0.7751 | 0.7643 | 0.9107 | 0.9312 | 0.9246 | 0.8764 | | 0.9759 | 0.2454 | 10000 | 0.5718 | 0.7778 | 0.7693 | 0.9116 | 0.9352 | 0.9240 | 0.8756 | | 1.0269 | 0.2576 | 10500 | 0.5584 | 0.7820 | 0.7691 | 0.9146 | 0.9355 | 0.9303 | 0.8782 | | 1.0052 | 0.2699 | 11000 | 0.5409 | 0.7868 | 0.7762 | 0.9183 | 0.9380 | 0.9336 | 0.8833 | | 0.9736 | 0.2822 | 11500 | 0.5483 | 0.7864 | 0.7707 | 0.9185 | 0.9377 | 0.9343 | 0.8835 | | 0.9673 | 0.2944 | 12000 | 0.5449 | 0.7869 | 0.7768 | 0.9189 | 0.9393 | 0.9335 | 0.8840 | | 0.9870 | 0.3067 | 12500 | 0.5487 | 0.7857 | 0.7737 | 0.9184 | 0.9366 | 0.9345 | 0.8841 | | 0.9597 | 0.3190 | 13000 | 0.5490 | 0.7883 | 0.7795 | 0.9196 | 0.9370 | 0.9342 | 0.8876 | | 0.9885 | 0.3312 | 13500 | 0.5532 | 0.7864 | 0.7764 | 0.9203 | 0.9392 | 0.9352 | 0.8865 | | 0.9791 | 0.3435 | 14000 | 0.5332 | 0.7940 | 0.7853 | 0.9224 | 0.9400 | 0.9377 | 0.8894 | | 0.9304 | 0.3558 | 14500 | 0.5508 | 0.7930 | 0.7790 | 0.9231 | 0.9400 | 0.9374 | 0.8919 | | 0.9576 | 0.3680 | 15000 | 0.5403 | 0.7954 | 0.7842 | 0.9241 | 0.9408 | 0.9375 | 0.8940 | | 0.9085 | 0.3803 | 15500 | 0.5327 | 0.7943 | 0.7860 | 0.9233 | 0.9396 | 0.9396 | 0.8907 | | 0.9422 | 0.3926 | 16000 | 0.5296 | 0.7986 | 0.7900 | 0.9257 | 0.9432 | 0.9394 | 0.8944 | | 0.9444 | 0.4048 | 16500 | 0.5108 | 0.8016 | 0.7905 | 0.9265 | 0.9445 | 0.9394 | 0.8957 | | 0.8995 | 0.4171 | 17000 | 0.5236 | 0.8014 | 0.7932 | 0.9253 | 0.9420 | 0.9412 | 0.8928 | | 0.9130 | 0.4294 | 17500 | 0.5190 | 0.8010 | 0.7907 | 0.9265 | 0.9427 | 0.9408 | 0.8960 | | 0.9136 | 0.4416 | 18000 | 0.5214 | 0.8033 | 0.7945 | 0.9277 | 0.9442 | 0.9418 | 0.8971 | | 0.8579 | 0.4539 | 18500 | 0.5147 | 0.8039 | 0.7940 | 0.9286 | 0.9449 | 0.9414 | 0.8996 | | 0.8914 | 0.4662 | 19000 | 0.5307 | 0.7969 | 0.7912 | 0.9276 | 0.9431 | 0.9408 | 0.8988 | | 0.8881 | 0.4784 | 19500 | 0.5401 | 0.7962 | 0.7895 | 0.9236 | 0.9440 | 0.9386 | 0.8883 | | 0.8726 | 0.4907 | 20000 | 0.5130 | 0.8023 | 0.7956 | 0.9289 | 0.9459 | 0.9410 | 0.8998 | | 0.8909 | 0.5030 | 20500 | 0.5075 | 0.8038 | 0.7955 | 0.9291 | 0.9454 | 0.9431 | 0.8989 | | 0.8748 | 0.5152 | 21000 | 0.5080 | 0.8073 | 0.7970 | 0.9301 | 0.9468 | 0.9435 | 0.9000 | | 0.8879 | 0.5275 | 21500 | 0.5117 | 0.8029 | 0.7952 | 0.9281 | 0.9429 | 0.9426 | 0.8988 | | 0.8578 | 0.5398 | 22000 | 0.5001 | 0.8081 | 0.7983 | 0.9308 | 0.9469 | 0.9457 | 0.8998 | | 0.8965 | 0.5520 | 22500 | 0.4930 | 0.8095 | 0.7991 | 0.9315 | 0.9478 | 0.9443 | 0.9023 | | 0.8542 | 0.5643 | 23000 | 0.5220 | 0.8059 | 0.7968 | 0.9297 | 0.9449 | 0.9428 | 0.9014 | | 0.8808 | 0.5766 | 23500 | 0.5017 | 0.8060 | 0.7960 | 0.9313 | 0.9468 | 0.9440 | 0.9030 | | 0.8487 | 0.5888 | 24000 | 0.5146 | 0.8100 | 0.8011 | 0.9317 | 0.9458 | 0.9460 | 0.9035 | | 0.8643 | 0.6011 | 24500 | 0.5091 | 0.8073 | 0.7981 | 0.9300 | 0.9464 | 0.9420 | 0.9017 | | 0.8851 | 0.6134 | 25000 | 0.4869 | 0.8136 | 0.8052 | 0.9334 | 0.9483 | 0.9468 | 0.9050 | | 0.9011 | 0.6256 | 25500 | 0.4874 | 0.8141 | 0.8061 | 0.9343 | 0.9491 | 0.9465 | 0.9072 | | 0.8509 | 0.6379 | 26000 | 0.4911 | 0.8114 | 0.7984 | 0.9342 | 0.9500 | 0.9474 | 0.9051 | | 0.8263 | 0.6502 | 26500 | 0.4925 | 0.8150 | 0.8069 | 0.9353 | 0.9502 | 0.9480 | 0.9078 | | 0.8017 | 0.6624 | 27000 | 0.5023 | 0.8118 | 0.8015 | 0.9331 | 0.9484 | 0.9465 | 0.9043 | | 0.8267 | 0.6747 | 27500 | 0.5126 | 0.8109 | 0.8040 | 0.9336 | 0.9475 | 0.9472 | 0.9062 | | 0.8766 | 0.6870 | 28000 | 0.4923 | 0.8141 | 0.8061 | 0.9351 | 0.9494 | 0.9473 | 0.9086 | | 0.8348 | 0.6992 | 28500 | 0.5225 | 0.8099 | 0.8019 | 0.9295 | 0.9456 | 0.9471 | 0.8957 | | 0.8355 | 0.7115 | 29000 | 0.5055 | 0.8143 | 0.8045 | 0.9351 | 0.9496 | 0.9487 | 0.9071 | ### Framework versions - Transformers 5.8.1 - Pytorch 2.11.0+cu130 - Datasets 5.0.0 - Tokenizers 0.22.2