Hugging Face's logo

Sphincz
/
monolang_roberta

Fill-Mask
Transformers
Safetensors
roberta
Generated from Trainer
Model card Files
xet
Community
monolang_roberta / README.md
Sphincz's picture
Sphincz
End of training
fa2fa06 verified
preview code
|
raw
history blame contribute delete
20.7 kB
metadata 
library_name: transformers
tags:
  - generated_from_trainer
model-index:
  - name: monolang_roberta
    results: []

Visualize in Weights & Biases

monolang_roberta

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9382

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 128
  • eval_batch_size: 128
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 1024
  • optimizer: Use OptimizerNames.ADAMW_HF with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 200
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
4.3759 0.5735 100 4.2809
3.8527 1.1434 200 3.7609
3.3289 1.7168 300 3.2264
2.9251 2.2867 400 2.8520
2.7058 2.8602 500 2.6128
2.5376 3.4301 600 2.4893
2.3017 4.0 700 2.3807
2.3375 4.5735 800 2.3030
2.2953 5.1434 900 2.2198
2.2365 5.7168 1000 2.1678
2.1709 6.2867 1100 2.1113
2.0904 6.8602 1200 2.0296
2.0443 7.4301 1300 1.9785
1.8951 8.0 1400 1.9323
1.9251 8.5735 1500 1.8842
1.8639 9.1434 1600 1.8533
1.8359 9.7168 1700 1.8028
1.8146 10.2867 1800 1.7665
1.8021 10.8602 1900 1.7489
1.7819 11.4301 2000 1.7204
1.6114 12.0 2100 1.6807
1.7171 12.5735 2200 1.6776
1.6929 13.1434 2300 1.6672
1.6784 13.7168 2400 1.6448
1.6724 14.2867 2500 1.5998
1.6372 14.8602 2600 1.5929
1.6156 15.4301 2700 1.5809
1.5284 16.0 2800 1.5574
1.5787 16.5735 2900 1.5392
1.5807 17.1434 3000 1.5195
1.5732 17.7168 3100 1.5140
1.5191 18.2867 3200 1.5144
1.5057 18.8602 3300 1.4768
1.5029 19.4301 3400 1.4749
1.3981 20.0 3500 1.4581
1.4977 20.5735 3600 1.4443
1.4729 21.1434 3700 1.4306
1.4637 21.7168 3800 1.4297
1.4432 22.2867 3900 1.4210
1.4343 22.8602 4000 1.4103
1.4179 23.4301 4100 1.3859
1.3212 24.0 4200 1.4049
1.4155 24.5735 4300 1.3742
1.3983 25.1434 4400 1.3840
1.3998 25.7168 4500 1.3647
1.3565 26.2867 4600 1.3581
1.3805 26.8602 4700 1.3466
1.3731 27.4301 4800 1.3374
1.2775 28.0 4900 1.3320
1.3217 28.5735 5000 1.3215
1.3404 29.1434 5100 1.3185
1.3235 29.7168 5200 1.3119
1.3445 30.2867 5300 1.3268
1.3243 30.8602 5400 1.3003
1.2853 31.4301 5500 1.2758
1.1974 32.0 5600 1.2880
1.2663 32.5735 5700 1.2987
1.2617 33.1434 5800 1.2877
1.2717 33.7168 5900 1.2772
1.2652 34.2867 6000 1.2786
1.2668 34.8602 6100 1.2532
1.2628 35.4301 6200 1.2404
1.1627 36.0 6300 1.2596
1.2476 36.5735 6400 1.2492
1.2457 37.1434 6500 1.2402
1.2393 37.7168 6600 1.2334
1.2378 38.2867 6700 1.2394
1.2272 38.8602 6800 1.2213
1.2204 39.4301 6900 1.2100
1.1453 40.0 7000 1.2134
1.1963 40.5735 7100 1.2253
1.2227 41.1434 7200 1.2188
1.2166 41.7168 7300 1.2008
1.2091 42.2867 7400 1.2055
1.2127 42.8602 7500 1.2055
1.1997 43.4301 7600 1.1971
1.0956 44.0 7700 1.1974
1.1672 44.5735 7800 1.1813
1.1746 45.1434 7900 1.1858
1.1569 45.7168 8000 1.1858
1.1644 46.2867 8100 1.1759
1.1629 46.8602 8200 1.1726
1.1744 47.4301 8300 1.1646
1.0728 48.0 8400 1.1666
1.1436 48.5735 8500 1.1801
1.126 49.1434 8600 1.1663
1.1428 49.7168 8700 1.1510
1.1298 50.2867 8800 1.1669
1.1083 50.8602 8900 1.1516
1.1404 51.4301 9000 1.1462
1.0507 52.0 9100 1.1323
1.1203 52.5735 9200 1.1381
1.1176 53.1434 9300 1.1368
1.1228 53.7168 9400 1.1321
1.1341 54.2867 9500 1.1388
1.1133 54.8602 9600 1.1385
1.0973 55.4301 9700 1.1329
1.0327 56.0 9800 1.1310
1.0939 56.5735 9900 1.1411
1.0856 57.1434 10000 1.1149
1.1024 57.7168 10100 1.1196
1.0865 58.2867 10200 1.1196
1.0842 58.8602 10300 1.1250
1.0786 59.4301 10400 1.1089
0.9986 60.0 10500 1.1002
1.0667 60.5735 10600 1.1080
1.064 61.1434 10700 1.0939
1.0784 61.7168 10800 1.1257
1.0663 62.2867 10900 1.0998
1.0794 62.8602 11000 1.0964
1.0457 63.4301 11100 1.0893
0.9931 64.0 11200 1.1005
1.0574 64.5735 11300 1.0909
1.0619 65.1434 11400 1.0873
1.0669 65.7168 11500 1.0914
1.0273 66.2867 11600 1.0926
1.0467 66.8602 11700 1.0774
1.0418 67.4301 11800 1.0900
0.9703 68.0 11900 1.0799
1.0118 68.5735 12000 1.0846
1.0261 69.1434 12100 1.0652
1.028 69.7168 12200 1.0750
1.0204 70.2867 12300 1.0674
1.0448 70.8602 12400 1.0690
0.9965 71.4301 12500 1.0509
0.9445 72.0 12600 1.0611
1.0125 72.5735 12700 1.0688
0.9895 73.1434 12800 1.0629
1.0051 73.7168 12900 1.0446
0.9876 74.2867 13000 1.0576
0.9867 74.8602 13100 1.0576
0.9902 75.4301 13200 1.0572
0.9248 76.0 13300 1.0636
0.9988 76.5735 13400 1.0623
0.998 77.1434 13500 1.0668
0.9777 77.7168 13600 1.0523
0.9915 78.2867 13700 1.0476
0.9914 78.8602 13800 1.0419
0.992 79.4301 13900 1.0407
0.922 80.0 14000 1.0459
0.9738 80.5735 14100 1.0424
0.9996 81.1434 14200 1.0459
0.9856 81.7168 14300 1.0328
0.9839 82.2867 14400 1.0587
0.9748 82.8602 14500 1.0369
0.9601 83.4301 14600 1.0350
0.8863 84.0 14700 1.0475
0.9639 84.5735 14800 1.0441
0.9671 85.1434 14900 1.0326
0.9515 85.7168 15000 1.0256
0.9309 86.2867 15100 1.0351
0.9479 86.8602 15200 1.0349
0.9738 87.4301 15300 1.0221
0.8831 88.0 15400 1.0311
0.9384 88.5735 15500 1.0231
0.9404 89.1434 15600 1.0369
0.9354 89.7168 15700 1.0220
0.9228 90.2867 15800 1.0354
0.9312 90.8602 15900 1.0232
0.9466 91.4301 16000 1.0174
0.8721 92.0 16100 1.0168
0.9328 92.5735 16200 1.0216
0.9331 93.1434 16300 1.0145
0.9435 93.7168 16400 1.0208
0.9253 94.2867 16500 1.0163
0.9231 94.8602 16600 1.0159
0.9273 95.4301 16700 1.0044
0.8571 96.0 16800 1.0047
0.9293 96.5735 16900 1.0111
0.9264 97.1434 17000 1.0088
0.9221 97.7168 17100 1.0074
0.9143 98.2867 17200 1.0123
0.9119 98.8602 17300 1.0049
0.9036 99.4301 17400 0.9973
0.869 100.0 17500 1.0031
0.9024 100.5735 17600 1.0054
0.896 101.1434 17700 0.9966
0.8883 101.7168 17800 1.0010
0.9053 102.2867 17900 1.0160
0.9263 102.8602 18000 1.0012
0.9042 103.4301 18100 1.0145
0.8341 104.0 18200 0.9961
0.9028 104.5735 18300 1.0040
0.9026 105.1434 18400 0.9844
0.8801 105.7168 18500 0.9907
0.8956 106.2867 18600 0.9983
0.9023 106.8602 18700 0.9840
0.8712 107.4301 18800 0.9946
0.8374 108.0 18900 0.9944
0.9102 108.5735 19000 0.9928
0.8914 109.1434 19100 0.9778
0.8852 109.7168 19200 0.9928
0.8826 110.2867 19300 0.9829
0.881 110.8602 19400 0.9747
0.878 111.4301 19500 0.9809
0.8219 112.0 19600 0.9707
0.8797 112.5735 19700 0.9727
0.8911 113.1434 19800 0.9759
0.8706 113.7168 19900 0.9809
0.8952 114.2867 20000 0.9871
0.8585 114.8602 20100 0.9777
0.8679 115.4301 20200 0.9745
0.8289 116.0 20300 0.9793
0.8647 116.5735 20400 0.9791
0.8711 117.1434 20500 0.9806
0.8734 117.7168 20600 0.9831
0.8427 118.2867 20700 0.9862
0.8678 118.8602 20800 0.9786
0.8687 119.4301 20900 0.9738
0.8072 120.0 21000 0.9758
0.854 120.5735 21100 0.9647
0.8679 121.1434 21200 0.9648
0.8652 121.7168 21300 0.9747
0.8621 122.2867 21400 0.9818
0.8446 122.8602 21500 0.9861
0.8618 123.4301 21600 0.9597
0.8102 124.0 21700 0.9710
0.8605 124.5735 21800 0.9672
0.845 125.1434 21900 0.9760
0.8452 125.7168 22000 0.9756
0.848 126.2867 22100 0.9624
0.8522 126.8602 22200 0.9559
0.8481 127.4301 22300 0.9682
0.7941 128.0 22400 0.9693
0.8381 128.5735 22500 0.9690
0.8498 129.1434 22600 0.9656
0.8558 129.7168 22700 0.9619
0.8241 130.2867 22800 0.9568
0.8422 130.8602 22900 0.9662
0.8446 131.4301 23000 0.9598
0.7789 132.0 23100 0.9545
0.8287 132.5735 23200 0.9655
0.8413 133.1434 23300 0.9517
0.8344 133.7168 23400 0.9638
0.8287 134.2867 23500 0.9715
0.8253 134.8602 23600 0.9619
0.8232 135.4301 23700 0.9617
0.7756 136.0 23800 0.9559
0.8306 136.5735 23900 0.9612
0.8383 137.1434 24000 0.9481
0.8176 137.7168 24100 0.9537
0.8292 138.2867 24200 0.9640
0.8332 138.8602 24300 0.9477
0.8296 139.4301 24400 0.9636
0.7812 140.0 24500 0.9598
0.8303 140.5735 24600 0.9559
0.8207 141.1434 24700 0.9397
0.8163 141.7168 24800 0.9461
0.8177 142.2867 24900 0.9570
0.8185 142.8602 25000 0.9555
0.821 143.4301 25100 0.9620
0.7833 144.0 25200 0.9386
0.8218 144.5735 25300 0.9523
0.8156 145.1434 25400 0.9508
0.8205 145.7168 25500 0.9559
0.8204 146.2867 25600 0.9569
0.8113 146.8602 25700 0.9489
0.8093 147.4301 25800 0.9483
0.7531 148.0 25900 0.9512
0.8023 148.5735 26000 0.9543
0.8088 149.1434 26100 0.9640
0.8054 149.7168 26200 0.9508
0.8164 150.2867 26300 0.9372
0.8307 150.8602 26400 0.9495
0.8084 151.4301 26500 0.9408
0.7723 152.0 26600 0.9561
0.7929 152.5735 26700 0.9448
0.7836 153.1434 26800 0.9495
0.8016 153.7168 26900 0.9471
0.8073 154.2867 27000 0.9490
0.811 154.8602 27100 0.9445
0.808 155.4301 27200 0.9497
0.7446 156.0 27300 0.9624
0.8005 156.5735 27400 0.9527
0.8003 157.1434 27500 0.9406
0.7985 157.7168 27600 0.9430
0.8044 158.2867 27700 0.9457
0.817 158.8602 27800 0.9419
0.8118 159.4301 27900 0.9360
0.7421 160.0 28000 0.9401
0.8 160.5735 28100 0.9437
0.7981 161.1434 28200 0.9223
0.7917 161.7168 28300 0.9423
0.8029 162.2867 28400 0.9357
0.7934 162.8602 28500 0.9504
0.7907 163.4301 28600 0.9382
0.7451 164.0 28700 0.9348
0.7871 164.5735 28800 0.9435
0.7918 165.1434 28900 0.9399
0.8033 165.7168 29000 0.9382
0.7913 166.2867 29100 0.9279
0.7902 166.8602 29200 0.9461
0.781 167.4301 29300 0.9376
0.7472 168.0 29400 0.9337
0.7756 168.5735 29500 0.9419
0.7944 169.1434 29600 0.9313
0.7868 169.7168 29700 0.9308
0.7914 170.2867 29800 0.9339
0.7904 170.8602 29900 0.9410
0.7923 171.4301 30000 0.9442
0.7428 172.0 30100 0.9383
0.7822 172.5735 30200 0.9369
0.7956 173.1434 30300 0.9367
0.7888 173.7168 30400 0.9332
0.79 174.2867 30500 0.9264
0.7922 174.8602 30600 0.9313
0.7817 175.4301 30700 0.9363
0.7282 176.0 30800 0.9198
0.7819 176.5735 30900 0.9448
0.7692 177.1434 31000 0.9359
0.7725 177.7168 31100 0.9350
0.7819 178.2867 31200 0.9376
0.7951 178.8602 31300 0.9235
0.7793 179.4301 31400 0.9309
0.7439 180.0 31500 0.9385
0.7843 180.5735 31600 0.9364
0.781 181.1434 31700 0.9339
0.7859 181.7168 31800 0.9422
0.7684 182.2867 31900 0.9404
0.7722 182.8602 32000 0.9209
0.7799 183.4301 32100 0.9293
0.7344 184.0 32200 0.9242
0.7847 184.5735 32300 0.9372
0.7825 185.1434 32400 0.9271
0.7733 185.7168 32500 0.9302
0.7603 186.2867 32600 0.9334
0.7825 186.8602 32700 0.9228
0.7806 187.4301 32800 0.9348
0.7184 188.0 32900 0.9262
0.7866 188.5735 33000 0.9271
0.7649 189.1434 33100 0.9236
0.7825 189.7168 33200 0.9378
0.7788 190.2867 33300 0.9328
0.7771 190.8602 33400 0.9311
0.7771 191.4301 33500 0.9356
0.722 192.0 33600 0.9329
0.7683 192.5735 33700 0.9266
0.7724 193.1434 33800 0.9198
0.7491 193.7168 33900 0.9304
0.7591 194.2867 34000 0.9245
0.7609 194.8602 34100 0.9380
0.7672 195.4301 34200 0.9298
0.7194 196.0 34300 0.9120
0.7618 196.5735 34400 0.9269
0.7847 197.1434 34500 0.9285
0.7746 197.7168 34600 0.9113
0.7885 198.2867 34700 0.9460
0.7768 198.8602 34800 0.9382

Framework versions

  • Transformers 4.49.0
  • Pytorch 2.6.0+cu124
  • Datasets 3.5.0
  • Tokenizers 0.21.1