--- library_name: transformers tags: - generated_from_trainer model-index: - name: hypa-tiny-keys results: [] --- # hypa-tiny-keys This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset. It achieves the following results on the evaluation set: - Loss: 1.1086 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 32 - eval_batch_size: 32 - seed: 3407 - gradient_accumulation_steps: 2 - total_train_batch_size: 64 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.95) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 500 - training_steps: 60000 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:-----:|:---------------:| | 2.8771 | 0.0083 | 500 | 2.8239 | | 2.3237 | 0.0167 | 1000 | 2.2951 | | 1.9399 | 0.025 | 1500 | 1.9368 | | 1.7840 | 0.0333 | 2000 | 1.7695 | | 1.6826 | 0.0417 | 2500 | 1.6788 | | 1.6208 | 0.05 | 3000 | 1.6156 | | 1.5774 | 0.0583 | 3500 | 1.5753 | | 1.5343 | 0.0667 | 4000 | 1.5389 | | 1.5108 | 0.075 | 4500 | 1.5089 | | 1.4918 | 0.0833 | 5000 | 1.4879 | | 1.4696 | 0.0917 | 5500 | 1.4679 | | 1.4439 | 0.1 | 6000 | 1.4490 | | 1.4353 | 0.1083 | 6500 | 1.4347 | | 1.4301 | 0.1167 | 7000 | 1.4197 | | 1.4114 | 0.125 | 7500 | 1.4098 | | 1.3965 | 0.1333 | 8000 | 1.3979 | | 1.3879 | 0.1417 | 8500 | 1.3883 | | 1.3754 | 0.15 | 9000 | 1.3759 | | 1.3746 | 0.1583 | 9500 | 1.3687 | | 1.3556 | 0.1667 | 10000 | 1.3607 | | 1.3897 | 0.175 | 10500 | 1.3605 | | 1.3735 | 0.1833 | 11000 | 1.3588 | | 1.3561 | 0.1917 | 11500 | 1.3561 | | 1.3514 | 0.2 | 12000 | 1.3529 | | 1.3370 | 0.2083 | 12500 | 1.3490 | | 1.3319 | 0.2167 | 13000 | 1.3470 | | 1.3339 | 0.225 | 13500 | 1.3440 | | 1.3161 | 0.2333 | 14000 | 1.3415 | | 1.3142 | 0.2417 | 14500 | 1.3338 | | 1.3061 | 0.25 | 15000 | 1.3280 | | 1.2954 | 0.2583 | 15500 | 1.3279 | | 1.2961 | 0.2667 | 16000 | 1.3253 | | 1.2917 | 0.275 | 16500 | 1.3156 | | 1.2850 | 0.2833 | 17000 | 1.3168 | | 1.2734 | 0.2917 | 17500 | 1.3135 | | 1.2637 | 0.3 | 18000 | 1.3080 | | 1.2708 | 0.3083 | 18500 | 1.3039 | | 1.2625 | 0.3167 | 19000 | 1.3002 | | 1.2623 | 0.325 | 19500 | 1.2992 | | 1.2482 | 0.3333 | 20000 | 1.2961 | | 1.2453 | 0.3417 | 20500 | 1.2909 | | 1.2873 | 0.35 | 21000 | 1.2822 | | 1.2766 | 0.3583 | 21500 | 1.2847 | | 1.2673 | 0.3667 | 22000 | 1.2796 | | 1.2618 | 0.375 | 22500 | 1.2755 | | 1.2543 | 0.3833 | 23000 | 1.2704 | | 1.2471 | 0.3917 | 23500 | 1.2728 | | 1.2345 | 0.4 | 24000 | 1.2703 | | 1.2350 | 0.4083 | 24500 | 1.2690 | | 1.2337 | 0.4167 | 25000 | 1.2657 | | 1.2304 | 0.425 | 25500 | 1.2619 | | 1.2202 | 0.4333 | 26000 | 1.2560 | | 1.2182 | 0.4417 | 26500 | 1.2521 | | 1.2137 | 0.45 | 27000 | 1.2488 | | 1.2118 | 0.4583 | 27500 | 1.2478 | | 1.2024 | 0.4667 | 28000 | 1.2449 | | 1.1989 | 0.475 | 28500 | 1.2446 | | 1.1929 | 0.4833 | 29000 | 1.2421 | | 1.1942 | 0.4917 | 29500 | 1.2386 | | 1.1842 | 0.5 | 30000 | 1.2354 | | 1.1798 | 0.5083 | 30500 | 1.2325 | | 1.2264 | 0.5167 | 31000 | 1.2270 | | 1.2130 | 0.525 | 31500 | 1.2215 | | 1.2058 | 0.5333 | 32000 | 1.2180 | | 1.1970 | 0.5417 | 32500 | 1.2136 | | 1.1886 | 0.55 | 33000 | 1.2118 | | 1.1826 | 0.5583 | 33500 | 1.2106 | | 1.1798 | 0.5667 | 34000 | 1.2090 | | 1.1714 | 0.575 | 34500 | 1.2061 | | 1.1707 | 0.5833 | 35000 | 1.2036 | | 1.1774 | 0.5917 | 35500 | 1.2013 | | 1.1602 | 0.6 | 36000 | 1.1985 | | 1.1625 | 0.6083 | 36500 | 1.1928 | | 1.1514 | 0.6167 | 37000 | 1.1921 | | 1.1507 | 0.625 | 37500 | 1.1902 | | 1.1444 | 0.6333 | 38000 | 1.1878 | | 1.1425 | 0.6417 | 38500 | 1.1865 | | 1.1393 | 0.65 | 39000 | 1.1819 | | 1.1355 | 0.6583 | 39500 | 1.1793 | | 1.1277 | 0.6667 | 40000 | 1.1765 | | 1.1314 | 0.675 | 40500 | 1.1780 | | 1.1272 | 0.6833 | 41000 | 1.1748 | | 1.1607 | 0.6917 | 41500 | 1.1643 | | 1.1553 | 0.7 | 42000 | 1.1613 | | 1.1484 | 0.7083 | 42500 | 1.1588 | | 1.1419 | 0.7167 | 43000 | 1.1546 | | 1.1298 | 0.725 | 43500 | 1.1523 | | 1.1313 | 0.7333 | 44000 | 1.1489 | | 1.1322 | 0.7417 | 44500 | 1.1474 | | 1.1266 | 0.75 | 45000 | 1.1453 | | 1.1190 | 0.7583 | 45500 | 1.1429 | | 1.1186 | 0.7667 | 46000 | 1.1399 | | 1.1122 | 0.775 | 46500 | 1.1377 | | 1.1145 | 0.7833 | 47000 | 1.1354 | | 1.1040 | 0.7917 | 47500 | 1.1348 | | 1.1126 | 0.8 | 48000 | 1.1331 | | 1.1042 | 0.8083 | 48500 | 1.1304 | | 1.0946 | 0.8167 | 49000 | 1.1298 | | 1.0894 | 0.825 | 49500 | 1.1280 | | 1.0993 | 0.8333 | 50000 | 1.1272 | | 1.0916 | 0.8417 | 50500 | 1.1261 | | 1.0844 | 0.85 | 51000 | 1.1234 | | 1.1222 | 0.8583 | 51500 | 1.1204 | | 1.1237 | 0.8667 | 52000 | 1.1179 | | 1.1307 | 0.875 | 52500 | 1.1154 | | 1.1195 | 0.8833 | 53000 | 1.1141 | | 1.1164 | 0.8917 | 53500 | 1.1130 | | 1.1139 | 0.9 | 54000 | 1.1124 | | 1.1159 | 0.9083 | 54500 | 1.1118 | | 1.1087 | 0.9167 | 55000 | 1.1112 | | 1.1082 | 0.925 | 55500 | 1.1104 | | 1.1036 | 0.9333 | 56000 | 1.1099 | | 1.1099 | 0.9417 | 56500 | 1.1097 | | 1.1088 | 0.95 | 57000 | 1.1094 | | 1.1091 | 0.9583 | 57500 | 1.1091 | | 1.1052 | 0.9667 | 58000 | 1.1088 | | 1.1012 | 0.975 | 58500 | 1.1087 | | 1.1072 | 0.9833 | 59000 | 1.1086 | | 1.1072 | 0.9917 | 59500 | 1.1086 | | 1.1053 | 1.0 | 60000 | 1.1086 | ### Framework versions - Transformers 5.17.0 - Pytorch 2.11.0+cu128 - Datasets 5.0.1 - Tokenizers 0.23.2