| Experiment: 2026-04-13-039-data-selection |
| Start time: 2026-04-13 14:19:49 |
|
|
| Device: cuda |
| Tokenizer vocab size: 50257 |
| Loading teacher model: /home/coder/experiments/finetuned-gpt2-large-50m |
| Teacher parameters: 774,030,080 (774.0M) |
| Loading pool data... |
| Total training tokens: 9999980 |
| Number of training chunks: 9765 |
| Student parameters: 162,148,608 (162.1M) |
| Student architecture: Llama (12L, 768H, 12A, 2048I) |
| Grad accum: 4, Effective batch: 16 |
| Total steps: 6110, Steps/epoch: 611 |
| Temperature: 2.0, Alpha: 0.5 |
|
|
| Epoch 1/10 | Step 10/6110 | Loss: 10.0095 (CE: 10.8165, KD: 9.1750) | Avg: 10.1923 | LR: 1.00e-05 |
| Epoch 1/10 | Step 20/6110 | Loss: 9.4791 (CE: 10.0368, KD: 8.9100) | Avg: 10.0206 | LR: 2.00e-05 |
| Epoch 1/10 | Step 30/6110 | Loss: 8.8191 (CE: 9.3620, KD: 8.2738) | Avg: 9.7973 | LR: 3.00e-05 |
| Epoch 1/10 | Step 40/6110 | Loss: 9.0437 (CE: 9.1246, KD: 8.9617) | Avg: 9.5745 | LR: 4.00e-05 |
| Epoch 1/10 | Step 50/6110 | Loss: 8.0996 (CE: 8.7685, KD: 7.4305) | Avg: 9.3570 | LR: 5.00e-05 |
| Epoch 1/10 | Step 60/6110 | Loss: 8.6687 (CE: 8.9492, KD: 8.3862) | Avg: 9.1420 | LR: 6.00e-05 |
| Epoch 1/10 | Step 70/6110 | Loss: 7.5424 (CE: 8.0205, KD: 7.0638) | Avg: 8.9232 | LR: 7.00e-05 |
| Epoch 1/10 | Step 80/6110 | Loss: 6.7640 (CE: 7.5897, KD: 5.9371) | Avg: 8.7187 | LR: 8.00e-05 |
| Epoch 1/10 | Step 90/6110 | Loss: 6.9664 (CE: 7.3822, KD: 6.5502) | Avg: 8.5287 | LR: 9.00e-05 |
| Epoch 1/10 | Step 100/6110 | Loss: 6.8416 (CE: 7.6013, KD: 6.0797) | Avg: 8.3521 | LR: 1.00e-04 |
| Epoch 1/10 | Step 110/6110 | Loss: 6.3619 (CE: 7.2609, KD: 5.4608) | Avg: 8.1969 | LR: 1.10e-04 |
| Epoch 1/10 | Step 120/6110 | Loss: 6.5781 (CE: 7.4313, KD: 5.7208) | Avg: 8.0653 | LR: 1.20e-04 |
| Epoch 1/10 | Step 130/6110 | Loss: 6.1693 (CE: 6.9595, KD: 5.3739) | Avg: 7.9363 | LR: 1.30e-04 |
| Epoch 1/10 | Step 140/6110 | Loss: 6.4805 (CE: 7.3669, KD: 5.5889) | Avg: 7.8249 | LR: 1.40e-04 |
| Epoch 1/10 | Step 150/6110 | Loss: 6.1767 (CE: 6.9384, KD: 5.4082) | Avg: 7.7193 | LR: 1.50e-04 |
| Epoch 1/10 | Step 160/6110 | Loss: 6.6880 (CE: 7.4261, KD: 5.9415) | Avg: 7.6239 | LR: 1.60e-04 |
| Epoch 1/10 | Step 170/6110 | Loss: 5.6760 (CE: 6.6147, KD: 4.7286) | Avg: 7.5289 | LR: 1.70e-04 |
| Epoch 1/10 | Step 180/6110 | Loss: 5.8842 (CE: 6.8969, KD: 4.8615) | Avg: 7.4389 | LR: 1.80e-04 |
| Epoch 1/10 | Step 190/6110 | Loss: 5.8432 (CE: 6.7480, KD: 4.9241) | Avg: 7.3537 | LR: 1.90e-04 |
| Epoch 1/10 | Step 200/6110 | Loss: 5.3198 (CE: 6.3265, KD: 4.3003) | Avg: 7.2710 | LR: 2.00e-04 |
| Epoch 1/10 | Step 210/6110 | Loss: 5.6219 (CE: 6.7199, KD: 4.5102) | Avg: 7.1901 | LR: 2.10e-04 |
| Epoch 1/10 | Step 220/6110 | Loss: 5.1295 (CE: 6.3912, KD: 3.8535) | Avg: 7.1153 | LR: 2.20e-04 |
| Epoch 1/10 | Step 230/6110 | Loss: 5.5542 (CE: 6.6510, KD: 4.4399) | Avg: 7.0422 | LR: 2.30e-04 |
| Epoch 1/10 | Step 240/6110 | Loss: 5.4157 (CE: 6.5491, KD: 4.2645) | Avg: 6.9736 | LR: 2.40e-04 |
| Epoch 1/10 | Step 250/6110 | Loss: 5.3719 (CE: 6.3092, KD: 4.4181) | Avg: 6.9073 | LR: 2.50e-04 |
| Epoch 1/10 | Step 260/6110 | Loss: 5.0125 (CE: 6.0095, KD: 3.9961) | Avg: 6.8457 | LR: 2.60e-04 |
| Epoch 1/10 | Step 270/6110 | Loss: 4.8035 (CE: 6.1126, KD: 3.4766) | Avg: 6.7873 | LR: 2.70e-04 |
| Epoch 1/10 | Step 280/6110 | Loss: 5.2260 (CE: 6.2132, KD: 4.2212) | Avg: 6.7306 | LR: 2.80e-04 |
| Epoch 1/10 | Step 290/6110 | Loss: 5.3053 (CE: 6.3681, KD: 4.2199) | Avg: 6.6776 | LR: 2.90e-04 |
| Epoch 1/10 | Step 300/6110 | Loss: 5.1180 (CE: 6.2219, KD: 3.9935) | Avg: 6.6242 | LR: 3.00e-04 |
| Epoch 1/10 | Step 310/6110 | Loss: 5.2901 (CE: 6.1394, KD: 4.4188) | Avg: 6.5755 | LR: 3.10e-04 |
| Epoch 1/10 | Step 320/6110 | Loss: 5.0376 (CE: 6.1504, KD: 3.9015) | Avg: 6.5287 | LR: 3.20e-04 |
| Epoch 1/10 | Step 330/6110 | Loss: 5.3339 (CE: 6.1049, KD: 4.5367) | Avg: 6.4815 | LR: 3.30e-04 |
| Epoch 1/10 | Step 340/6110 | Loss: 4.9189 (CE: 6.2378, KD: 3.5738) | Avg: 6.4359 | LR: 3.40e-04 |
| Epoch 1/10 | Step 350/6110 | Loss: 4.8344 (CE: 5.9357, KD: 3.7065) | Avg: 6.3945 | LR: 3.50e-04 |
| Epoch 1/10 | Step 360/6110 | Loss: 4.5822 (CE: 5.8580, KD: 3.2774) | Avg: 6.3517 | LR: 3.60e-04 |
| Epoch 1/10 | Step 370/6110 | Loss: 4.8283 (CE: 6.1172, KD: 3.5085) | Avg: 6.3136 | LR: 3.70e-04 |
| Epoch 1/10 | Step 380/6110 | Loss: 5.1314 (CE: 6.1138, KD: 4.1172) | Avg: 6.2739 | LR: 3.80e-04 |
| Epoch 1/10 | Step 390/6110 | Loss: 4.8990 (CE: 6.2449, KD: 3.5251) | Avg: 6.2359 | LR: 3.90e-04 |
| Epoch 1/10 | Step 400/6110 | Loss: 5.1450 (CE: 6.2734, KD: 3.9890) | Avg: 6.2004 | LR: 4.00e-04 |
| Epoch 1/10 | Step 410/6110 | Loss: 4.6193 (CE: 5.8633, KD: 3.3461) | Avg: 6.1638 | LR: 4.10e-04 |
| Epoch 1/10 | Step 420/6110 | Loss: 4.7160 (CE: 5.7290, KD: 3.6725) | Avg: 6.1291 | LR: 4.20e-04 |
| Epoch 1/10 | Step 430/6110 | Loss: 4.5967 (CE: 5.8952, KD: 3.2630) | Avg: 6.0965 | LR: 4.30e-04 |
| Epoch 1/10 | Step 440/6110 | Loss: 4.6082 (CE: 5.6211, KD: 3.5662) | Avg: 6.0657 | LR: 4.40e-04 |
| Epoch 1/10 | Step 450/6110 | Loss: 4.4810 (CE: 5.7127, KD: 3.2146) | Avg: 6.0350 | LR: 4.50e-04 |
| Epoch 1/10 | Step 460/6110 | Loss: 4.5194 (CE: 5.8112, KD: 3.1949) | Avg: 6.0054 | LR: 4.60e-04 |
| Epoch 1/10 | Step 470/6110 | Loss: 4.7304 (CE: 5.9381, KD: 3.4902) | Avg: 5.9760 | LR: 4.70e-04 |
| Epoch 1/10 | Step 480/6110 | Loss: 4.5235 (CE: 5.6892, KD: 3.3238) | Avg: 5.9467 | LR: 4.80e-04 |
| Epoch 1/10 | Step 490/6110 | Loss: 4.5714 (CE: 5.7232, KD: 3.3901) | Avg: 5.9210 | LR: 4.90e-04 |
| Epoch 1/10 | Step 500/6110 | Loss: 4.5537 (CE: 5.9697, KD: 3.1007) | Avg: 5.8941 | LR: 5.00e-04 |
| Epoch 1/10 | Step 510/6110 | Loss: 4.6956 (CE: 5.8971, KD: 3.4578) | Avg: 5.8692 | LR: 5.10e-04 |
| Epoch 1/10 | Step 520/6110 | Loss: 4.5815 (CE: 5.9869, KD: 3.1409) | Avg: 5.8432 | LR: 5.20e-04 |
| Epoch 1/10 | Step 530/6110 | Loss: 5.1682 (CE: 6.0768, KD: 4.2276) | Avg: 5.8206 | LR: 5.30e-04 |
| Epoch 1/10 | Step 540/6110 | Loss: 4.6757 (CE: 5.8387, KD: 3.4770) | Avg: 5.7959 | LR: 5.40e-04 |
| Epoch 1/10 | Step 550/6110 | Loss: 4.3247 (CE: 5.6809, KD: 2.9256) | Avg: 5.7712 | LR: 5.50e-04 |
| Epoch 1/10 | Step 560/6110 | Loss: 4.5805 (CE: 5.7530, KD: 3.3681) | Avg: 5.7461 | LR: 5.60e-04 |
| Epoch 1/10 | Step 570/6110 | Loss: 4.5477 (CE: 5.5472, KD: 3.5098) | Avg: 5.7217 | LR: 5.70e-04 |
| Epoch 1/10 | Step 580/6110 | Loss: 4.3644 (CE: 5.6377, KD: 3.0535) | Avg: 5.6997 | LR: 5.80e-04 |
| Epoch 1/10 | Step 590/6110 | Loss: 4.4682 (CE: 5.5662, KD: 3.3248) | Avg: 5.6780 | LR: 5.90e-04 |
| Epoch 1/10 | Step 600/6110 | Loss: 4.1805 (CE: 5.4688, KD: 2.8509) | Avg: 5.6561 | LR: 6.00e-04 |
| Epoch 1/10 | Step 610/6110 | Loss: 4.6669 (CE: 5.7995, KD: 3.4927) | Avg: 5.6364 | LR: 6.10e-04 |
|
|
| Epoch 1/10 complete | Avg Loss: 5.6358 (CE: 6.6260, KD: 4.6237) | Perplexity: 754.48 | Time: 391.6s |
|
|
| Epoch 2/10 | Step 620/6110 | Loss: 4.3952 (CE: 5.6550, KD: 3.0908) | Avg: 4.3541 | LR: 6.20e-04 |
| Epoch 2/10 | Step 630/6110 | Loss: 4.3718 (CE: 5.3498, KD: 3.3527) | Avg: 4.3422 | LR: 6.30e-04 |
| Epoch 2/10 | Step 640/6110 | Loss: 4.3700 (CE: 5.4416, KD: 3.2480) | Avg: 4.3422 | LR: 6.40e-04 |
| Epoch 2/10 | Step 650/6110 | Loss: 4.3716 (CE: 5.6320, KD: 3.0612) | Avg: 4.3191 | LR: 6.50e-04 |
| Epoch 2/10 | Step 660/6110 | Loss: 4.1912 (CE: 5.5784, KD: 2.7614) | Avg: 4.3108 | LR: 6.60e-04 |
| Epoch 2/10 | Step 670/6110 | Loss: 4.2567 (CE: 5.3470, KD: 3.1207) | Avg: 4.2962 | LR: 6.70e-04 |
| Epoch 2/10 | Step 680/6110 | Loss: 4.3347 (CE: 5.4979, KD: 3.1234) | Avg: 4.2862 | LR: 6.80e-04 |
| Epoch 2/10 | Step 690/6110 | Loss: 4.1485 (CE: 5.3520, KD: 2.8985) | Avg: 4.2733 | LR: 6.90e-04 |
| Epoch 2/10 | Step 700/6110 | Loss: 4.0883 (CE: 5.3525, KD: 2.7818) | Avg: 4.2674 | LR: 7.00e-04 |
| Epoch 2/10 | Step 710/6110 | Loss: 4.2065 (CE: 5.0272, KD: 3.3471) | Avg: 4.2581 | LR: 7.10e-04 |
| Epoch 2/10 | Step 720/6110 | Loss: 4.2277 (CE: 5.4404, KD: 2.9701) | Avg: 4.2535 | LR: 7.20e-04 |
| Epoch 2/10 | Step 730/6110 | Loss: 4.1106 (CE: 5.3838, KD: 2.7936) | Avg: 4.2455 | LR: 7.30e-04 |
| Epoch 2/10 | Step 740/6110 | Loss: 4.0787 (CE: 5.3683, KD: 2.7453) | Avg: 4.2351 | LR: 7.40e-04 |
| Epoch 2/10 | Step 750/6110 | Loss: 4.5035 (CE: 5.1785, KD: 3.7808) | Avg: 4.2313 | LR: 7.50e-04 |
| Epoch 2/10 | Step 760/6110 | Loss: 4.1541 (CE: 5.6041, KD: 2.6546) | Avg: 4.2232 | LR: 7.60e-04 |
| Epoch 2/10 | Step 770/6110 | Loss: 3.9159 (CE: 5.0995, KD: 2.6824) | Avg: 4.2187 | LR: 7.70e-04 |
| Epoch 2/10 | Step 780/6110 | Loss: 4.1745 (CE: 5.5839, KD: 2.7166) | Avg: 4.2180 | LR: 7.80e-04 |
| Epoch 2/10 | Step 790/6110 | Loss: 4.2365 (CE: 5.4057, KD: 3.0253) | Avg: 4.2140 | LR: 7.90e-04 |
| Epoch 2/10 | Step 800/6110 | Loss: 4.4148 (CE: 5.4164, KD: 3.3671) | Avg: 4.2085 | LR: 8.00e-04 |
| Epoch 2/10 | Step 810/6110 | Loss: 4.0974 (CE: 5.1825, KD: 2.9564) | Avg: 4.2004 | LR: 8.10e-04 |
| Epoch 2/10 | Step 820/6110 | Loss: 3.9985 (CE: 5.3049, KD: 2.6457) | Avg: 4.1958 | LR: 8.20e-04 |
| Epoch 2/10 | Step 830/6110 | Loss: 4.2432 (CE: 5.5562, KD: 2.8811) | Avg: 4.1917 | LR: 8.30e-04 |
| Epoch 2/10 | Step 840/6110 | Loss: 3.9509 (CE: 5.1848, KD: 2.6744) | Avg: 4.1877 | LR: 8.40e-04 |
| Epoch 2/10 | Step 850/6110 | Loss: 4.0527 (CE: 5.3541, KD: 2.7051) | Avg: 4.1836 | LR: 8.50e-04 |
| Epoch 2/10 | Step 860/6110 | Loss: 4.4924 (CE: 5.4756, KD: 3.4484) | Avg: 4.1751 | LR: 8.60e-04 |
| Epoch 2/10 | Step 870/6110 | Loss: 4.2432 (CE: 5.5864, KD: 2.8487) | Avg: 4.1712 | LR: 8.70e-04 |
| Epoch 2/10 | Step 880/6110 | Loss: 4.0963 (CE: 5.4944, KD: 2.6482) | Avg: 4.1658 | LR: 8.80e-04 |
| Epoch 2/10 | Step 890/6110 | Loss: 3.9985 (CE: 5.2441, KD: 2.6992) | Avg: 4.1611 | LR: 8.90e-04 |
| Epoch 2/10 | Step 900/6110 | Loss: 3.9063 (CE: 5.2597, KD: 2.5083) | Avg: 4.1555 | LR: 9.00e-04 |
| Epoch 2/10 | Step 910/6110 | Loss: 4.1125 (CE: 5.2477, KD: 2.9223) | Avg: 4.1514 | LR: 9.10e-04 |
| Epoch 2/10 | Step 920/6110 | Loss: 5.0279 (CE: 5.5419, KD: 4.4632) | Avg: 4.1485 | LR: 9.20e-04 |
| Epoch 2/10 | Step 930/6110 | Loss: 3.8922 (CE: 5.0538, KD: 2.6787) | Avg: 4.1459 | LR: 9.30e-04 |
| Epoch 2/10 | Step 940/6110 | Loss: 4.1754 (CE: 5.2692, KD: 3.0306) | Avg: 4.1421 | LR: 9.40e-04 |
| Epoch 2/10 | Step 950/6110 | Loss: 4.1051 (CE: 4.7473, KD: 3.4095) | Avg: 4.1363 | LR: 9.50e-04 |
| Epoch 2/10 | Step 960/6110 | Loss: 4.1668 (CE: 5.2652, KD: 3.0164) | Avg: 4.1325 | LR: 9.60e-04 |
| Epoch 2/10 | Step 970/6110 | Loss: 3.8798 (CE: 5.1742, KD: 2.5256) | Avg: 4.1305 | LR: 9.70e-04 |
| Epoch 2/10 | Step 980/6110 | Loss: 3.9252 (CE: 4.7455, KD: 3.0512) | Avg: 4.1263 | LR: 9.80e-04 |
| Epoch 2/10 | Step 990/6110 | Loss: 3.7818 (CE: 4.9914, KD: 2.5218) | Avg: 4.1217 | LR: 9.90e-04 |
| Epoch 2/10 | Step 1000/6110 | Loss: 4.3031 (CE: 5.5679, KD: 2.9799) | Avg: 4.1177 | LR: 1.00e-03 |
| Epoch 2/10 | Step 1010/6110 | Loss: 4.0736 (CE: 5.1742, KD: 2.9188) | Avg: 4.1105 | LR: 1.01e-03 |
| Epoch 2/10 | Step 1020/6110 | Loss: 3.7023 (CE: 5.0457, KD: 2.3104) | Avg: 4.1050 | LR: 1.02e-03 |
| Epoch 2/10 | Step 1030/6110 | Loss: 3.7088 (CE: 5.0070, KD: 2.3591) | Avg: 4.0977 | LR: 1.03e-03 |
| Epoch 2/10 | Step 1040/6110 | Loss: 3.9039 (CE: 5.1077, KD: 2.6451) | Avg: 4.0920 | LR: 1.04e-03 |
| Epoch 2/10 | Step 1050/6110 | Loss: 3.6023 (CE: 4.8453, KD: 2.3071) | Avg: 4.0888 | LR: 1.05e-03 |
| Epoch 2/10 | Step 1060/6110 | Loss: 4.4078 (CE: 5.2453, KD: 3.5077) | Avg: 4.0846 | LR: 1.06e-03 |
| Epoch 2/10 | Step 1070/6110 | Loss: 3.9580 (CE: 5.2204, KD: 2.6406) | Avg: 4.0815 | LR: 1.07e-03 |
| Epoch 2/10 | Step 1080/6110 | Loss: 3.6123 (CE: 4.8634, KD: 2.3052) | Avg: 4.0755 | LR: 1.08e-03 |
| Epoch 2/10 | Step 1090/6110 | Loss: 3.6877 (CE: 4.9975, KD: 2.3278) | Avg: 4.0706 | LR: 1.09e-03 |
| Epoch 2/10 | Step 1100/6110 | Loss: 3.5887 (CE: 4.6222, KD: 2.5170) | Avg: 4.0647 | LR: 1.10e-03 |
| Epoch 2/10 | Step 1110/6110 | Loss: 3.6177 (CE: 4.8991, KD: 2.2829) | Avg: 4.0604 | LR: 1.11e-03 |
| Epoch 2/10 | Step 1120/6110 | Loss: 3.8883 (CE: 5.1842, KD: 2.5328) | Avg: 4.0554 | LR: 1.12e-03 |
| Epoch 2/10 | Step 1130/6110 | Loss: 3.7959 (CE: 5.0331, KD: 2.5036) | Avg: 4.0506 | LR: 1.13e-03 |
| Epoch 2/10 | Step 1140/6110 | Loss: 4.0235 (CE: 5.3149, KD: 2.6729) | Avg: 4.0460 | LR: 1.14e-03 |
| Epoch 2/10 | Step 1150/6110 | Loss: 3.8719 (CE: 5.1053, KD: 2.5820) | Avg: 4.0428 | LR: 1.15e-03 |
| Epoch 2/10 | Step 1160/6110 | Loss: 3.4668 (CE: 4.6703, KD: 2.2147) | Avg: 4.0391 | LR: 1.16e-03 |
| Epoch 2/10 | Step 1170/6110 | Loss: 3.7842 (CE: 5.1568, KD: 2.3553) | Avg: 4.0351 | LR: 1.17e-03 |
| Epoch 2/10 | Step 1180/6110 | Loss: 3.5037 (CE: 4.7108, KD: 2.2483) | Avg: 4.0296 | LR: 1.18e-03 |
| Epoch 2/10 | Step 1190/6110 | Loss: 4.0367 (CE: 5.1535, KD: 2.8569) | Avg: 4.0251 | LR: 1.19e-03 |
| Epoch 2/10 | Step 1200/6110 | Loss: 3.7230 (CE: 4.9761, KD: 2.4125) | Avg: 4.0212 | LR: 1.20e-03 |
| Epoch 2/10 | Step 1210/6110 | Loss: 3.8022 (CE: 5.2460, KD: 2.3043) | Avg: 4.0166 | LR: 1.21e-03 |
| Epoch 2/10 | Step 1220/6110 | Loss: 3.0183 (CE: 4.1370, KD: 1.8567) | Avg: 4.0120 | LR: 1.22e-03 |
|
|
| Epoch 2/10 complete | Avg Loss: 4.0116 (CE: 5.2089, KD: 2.7639) | Perplexity: 182.88 | Time: 391.8s |
|
|
| Epoch 3/10 | Step 1230/6110 | Loss: 3.6393 (CE: 4.7902, KD: 2.4374) | Avg: 3.6913 | LR: 1.23e-03 |
| Epoch 3/10 | Step 1240/6110 | Loss: 3.6420 (CE: 4.8549, KD: 2.3687) | Avg: 3.6381 | LR: 1.24e-03 |
| Epoch 3/10 | Step 1250/6110 | Loss: 3.5993 (CE: 4.7760, KD: 2.3733) | Avg: 3.6592 | LR: 1.25e-03 |
| Epoch 3/10 | Step 1260/6110 | Loss: 3.5793 (CE: 4.8855, KD: 2.2180) | Avg: 3.6453 | LR: 1.26e-03 |
| Epoch 3/10 | Step 1270/6110 | Loss: 3.7619 (CE: 5.0176, KD: 2.4444) | Avg: 3.6533 | LR: 1.27e-03 |
| Epoch 3/10 | Step 1280/6110 | Loss: 3.7262 (CE: 5.0249, KD: 2.3716) | Avg: 3.6421 | LR: 1.28e-03 |
| Epoch 3/10 | Step 1290/6110 | Loss: 3.6485 (CE: 4.9006, KD: 2.3393) | Avg: 3.6432 | LR: 1.29e-03 |
| Epoch 3/10 | Step 1300/6110 | Loss: 3.7645 (CE: 5.0027, KD: 2.4659) | Avg: 3.6532 | LR: 1.30e-03 |
| Epoch 3/10 | Step 1310/6110 | Loss: 3.7348 (CE: 5.1170, KD: 2.2920) | Avg: 3.6490 | LR: 1.31e-03 |
| Epoch 3/10 | Step 1320/6110 | Loss: 3.6368 (CE: 4.9816, KD: 2.2278) | Avg: 3.6511 | LR: 1.32e-03 |
| Epoch 3/10 | Step 1330/6110 | Loss: 4.0184 (CE: 4.7686, KD: 3.1926) | Avg: 3.6540 | LR: 1.33e-03 |
| Epoch 3/10 | Step 1340/6110 | Loss: 3.4699 (CE: 4.6044, KD: 2.2773) | Avg: 3.6542 | LR: 1.34e-03 |
| Epoch 3/10 | Step 1350/6110 | Loss: 3.7567 (CE: 4.9005, KD: 2.5591) | Avg: 3.6520 | LR: 1.35e-03 |
| Epoch 3/10 | Step 1360/6110 | Loss: 3.7481 (CE: 4.8594, KD: 2.5790) | Avg: 3.6528 | LR: 1.36e-03 |
| Epoch 3/10 | Step 1370/6110 | Loss: 3.4395 (CE: 4.6611, KD: 2.1610) | Avg: 3.6565 | LR: 1.37e-03 |
| Epoch 3/10 | Step 1380/6110 | Loss: 3.6638 (CE: 4.8111, KD: 2.4579) | Avg: 3.6654 | LR: 1.38e-03 |
| Epoch 3/10 | Step 1390/6110 | Loss: 3.5698 (CE: 5.0351, KD: 2.0429) | Avg: 3.6709 | LR: 1.39e-03 |
| Epoch 3/10 | Step 1400/6110 | Loss: 3.5794 (CE: 4.8573, KD: 2.2351) | Avg: 3.6691 | LR: 1.40e-03 |
| Epoch 3/10 | Step 1410/6110 | Loss: 3.6682 (CE: 4.7810, KD: 2.4916) | Avg: 3.6703 | LR: 1.41e-03 |
| Epoch 3/10 | Step 1420/6110 | Loss: 4.0681 (CE: 4.6584, KD: 3.4157) | Avg: 3.6742 | LR: 1.42e-03 |
| Epoch 3/10 | Step 1430/6110 | Loss: 3.8973 (CE: 5.1064, KD: 2.6129) | Avg: 3.6746 | LR: 1.43e-03 |
| Epoch 3/10 | Step 1440/6110 | Loss: 3.7946 (CE: 5.0162, KD: 2.5040) | Avg: 3.6739 | LR: 1.44e-03 |
| Epoch 3/10 | Step 1450/6110 | Loss: 3.7539 (CE: 5.1244, KD: 2.3117) | Avg: 3.6735 | LR: 1.45e-03 |
| Epoch 3/10 | Step 1460/6110 | Loss: 3.5273 (CE: 4.8485, KD: 2.1492) | Avg: 3.6724 | LR: 1.46e-03 |
| Epoch 3/10 | Step 1470/6110 | Loss: 4.2066 (CE: 4.3072, KD: 4.0478) | Avg: 3.6701 | LR: 1.47e-03 |
| Epoch 3/10 | Step 1480/6110 | Loss: 3.3750 (CE: 4.4639, KD: 2.2263) | Avg: 3.6694 | LR: 1.48e-03 |
| Epoch 3/10 | Step 1490/6110 | Loss: 4.0880 (CE: 5.1307, KD: 2.9729) | Avg: 3.6673 | LR: 1.49e-03 |
| Epoch 3/10 | Step 1500/6110 | Loss: 3.4516 (CE: 4.8150, KD: 2.0313) | Avg: 3.6659 | LR: 1.50e-03 |
| Epoch 3/10 | Step 1510/6110 | Loss: 3.6491 (CE: 4.9348, KD: 2.3018) | Avg: 3.6642 | LR: 1.51e-03 |
| Epoch 3/10 | Step 1520/6110 | Loss: 3.5215 (CE: 4.4120, KD: 2.5694) | Avg: 3.6632 | LR: 1.52e-03 |
| Epoch 3/10 | Step 1530/6110 | Loss: 3.6316 (CE: 4.9956, KD: 2.2041) | Avg: 3.6622 | LR: 1.53e-03 |
| Epoch 3/10 | Step 1540/6110 | Loss: 3.4655 (CE: 4.7894, KD: 2.0834) | Avg: 3.6611 | LR: 1.54e-03 |
| Epoch 3/10 | Step 1550/6110 | Loss: 3.5704 (CE: 4.7457, KD: 2.3346) | Avg: 3.6585 | LR: 1.55e-03 |
| Epoch 3/10 | Step 1560/6110 | Loss: 3.7920 (CE: 4.8509, KD: 2.6614) | Avg: 3.6547 | LR: 1.56e-03 |
| Epoch 3/10 | Step 1570/6110 | Loss: 3.2134 (CE: 4.4198, KD: 1.9537) | Avg: 3.6519 | LR: 1.57e-03 |
| Epoch 3/10 | Step 1580/6110 | Loss: 3.4708 (CE: 4.5262, KD: 2.3563) | Avg: 3.6510 | LR: 1.58e-03 |
| Epoch 3/10 | Step 1590/6110 | Loss: 3.5838 (CE: 4.6541, KD: 2.4503) | Avg: 3.6490 | LR: 1.59e-03 |
| Epoch 3/10 | Step 1600/6110 | Loss: 3.4422 (CE: 4.6740, KD: 2.1508) | Avg: 3.6464 | LR: 1.60e-03 |
| Epoch 3/10 | Step 1610/6110 | Loss: 3.4423 (CE: 4.7846, KD: 2.0383) | Avg: 3.6437 | LR: 1.61e-03 |
| Epoch 3/10 | Step 1620/6110 | Loss: 3.6545 (CE: 4.7806, KD: 2.4679) | Avg: 3.6410 | LR: 1.62e-03 |
| Epoch 3/10 | Step 1630/6110 | Loss: 3.4391 (CE: 4.7042, KD: 2.1200) | Avg: 3.6385 | LR: 1.63e-03 |
| Epoch 3/10 | Step 1640/6110 | Loss: 3.5725 (CE: 4.8524, KD: 2.2369) | Avg: 3.6338 | LR: 1.64e-03 |
| Epoch 3/10 | Step 1650/6110 | Loss: 3.2139 (CE: 4.4683, KD: 1.9050) | Avg: 3.6315 | LR: 1.65e-03 |
| Epoch 3/10 | Step 1660/6110 | Loss: 3.8275 (CE: 5.0958, KD: 2.4942) | Avg: 3.6313 | LR: 1.66e-03 |
| Epoch 3/10 | Step 1670/6110 | Loss: 3.6579 (CE: 4.9758, KD: 2.2768) | Avg: 3.6292 | LR: 1.67e-03 |
| Epoch 3/10 | Step 1680/6110 | Loss: 3.7783 (CE: 5.0011, KD: 2.4886) | Avg: 3.6282 | LR: 1.68e-03 |
| Epoch 3/10 | Step 1690/6110 | Loss: 3.5651 (CE: 4.1703, KD: 2.9098) | Avg: 3.6260 | LR: 1.69e-03 |
| Epoch 3/10 | Step 1700/6110 | Loss: 3.6369 (CE: 4.8817, KD: 2.3263) | Avg: 3.6247 | LR: 1.70e-03 |
| Epoch 3/10 | Step 1710/6110 | Loss: 3.5577 (CE: 4.7291, KD: 2.2968) | Avg: 3.6225 | LR: 1.71e-03 |
| Epoch 3/10 | Step 1720/6110 | Loss: 3.9495 (CE: 5.2735, KD: 2.5606) | Avg: 3.6217 | LR: 1.72e-03 |
| Epoch 3/10 | Step 1730/6110 | Loss: 3.6759 (CE: 4.5735, KD: 2.7092) | Avg: 3.6214 | LR: 1.73e-03 |
| Epoch 3/10 | Step 1740/6110 | Loss: 3.5783 (CE: 4.8214, KD: 2.2722) | Avg: 3.6192 | LR: 1.74e-03 |
| Epoch 3/10 | Step 1750/6110 | Loss: 3.6693 (CE: 4.8907, KD: 2.3818) | Avg: 3.6196 | LR: 1.75e-03 |
| Epoch 3/10 | Step 1760/6110 | Loss: 3.6120 (CE: 4.8534, KD: 2.2989) | Avg: 3.6177 | LR: 1.76e-03 |
| Epoch 3/10 | Step 1770/6110 | Loss: 3.5497 (CE: 4.8295, KD: 2.2082) | Avg: 3.6152 | LR: 1.77e-03 |
| Epoch 3/10 | Step 1780/6110 | Loss: 3.5762 (CE: 4.8484, KD: 2.2413) | Avg: 3.6127 | LR: 1.78e-03 |
| Epoch 3/10 | Step 1790/6110 | Loss: 3.7668 (CE: 5.0306, KD: 2.4391) | Avg: 3.6101 | LR: 1.79e-03 |
| Epoch 3/10 | Step 1800/6110 | Loss: 3.5355 (CE: 4.9154, KD: 2.0946) | Avg: 3.6071 | LR: 1.80e-03 |
| Epoch 3/10 | Step 1810/6110 | Loss: 3.4593 (CE: 4.7883, KD: 2.0756) | Avg: 3.6044 | LR: 1.81e-03 |
| Epoch 3/10 | Step 1820/6110 | Loss: 3.6176 (CE: 4.7998, KD: 2.3742) | Avg: 3.6020 | LR: 1.82e-03 |
| Epoch 3/10 | Step 1830/6110 | Loss: 3.5370 (CE: 4.7491, KD: 2.2565) | Avg: 3.5994 | LR: 1.83e-03 |
|
|
| Epoch 3/10 complete | Avg Loss: 3.5986 (CE: 4.7880, KD: 2.3460) | Perplexity: 120.07 | Time: 390.9s |
|
|
| Epoch 4/10 | Step 1840/6110 | Loss: 3.1468 (CE: 4.3609, KD: 1.8763) | Avg: 3.2745 | LR: 1.84e-03 |
| Epoch 4/10 | Step 1850/6110 | Loss: 3.1855 (CE: 3.9896, KD: 2.3142) | Avg: 3.2727 | LR: 1.85e-03 |
| Epoch 4/10 | Step 1860/6110 | Loss: 3.4503 (CE: 4.6014, KD: 2.2237) | Avg: 3.2907 | LR: 1.86e-03 |
| Epoch 4/10 | Step 1870/6110 | Loss: 3.2453 (CE: 4.3963, KD: 2.0250) | Avg: 3.3068 | LR: 1.87e-03 |
| Epoch 4/10 | Step 1880/6110 | Loss: 3.4785 (CE: 4.6597, KD: 2.2328) | Avg: 3.3113 | LR: 1.88e-03 |
| Epoch 4/10 | Step 1890/6110 | Loss: 3.2415 (CE: 4.3519, KD: 2.0748) | Avg: 3.3058 | LR: 1.89e-03 |
| Epoch 4/10 | Step 1900/6110 | Loss: 3.1977 (CE: 4.1986, KD: 2.1390) | Avg: 3.3060 | LR: 1.90e-03 |
| Epoch 4/10 | Step 1910/6110 | Loss: 3.1986 (CE: 4.3148, KD: 2.0263) | Avg: 3.3027 | LR: 1.91e-03 |
| Epoch 4/10 | Step 1920/6110 | Loss: 3.6735 (CE: 5.0496, KD: 2.2269) | Avg: 3.3100 | LR: 1.92e-03 |
| Epoch 4/10 | Step 1930/6110 | Loss: 2.8035 (CE: 3.3766, KD: 2.1818) | Avg: 3.3145 | LR: 1.93e-03 |
| Epoch 4/10 | Step 1940/6110 | Loss: 3.4479 (CE: 4.7127, KD: 2.1190) | Avg: 3.3173 | LR: 1.94e-03 |
| Epoch 4/10 | Step 1950/6110 | Loss: 3.2291 (CE: 4.5309, KD: 1.8708) | Avg: 3.3218 | LR: 1.95e-03 |
| Epoch 4/10 | Step 1960/6110 | Loss: 3.2428 (CE: 4.5266, KD: 1.9023) | Avg: 3.3247 | LR: 1.96e-03 |
| Epoch 4/10 | Step 1970/6110 | Loss: 3.3697 (CE: 4.5957, KD: 2.0834) | Avg: 3.3256 | LR: 1.97e-03 |
| Epoch 4/10 | Step 1980/6110 | Loss: 3.3960 (CE: 4.5174, KD: 2.2126) | Avg: 3.3263 | LR: 1.98e-03 |
| Epoch 4/10 | Step 1990/6110 | Loss: 3.3252 (CE: 4.6793, KD: 1.9094) | Avg: 3.3226 | LR: 1.99e-03 |
| Epoch 4/10 | Step 2000/6110 | Loss: 3.4158 (CE: 4.6701, KD: 2.1026) | Avg: 3.3212 | LR: 2.00e-03 |
| Epoch 4/10 | Step 2010/6110 | Loss: 3.2204 (CE: 4.5188, KD: 1.8698) | Avg: 3.3202 | LR: 2.01e-03 |
| Epoch 4/10 | Step 2020/6110 | Loss: 3.0092 (CE: 3.9420, KD: 2.0245) | Avg: 3.3211 | LR: 2.02e-03 |
| Epoch 4/10 | Step 2030/6110 | Loss: 3.4082 (CE: 4.6547, KD: 2.1040) | Avg: 3.3224 | LR: 2.03e-03 |
| Epoch 4/10 | Step 2040/6110 | Loss: 3.2326 (CE: 4.5033, KD: 1.9029) | Avg: 3.3232 | LR: 2.04e-03 |
| Epoch 4/10 | Step 2050/6110 | Loss: 3.1562 (CE: 4.4085, KD: 1.8489) | Avg: 3.3208 | LR: 2.05e-03 |
| Epoch 4/10 | Step 2060/6110 | Loss: 3.4411 (CE: 4.4500, KD: 2.3790) | Avg: 3.3199 | LR: 2.06e-03 |
| Epoch 4/10 | Step 2070/6110 | Loss: 3.3045 (CE: 4.4048, KD: 2.1445) | Avg: 3.3208 | LR: 2.07e-03 |
| Epoch 4/10 | Step 2080/6110 | Loss: 3.3767 (CE: 4.6177, KD: 2.0780) | Avg: 3.3246 | LR: 2.08e-03 |
| Epoch 4/10 | Step 2090/6110 | Loss: 3.3193 (CE: 4.6545, KD: 1.9253) | Avg: 3.3244 | LR: 2.09e-03 |
| Epoch 4/10 | Step 2100/6110 | Loss: 3.3931 (CE: 4.3896, KD: 2.3375) | Avg: 3.3245 | LR: 2.10e-03 |
| Epoch 4/10 | Step 2110/6110 | Loss: 3.2187 (CE: 4.5657, KD: 1.8179) | Avg: 3.3253 | LR: 2.11e-03 |
| Epoch 4/10 | Step 2120/6110 | Loss: 3.3228 (CE: 4.5005, KD: 2.0853) | Avg: 3.3280 | LR: 2.12e-03 |
| Epoch 4/10 | Step 2130/6110 | Loss: 3.1935 (CE: 4.2952, KD: 2.0369) | Avg: 3.3285 | LR: 2.13e-03 |
| Epoch 4/10 | Step 2140/6110 | Loss: 3.3434 (CE: 4.5786, KD: 2.0500) | Avg: 3.3293 | LR: 2.14e-03 |
| Epoch 4/10 | Step 2150/6110 | Loss: 3.4753 (CE: 4.5062, KD: 2.3818) | Avg: 3.3286 | LR: 2.15e-03 |
| Epoch 4/10 | Step 2160/6110 | Loss: 3.2324 (CE: 4.4594, KD: 1.9495) | Avg: 3.3268 | LR: 2.16e-03 |
| Epoch 4/10 | Step 2170/6110 | Loss: 3.3395 (CE: 4.6073, KD: 2.0110) | Avg: 3.3277 | LR: 2.17e-03 |
| Epoch 4/10 | Step 2180/6110 | Loss: 3.3230 (CE: 4.5232, KD: 2.0626) | Avg: 3.3281 | LR: 2.18e-03 |
| Epoch 4/10 | Step 2190/6110 | Loss: 3.4852 (CE: 4.7107, KD: 2.1954) | Avg: 3.3292 | LR: 2.19e-03 |
| Epoch 4/10 | Step 2200/6110 | Loss: 3.5059 (CE: 4.7110, KD: 2.2407) | Avg: 3.3297 | LR: 2.20e-03 |
| Epoch 4/10 | Step 2210/6110 | Loss: 3.4255 (CE: 4.7743, KD: 2.0157) | Avg: 3.3303 | LR: 2.21e-03 |
| Epoch 4/10 | Step 2220/6110 | Loss: 3.0217 (CE: 4.1697, KD: 1.8252) | Avg: 3.3297 | LR: 2.22e-03 |
| Epoch 4/10 | Step 2230/6110 | Loss: 3.2396 (CE: 4.4784, KD: 1.9482) | Avg: 3.3299 | LR: 2.23e-03 |
| Epoch 4/10 | Step 2240/6110 | Loss: 3.1762 (CE: 4.3722, KD: 1.9309) | Avg: 3.3275 | LR: 2.24e-03 |
| Epoch 4/10 | Step 2250/6110 | Loss: 3.2819 (CE: 4.4879, KD: 2.0200) | Avg: 3.3270 | LR: 2.25e-03 |
| Epoch 4/10 | Step 2260/6110 | Loss: 3.1990 (CE: 4.5811, KD: 1.7677) | Avg: 3.3263 | LR: 2.26e-03 |
| Epoch 4/10 | Step 2270/6110 | Loss: 3.2924 (CE: 4.4614, KD: 2.0676) | Avg: 3.3258 | LR: 2.27e-03 |
| Epoch 4/10 | Step 2280/6110 | Loss: 3.1755 (CE: 4.3788, KD: 1.9142) | Avg: 3.3243 | LR: 2.28e-03 |
| Epoch 4/10 | Step 2290/6110 | Loss: 3.2386 (CE: 4.3654, KD: 2.0581) | Avg: 3.3222 | LR: 2.29e-03 |
| Epoch 4/10 | Step 2300/6110 | Loss: 3.3516 (CE: 4.6647, KD: 1.9843) | Avg: 3.3213 | LR: 2.30e-03 |
| Epoch 4/10 | Step 2310/6110 | Loss: 3.1634 (CE: 4.3253, KD: 1.9437) | Avg: 3.3211 | LR: 2.31e-03 |
| Epoch 4/10 | Step 2320/6110 | Loss: 3.2554 (CE: 4.5230, KD: 1.9369) | Avg: 3.3207 | LR: 2.32e-03 |
| Epoch 4/10 | Step 2330/6110 | Loss: 3.2167 (CE: 4.4748, KD: 1.9104) | Avg: 3.3200 | LR: 2.33e-03 |
| Epoch 4/10 | Step 2340/6110 | Loss: 3.0028 (CE: 4.1771, KD: 1.7817) | Avg: 3.3201 | LR: 2.34e-03 |
| Epoch 4/10 | Step 2350/6110 | Loss: 3.0957 (CE: 4.2502, KD: 1.8897) | Avg: 3.3189 | LR: 2.35e-03 |
| Epoch 4/10 | Step 2360/6110 | Loss: 3.2681 (CE: 4.4479, KD: 2.0312) | Avg: 3.3180 | LR: 2.36e-03 |
| Epoch 4/10 | Step 2370/6110 | Loss: 3.2306 (CE: 4.5146, KD: 1.8881) | Avg: 3.3167 | LR: 2.37e-03 |
| Epoch 4/10 | Step 2380/6110 | Loss: 3.2763 (CE: 4.5589, KD: 1.9436) | Avg: 3.3155 | LR: 2.38e-03 |
| Epoch 4/10 | Step 2390/6110 | Loss: 3.0910 (CE: 4.3652, KD: 1.7657) | Avg: 3.3142 | LR: 2.39e-03 |
| Epoch 4/10 | Step 2400/6110 | Loss: 3.4301 (CE: 4.7022, KD: 2.1001) | Avg: 3.3128 | LR: 2.40e-03 |
| Epoch 4/10 | Step 2410/6110 | Loss: 3.2632 (CE: 4.6591, KD: 1.8177) | Avg: 3.3113 | LR: 2.41e-03 |
| Epoch 4/10 | Step 2420/6110 | Loss: 3.4477 (CE: 4.7105, KD: 2.1292) | Avg: 3.3102 | LR: 2.42e-03 |
| Epoch 4/10 | Step 2430/6110 | Loss: 2.9438 (CE: 4.1333, KD: 1.7043) | Avg: 3.3089 | LR: 2.43e-03 |
| Epoch 4/10 | Step 2440/6110 | Loss: 3.2400 (CE: 4.5325, KD: 1.8977) | Avg: 3.3076 | LR: 2.44e-03 |
|
|
| Epoch 4/10 complete | Avg Loss: 3.3072 (CE: 4.4932, KD: 2.0635) | Perplexity: 89.41 | Time: 390.3s |
|
|
| Epoch 5/10 | Step 2450/6110 | Loss: 2.5979 (CE: 3.7310, KD: 1.4249) | Avg: 3.0622 | LR: 2.45e-03 |
| Epoch 5/10 | Step 2460/6110 | Loss: 3.0226 (CE: 4.2741, KD: 1.7143) | Avg: 3.0576 | LR: 2.46e-03 |
| Epoch 5/10 | Step 2470/6110 | Loss: 3.0426 (CE: 3.9985, KD: 2.0305) | Avg: 3.0684 | LR: 2.47e-03 |
| Epoch 5/10 | Step 2480/6110 | Loss: 3.0719 (CE: 4.1684, KD: 1.9221) | Avg: 3.0553 | LR: 2.48e-03 |
| Epoch 5/10 | Step 2490/6110 | Loss: 3.2825 (CE: 4.4782, KD: 2.0221) | Avg: 3.0628 | LR: 2.49e-03 |
| Epoch 5/10 | Step 2500/6110 | Loss: 3.1974 (CE: 4.3640, KD: 1.9724) | Avg: 3.0701 | LR: 2.50e-03 |
| Epoch 5/10 | Step 2510/6110 | Loss: 2.7047 (CE: 3.7179, KD: 1.6465) | Avg: 3.0751 | LR: 2.51e-03 |
| Epoch 5/10 | Step 2520/6110 | Loss: 3.0611 (CE: 4.2535, KD: 1.8158) | Avg: 3.0763 | LR: 2.52e-03 |
| Epoch 5/10 | Step 2530/6110 | Loss: 3.1417 (CE: 4.2201, KD: 2.0101) | Avg: 3.0751 | LR: 2.53e-03 |
| Epoch 5/10 | Step 2540/6110 | Loss: 3.1331 (CE: 4.2638, KD: 1.9443) | Avg: 3.0826 | LR: 2.54e-03 |
| Epoch 5/10 | Step 2550/6110 | Loss: 3.1571 (CE: 4.3921, KD: 1.8717) | Avg: 3.0883 | LR: 2.55e-03 |
| Epoch 5/10 | Step 2560/6110 | Loss: 3.1456 (CE: 4.1602, KD: 2.0688) | Avg: 3.0888 | LR: 2.56e-03 |
| Epoch 5/10 | Step 2570/6110 | Loss: 3.1037 (CE: 4.3211, KD: 1.8343) | Avg: 3.0881 | LR: 2.57e-03 |
| Epoch 5/10 | Step 2580/6110 | Loss: 3.0675 (CE: 4.4294, KD: 1.6533) | Avg: 3.0859 | LR: 2.58e-03 |
| Epoch 5/10 | Step 2590/6110 | Loss: 3.0849 (CE: 4.3558, KD: 1.7637) | Avg: 3.0862 | LR: 2.59e-03 |
| Epoch 5/10 | Step 2600/6110 | Loss: 3.1530 (CE: 3.9923, KD: 2.2474) | Avg: 3.0865 | LR: 2.60e-03 |
| Epoch 5/10 | Step 2610/6110 | Loss: 3.1971 (CE: 4.4086, KD: 1.9340) | Avg: 3.0894 | LR: 2.61e-03 |
| Epoch 5/10 | Step 2620/6110 | Loss: 3.1928 (CE: 4.4705, KD: 1.8651) | Avg: 3.0931 | LR: 2.62e-03 |
| Epoch 5/10 | Step 2630/6110 | Loss: 3.0798 (CE: 4.2394, KD: 1.8682) | Avg: 3.0954 | LR: 2.63e-03 |
| Epoch 5/10 | Step 2640/6110 | Loss: 3.1719 (CE: 4.2518, KD: 2.0312) | Avg: 3.0992 | LR: 2.64e-03 |
| Epoch 5/10 | Step 2650/6110 | Loss: 3.2114 (CE: 4.4896, KD: 1.8693) | Avg: 3.1015 | LR: 2.65e-03 |
| Epoch 5/10 | Step 2660/6110 | Loss: 3.2282 (CE: 4.5162, KD: 1.8811) | Avg: 3.1049 | LR: 2.66e-03 |
| Epoch 5/10 | Step 2670/6110 | Loss: 3.1633 (CE: 4.3457, KD: 1.9212) | Avg: 3.1083 | LR: 2.67e-03 |
| Epoch 5/10 | Step 2680/6110 | Loss: 3.1735 (CE: 4.3599, KD: 1.9332) | Avg: 3.1128 | LR: 2.68e-03 |
| Epoch 5/10 | Step 2690/6110 | Loss: 3.2305 (CE: 4.4215, KD: 1.9788) | Avg: 3.1180 | LR: 2.69e-03 |
| Epoch 5/10 | Step 2700/6110 | Loss: 3.1267 (CE: 4.2703, KD: 1.9125) | Avg: 3.1224 | LR: 2.70e-03 |
| Epoch 5/10 | Step 2710/6110 | Loss: 3.2890 (CE: 4.3902, KD: 2.1250) | Avg: 3.1257 | LR: 2.71e-03 |
| Epoch 5/10 | Step 2720/6110 | Loss: 3.2116 (CE: 4.4451, KD: 1.9149) | Avg: 3.1292 | LR: 2.72e-03 |
| Epoch 5/10 | Step 2730/6110 | Loss: 3.3934 (CE: 4.2835, KD: 2.4403) | Avg: 3.1324 | LR: 2.73e-03 |
| Epoch 5/10 | Step 2740/6110 | Loss: 3.3251 (CE: 4.1777, KD: 2.3975) | Avg: 3.1353 | LR: 2.74e-03 |
| Epoch 5/10 | Step 2750/6110 | Loss: 3.1267 (CE: 4.4174, KD: 1.7739) | Avg: 3.1370 | LR: 2.75e-03 |
| Epoch 5/10 | Step 2760/6110 | Loss: 3.2462 (CE: 4.4142, KD: 2.0028) | Avg: 3.1382 | LR: 2.76e-03 |
| Epoch 5/10 | Step 2770/6110 | Loss: 3.1540 (CE: 4.2565, KD: 1.9865) | Avg: 3.1389 | LR: 2.77e-03 |
| Epoch 5/10 | Step 2780/6110 | Loss: 3.1425 (CE: 4.3128, KD: 1.9094) | Avg: 3.1398 | LR: 2.78e-03 |
| Epoch 5/10 | Step 2790/6110 | Loss: 3.2888 (CE: 4.5817, KD: 1.9349) | Avg: 3.1420 | LR: 2.79e-03 |
| Epoch 5/10 | Step 2800/6110 | Loss: 3.2579 (CE: 4.4007, KD: 2.0506) | Avg: 3.1443 | LR: 2.80e-03 |
| Epoch 5/10 | Step 2810/6110 | Loss: 3.4961 (CE: 4.2068, KD: 2.7321) | Avg: 3.1462 | LR: 2.81e-03 |
| Epoch 5/10 | Step 2820/6110 | Loss: 3.3755 (CE: 4.5903, KD: 2.1032) | Avg: 3.1503 | LR: 2.82e-03 |
| Epoch 5/10 | Step 2830/6110 | Loss: 3.2118 (CE: 4.4557, KD: 1.9125) | Avg: 3.1526 | LR: 2.83e-03 |
| Epoch 5/10 | Step 2840/6110 | Loss: 3.2794 (CE: 4.4314, KD: 2.0619) | Avg: 3.1547 | LR: 2.84e-03 |
| Epoch 5/10 | Step 2850/6110 | Loss: 3.3227 (CE: 4.4571, KD: 2.1267) | Avg: 3.1587 | LR: 2.85e-03 |
| Epoch 5/10 | Step 2860/6110 | Loss: 3.2388 (CE: 4.4502, KD: 1.9678) | Avg: 3.1607 | LR: 2.86e-03 |
| Epoch 5/10 | Step 2870/6110 | Loss: 3.3635 (CE: 4.5279, KD: 2.1277) | Avg: 3.1626 | LR: 2.87e-03 |
| Epoch 5/10 | Step 2880/6110 | Loss: 3.1448 (CE: 4.4361, KD: 1.7925) | Avg: 3.1646 | LR: 2.88e-03 |
| Epoch 5/10 | Step 2890/6110 | Loss: 3.2825 (CE: 4.5581, KD: 1.9423) | Avg: 3.1656 | LR: 2.89e-03 |
| Epoch 5/10 | Step 2900/6110 | Loss: 3.4417 (CE: 4.3239, KD: 2.4785) | Avg: 3.1659 | LR: 2.90e-03 |
| Epoch 5/10 | Step 2910/6110 | Loss: 3.2567 (CE: 4.4876, KD: 1.9707) | Avg: 3.1671 | LR: 2.91e-03 |
| Epoch 5/10 | Step 2920/6110 | Loss: 3.1402 (CE: 4.3072, KD: 1.9178) | Avg: 3.1675 | LR: 2.92e-03 |
| Epoch 5/10 | Step 2930/6110 | Loss: 3.1086 (CE: 4.3040, KD: 1.8656) | Avg: 3.1676 | LR: 2.93e-03 |
| Epoch 5/10 | Step 2940/6110 | Loss: 3.1146 (CE: 4.1849, KD: 1.9886) | Avg: 3.1671 | LR: 2.94e-03 |
| Epoch 5/10 | Step 2950/6110 | Loss: 3.1096 (CE: 4.1969, KD: 1.9599) | Avg: 3.1676 | LR: 2.95e-03 |
| Epoch 5/10 | Step 2960/6110 | Loss: 3.1609 (CE: 4.5245, KD: 1.7373) | Avg: 3.1698 | LR: 2.96e-03 |
| Epoch 5/10 | Step 2970/6110 | Loss: 3.2301 (CE: 4.4672, KD: 1.9366) | Avg: 3.1710 | LR: 2.97e-03 |
| Epoch 5/10 | Step 2980/6110 | Loss: 3.5345 (CE: 4.6513, KD: 2.3418) | Avg: 3.1730 | LR: 2.98e-03 |
| Epoch 5/10 | Step 2990/6110 | Loss: 3.2813 (CE: 4.3532, KD: 2.1488) | Avg: 3.1743 | LR: 2.99e-03 |
| Epoch 5/10 | Step 3000/6110 | Loss: 3.3196 (CE: 4.5863, KD: 1.9903) | Avg: 3.1760 | LR: 3.00e-03 |
| Epoch 5/10 | Step 3010/6110 | Loss: 3.1297 (CE: 4.2945, KD: 1.9074) | Avg: 3.1765 | LR: 3.00e-03 |
| Epoch 5/10 | Step 3020/6110 | Loss: 3.2527 (CE: 4.5081, KD: 1.9374) | Avg: 3.1768 | LR: 3.00e-03 |
| Epoch 5/10 | Step 3030/6110 | Loss: 3.1804 (CE: 4.4323, KD: 1.8734) | Avg: 3.1772 | LR: 3.00e-03 |
| Epoch 5/10 | Step 3040/6110 | Loss: 3.3926 (CE: 4.7635, KD: 1.9649) | Avg: 3.1774 | LR: 3.00e-03 |
| Epoch 5/10 | Step 3050/6110 | Loss: 3.1426 (CE: 4.3570, KD: 1.8680) | Avg: 3.1778 | LR: 3.00e-03 |
|
|
| Epoch 5/10 complete | Avg Loss: 3.1777 (CE: 4.3462, KD: 1.9492) | Perplexity: 77.19 | Time: 390.8s |
|
|
| SWA: saving checkpoint for epoch 5 |
| Epoch 6/10 | Step 3060/6110 | Loss: 2.9041 (CE: 4.0513, KD: 1.6991) | Avg: 3.0504 | LR: 3.00e-03 |
| Epoch 6/10 | Step 3070/6110 | Loss: 2.9513 (CE: 4.1103, KD: 1.7367) | Avg: 3.0281 | LR: 3.00e-03 |
| Epoch 6/10 | Step 3080/6110 | Loss: 3.0976 (CE: 4.1462, KD: 1.9921) | Avg: 3.0334 | LR: 3.00e-03 |
| Epoch 6/10 | Step 3090/6110 | Loss: 2.8942 (CE: 3.9455, KD: 1.7641) | Avg: 3.0289 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3100/6110 | Loss: 3.0004 (CE: 4.0942, KD: 1.8359) | Avg: 3.0316 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3110/6110 | Loss: 3.1658 (CE: 4.3323, KD: 1.9394) | Avg: 3.0225 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3120/6110 | Loss: 2.9746 (CE: 3.9059, KD: 1.9910) | Avg: 3.0332 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3130/6110 | Loss: 3.2663 (CE: 4.4755, KD: 1.9913) | Avg: 3.0406 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3140/6110 | Loss: 3.0704 (CE: 4.1090, KD: 1.9777) | Avg: 3.0419 | LR: 2.99e-03 |
| Epoch 6/10 | Step 3150/6110 | Loss: 2.9538 (CE: 3.9661, KD: 1.8799) | Avg: 3.0428 | LR: 2.98e-03 |
| Epoch 6/10 | Step 3160/6110 | Loss: 3.1692 (CE: 4.3343, KD: 1.9401) | Avg: 3.0496 | LR: 2.98e-03 |
| Epoch 6/10 | Step 3170/6110 | Loss: 2.8961 (CE: 3.9693, KD: 1.7666) | Avg: 3.0559 | LR: 2.98e-03 |
| Epoch 6/10 | Step 3180/6110 | Loss: 3.1943 (CE: 4.4424, KD: 1.8810) | Avg: 3.0596 | LR: 2.98e-03 |
| Epoch 6/10 | Step 3190/6110 | Loss: 2.9851 (CE: 4.1584, KD: 1.7533) | Avg: 3.0656 | LR: 2.97e-03 |
| Epoch 6/10 | Step 3200/6110 | Loss: 2.9866 (CE: 3.7628, KD: 2.1448) | Avg: 3.0725 | LR: 2.97e-03 |
| Epoch 6/10 | Step 3210/6110 | Loss: 2.9078 (CE: 4.0941, KD: 1.6713) | Avg: 3.0727 | LR: 2.97e-03 |
| Epoch 6/10 | Step 3220/6110 | Loss: 3.0388 (CE: 4.3022, KD: 1.7147) | Avg: 3.0712 | LR: 2.96e-03 |
| Epoch 6/10 | Step 3230/6110 | Loss: 3.0208 (CE: 4.0938, KD: 1.8876) | Avg: 3.0663 | LR: 2.96e-03 |
| Epoch 6/10 | Step 3240/6110 | Loss: 3.0793 (CE: 4.3172, KD: 1.7932) | Avg: 3.0652 | LR: 2.96e-03 |
| Epoch 6/10 | Step 3250/6110 | Loss: 3.2092 (CE: 4.4357, KD: 1.9226) | Avg: 3.0635 | LR: 2.95e-03 |
| Epoch 6/10 | Step 3260/6110 | Loss: 2.9994 (CE: 4.0937, KD: 1.8454) | Avg: 3.0635 | LR: 2.95e-03 |
| Epoch 6/10 | Step 3270/6110 | Loss: 2.9465 (CE: 4.2490, KD: 1.5928) | Avg: 3.0598 | LR: 2.94e-03 |
| Epoch 6/10 | Step 3280/6110 | Loss: 2.9594 (CE: 4.2507, KD: 1.6157) | Avg: 3.0589 | LR: 2.94e-03 |
| Epoch 6/10 | Step 3290/6110 | Loss: 2.9768 (CE: 3.8592, KD: 2.0403) | Avg: 3.0570 | LR: 2.94e-03 |
| Epoch 6/10 | Step 3300/6110 | Loss: 3.0861 (CE: 4.4111, KD: 1.7049) | Avg: 3.0572 | LR: 2.93e-03 |
| Epoch 6/10 | Step 3310/6110 | Loss: 2.9728 (CE: 4.2905, KD: 1.6075) | Avg: 3.0557 | LR: 2.93e-03 |
| Epoch 6/10 | Step 3320/6110 | Loss: 3.1926 (CE: 4.2864, KD: 2.0369) | Avg: 3.0541 | LR: 2.92e-03 |
| Epoch 6/10 | Step 3330/6110 | Loss: 3.1280 (CE: 4.4284, KD: 1.7758) | Avg: 3.0525 | LR: 2.92e-03 |
| Epoch 6/10 | Step 3340/6110 | Loss: 2.9896 (CE: 4.1049, KD: 1.8260) | Avg: 3.0504 | LR: 2.91e-03 |
| Epoch 6/10 | Step 3350/6110 | Loss: 2.9202 (CE: 4.1572, KD: 1.6340) | Avg: 3.0491 | LR: 2.91e-03 |
| Epoch 6/10 | Step 3360/6110 | Loss: 2.9770 (CE: 4.1477, KD: 1.7561) | Avg: 3.0478 | LR: 2.90e-03 |
| Epoch 6/10 | Step 3370/6110 | Loss: 2.9022 (CE: 4.0127, KD: 1.7416) | Avg: 3.0463 | LR: 2.90e-03 |
| Epoch 6/10 | Step 3380/6110 | Loss: 2.8586 (CE: 3.9756, KD: 1.6936) | Avg: 3.0445 | LR: 2.89e-03 |
| Epoch 6/10 | Step 3390/6110 | Loss: 3.1223 (CE: 4.2854, KD: 1.9092) | Avg: 3.0431 | LR: 2.89e-03 |
| Epoch 6/10 | Step 3400/6110 | Loss: 2.9098 (CE: 3.9410, KD: 1.8251) | Avg: 3.0407 | LR: 2.88e-03 |
| Epoch 6/10 | Step 3410/6110 | Loss: 2.9485 (CE: 3.9801, KD: 1.8698) | Avg: 3.0394 | LR: 2.87e-03 |
| Epoch 6/10 | Step 3420/6110 | Loss: 3.0287 (CE: 4.2893, KD: 1.7151) | Avg: 3.0384 | LR: 2.87e-03 |
| Epoch 6/10 | Step 3430/6110 | Loss: 2.9388 (CE: 4.1698, KD: 1.6613) | Avg: 3.0365 | LR: 2.86e-03 |
| Epoch 6/10 | Step 3440/6110 | Loss: 2.8192 (CE: 3.9423, KD: 1.6461) | Avg: 3.0360 | LR: 2.85e-03 |
| Epoch 6/10 | Step 3450/6110 | Loss: 3.0203 (CE: 4.3842, KD: 1.6042) | Avg: 3.0347 | LR: 2.85e-03 |
| Epoch 6/10 | Step 3460/6110 | Loss: 3.0881 (CE: 4.3151, KD: 1.8081) | Avg: 3.0342 | LR: 2.84e-03 |
| Epoch 6/10 | Step 3470/6110 | Loss: 3.1180 (CE: 4.2134, KD: 1.9710) | Avg: 3.0336 | LR: 2.83e-03 |
| Epoch 6/10 | Step 3480/6110 | Loss: 3.0808 (CE: 4.2523, KD: 1.8555) | Avg: 3.0318 | LR: 2.83e-03 |
| Epoch 6/10 | Step 3490/6110 | Loss: 2.8680 (CE: 4.0964, KD: 1.5925) | Avg: 3.0303 | LR: 2.82e-03 |
| Epoch 6/10 | Step 3500/6110 | Loss: 3.0009 (CE: 4.0446, KD: 1.9108) | Avg: 3.0294 | LR: 2.81e-03 |
| Epoch 6/10 | Step 3510/6110 | Loss: 3.0641 (CE: 4.2568, KD: 1.8168) | Avg: 3.0288 | LR: 2.81e-03 |
| Epoch 6/10 | Step 3520/6110 | Loss: 3.0790 (CE: 4.4084, KD: 1.6957) | Avg: 3.0277 | LR: 2.80e-03 |
| Epoch 6/10 | Step 3530/6110 | Loss: 3.0343 (CE: 4.2241, KD: 1.7968) | Avg: 3.0262 | LR: 2.79e-03 |
| Epoch 6/10 | Step 3540/6110 | Loss: 2.8964 (CE: 4.1705, KD: 1.5747) | Avg: 3.0247 | LR: 2.78e-03 |
| Epoch 6/10 | Step 3550/6110 | Loss: 2.9307 (CE: 4.0915, KD: 1.7216) | Avg: 3.0239 | LR: 2.77e-03 |
| Epoch 6/10 | Step 3560/6110 | Loss: 3.0972 (CE: 4.3263, KD: 1.8165) | Avg: 3.0229 | LR: 2.77e-03 |
| Epoch 6/10 | Step 3570/6110 | Loss: 2.9724 (CE: 4.1328, KD: 1.7612) | Avg: 3.0227 | LR: 2.76e-03 |
| Epoch 6/10 | Step 3580/6110 | Loss: 3.0179 (CE: 4.1647, KD: 1.8236) | Avg: 3.0216 | LR: 2.75e-03 |
| Epoch 6/10 | Step 3590/6110 | Loss: 3.0925 (CE: 4.1919, KD: 1.9441) | Avg: 3.0207 | LR: 2.74e-03 |
| Epoch 6/10 | Step 3600/6110 | Loss: 2.9197 (CE: 3.9708, KD: 1.8284) | Avg: 3.0203 | LR: 2.73e-03 |
| Epoch 6/10 | Step 3610/6110 | Loss: 2.9739 (CE: 4.1068, KD: 1.7924) | Avg: 3.0197 | LR: 2.72e-03 |
| Epoch 6/10 | Step 3620/6110 | Loss: 3.0614 (CE: 4.2516, KD: 1.8244) | Avg: 3.0188 | LR: 2.72e-03 |
| Epoch 6/10 | Step 3630/6110 | Loss: 2.8492 (CE: 4.0090, KD: 1.6435) | Avg: 3.0179 | LR: 2.71e-03 |
| Epoch 6/10 | Step 3640/6110 | Loss: 2.9943 (CE: 4.2663, KD: 1.6698) | Avg: 3.0170 | LR: 2.70e-03 |
| Epoch 6/10 | Step 3650/6110 | Loss: 3.0822 (CE: 4.3420, KD: 1.7728) | Avg: 3.0169 | LR: 2.69e-03 |
| Epoch 6/10 | Step 3660/6110 | Loss: 2.9119 (CE: 3.9501, KD: 1.8280) | Avg: 3.0155 | LR: 2.68e-03 |
|
|
| Epoch 6/10 complete | Avg Loss: 3.0147 (CE: 4.1763, KD: 1.8000) | Perplexity: 65.13 | Time: 390.8s |
|
|
| SWA: saving checkpoint for epoch 6 |
| Epoch 7/10 | Step 3670/6110 | Loss: 2.8325 (CE: 3.8178, KD: 1.7947) | Avg: 2.8396 | LR: 2.67e-03 |
| Epoch 7/10 | Step 3680/6110 | Loss: 2.7853 (CE: 3.7598, KD: 1.7592) | Avg: 2.8124 | LR: 2.66e-03 |
| Epoch 7/10 | Step 3690/6110 | Loss: 2.9119 (CE: 4.0786, KD: 1.6872) | Avg: 2.8039 | LR: 2.65e-03 |
| Epoch 7/10 | Step 3700/6110 | Loss: 2.7586 (CE: 3.9236, KD: 1.5437) | Avg: 2.8112 | LR: 2.64e-03 |
| Epoch 7/10 | Step 3710/6110 | Loss: 2.6870 (CE: 3.6951, KD: 1.6276) | Avg: 2.8038 | LR: 2.63e-03 |
| Epoch 7/10 | Step 3720/6110 | Loss: 2.7973 (CE: 3.8631, KD: 1.6802) | Avg: 2.8021 | LR: 2.62e-03 |
| Epoch 7/10 | Step 3730/6110 | Loss: 2.8260 (CE: 3.8925, KD: 1.7083) | Avg: 2.7981 | LR: 2.61e-03 |
| Epoch 7/10 | Step 3740/6110 | Loss: 2.8264 (CE: 3.9355, KD: 1.6668) | Avg: 2.8019 | LR: 2.60e-03 |
| Epoch 7/10 | Step 3750/6110 | Loss: 2.9706 (CE: 4.1996, KD: 1.6870) | Avg: 2.8093 | LR: 2.59e-03 |
| Epoch 7/10 | Step 3760/6110 | Loss: 2.8907 (CE: 3.9664, KD: 1.7641) | Avg: 2.8094 | LR: 2.58e-03 |
| Epoch 7/10 | Step 3770/6110 | Loss: 2.6220 (CE: 3.6901, KD: 1.5089) | Avg: 2.8099 | LR: 2.57e-03 |
| Epoch 7/10 | Step 3780/6110 | Loss: 2.5471 (CE: 3.5479, KD: 1.5004) | Avg: 2.8101 | LR: 2.56e-03 |
| Epoch 7/10 | Step 3790/6110 | Loss: 2.8352 (CE: 3.9191, KD: 1.6963) | Avg: 2.8100 | LR: 2.55e-03 |
| Epoch 7/10 | Step 3800/6110 | Loss: 2.7650 (CE: 3.9580, KD: 1.5241) | Avg: 2.8128 | LR: 2.54e-03 |
| Epoch 7/10 | Step 3810/6110 | Loss: 2.9509 (CE: 4.0488, KD: 1.7902) | Avg: 2.8096 | LR: 2.53e-03 |
| Epoch 7/10 | Step 3820/6110 | Loss: 2.8289 (CE: 4.0434, KD: 1.5650) | Avg: 2.8077 | LR: 2.51e-03 |
| Epoch 7/10 | Step 3830/6110 | Loss: 2.3882 (CE: 3.4824, KD: 1.2637) | Avg: 2.8066 | LR: 2.50e-03 |
| Epoch 7/10 | Step 3840/6110 | Loss: 2.6917 (CE: 3.7362, KD: 1.5942) | Avg: 2.8064 | LR: 2.49e-03 |
| Epoch 7/10 | Step 3850/6110 | Loss: 2.7828 (CE: 3.9787, KD: 1.5394) | Avg: 2.8033 | LR: 2.48e-03 |
| Epoch 7/10 | Step 3860/6110 | Loss: 2.7336 (CE: 3.8479, KD: 1.5664) | Avg: 2.8033 | LR: 2.47e-03 |
| Epoch 7/10 | Step 3870/6110 | Loss: 2.9455 (CE: 4.0839, KD: 1.7564) | Avg: 2.8039 | LR: 2.46e-03 |
| Epoch 7/10 | Step 3880/6110 | Loss: 2.9220 (CE: 4.0373, KD: 1.7538) | Avg: 2.8062 | LR: 2.45e-03 |
| Epoch 7/10 | Step 3890/6110 | Loss: 2.8548 (CE: 4.1052, KD: 1.5543) | Avg: 2.8060 | LR: 2.43e-03 |
| Epoch 7/10 | Step 3900/6110 | Loss: 2.7977 (CE: 3.9686, KD: 1.5770) | Avg: 2.8067 | LR: 2.42e-03 |
| Epoch 7/10 | Step 3910/6110 | Loss: 2.6396 (CE: 3.6667, KD: 1.5642) | Avg: 2.8070 | LR: 2.41e-03 |
| Epoch 7/10 | Step 3920/6110 | Loss: 3.0216 (CE: 4.2919, KD: 1.6978) | Avg: 2.8070 | LR: 2.40e-03 |
| Epoch 7/10 | Step 3930/6110 | Loss: 2.8682 (CE: 3.9892, KD: 1.6980) | Avg: 2.8075 | LR: 2.39e-03 |
| Epoch 7/10 | Step 3940/6110 | Loss: 2.7479 (CE: 3.7753, KD: 1.6713) | Avg: 2.8075 | LR: 2.37e-03 |
| Epoch 7/10 | Step 3950/6110 | Loss: 2.9342 (CE: 4.1436, KD: 1.6735) | Avg: 2.8075 | LR: 2.36e-03 |
| Epoch 7/10 | Step 3960/6110 | Loss: 2.9468 (CE: 4.0771, KD: 1.7652) | Avg: 2.8074 | LR: 2.35e-03 |
| Epoch 7/10 | Step 3970/6110 | Loss: 2.3585 (CE: 3.0484, KD: 1.6300) | Avg: 2.8084 | LR: 2.34e-03 |
| Epoch 7/10 | Step 3980/6110 | Loss: 2.7642 (CE: 4.0129, KD: 1.4706) | Avg: 2.8082 | LR: 2.32e-03 |
| Epoch 7/10 | Step 3990/6110 | Loss: 2.6203 (CE: 3.6882, KD: 1.5025) | Avg: 2.8085 | LR: 2.31e-03 |
| Epoch 7/10 | Step 4000/6110 | Loss: 2.8228 (CE: 3.9649, KD: 1.6290) | Avg: 2.8089 | LR: 2.30e-03 |
| Epoch 7/10 | Step 4010/6110 | Loss: 2.8738 (CE: 4.2149, KD: 1.4854) | Avg: 2.8089 | LR: 2.28e-03 |
| Epoch 7/10 | Step 4020/6110 | Loss: 2.7614 (CE: 3.9100, KD: 1.5617) | Avg: 2.8087 | LR: 2.27e-03 |
| Epoch 7/10 | Step 4030/6110 | Loss: 2.8912 (CE: 4.0412, KD: 1.6888) | Avg: 2.8094 | LR: 2.26e-03 |
| Epoch 7/10 | Step 4040/6110 | Loss: 2.8598 (CE: 4.1658, KD: 1.5069) | Avg: 2.8102 | LR: 2.25e-03 |
| Epoch 7/10 | Step 4050/6110 | Loss: 2.9264 (CE: 3.9519, KD: 1.8548) | Avg: 2.8098 | LR: 2.23e-03 |
| Epoch 7/10 | Step 4060/6110 | Loss: 2.7861 (CE: 3.9804, KD: 1.5455) | Avg: 2.8099 | LR: 2.22e-03 |
| Epoch 7/10 | Step 4070/6110 | Loss: 2.7446 (CE: 3.9968, KD: 1.4495) | Avg: 2.8107 | LR: 2.21e-03 |
| Epoch 7/10 | Step 4080/6110 | Loss: 2.7249 (CE: 3.9380, KD: 1.4645) | Avg: 2.8109 | LR: 2.19e-03 |
| Epoch 7/10 | Step 4090/6110 | Loss: 2.6729 (CE: 3.6034, KD: 1.7009) | Avg: 2.8105 | LR: 2.18e-03 |
| Epoch 7/10 | Step 4100/6110 | Loss: 2.8196 (CE: 4.1085, KD: 1.4816) | Avg: 2.8105 | LR: 2.17e-03 |
| Epoch 7/10 | Step 4110/6110 | Loss: 2.7563 (CE: 3.8933, KD: 1.5704) | Avg: 2.8100 | LR: 2.15e-03 |
| Epoch 7/10 | Step 4120/6110 | Loss: 2.8639 (CE: 4.1077, KD: 1.5745) | Avg: 2.8096 | LR: 2.14e-03 |
| Epoch 7/10 | Step 4130/6110 | Loss: 2.9489 (CE: 3.9706, KD: 1.8718) | Avg: 2.8089 | LR: 2.12e-03 |
| Epoch 7/10 | Step 4140/6110 | Loss: 2.9854 (CE: 4.2536, KD: 1.6689) | Avg: 2.8083 | LR: 2.11e-03 |
| Epoch 7/10 | Step 4150/6110 | Loss: 2.8237 (CE: 4.0690, KD: 1.5274) | Avg: 2.8072 | LR: 2.10e-03 |
| Epoch 7/10 | Step 4160/6110 | Loss: 2.9711 (CE: 4.1866, KD: 1.7057) | Avg: 2.8072 | LR: 2.08e-03 |
| Epoch 7/10 | Step 4170/6110 | Loss: 2.7535 (CE: 3.9474, KD: 1.5136) | Avg: 2.8068 | LR: 2.07e-03 |
| Epoch 7/10 | Step 4180/6110 | Loss: 2.7880 (CE: 4.0134, KD: 1.5168) | Avg: 2.8071 | LR: 2.05e-03 |
| Epoch 7/10 | Step 4190/6110 | Loss: 2.6918 (CE: 3.8884, KD: 1.4484) | Avg: 2.8068 | LR: 2.04e-03 |
| Epoch 7/10 | Step 4200/6110 | Loss: 2.8624 (CE: 4.1011, KD: 1.5759) | Avg: 2.8068 | LR: 2.03e-03 |
| Epoch 7/10 | Step 4210/6110 | Loss: 2.8254 (CE: 3.9621, KD: 1.6366) | Avg: 2.8068 | LR: 2.01e-03 |
| Epoch 7/10 | Step 4220/6110 | Loss: 2.4550 (CE: 3.5118, KD: 1.3589) | Avg: 2.8066 | LR: 2.00e-03 |
| Epoch 7/10 | Step 4230/6110 | Loss: 2.6551 (CE: 3.8469, KD: 1.4194) | Avg: 2.8059 | LR: 1.98e-03 |
| Epoch 7/10 | Step 4240/6110 | Loss: 2.9312 (CE: 4.0450, KD: 1.7678) | Avg: 2.8058 | LR: 1.97e-03 |
| Epoch 7/10 | Step 4250/6110 | Loss: 2.8662 (CE: 4.1078, KD: 1.5787) | Avg: 2.8056 | LR: 1.95e-03 |
| Epoch 7/10 | Step 4260/6110 | Loss: 3.0102 (CE: 3.9998, KD: 1.9688) | Avg: 2.8053 | LR: 1.94e-03 |
| Epoch 7/10 | Step 4270/6110 | Loss: 2.7695 (CE: 4.0820, KD: 1.4090) | Avg: 2.8046 | LR: 1.93e-03 |
|
|
| Epoch 7/10 complete | Avg Loss: 2.8042 (CE: 3.9357, KD: 1.6232) | Perplexity: 51.20 | Time: 390.4s |
|
|
| SWA: saving checkpoint for epoch 7 |
| Epoch 8/10 | Step 4280/6110 | Loss: 2.5225 (CE: 3.5333, KD: 1.4622) | Avg: 2.4840 | LR: 1.91e-03 |
| Epoch 8/10 | Step 4290/6110 | Loss: 2.2162 (CE: 2.7873, KD: 1.6032) | Avg: 2.5270 | LR: 1.90e-03 |
| Epoch 8/10 | Step 4300/6110 | Loss: 2.4756 (CE: 3.5194, KD: 1.3817) | Avg: 2.5478 | LR: 1.88e-03 |
| Epoch 8/10 | Step 4310/6110 | Loss: 2.5146 (CE: 3.5711, KD: 1.4082) | Avg: 2.5513 | LR: 1.87e-03 |
| Epoch 8/10 | Step 4320/6110 | Loss: 2.6472 (CE: 3.7550, KD: 1.4868) | Avg: 2.5574 | LR: 1.85e-03 |
| Epoch 8/10 | Step 4330/6110 | Loss: 2.3188 (CE: 3.0589, KD: 1.5313) | Avg: 2.5633 | LR: 1.84e-03 |
| Epoch 8/10 | Step 4340/6110 | Loss: 2.6235 (CE: 3.6980, KD: 1.4959) | Avg: 2.5690 | LR: 1.82e-03 |
| Epoch 8/10 | Step 4350/6110 | Loss: 2.9394 (CE: 3.4706, KD: 2.3500) | Avg: 2.5672 | LR: 1.81e-03 |
| Epoch 8/10 | Step 4360/6110 | Loss: 2.5727 (CE: 3.6169, KD: 1.4762) | Avg: 2.5708 | LR: 1.79e-03 |
| Epoch 8/10 | Step 4370/6110 | Loss: 2.7928 (CE: 3.9037, KD: 1.6282) | Avg: 2.5760 | LR: 1.78e-03 |
| Epoch 8/10 | Step 4380/6110 | Loss: 2.6485 (CE: 3.7256, KD: 1.5209) | Avg: 2.5766 | LR: 1.76e-03 |
| Epoch 8/10 | Step 4390/6110 | Loss: 2.6203 (CE: 3.5548, KD: 1.6349) | Avg: 2.5760 | LR: 1.75e-03 |
| Epoch 8/10 | Step 4400/6110 | Loss: 2.6231 (CE: 3.6358, KD: 1.5587) | Avg: 2.5758 | LR: 1.73e-03 |
| Epoch 8/10 | Step 4410/6110 | Loss: 2.5799 (CE: 3.6501, KD: 1.4551) | Avg: 2.5793 | LR: 1.72e-03 |
| Epoch 8/10 | Step 4420/6110 | Loss: 2.5667 (CE: 3.7439, KD: 1.3389) | Avg: 2.5810 | LR: 1.70e-03 |
| Epoch 8/10 | Step 4430/6110 | Loss: 2.6347 (CE: 3.8442, KD: 1.3783) | Avg: 2.5820 | LR: 1.69e-03 |
| Epoch 8/10 | Step 4440/6110 | Loss: 2.5959 (CE: 3.7697, KD: 1.3720) | Avg: 2.5828 | LR: 1.67e-03 |
| Epoch 8/10 | Step 4450/6110 | Loss: 2.2482 (CE: 3.1108, KD: 1.3486) | Avg: 2.5838 | LR: 1.66e-03 |
| Epoch 8/10 | Step 4460/6110 | Loss: 2.5773 (CE: 3.5642, KD: 1.5365) | Avg: 2.5868 | LR: 1.64e-03 |
| Epoch 8/10 | Step 4470/6110 | Loss: 2.5359 (CE: 3.6169, KD: 1.4059) | Avg: 2.5869 | LR: 1.63e-03 |
| Epoch 8/10 | Step 4480/6110 | Loss: 2.5404 (CE: 3.7260, KD: 1.3077) | Avg: 2.5892 | LR: 1.61e-03 |
| Epoch 8/10 | Step 4490/6110 | Loss: 2.4767 (CE: 3.3909, KD: 1.5142) | Avg: 2.5900 | LR: 1.60e-03 |
| Epoch 8/10 | Step 4500/6110 | Loss: 2.3898 (CE: 3.3127, KD: 1.4251) | Avg: 2.5913 | LR: 1.58e-03 |
| Epoch 8/10 | Step 4510/6110 | Loss: 2.6343 (CE: 3.9349, KD: 1.2828) | Avg: 2.5916 | LR: 1.57e-03 |
| Epoch 8/10 | Step 4520/6110 | Loss: 2.6019 (CE: 3.7536, KD: 1.4000) | Avg: 2.5927 | LR: 1.55e-03 |
| Epoch 8/10 | Step 4530/6110 | Loss: 2.5827 (CE: 3.6035, KD: 1.5098) | Avg: 2.5924 | LR: 1.54e-03 |
| Epoch 8/10 | Step 4540/6110 | Loss: 2.4529 (CE: 3.3913, KD: 1.4705) | Avg: 2.5921 | LR: 1.52e-03 |
| Epoch 8/10 | Step 4550/6110 | Loss: 2.6059 (CE: 3.7173, KD: 1.4446) | Avg: 2.5922 | LR: 1.51e-03 |
| Epoch 8/10 | Step 4560/6110 | Loss: 2.7821 (CE: 4.0619, KD: 1.4528) | Avg: 2.5921 | LR: 1.49e-03 |
| Epoch 8/10 | Step 4570/6110 | Loss: 2.5338 (CE: 3.6237, KD: 1.4030) | Avg: 2.5935 | LR: 1.48e-03 |
| Epoch 8/10 | Step 4580/6110 | Loss: 2.5645 (CE: 3.6775, KD: 1.4050) | Avg: 2.5928 | LR: 1.46e-03 |
| Epoch 8/10 | Step 4590/6110 | Loss: 2.5623 (CE: 3.6117, KD: 1.4648) | Avg: 2.5931 | LR: 1.45e-03 |
| Epoch 8/10 | Step 4600/6110 | Loss: 2.4907 (CE: 3.5837, KD: 1.3476) | Avg: 2.5918 | LR: 1.43e-03 |
| Epoch 8/10 | Step 4610/6110 | Loss: 2.7701 (CE: 3.8816, KD: 1.6081) | Avg: 2.5922 | LR: 1.42e-03 |
| Epoch 8/10 | Step 4620/6110 | Loss: 2.5337 (CE: 3.4394, KD: 1.5764) | Avg: 2.5919 | LR: 1.40e-03 |
| Epoch 8/10 | Step 4630/6110 | Loss: 2.3905 (CE: 3.4356, KD: 1.3030) | Avg: 2.5923 | LR: 1.39e-03 |
| Epoch 8/10 | Step 4640/6110 | Loss: 2.6189 (CE: 3.7980, KD: 1.3916) | Avg: 2.5927 | LR: 1.37e-03 |
| Epoch 8/10 | Step 4650/6110 | Loss: 2.5973 (CE: 3.6988, KD: 1.4481) | Avg: 2.5932 | LR: 1.36e-03 |
| Epoch 8/10 | Step 4660/6110 | Loss: 2.6757 (CE: 3.8811, KD: 1.4229) | Avg: 2.5938 | LR: 1.34e-03 |
| Epoch 8/10 | Step 4670/6110 | Loss: 2.5761 (CE: 3.5991, KD: 1.5060) | Avg: 2.5941 | LR: 1.33e-03 |
| Epoch 8/10 | Step 4680/6110 | Loss: 1.9928 (CE: 2.9158, KD: 1.0375) | Avg: 2.5940 | LR: 1.31e-03 |
| Epoch 8/10 | Step 4690/6110 | Loss: 2.6431 (CE: 3.4492, KD: 1.7896) | Avg: 2.5940 | LR: 1.30e-03 |
| Epoch 8/10 | Step 4700/6110 | Loss: 2.5954 (CE: 3.6101, KD: 1.5259) | Avg: 2.5942 | LR: 1.28e-03 |
| Epoch 8/10 | Step 4710/6110 | Loss: 2.6719 (CE: 3.8018, KD: 1.4935) | Avg: 2.5933 | LR: 1.27e-03 |
| Epoch 8/10 | Step 4720/6110 | Loss: 2.6415 (CE: 3.9057, KD: 1.3297) | Avg: 2.5934 | LR: 1.25e-03 |
| Epoch 8/10 | Step 4730/6110 | Loss: 2.5432 (CE: 3.7040, KD: 1.3370) | Avg: 2.5932 | LR: 1.24e-03 |
| Epoch 8/10 | Step 4740/6110 | Loss: 2.5250 (CE: 3.6151, KD: 1.3885) | Avg: 2.5938 | LR: 1.22e-03 |
| Epoch 8/10 | Step 4750/6110 | Loss: 2.6593 (CE: 3.7503, KD: 1.5213) | Avg: 2.5937 | LR: 1.21e-03 |
| Epoch 8/10 | Step 4760/6110 | Loss: 2.6706 (CE: 3.7905, KD: 1.5038) | Avg: 2.5931 | LR: 1.19e-03 |
| Epoch 8/10 | Step 4770/6110 | Loss: 2.5936 (CE: 3.7330, KD: 1.4043) | Avg: 2.5931 | LR: 1.18e-03 |
| Epoch 8/10 | Step 4780/6110 | Loss: 2.3070 (CE: 3.2978, KD: 1.2754) | Avg: 2.5939 | LR: 1.16e-03 |
| Epoch 8/10 | Step 4790/6110 | Loss: 2.7416 (CE: 3.9025, KD: 1.5305) | Avg: 2.5936 | LR: 1.15e-03 |
| Epoch 8/10 | Step 4800/6110 | Loss: 2.6376 (CE: 3.7010, KD: 1.5208) | Avg: 2.5939 | LR: 1.13e-03 |
| Epoch 8/10 | Step 4810/6110 | Loss: 2.6077 (CE: 3.8372, KD: 1.3296) | Avg: 2.5940 | LR: 1.12e-03 |
| Epoch 8/10 | Step 4820/6110 | Loss: 2.6216 (CE: 3.7678, KD: 1.4252) | Avg: 2.5940 | LR: 1.10e-03 |
| Epoch 8/10 | Step 4830/6110 | Loss: 2.6682 (CE: 3.7894, KD: 1.4995) | Avg: 2.5937 | LR: 1.09e-03 |
| Epoch 8/10 | Step 4840/6110 | Loss: 2.6347 (CE: 3.7284, KD: 1.4902) | Avg: 2.5936 | LR: 1.07e-03 |
| Epoch 8/10 | Step 4850/6110 | Loss: 2.5729 (CE: 3.8105, KD: 1.2896) | Avg: 2.5928 | LR: 1.06e-03 |
| Epoch 8/10 | Step 4860/6110 | Loss: 2.4512 (CE: 3.5108, KD: 1.3445) | Avg: 2.5926 | LR: 1.05e-03 |
| Epoch 8/10 | Step 4870/6110 | Loss: 2.4624 (CE: 3.4696, KD: 1.4082) | Avg: 2.5928 | LR: 1.03e-03 |
| Epoch 8/10 | Step 4880/6110 | Loss: 2.5006 (CE: 3.5810, KD: 1.3679) | Avg: 2.5924 | LR: 1.02e-03 |
|
|
| Epoch 8/10 complete | Avg Loss: 2.5924 (CE: 3.6706, KD: 1.4647) | Perplexity: 39.28 | Time: 393.4s |
|
|
| SWA: saving checkpoint for epoch 8 |
| Epoch 9/10 | Step 4890/6110 | Loss: 2.3126 (CE: 3.3670, KD: 1.2176) | Avg: 2.3700 | LR: 1.00e-03 |
| Epoch 9/10 | Step 4900/6110 | Loss: 2.3426 (CE: 3.3015, KD: 1.3340) | Avg: 2.3756 | LR: 9.88e-04 |
| Epoch 9/10 | Step 4910/6110 | Loss: 2.3793 (CE: 3.4382, KD: 1.2700) | Avg: 2.3679 | LR: 9.74e-04 |
| Epoch 9/10 | Step 4920/6110 | Loss: 2.3733 (CE: 3.4269, KD: 1.2681) | Avg: 2.3633 | LR: 9.59e-04 |
| Epoch 9/10 | Step 4930/6110 | Loss: 2.3610 (CE: 3.4177, KD: 1.2517) | Avg: 2.3615 | LR: 9.45e-04 |
| Epoch 9/10 | Step 4940/6110 | Loss: 2.3526 (CE: 3.3660, KD: 1.2875) | Avg: 2.3662 | LR: 9.31e-04 |
| Epoch 9/10 | Step 4950/6110 | Loss: 2.2856 (CE: 3.2787, KD: 1.2378) | Avg: 2.3638 | LR: 9.17e-04 |
| Epoch 9/10 | Step 4960/6110 | Loss: 2.3142 (CE: 3.2711, KD: 1.3026) | Avg: 2.3588 | LR: 9.03e-04 |
| Epoch 9/10 | Step 4970/6110 | Loss: 2.4049 (CE: 3.5337, KD: 1.2261) | Avg: 2.3564 | LR: 8.89e-04 |
| Epoch 9/10 | Step 4980/6110 | Loss: 2.3241 (CE: 3.3119, KD: 1.2829) | Avg: 2.3528 | LR: 8.76e-04 |
| Epoch 9/10 | Step 4990/6110 | Loss: 2.1323 (CE: 2.7216, KD: 1.4958) | Avg: 2.3521 | LR: 8.62e-04 |
| Epoch 9/10 | Step 5000/6110 | Loss: 2.4731 (CE: 3.5526, KD: 1.3394) | Avg: 2.3540 | LR: 8.48e-04 |
| Epoch 9/10 | Step 5010/6110 | Loss: 2.4943 (CE: 3.4949, KD: 1.4421) | Avg: 2.3541 | LR: 8.35e-04 |
| Epoch 9/10 | Step 5020/6110 | Loss: 2.3826 (CE: 3.4278, KD: 1.2855) | Avg: 2.3560 | LR: 8.21e-04 |
| Epoch 9/10 | Step 5030/6110 | Loss: 2.3835 (CE: 3.3161, KD: 1.3956) | Avg: 2.3569 | LR: 8.08e-04 |
| Epoch 9/10 | Step 5040/6110 | Loss: 2.4239 (CE: 3.4763, KD: 1.3182) | Avg: 2.3576 | LR: 7.94e-04 |
| Epoch 9/10 | Step 5050/6110 | Loss: 2.3932 (CE: 3.4922, KD: 1.2399) | Avg: 2.3595 | LR: 7.81e-04 |
| Epoch 9/10 | Step 5060/6110 | Loss: 2.3728 (CE: 3.2624, KD: 1.4325) | Avg: 2.3615 | LR: 7.68e-04 |
| Epoch 9/10 | Step 5070/6110 | Loss: 2.2965 (CE: 3.2631, KD: 1.2799) | Avg: 2.3630 | LR: 7.54e-04 |
| Epoch 9/10 | Step 5080/6110 | Loss: 2.3507 (CE: 3.2407, KD: 1.4109) | Avg: 2.3631 | LR: 7.41e-04 |
| Epoch 9/10 | Step 5090/6110 | Loss: 2.1938 (CE: 3.0700, KD: 1.2672) | Avg: 2.3638 | LR: 7.28e-04 |
| Epoch 9/10 | Step 5100/6110 | Loss: 2.4684 (CE: 3.4467, KD: 1.4359) | Avg: 2.3637 | LR: 7.15e-04 |
| Epoch 9/10 | Step 5110/6110 | Loss: 2.3726 (CE: 3.2809, KD: 1.4170) | Avg: 2.3635 | LR: 7.02e-04 |
| Epoch 9/10 | Step 5120/6110 | Loss: 2.2916 (CE: 3.2301, KD: 1.3021) | Avg: 2.3634 | LR: 6.90e-04 |
| Epoch 9/10 | Step 5130/6110 | Loss: 2.4858 (CE: 3.4587, KD: 1.4601) | Avg: 2.3637 | LR: 6.77e-04 |
| Epoch 9/10 | Step 5140/6110 | Loss: 2.5844 (CE: 3.7464, KD: 1.3683) | Avg: 2.3649 | LR: 6.64e-04 |
| Epoch 9/10 | Step 5150/6110 | Loss: 2.3441 (CE: 3.3778, KD: 1.2636) | Avg: 2.3644 | LR: 6.52e-04 |
| Epoch 9/10 | Step 5160/6110 | Loss: 2.4480 (CE: 3.6095, KD: 1.2329) | Avg: 2.3651 | LR: 6.39e-04 |
| Epoch 9/10 | Step 5170/6110 | Loss: 2.3186 (CE: 3.2888, KD: 1.2952) | Avg: 2.3652 | LR: 6.27e-04 |
| Epoch 9/10 | Step 5180/6110 | Loss: 2.5049 (CE: 3.4917, KD: 1.4670) | Avg: 2.3658 | LR: 6.15e-04 |
| Epoch 9/10 | Step 5190/6110 | Loss: 2.3946 (CE: 3.3198, KD: 1.4161) | Avg: 2.3656 | LR: 6.02e-04 |
| Epoch 9/10 | Step 5200/6110 | Loss: 2.2762 (CE: 3.1788, KD: 1.3231) | Avg: 2.3655 | LR: 5.90e-04 |
| Epoch 9/10 | Step 5210/6110 | Loss: 2.3549 (CE: 3.2690, KD: 1.3843) | Avg: 2.3654 | LR: 5.78e-04 |
| Epoch 9/10 | Step 5220/6110 | Loss: 2.2785 (CE: 3.2201, KD: 1.2917) | Avg: 2.3652 | LR: 5.66e-04 |
| Epoch 9/10 | Step 5230/6110 | Loss: 2.3538 (CE: 3.3699, KD: 1.2849) | Avg: 2.3648 | LR: 5.55e-04 |
| Epoch 9/10 | Step 5240/6110 | Loss: 2.3796 (CE: 3.4091, KD: 1.2997) | Avg: 2.3652 | LR: 5.43e-04 |
| Epoch 9/10 | Step 5250/6110 | Loss: 2.3346 (CE: 3.3322, KD: 1.2860) | Avg: 2.3653 | LR: 5.31e-04 |
| Epoch 9/10 | Step 5260/6110 | Loss: 2.3478 (CE: 3.3759, KD: 1.2688) | Avg: 2.3647 | LR: 5.20e-04 |
| Epoch 9/10 | Step 5270/6110 | Loss: 2.4074 (CE: 3.4284, KD: 1.3345) | Avg: 2.3643 | LR: 5.08e-04 |
| Epoch 9/10 | Step 5280/6110 | Loss: 2.4458 (CE: 3.4432, KD: 1.3945) | Avg: 2.3642 | LR: 4.97e-04 |
| Epoch 9/10 | Step 5290/6110 | Loss: 2.4358 (CE: 3.5374, KD: 1.2820) | Avg: 2.3642 | LR: 4.86e-04 |
| Epoch 9/10 | Step 5300/6110 | Loss: 2.3565 (CE: 3.2342, KD: 1.4255) | Avg: 2.3635 | LR: 4.75e-04 |
| Epoch 9/10 | Step 5310/6110 | Loss: 2.4239 (CE: 3.4406, KD: 1.3529) | Avg: 2.3627 | LR: 4.64e-04 |
| Epoch 9/10 | Step 5320/6110 | Loss: 2.3182 (CE: 3.4059, KD: 1.1849) | Avg: 2.3623 | LR: 4.53e-04 |
| Epoch 9/10 | Step 5330/6110 | Loss: 2.4493 (CE: 3.5777, KD: 1.2685) | Avg: 2.3624 | LR: 4.42e-04 |
| Epoch 9/10 | Step 5340/6110 | Loss: 2.3148 (CE: 3.3568, KD: 1.2200) | Avg: 2.3632 | LR: 4.31e-04 |
| Epoch 9/10 | Step 5350/6110 | Loss: 2.5478 (CE: 3.6239, KD: 1.4203) | Avg: 2.3631 | LR: 4.21e-04 |
| Epoch 9/10 | Step 5360/6110 | Loss: 2.5023 (CE: 3.5528, KD: 1.3992) | Avg: 2.3635 | LR: 4.10e-04 |
| Epoch 9/10 | Step 5370/6110 | Loss: 2.3442 (CE: 3.4142, KD: 1.2275) | Avg: 2.3633 | LR: 4.00e-04 |
| Epoch 9/10 | Step 5380/6110 | Loss: 2.4568 (CE: 3.5083, KD: 1.3508) | Avg: 2.3627 | LR: 3.90e-04 |
| Epoch 9/10 | Step 5390/6110 | Loss: 2.2568 (CE: 3.1056, KD: 1.3527) | Avg: 2.3621 | LR: 3.80e-04 |
| Epoch 9/10 | Step 5400/6110 | Loss: 2.3634 (CE: 3.1648, KD: 1.5107) | Avg: 2.3619 | LR: 3.70e-04 |
| Epoch 9/10 | Step 5410/6110 | Loss: 2.3988 (CE: 3.4545, KD: 1.2934) | Avg: 2.3614 | LR: 3.60e-04 |
| Epoch 9/10 | Step 5420/6110 | Loss: 2.3756 (CE: 3.4812, KD: 1.2190) | Avg: 2.3615 | LR: 3.50e-04 |
| Epoch 9/10 | Step 5430/6110 | Loss: 2.3887 (CE: 3.4903, KD: 1.2402) | Avg: 2.3611 | LR: 3.40e-04 |
| Epoch 9/10 | Step 5440/6110 | Loss: 2.3157 (CE: 3.4266, KD: 1.1578) | Avg: 2.3610 | LR: 3.31e-04 |
| Epoch 9/10 | Step 5450/6110 | Loss: 2.4088 (CE: 3.4508, KD: 1.3134) | Avg: 2.3608 | LR: 3.21e-04 |
| Epoch 9/10 | Step 5460/6110 | Loss: 2.3812 (CE: 3.4791, KD: 1.2348) | Avg: 2.3610 | LR: 3.12e-04 |
| Epoch 9/10 | Step 5470/6110 | Loss: 2.4940 (CE: 3.4171, KD: 1.5197) | Avg: 2.3608 | LR: 3.03e-04 |
| Epoch 9/10 | Step 5480/6110 | Loss: 2.3509 (CE: 3.2199, KD: 1.4259) | Avg: 2.3606 | LR: 2.94e-04 |
| Epoch 9/10 | Step 5490/6110 | Loss: 2.2035 (CE: 3.0979, KD: 1.2597) | Avg: 2.3601 | LR: 2.85e-04 |
|
|
| Epoch 9/10 complete | Avg Loss: 2.3594 (CE: 3.3477, KD: 1.3196) | Perplexity: 28.44 | Time: 391.0s |
|
|
| SWA: saving checkpoint for epoch 9 |
| Epoch 10/10 | Step 5500/6110 | Loss: 2.2356 (CE: 3.2778, KD: 1.1454) | Avg: 2.1881 | LR: 2.76e-04 |
| Epoch 10/10 | Step 5510/6110 | Loss: 1.9552 (CE: 2.7312, KD: 1.1306) | Avg: 2.1539 | LR: 2.67e-04 |
| Epoch 10/10 | Step 5520/6110 | Loss: 2.2291 (CE: 3.0618, KD: 1.3414) | Avg: 2.1512 | LR: 2.59e-04 |
| Epoch 10/10 | Step 5530/6110 | Loss: 2.2816 (CE: 3.3372, KD: 1.1703) | Avg: 2.1569 | LR: 2.50e-04 |
| Epoch 10/10 | Step 5540/6110 | Loss: 2.1670 (CE: 3.1617, KD: 1.1202) | Avg: 2.1641 | LR: 2.42e-04 |
| Epoch 10/10 | Step 5550/6110 | Loss: 2.1799 (CE: 3.0856, KD: 1.2195) | Avg: 2.1670 | LR: 2.34e-04 |
| Epoch 10/10 | Step 5560/6110 | Loss: 2.3055 (CE: 3.3022, KD: 1.2583) | Avg: 2.1668 | LR: 2.26e-04 |
| Epoch 10/10 | Step 5570/6110 | Loss: 2.2204 (CE: 2.9263, KD: 1.4589) | Avg: 2.1675 | LR: 2.18e-04 |
| Epoch 10/10 | Step 5580/6110 | Loss: 2.2704 (CE: 3.2321, KD: 1.2530) | Avg: 2.1691 | LR: 2.10e-04 |
| Epoch 10/10 | Step 5590/6110 | Loss: 2.0258 (CE: 2.8840, KD: 1.1177) | Avg: 2.1685 | LR: 2.02e-04 |
| Epoch 10/10 | Step 5600/6110 | Loss: 2.1917 (CE: 3.0486, KD: 1.2790) | Avg: 2.1686 | LR: 1.95e-04 |
| Epoch 10/10 | Step 5610/6110 | Loss: 2.1538 (CE: 3.0167, KD: 1.2329) | Avg: 2.1705 | LR: 1.87e-04 |
| Epoch 10/10 | Step 5620/6110 | Loss: 2.1407 (CE: 2.9247, KD: 1.2983) | Avg: 2.1715 | LR: 1.80e-04 |
| Epoch 10/10 | Step 5630/6110 | Loss: 2.0886 (CE: 2.9435, KD: 1.1828) | Avg: 2.1735 | LR: 1.73e-04 |
| Epoch 10/10 | Step 5640/6110 | Loss: 2.1539 (CE: 3.0511, KD: 1.1977) | Avg: 2.1720 | LR: 1.66e-04 |
| Epoch 10/10 | Step 5650/6110 | Loss: 2.2263 (CE: 3.0161, KD: 1.3744) | Avg: 2.1722 | LR: 1.59e-04 |
| Epoch 10/10 | Step 5660/6110 | Loss: 2.1912 (CE: 3.0076, KD: 1.3185) | Avg: 2.1729 | LR: 1.52e-04 |
| Epoch 10/10 | Step 5670/6110 | Loss: 2.2302 (CE: 3.1084, KD: 1.2947) | Avg: 2.1740 | LR: 1.46e-04 |
| Epoch 10/10 | Step 5680/6110 | Loss: 2.1834 (CE: 3.0169, KD: 1.2925) | Avg: 2.1743 | LR: 1.39e-04 |
| Epoch 10/10 | Step 5690/6110 | Loss: 1.8121 (CE: 2.5006, KD: 1.0811) | Avg: 2.1733 | LR: 1.33e-04 |
| Epoch 10/10 | Step 5700/6110 | Loss: 2.1312 (CE: 2.9587, KD: 1.2448) | Avg: 2.1730 | LR: 1.27e-04 |
| Epoch 10/10 | Step 5710/6110 | Loss: 2.0956 (CE: 2.9872, KD: 1.1474) | Avg: 2.1732 | LR: 1.21e-04 |
| Epoch 10/10 | Step 5720/6110 | Loss: 2.3243 (CE: 3.2583, KD: 1.3303) | Avg: 2.1737 | LR: 1.15e-04 |
| Epoch 10/10 | Step 5730/6110 | Loss: 2.1810 (CE: 3.0284, KD: 1.2742) | Avg: 2.1756 | LR: 1.09e-04 |
| Epoch 10/10 | Step 5740/6110 | Loss: 2.1793 (CE: 3.1327, KD: 1.1700) | Avg: 2.1756 | LR: 1.04e-04 |
| Epoch 10/10 | Step 5750/6110 | Loss: 2.2518 (CE: 3.0782, KD: 1.3693) | Avg: 2.1751 | LR: 9.81e-05 |
| Epoch 10/10 | Step 5760/6110 | Loss: 2.1413 (CE: 3.0665, KD: 1.1600) | Avg: 2.1738 | LR: 9.28e-05 |
| Epoch 10/10 | Step 5770/6110 | Loss: 2.2368 (CE: 3.1136, KD: 1.3005) | Avg: 2.1740 | LR: 8.76e-05 |
| Epoch 10/10 | Step 5780/6110 | Loss: 2.3628 (CE: 3.4242, KD: 1.2430) | Avg: 2.1738 | LR: 8.26e-05 |
| Epoch 10/10 | Step 5790/6110 | Loss: 2.1762 (CE: 3.1421, KD: 1.1535) | Avg: 2.1744 | LR: 7.77e-05 |
| Epoch 10/10 | Step 5800/6110 | Loss: 2.0873 (CE: 2.9744, KD: 1.1457) | Avg: 2.1734 | LR: 7.29e-05 |
| Epoch 10/10 | Step 5810/6110 | Loss: 2.2112 (CE: 3.0972, KD: 1.2671) | Avg: 2.1732 | LR: 6.84e-05 |
| Epoch 10/10 | Step 5820/6110 | Loss: 2.2452 (CE: 3.1678, KD: 1.2617) | Avg: 2.1732 | LR: 6.39e-05 |
| Epoch 10/10 | Step 5830/6110 | Loss: 2.2227 (CE: 2.9912, KD: 1.3889) | Avg: 2.1738 | LR: 5.96e-05 |
| Epoch 10/10 | Step 5840/6110 | Loss: 2.3122 (CE: 3.2970, KD: 1.2717) | Avg: 2.1734 | LR: 5.54e-05 |
| Epoch 10/10 | Step 5850/6110 | Loss: 2.2492 (CE: 3.1601, KD: 1.2819) | Avg: 2.1738 | LR: 5.14e-05 |
| Epoch 10/10 | Step 5860/6110 | Loss: 2.5140 (CE: 3.5230, KD: 1.4544) | Avg: 2.1751 | LR: 4.76e-05 |
| Epoch 10/10 | Step 5870/6110 | Loss: 2.1619 (CE: 3.0598, KD: 1.2087) | Avg: 2.1753 | LR: 4.39e-05 |
| Epoch 10/10 | Step 5880/6110 | Loss: 2.0939 (CE: 2.9471, KD: 1.1898) | Avg: 2.1753 | LR: 4.03e-05 |
| Epoch 10/10 | Step 5890/6110 | Loss: 2.1871 (CE: 3.1116, KD: 1.2056) | Avg: 2.1754 | LR: 3.69e-05 |
| Epoch 10/10 | Step 5900/6110 | Loss: 2.0103 (CE: 2.7005, KD: 1.2702) | Avg: 2.1755 | LR: 3.36e-05 |
| Epoch 10/10 | Step 5910/6110 | Loss: 2.1675 (CE: 3.0303, KD: 1.2470) | Avg: 2.1756 | LR: 3.05e-05 |
| Epoch 10/10 | Step 5920/6110 | Loss: 2.3125 (CE: 3.3099, KD: 1.2566) | Avg: 2.1758 | LR: 2.75e-05 |
| Epoch 10/10 | Step 5930/6110 | Loss: 2.1421 (CE: 3.0073, KD: 1.2172) | Avg: 2.1755 | LR: 2.47e-05 |
| Epoch 10/10 | Step 5940/6110 | Loss: 2.2066 (CE: 3.0651, KD: 1.2882) | Avg: 2.1756 | LR: 2.21e-05 |
| Epoch 10/10 | Step 5950/6110 | Loss: 2.1212 (CE: 2.9328, KD: 1.2573) | Avg: 2.1759 | LR: 1.95e-05 |
| Epoch 10/10 | Step 5960/6110 | Loss: 2.1086 (CE: 2.9394, KD: 1.2238) | Avg: 2.1762 | LR: 1.72e-05 |
| Epoch 10/10 | Step 5970/6110 | Loss: 2.1333 (CE: 2.9764, KD: 1.2382) | Avg: 2.1761 | LR: 1.50e-05 |
| Epoch 10/10 | Step 5980/6110 | Loss: 2.2088 (CE: 3.1960, KD: 1.1669) | Avg: 2.1761 | LR: 1.29e-05 |
| Epoch 10/10 | Step 5990/6110 | Loss: 2.1265 (CE: 2.9887, KD: 1.2109) | Avg: 2.1760 | LR: 1.10e-05 |
| Epoch 10/10 | Step 6000/6110 | Loss: 2.1872 (CE: 3.1368, KD: 1.1848) | Avg: 2.1765 | LR: 9.25e-06 |
| Epoch 10/10 | Step 6010/6110 | Loss: 2.3102 (CE: 3.2566, KD: 1.3065) | Avg: 2.1766 | LR: 7.65e-06 |
| Epoch 10/10 | Step 6020/6110 | Loss: 2.2689 (CE: 3.2041, KD: 1.2757) | Avg: 2.1769 | LR: 6.19e-06 |
| Epoch 10/10 | Step 6030/6110 | Loss: 2.1211 (CE: 3.0124, KD: 1.1763) | Avg: 2.1767 | LR: 4.90e-06 |
| Epoch 10/10 | Step 6040/6110 | Loss: 2.1131 (CE: 3.0229, KD: 1.1473) | Avg: 2.1766 | LR: 3.75e-06 |
| Epoch 10/10 | Step 6050/6110 | Loss: 2.2380 (CE: 3.1368, KD: 1.2829) | Avg: 2.1768 | LR: 2.75e-06 |
| Epoch 10/10 | Step 6060/6110 | Loss: 2.1146 (CE: 2.9434, KD: 1.2279) | Avg: 2.1768 | LR: 1.91e-06 |
| Epoch 10/10 | Step 6070/6110 | Loss: 2.2949 (CE: 3.2055, KD: 1.3268) | Avg: 2.1769 | LR: 1.22e-06 |
| Epoch 10/10 | Step 6080/6110 | Loss: 2.1673 (CE: 2.9490, KD: 1.3281) | Avg: 2.1764 | LR: 6.89e-07 |
| Epoch 10/10 | Step 6090/6110 | Loss: 2.2638 (CE: 3.1449, KD: 1.3227) | Avg: 2.1761 | LR: 3.06e-07 |
| Epoch 10/10 | Step 6100/6110 | Loss: 2.1156 (CE: 3.0433, KD: 1.1330) | Avg: 2.1767 | LR: 7.65e-08 |
| Epoch 10/10 | Step 6110/6110 | Loss: 1.5971 (CE: 1.5570, KD: 1.5892) | Avg: 2.1767 | LR: 0.00e+00 |
|
|
| Epoch 10/10 complete | Avg Loss: 2.1767 (CE: 3.0617, KD: 1.2359) | Perplexity: 21.36 | Time: 391.3s |
|
|
| SWA: saving checkpoint for epoch 10 |
| SWA: averaging 6 checkpoints (epochs 5-10) |
| Saving model... |
| Training complete. Model saved to /home/coder/experiments/2026-04-13-039-data-selection |
| End time: 2026-04-13 15:25:35 |
|
|
