File size: 119,994 Bytes
5e571cd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 | W0824 23:13:21.569000 5193 torch/distributed/run.py:774]
W0824 23:13:21.569000 5193 torch/distributed/run.py:774] *****************************************
W0824 23:13:21.569000 5193 torch/distributed/run.py:774] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W0824 23:13:21.569000 5193 torch/distributed/run.py:774] *****************************************
Overriding config with configs/training/minimal-en-125m-4ep.py:
# Approximately 125M-parameter pretraining run on Minimal EN Corpus 2.5B.
# I/O
out_dir = "../runs/minimal-en-125m-4ep"
eval_interval = 1000
eval_iters = 100
log_interval = 10
always_save_checkpoint = True
init_from = "scratch"
# Logging
wandb_log = True
wandb_project = "minimal-en"
wandb_run_name = "125m-12k-4ep"
# Data
dataset = "minimal-en-corpus-2.5b"
gradient_accumulation_steps = 40
batch_size = 12
block_size = 1024
# Model (127.6M parameters with the 12,288-token vocabulary)
n_layer = 12
n_head = 14
n_embd = 896
dropout = 0.0
bias = False
# AdamW
learning_rate = 3e-4
max_iters = 22003
weight_decay = 0.1
beta1 = 0.9
beta2 = 0.95
grad_clip = 1.0
# Cosine learning-rate schedule
decay_lr = True
warmup_iters = 440
lr_decay_iters = 22003
min_lr = 3e-5
# Runtime
backend = "nccl"
device = "cuda"
dtype = "bfloat16"
compile = True
Overriding config with configs/training/minimal-en-125m-4ep.py:
# Approximately 125M-parameter pretraining run on Minimal EN Corpus 2.5B.
# I/O
out_dir = "../runs/minimal-en-125m-4ep"
eval_interval = 1000
eval_iters = 100
log_interval = 10
always_save_checkpoint = True
init_from = "scratch"
# Logging
wandb_log = True
wandb_project = "minimal-en"
wandb_run_name = "125m-12k-4ep"
# Data
dataset = "minimal-en-corpus-2.5b"
gradient_accumulation_steps = 40
batch_size = 12
block_size = 1024
# Model (127.6M parameters with the 12,288-token vocabulary)
n_layer = 12
n_head = 14
n_embd = 896
dropout = 0.0
bias = False
# AdamW
learning_rate = 3e-4
max_iters = 22003
weight_decay = 0.1
beta1 = 0.9
beta2 = 0.95
grad_clip = 1.0
# Cosine learning-rate schedule
decay_lr = True
warmup_iters = 440
lr_decay_iters = 22003
min_lr = 3e-5
# Runtime
backend = "nccl"
device = "cuda"
dtype = "bfloat16"
compile = True
tokens per iteration will be: 491,520
found vocab_size = 12288 (inside data/minimal-en-corpus-2.5b/meta.pkl)
Initializing a new model from scratch
tokens per iteration will be: 491,520
found vocab_size = 12288 (inside data/minimal-en-corpus-2.5b/meta.pkl)
Initializing a new model from scratch
number of parameters: 126.64M
number of parameters: 126.64M
/workspace/nanogpt/train.py:196: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
scaler = torch.cuda.amp.GradScaler(enabled=(dtype == 'float16'))
num decayed parameter tensors: 50, with 127,533,056 parameters
num non-decayed parameter tensors: 25, with 22,400 parameters
/workspace/nanogpt/train.py:196: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
scaler = torch.cuda.amp.GradScaler(enabled=(dtype == 'float16'))
num decayed parameter tensors: 50, with 127,533,056 parameters
num non-decayed parameter tensors: 25, with 22,400 parameters
using fused AdamW: True
compiling the model... (takes a ~minute)
using fused AdamW: True
compiling the model... (takes a ~minute)
NCCL version 2.27.3+cuda12.9
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from /root/.netrc.
wandb: Currently logged in as: dawidmajewski (dawidmajewski-ezotic) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: setting up run hlbpqpoe
wandb: Tracking run with wandb version 0.28.2
wandb: Run data is saved locally in /workspace/logs/wandb/wandb/run-20260824_231331-hlbpqpoe
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run 125m-12k-4ep
wandb: ⭐️ View project at https://wandb.ai/dawidmajewski-ezotic/minimal-en
wandb: 🚀 View run at https://wandb.ai/dawidmajewski-ezotic/minimal-en/runs/hlbpqpoe
step 0: train loss 9.5797, val loss 9.5732
iter 0: loss 9.5701, time 36799.30ms, mfu -100.00%
iter 10: loss 8.6515, time 1993.32ms, mfu 35.25%
iter 20: loss 8.4448, time 2017.84ms, mfu 35.20%
iter 30: loss 8.2446, time 2039.18ms, mfu 35.13%
iter 40: loss 7.9333, time 2044.29ms, mfu 35.05%
iter 50: loss 7.8167, time 2061.04ms, mfu 34.96%
iter 60: loss 7.4723, time 2066.40ms, mfu 34.86%
iter 70: loss 7.1812, time 2067.45ms, mfu 34.77%
iter 80: loss 7.0585, time 2085.67ms, mfu 34.66%
iter 90: loss 6.8916, time 2071.91ms, mfu 34.59%
iter 100: loss 6.9671, time 2082.79ms, mfu 34.50%
iter 110: loss 6.6913, time 2083.09ms, mfu 34.43%
iter 120: loss 6.4896, time 2070.43ms, mfu 34.38%
iter 130: loss 6.4626, time 2069.20ms, mfu 34.33%
iter 140: loss 6.3726, time 2095.37ms, mfu 34.25%
iter 150: loss 6.2077, time 2093.91ms, mfu 34.18%
iter 160: loss 6.1422, time 2093.72ms, mfu 34.12%
iter 170: loss 5.9801, time 2101.12ms, mfu 34.05%
iter 180: loss 6.2115, time 2087.54ms, mfu 34.01%
iter 190: loss 5.8301, time 2103.76ms, mfu 33.95%
iter 200: loss 5.9559, time 2084.75ms, mfu 33.93%
iter 210: loss 5.8527, time 2090.91ms, mfu 33.89%
iter 220: loss 5.8517, time 2100.90ms, mfu 33.85%
iter 230: loss 5.6927, time 2080.43ms, mfu 33.84%
iter 240: loss 5.5679, time 2077.56ms, mfu 33.84%
iter 250: loss 5.6053, time 2106.90ms, mfu 33.79%
iter 260: loss 5.4759, time 2077.33ms, mfu 33.79%
iter 270: loss 5.3678, time 2076.06ms, mfu 33.80%
iter 280: loss 5.4428, time 2097.76ms, mfu 33.77%
iter 290: loss 5.2989, time 2086.06ms, mfu 33.76%
iter 300: loss 5.0415, time 2098.62ms, mfu 33.73%
iter 310: loss 4.9178, time 2078.58ms, mfu 33.74%
iter 320: loss 5.1031, time 2080.53ms, mfu 33.74%
iter 330: loss 5.3207, time 2104.51ms, mfu 33.70%
iter 340: loss 5.2468, time 2104.70ms, mfu 33.67%
iter 350: loss 5.2147, time 2086.30ms, mfu 33.67%
iter 360: loss 5.1747, time 2085.95ms, mfu 33.67%
iter 370: loss 5.0355, time 2079.08ms, mfu 33.69%
iter 380: loss 5.0886, time 2089.18ms, mfu 33.68%
iter 390: loss 5.0313, time 2095.34ms, mfu 33.67%
iter 400: loss 4.9797, time 2078.22ms, mfu 33.68%
iter 410: loss 5.0387, time 2069.48ms, mfu 33.71%
iter 420: loss 4.8122, time 2077.42ms, mfu 33.72%
iter 430: loss 5.1012, time 2091.23ms, mfu 33.71%
iter 440: loss 5.0285, time 2090.85ms, mfu 33.70%
iter 450: loss 4.7803, time 2085.61ms, mfu 33.69%
iter 460: loss 4.9936, time 2100.78ms, mfu 33.67%
iter 470: loss 4.2048, time 2130.72ms, mfu 33.60%
iter 480: loss 4.6369, time 2100.40ms, mfu 33.58%
iter 490: loss 4.6676, time 2104.42ms, mfu 33.56%
iter 500: loss 4.8083, time 2109.12ms, mfu 33.54%
iter 510: loss 4.4331, time 2074.72ms, mfu 33.57%
iter 520: loss 4.6347, time 2106.84ms, mfu 33.55%
iter 530: loss 4.7749, time 2081.79ms, mfu 33.57%
iter 540: loss 4.6217, time 2099.14ms, mfu 33.56%
iter 550: loss 4.4999, time 2101.37ms, mfu 33.55%
iter 560: loss 4.3056, time 2070.50ms, mfu 33.59%
iter 570: loss 4.4181, time 2104.49ms, mfu 33.57%
iter 580: loss 4.3402, time 2083.84ms, mfu 33.58%
iter 590: loss 4.5584, time 2103.17ms, mfu 33.56%
iter 600: loss 4.4462, time 2078.00ms, mfu 33.59%
iter 610: loss 4.2681, time 2104.35ms, mfu 33.57%
iter 620: loss 3.8997, time 2119.20ms, mfu 33.53%
iter 630: loss 4.1709, time 2084.69ms, mfu 33.54%
iter 640: loss 4.0962, time 2104.52ms, mfu 33.53%
iter 650: loss 3.9719, time 2088.32ms, mfu 33.54%
iter 660: loss 4.1625, time 2102.31ms, mfu 33.53%
iter 670: loss 4.0754, time 2086.50ms, mfu 33.54%
iter 680: loss 3.7406, time 2094.57ms, mfu 33.54%
iter 690: loss 3.9339, time 2085.20ms, mfu 33.56%
iter 700: loss 3.8437, time 2074.83ms, mfu 33.59%
iter 710: loss 3.9143, time 2098.15ms, mfu 33.58%
iter 720: loss 3.9684, time 2100.45ms, mfu 33.56%
iter 730: loss 3.5712, time 2098.23ms, mfu 33.56%
iter 740: loss 4.0678, time 2093.66ms, mfu 33.56%
iter 750: loss 4.0130, time 2084.70ms, mfu 33.57%
iter 760: loss 3.8476, time 2105.70ms, mfu 33.55%
iter 770: loss 3.9298, time 2086.56ms, mfu 33.56%
iter 780: loss 3.6444, time 2093.14ms, mfu 33.56%
iter 790: loss 3.5690, time 2090.08ms, mfu 33.57%
iter 800: loss 3.4821, time 2103.30ms, mfu 33.55%
iter 810: loss 3.7268, time 2082.37ms, mfu 33.57%
iter 820: loss 3.5891, time 2085.35ms, mfu 33.58%
iter 830: loss 3.7140, time 2083.72ms, mfu 33.60%
iter 840: loss 3.2428, time 2111.91ms, mfu 33.56%
iter 850: loss 3.3661, time 2104.27ms, mfu 33.55%
iter 860: loss 3.4586, time 2098.45ms, mfu 33.54%
iter 870: loss 3.6945, time 2090.80ms, mfu 33.55%
iter 880: loss 3.4731, time 2101.89ms, mfu 33.53%
iter 890: loss 3.5809, time 2074.95ms, mfu 33.57%
iter 900: loss 3.6004, time 2099.76ms, mfu 33.56%
iter 910: loss 3.5649, time 2095.65ms, mfu 33.55%
iter 920: loss 3.7382, time 2105.51ms, mfu 33.53%
iter 930: loss 3.5281, time 2083.98ms, mfu 33.55%
iter 940: loss 3.8531, time 2115.73ms, mfu 33.52%
iter 950: loss 3.6362, time 2108.05ms, mfu 33.50%
iter 960: loss 3.4495, time 2112.52ms, mfu 33.47%
iter 970: loss 3.5198, time 2104.74ms, mfu 33.47%
iter 980: loss 3.3078, time 2089.55ms, mfu 33.48%
iter 990: loss 3.4061, time 2092.22ms, mfu 33.49%
step 1000: train loss 3.4769, val loss 3.5219
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 1000: loss 3.3850, time 9006.11ms, mfu 30.92%
iter 1010: loss 3.2370, time 2078.63ms, mfu 31.21%
iter 1020: loss 3.3580, time 2100.29ms, mfu 31.43%
iter 1030: loss 3.6355, time 2092.06ms, mfu 31.65%
iter 1040: loss 3.2284, time 2087.74ms, mfu 31.85%
iter 1050: loss 3.3297, time 2097.62ms, mfu 32.01%
iter 1060: loss 3.2152, time 2086.47ms, mfu 32.18%
iter 1070: loss 3.5770, time 2076.36ms, mfu 32.35%
iter 1080: loss 3.4146, time 2106.38ms, mfu 32.45%
iter 1090: loss 2.9954, time 2085.45ms, mfu 32.57%
iter 1100: loss 3.6749, time 2109.49ms, mfu 32.64%
iter 1110: loss 3.2327, time 2106.02ms, mfu 32.72%
iter 1120: loss 3.2230, time 2088.08ms, mfu 32.81%
iter 1130: loss 3.4330, time 2106.66ms, mfu 32.86%
iter 1140: loss 3.2531, time 2090.60ms, mfu 32.94%
iter 1150: loss 3.3821, time 2092.74ms, mfu 33.00%
iter 1160: loss 3.4423, time 2105.26ms, mfu 33.04%
iter 1170: loss 3.3262, time 2087.35ms, mfu 33.10%
iter 1180: loss 3.5381, time 2091.57ms, mfu 33.15%
iter 1190: loss 3.2797, time 2110.25ms, mfu 33.16%
iter 1200: loss 3.3246, time 2075.26ms, mfu 33.23%
iter 1210: loss 3.4333, time 2094.02ms, mfu 33.26%
iter 1220: loss 3.4259, time 2088.27ms, mfu 33.30%
iter 1230: loss 3.2118, time 2091.79ms, mfu 33.33%
iter 1240: loss 2.9138, time 2097.08ms, mfu 33.35%
iter 1250: loss 3.2667, time 2088.42ms, mfu 33.38%
iter 1260: loss 2.9204, time 2099.43ms, mfu 33.39%
iter 1270: loss 3.0572, time 2086.39ms, mfu 33.42%
iter 1280: loss 3.1568, time 2117.32ms, mfu 33.39%
iter 1290: loss 3.4388, time 2086.31ms, mfu 33.42%
iter 1300: loss 3.1613, time 2086.44ms, mfu 33.45%
iter 1310: loss 3.4910, time 2088.56ms, mfu 33.47%
iter 1320: loss 3.1674, time 2080.59ms, mfu 33.50%
iter 1330: loss 3.0806, time 2086.68ms, mfu 33.51%
iter 1340: loss 3.3800, time 2108.42ms, mfu 33.49%
iter 1350: loss 3.3195, time 2089.45ms, mfu 33.51%
iter 1360: loss 3.5254, time 2096.98ms, mfu 33.51%
iter 1370: loss 3.3237, time 2108.79ms, mfu 33.49%
iter 1380: loss 3.2951, time 2104.57ms, mfu 33.48%
iter 1390: loss 3.2490, time 2087.63ms, mfu 33.50%
iter 1400: loss 3.2035, time 2099.25ms, mfu 33.49%
iter 1410: loss 3.2028, time 2096.14ms, mfu 33.49%
iter 1420: loss 2.9834, time 2091.12ms, mfu 33.51%
iter 1430: loss 3.1010, time 2106.93ms, mfu 33.49%
iter 1440: loss 3.0424, time 2083.61ms, mfu 33.51%
iter 1450: loss 2.9155, time 2098.67ms, mfu 33.51%
iter 1460: loss 3.1797, time 2084.02ms, mfu 33.53%
iter 1470: loss 3.1364, time 2088.68ms, mfu 33.54%
iter 1480: loss 3.5031, time 2089.95ms, mfu 33.55%
iter 1490: loss 3.3129, time 2098.83ms, mfu 33.54%
iter 1500: loss 3.3140, time 2078.42ms, mfu 33.57%
iter 1510: loss 3.3269, time 2096.52ms, mfu 33.56%
iter 1520: loss 3.1147, time 2094.67ms, mfu 33.56%
iter 1530: loss 3.5681, time 2081.67ms, mfu 33.58%
iter 1540: loss 3.3100, time 2091.29ms, mfu 33.58%
iter 1550: loss 3.1636, time 2082.59ms, mfu 33.60%
iter 1560: loss 3.2076, time 2081.72ms, mfu 33.61%
iter 1570: loss 3.2339, time 2082.92ms, mfu 33.62%
iter 1580: loss 2.9632, time 2081.89ms, mfu 33.64%
iter 1590: loss 3.2501, time 2090.25ms, mfu 33.63%
iter 1600: loss 3.0978, time 2086.36ms, mfu 33.64%
iter 1610: loss 3.0183, time 2095.82ms, mfu 33.63%
iter 1620: loss 3.2033, time 2103.42ms, mfu 33.60%
iter 1630: loss 3.3264, time 2082.80ms, mfu 33.62%
iter 1640: loss 2.8962, time 2080.24ms, mfu 33.63%
iter 1650: loss 3.2974, time 2082.40ms, mfu 33.64%
iter 1660: loss 3.1878, time 2077.11ms, mfu 33.66%
iter 1670: loss 2.9475, time 2088.90ms, mfu 33.66%
iter 1680: loss 3.1013, time 2079.50ms, mfu 33.67%
iter 1690: loss 3.0734, time 2073.39ms, mfu 33.69%
iter 1700: loss 3.5066, time 2079.11ms, mfu 33.70%
iter 1710: loss 3.1908, time 2088.10ms, mfu 33.70%
iter 1720: loss 3.2536, time 2108.03ms, mfu 33.66%
iter 1730: loss 3.2540, time 2107.71ms, mfu 33.63%
iter 1740: loss 2.9262, time 2088.94ms, mfu 33.63%
iter 1750: loss 3.2834, time 2091.66ms, mfu 33.62%
iter 1760: loss 3.4747, time 2087.66ms, mfu 33.63%
iter 1770: loss 3.2402, time 2081.42ms, mfu 33.64%
iter 1780: loss 2.9697, time 2075.73ms, mfu 33.66%
iter 1790: loss 2.9834, time 2087.18ms, mfu 33.66%
iter 1800: loss 3.1758, time 2095.63ms, mfu 33.65%
iter 1810: loss 3.0526, time 2105.01ms, mfu 33.62%
iter 1820: loss 2.9602, time 2089.51ms, mfu 33.62%
iter 1830: loss 2.8927, time 2103.48ms, mfu 33.60%
iter 1840: loss 3.2547, time 2103.57ms, mfu 33.58%
iter 1850: loss 3.1535, time 2078.31ms, mfu 33.60%
iter 1860: loss 3.1056, time 2091.17ms, mfu 33.60%
iter 1870: loss 3.2162, time 2073.97ms, mfu 33.63%
iter 1880: loss 3.0082, time 2083.10ms, mfu 33.64%
iter 1890: loss 3.0484, time 2075.17ms, mfu 33.66%
iter 1900: loss 3.1529, time 2081.50ms, mfu 33.67%
iter 1910: loss 3.0617, time 2108.79ms, mfu 33.63%
iter 1920: loss 2.8841, time 2093.34ms, mfu 33.63%
iter 1930: loss 2.9809, time 2109.44ms, mfu 33.60%
iter 1940: loss 3.0346, time 2087.30ms, mfu 33.60%
iter 1950: loss 3.2623, time 2075.74ms, mfu 33.63%
iter 1960: loss 3.0727, time 2084.41ms, mfu 33.63%
iter 1970: loss 3.2350, time 2091.24ms, mfu 33.63%
iter 1980: loss 2.7521, time 2094.86ms, mfu 33.62%
iter 1990: loss 3.0140, time 2087.20ms, mfu 33.63%
step 2000: train loss 3.0943, val loss 3.1266
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 2000: loss 3.0097, time 9108.18ms, mfu 31.03%
iter 2010: loss 2.9544, time 2086.93ms, mfu 31.30%
iter 2020: loss 3.1589, time 2078.56ms, mfu 31.55%
iter 2030: loss 3.1263, time 2097.99ms, mfu 31.74%
iter 2040: loss 2.9587, time 2099.60ms, mfu 31.91%
iter 2050: loss 3.0223, time 2097.16ms, mfu 32.07%
iter 2060: loss 2.9487, time 2088.10ms, mfu 32.23%
iter 2070: loss 2.9620, time 2082.66ms, mfu 32.38%
iter 2080: loss 3.0641, time 2087.66ms, mfu 32.51%
iter 2090: loss 2.7195, time 2074.76ms, mfu 32.64%
iter 2100: loss 3.2061, time 2072.21ms, mfu 32.77%
iter 2110: loss 2.8790, time 2098.70ms, mfu 32.84%
iter 2120: loss 3.2879, time 2075.23ms, mfu 32.94%
iter 2130: loss 3.1826, time 2090.97ms, mfu 33.01%
iter 2140: loss 3.4425, time 2082.97ms, mfu 33.08%
iter 2150: loss 2.9741, time 2072.72ms, mfu 33.16%
iter 2160: loss 3.2976, time 2078.42ms, mfu 33.23%
iter 2170: loss 2.9592, time 2085.85ms, mfu 33.27%
iter 2180: loss 3.4325, time 2083.51ms, mfu 33.32%
iter 2190: loss 3.1819, time 2098.03ms, mfu 33.33%
iter 2200: loss 3.1337, time 2071.30ms, mfu 33.39%
iter 2210: loss 3.0284, time 2103.11ms, mfu 33.39%
iter 2220: loss 2.9793, time 2081.02ms, mfu 33.43%
iter 2230: loss 3.0734, time 2094.47ms, mfu 33.44%
iter 2240: loss 3.1064, time 2074.89ms, mfu 33.48%
iter 2250: loss 3.2566, time 2083.70ms, mfu 33.51%
iter 2260: loss 2.8806, time 2086.37ms, mfu 33.52%
iter 2270: loss 3.0763, time 2091.94ms, mfu 33.53%
iter 2280: loss 3.0993, time 2082.17ms, mfu 33.55%
iter 2290: loss 2.7687, time 2103.77ms, mfu 33.54%
iter 2300: loss 3.1544, time 2086.01ms, mfu 33.55%
iter 2310: loss 3.0680, time 2081.71ms, mfu 33.57%
iter 2320: loss 2.9684, time 2097.08ms, mfu 33.56%
iter 2330: loss 2.7440, time 2096.52ms, mfu 33.56%
iter 2340: loss 3.0904, time 2076.54ms, mfu 33.59%
iter 2350: loss 3.0910, time 2079.44ms, mfu 33.61%
iter 2360: loss 3.1517, time 2098.28ms, mfu 33.59%
iter 2370: loss 3.2139, time 2067.59ms, mfu 33.63%
iter 2380: loss 3.2552, time 2086.13ms, mfu 33.64%
iter 2390: loss 2.9476, time 2096.71ms, mfu 33.62%
iter 2400: loss 3.0786, time 2093.13ms, mfu 33.62%
iter 2410: loss 2.8561, time 2068.74ms, mfu 33.65%
iter 2420: loss 2.9681, time 2079.62ms, mfu 33.67%
iter 2430: loss 3.0639, time 2077.07ms, mfu 33.68%
iter 2440: loss 2.9306, time 2074.17ms, mfu 33.70%
iter 2450: loss 2.7985, time 2096.54ms, mfu 33.68%
iter 2460: loss 2.9853, time 2068.47ms, mfu 33.71%
iter 2470: loss 2.9712, time 2067.52ms, mfu 33.74%
iter 2480: loss 2.7578, time 2075.79ms, mfu 33.75%
iter 2490: loss 2.9264, time 2094.64ms, mfu 33.73%
iter 2500: loss 2.6382, time 2072.98ms, mfu 33.74%
iter 2510: loss 2.9940, time 2086.30ms, mfu 33.74%
iter 2520: loss 2.7323, time 2069.35ms, mfu 33.76%
iter 2530: loss 2.6686, time 2079.04ms, mfu 33.76%
iter 2540: loss 2.7341, time 2090.39ms, mfu 33.75%
iter 2550: loss 2.7594, time 2081.52ms, mfu 33.75%
iter 2560: loss 2.8333, time 2086.59ms, mfu 33.74%
iter 2570: loss 2.9103, time 2095.76ms, mfu 33.72%
iter 2580: loss 2.8612, time 2070.09ms, mfu 33.74%
iter 2590: loss 2.8191, time 2077.10ms, mfu 33.75%
iter 2600: loss 2.7326, time 2089.63ms, mfu 33.74%
iter 2610: loss 2.7020, time 2068.17ms, mfu 33.76%
iter 2620: loss 2.7047, time 2095.00ms, mfu 33.74%
iter 2630: loss 3.0022, time 2082.40ms, mfu 33.74%
iter 2640: loss 3.3161, time 2085.93ms, mfu 33.73%
iter 2650: loss 3.1273, time 2084.28ms, mfu 33.73%
iter 2660: loss 2.9416, time 2096.91ms, mfu 33.71%
iter 2670: loss 2.7499, time 2079.56ms, mfu 33.72%
iter 2680: loss 3.1701, time 2088.46ms, mfu 33.71%
iter 2690: loss 2.8368, time 2081.15ms, mfu 33.71%
iter 2700: loss 2.7531, time 2099.55ms, mfu 33.69%
iter 2710: loss 3.0352, time 2079.42ms, mfu 33.70%
iter 2720: loss 2.8548, time 2082.25ms, mfu 33.70%
iter 2730: loss 2.9761, time 2073.71ms, mfu 33.72%
iter 2740: loss 3.1069, time 2095.52ms, mfu 33.70%
iter 2750: loss 2.7885, time 2105.93ms, mfu 33.67%
iter 2760: loss 2.9944, time 2073.42ms, mfu 33.69%
iter 2770: loss 3.1238, time 2092.19ms, mfu 33.68%
iter 2780: loss 2.9192, time 2088.67ms, mfu 33.67%
iter 2790: loss 3.1387, time 2093.42ms, mfu 33.66%
iter 2800: loss 3.0954, time 2092.68ms, mfu 33.65%
iter 2810: loss 3.2072, time 2093.31ms, mfu 33.64%
iter 2820: loss 2.9308, time 2083.50ms, mfu 33.65%
iter 2830: loss 2.9661, time 2090.68ms, mfu 33.65%
iter 2840: loss 2.6713, time 2076.45ms, mfu 33.67%
iter 2850: loss 2.7229, time 2075.23ms, mfu 33.69%
iter 2860: loss 3.2801, time 2084.72ms, mfu 33.69%
iter 2870: loss 2.9257, time 2102.12ms, mfu 33.66%
iter 2880: loss 2.8821, time 2082.41ms, mfu 33.67%
iter 2890: loss 2.9680, time 2079.90ms, mfu 33.68%
iter 2900: loss 3.0326, time 2074.65ms, mfu 33.70%
iter 2910: loss 2.5116, time 2080.17ms, mfu 33.71%
iter 2920: loss 3.1245, time 2065.37ms, mfu 33.74%
iter 2930: loss 3.0239, time 2092.04ms, mfu 33.72%
iter 2940: loss 3.0774, time 2081.26ms, mfu 33.73%
iter 2950: loss 2.8134, time 2075.49ms, mfu 33.74%
iter 2960: loss 2.9250, time 2097.11ms, mfu 33.71%
iter 2970: loss 3.1224, time 2079.51ms, mfu 33.72%
iter 2980: loss 2.8348, time 2084.81ms, mfu 33.72%
iter 2990: loss 3.0060, time 2109.60ms, mfu 33.68%
step 3000: train loss 2.9281, val loss 2.9812
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 3000: loss 3.0601, time 9151.48ms, mfu 31.08%
iter 3010: loss 2.7150, time 2084.09ms, mfu 31.34%
iter 3020: loss 2.7070, time 2070.79ms, mfu 31.60%
iter 3030: loss 2.7966, time 2095.53ms, mfu 31.79%
iter 3040: loss 2.9238, time 2077.16ms, mfu 32.00%
iter 3050: loss 2.6938, time 2068.39ms, mfu 32.19%
iter 3060: loss 2.8782, time 2080.72ms, mfu 32.35%
iter 3070: loss 2.9914, time 2080.87ms, mfu 32.49%
iter 3080: loss 2.7994, time 2076.33ms, mfu 32.63%
iter 3090: loss 2.9092, time 2083.13ms, mfu 32.74%
iter 3100: loss 3.0294, time 2086.56ms, mfu 32.83%
iter 3110: loss 3.1543, time 2089.83ms, mfu 32.91%
iter 3120: loss 2.8346, time 2076.61ms, mfu 33.00%
iter 3130: loss 2.6862, time 2070.32ms, mfu 33.09%
iter 3140: loss 3.1379, time 2081.03ms, mfu 33.16%
iter 3150: loss 2.5061, time 2074.49ms, mfu 33.23%
iter 3160: loss 2.6616, time 2080.76ms, mfu 33.29%
iter 3170: loss 2.9740, time 2069.50ms, mfu 33.35%
iter 3180: loss 2.6214, time 2063.92ms, mfu 33.42%
iter 3190: loss 3.0916, time 2091.78ms, mfu 33.44%
iter 3200: loss 2.9823, time 2078.97ms, mfu 33.47%
iter 3210: loss 2.7964, time 2079.92ms, mfu 33.50%
iter 3220: loss 3.0369, time 2099.36ms, mfu 33.50%
iter 3230: loss 2.4957, time 2081.40ms, mfu 33.53%
iter 3240: loss 2.8929, time 2063.21ms, mfu 33.58%
iter 3250: loss 2.8439, time 2064.79ms, mfu 33.62%
iter 3260: loss 3.0494, time 2096.57ms, mfu 33.61%
iter 3270: loss 2.9273, time 2080.16ms, mfu 33.63%
iter 3280: loss 2.9881, time 2088.97ms, mfu 33.63%
iter 3290: loss 3.3021, time 2068.27ms, mfu 33.66%
iter 3300: loss 2.7670, time 2067.16ms, mfu 33.70%
iter 3310: loss 2.9800, time 2097.20ms, mfu 33.68%
iter 3320: loss 3.2683, time 2071.10ms, mfu 33.70%
iter 3330: loss 3.1901, time 2068.84ms, mfu 33.73%
iter 3340: loss 2.9526, time 2092.34ms, mfu 33.71%
iter 3350: loss 2.8863, time 2079.09ms, mfu 33.72%
iter 3360: loss 2.8820, time 2072.72ms, mfu 33.74%
iter 3370: loss 3.0160, time 2082.08ms, mfu 33.74%
iter 3380: loss 3.1090, time 2088.08ms, mfu 33.73%
iter 3390: loss 2.9832, time 2076.39ms, mfu 33.74%
iter 3400: loss 3.0788, time 2095.70ms, mfu 33.72%
iter 3410: loss 3.0431, time 2084.59ms, mfu 33.72%
iter 3420: loss 3.0189, time 2100.32ms, mfu 33.69%
iter 3430: loss 2.8774, time 2076.34ms, mfu 33.71%
iter 3440: loss 2.7842, time 2084.61ms, mfu 33.70%
iter 3450: loss 3.1838, time 2082.34ms, mfu 33.71%
iter 3460: loss 2.8264, time 2072.92ms, mfu 33.73%
iter 3470: loss 2.9276, time 2100.59ms, mfu 33.70%
iter 3480: loss 3.0675, time 2094.04ms, mfu 33.68%
iter 3490: loss 3.2108, time 2086.26ms, mfu 33.68%
iter 3500: loss 2.5512, time 2073.21ms, mfu 33.70%
iter 3510: loss 3.1138, time 2076.55ms, mfu 33.72%
iter 3520: loss 2.9521, time 2058.69ms, mfu 33.76%
iter 3530: loss 2.6554, time 2067.60ms, mfu 33.78%
iter 3540: loss 2.8287, time 2069.90ms, mfu 33.80%
iter 3550: loss 2.9961, time 2085.06ms, mfu 33.79%
iter 3560: loss 2.7529, time 2085.13ms, mfu 33.78%
iter 3570: loss 2.8463, time 2071.72ms, mfu 33.79%
iter 3580: loss 2.8391, time 2074.18ms, mfu 33.80%
iter 3590: loss 2.6227, time 2063.66ms, mfu 33.82%
iter 3600: loss 3.0554, time 2079.61ms, mfu 33.82%
iter 3610: loss 2.6545, time 2068.14ms, mfu 33.83%
iter 3620: loss 3.1233, time 2077.02ms, mfu 33.83%
iter 3630: loss 2.8129, time 2068.63ms, mfu 33.85%
iter 3640: loss 2.6925, time 2079.76ms, mfu 33.84%
iter 3650: loss 2.7285, time 2072.39ms, mfu 33.85%
iter 3660: loss 2.9807, time 2066.40ms, mfu 33.86%
iter 3670: loss 3.0230, time 2080.69ms, mfu 33.85%
iter 3680: loss 2.6756, time 2064.03ms, mfu 33.87%
iter 3690: loss 2.8071, time 2081.33ms, mfu 33.86%
iter 3700: loss 2.6854, time 2070.27ms, mfu 33.87%
iter 3710: loss 2.9006, time 2075.38ms, mfu 33.87%
iter 3720: loss 2.9650, time 2086.78ms, mfu 33.85%
iter 3730: loss 2.8955, time 2081.14ms, mfu 33.84%
iter 3740: loss 3.1014, time 2082.24ms, mfu 33.83%
iter 3750: loss 3.1095, time 2070.20ms, mfu 33.84%
iter 3760: loss 2.9557, time 2071.86ms, mfu 33.85%
iter 3770: loss 2.8755, time 2075.61ms, mfu 33.85%
iter 3780: loss 2.7802, time 2073.01ms, mfu 33.85%
iter 3790: loss 3.0165, time 2086.68ms, mfu 33.83%
iter 3800: loss 2.9937, time 2086.96ms, mfu 33.82%
iter 3810: loss 2.9377, time 2077.09ms, mfu 33.82%
iter 3820: loss 2.9562, time 2097.84ms, mfu 33.78%
iter 3830: loss 2.8494, time 2062.39ms, mfu 33.81%
iter 3840: loss 2.8790, time 2076.07ms, mfu 33.82%
iter 3850: loss 2.9325, time 2067.72ms, mfu 33.83%
iter 3860: loss 3.0470, time 2082.84ms, mfu 33.82%
iter 3870: loss 2.6130, time 2075.21ms, mfu 33.83%
iter 3880: loss 2.3148, time 2070.67ms, mfu 33.84%
iter 3890: loss 2.8490, time 2068.97ms, mfu 33.85%
iter 3900: loss 2.5053, time 2081.71ms, mfu 33.84%
iter 3910: loss 2.6167, time 2075.10ms, mfu 33.84%
iter 3920: loss 2.8977, time 2068.89ms, mfu 33.85%
iter 3930: loss 2.9963, time 2055.53ms, mfu 33.88%
iter 3940: loss 2.7006, time 2073.11ms, mfu 33.89%
iter 3950: loss 3.0808, time 2071.63ms, mfu 33.89%
iter 3960: loss 2.7748, time 2074.35ms, mfu 33.89%
iter 3970: loss 2.9384, time 2067.47ms, mfu 33.90%
iter 3980: loss 3.1498, time 2081.32ms, mfu 33.88%
iter 3990: loss 3.2189, time 2082.19ms, mfu 33.87%
step 4000: train loss 2.8648, val loss 2.9117
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 4000: loss 2.8254, time 9065.40ms, mfu 31.26%
iter 4010: loss 2.9850, time 2068.52ms, mfu 31.53%
iter 4020: loss 3.1507, time 2066.73ms, mfu 31.77%
iter 4030: loss 3.0200, time 2070.10ms, mfu 31.99%
iter 4040: loss 2.9688, time 2061.25ms, mfu 32.20%
iter 4050: loss 2.8385, time 2062.97ms, mfu 32.39%
iter 4060: loss 3.1681, time 2061.49ms, mfu 32.56%
iter 4070: loss 2.8123, time 2065.62ms, mfu 32.70%
iter 4080: loss 2.6837, time 2082.52ms, mfu 32.80%
iter 4090: loss 2.9108, time 2066.67ms, mfu 32.92%
iter 4100: loss 2.9533, time 2074.16ms, mfu 33.02%
iter 4110: loss 2.4704, time 2074.39ms, mfu 33.10%
iter 4120: loss 3.0496, time 2075.79ms, mfu 33.18%
iter 4130: loss 2.8591, time 2069.94ms, mfu 33.25%
iter 4140: loss 2.9128, time 2077.01ms, mfu 33.31%
iter 4150: loss 3.1376, time 2076.29ms, mfu 33.36%
iter 4160: loss 3.0003, time 2061.50ms, mfu 33.44%
iter 4170: loss 2.7843, time 2064.36ms, mfu 33.50%
iter 4180: loss 2.5587, time 2066.67ms, mfu 33.55%
iter 4190: loss 2.6196, time 2064.97ms, mfu 33.59%
iter 4200: loss 2.8213, time 2064.89ms, mfu 33.64%
iter 4210: loss 3.1516, time 2062.18ms, mfu 33.68%
iter 4220: loss 2.5794, time 2079.40ms, mfu 33.69%
iter 4230: loss 3.1111, time 2068.03ms, mfu 33.72%
iter 4240: loss 2.7717, time 2067.29ms, mfu 33.75%
iter 4250: loss 2.6470, time 2063.30ms, mfu 33.78%
iter 4260: loss 2.6162, time 2074.66ms, mfu 33.79%
iter 4270: loss 2.8615, time 2071.04ms, mfu 33.80%
iter 4280: loss 2.7625, time 2061.29ms, mfu 33.83%
iter 4290: loss 3.0987, time 2059.62ms, mfu 33.86%
iter 4300: loss 2.8871, time 2067.75ms, mfu 33.87%
iter 4310: loss 2.7844, time 2055.51ms, mfu 33.90%
iter 4320: loss 2.8825, time 2054.72ms, mfu 33.93%
iter 4330: loss 2.8165, time 2076.42ms, mfu 33.92%
iter 4340: loss 3.0207, time 2062.89ms, mfu 33.93%
iter 4350: loss 2.8004, time 2073.50ms, mfu 33.93%
iter 4360: loss 2.9181, time 2074.57ms, mfu 33.92%
iter 4370: loss 2.8804, time 2069.34ms, mfu 33.93%
iter 4380: loss 2.9741, time 2074.38ms, mfu 33.92%
iter 4390: loss 2.8343, time 2063.98ms, mfu 33.93%
iter 4400: loss 2.7576, time 2063.77ms, mfu 33.94%
iter 4410: loss 2.7447, time 2075.10ms, mfu 33.93%
iter 4420: loss 2.8071, time 2080.23ms, mfu 33.92%
iter 4430: loss 2.6349, time 2066.90ms, mfu 33.93%
iter 4440: loss 2.7790, time 2079.09ms, mfu 33.91%
iter 4450: loss 2.4973, time 2074.52ms, mfu 33.91%
iter 4460: loss 2.7604, time 2054.86ms, mfu 33.94%
iter 4470: loss 2.9552, time 2061.55ms, mfu 33.95%
iter 4480: loss 2.6731, time 2060.41ms, mfu 33.97%
iter 4490: loss 2.7743, time 2072.27ms, mfu 33.96%
iter 4500: loss 2.8437, time 2064.80ms, mfu 33.97%
iter 4510: loss 2.5555, time 2064.05ms, mfu 33.97%
iter 4520: loss 2.7099, time 2058.34ms, mfu 33.99%
iter 4530: loss 2.9083, time 2062.84ms, mfu 34.00%
iter 4540: loss 2.6443, time 2063.40ms, mfu 34.00%
iter 4550: loss 2.7532, time 2059.47ms, mfu 34.01%
iter 4560: loss 2.7980, time 2064.34ms, mfu 34.02%
iter 4570: loss 2.6229, time 2062.99ms, mfu 34.02%
iter 4580: loss 2.8752, time 2055.44ms, mfu 34.04%
iter 4590: loss 2.9128, time 2066.59ms, mfu 34.03%
iter 4600: loss 2.8879, time 2053.29ms, mfu 34.05%
iter 4610: loss 2.8667, time 2075.48ms, mfu 34.03%
iter 4620: loss 2.9390, time 2066.45ms, mfu 34.03%
iter 4630: loss 2.6457, time 2065.42ms, mfu 34.03%
iter 4640: loss 3.0657, time 2078.80ms, mfu 34.00%
iter 4650: loss 2.9372, time 2067.81ms, mfu 34.00%
iter 4660: loss 2.6715, time 2074.73ms, mfu 33.99%
iter 4670: loss 2.9651, time 2063.95ms, mfu 33.99%
iter 4680: loss 2.8179, time 2063.05ms, mfu 34.00%
iter 4690: loss 2.6349, time 2066.46ms, mfu 34.00%
iter 4700: loss 2.8116, time 2061.80ms, mfu 34.01%
iter 4710: loss 2.9767, time 2075.57ms, mfu 33.99%
iter 4720: loss 2.9309, time 2063.79ms, mfu 34.00%
iter 4730: loss 2.8098, time 2067.92ms, mfu 33.99%
iter 4740: loss 2.5890, time 2057.94ms, mfu 34.01%
iter 4750: loss 2.7399, time 2067.88ms, mfu 34.01%
iter 4760: loss 2.5801, time 2071.37ms, mfu 34.00%
iter 4770: loss 2.9965, time 2062.90ms, mfu 34.00%
iter 4780: loss 2.3116, time 2065.53ms, mfu 34.00%
iter 4790: loss 2.4977, time 2070.60ms, mfu 34.00%
iter 4800: loss 3.0623, time 2053.57ms, mfu 34.02%
iter 4810: loss 2.9972, time 2078.39ms, mfu 34.00%
iter 4820: loss 2.8860, time 2062.83ms, mfu 34.00%
iter 4830: loss 2.7598, time 2070.66ms, mfu 34.00%
iter 4840: loss 2.6594, time 2077.25ms, mfu 33.98%
iter 4850: loss 2.8354, time 2073.79ms, mfu 33.97%
iter 4860: loss 2.6458, time 2059.65ms, mfu 33.98%
iter 4870: loss 2.5955, time 2055.48ms, mfu 34.00%
iter 4880: loss 2.5666, time 2069.63ms, mfu 34.00%
iter 4890: loss 2.2673, time 2078.16ms, mfu 33.98%
iter 4900: loss 2.9341, time 2063.25ms, mfu 33.99%
iter 4910: loss 2.7086, time 2076.25ms, mfu 33.97%
iter 4920: loss 2.8378, time 2059.09ms, mfu 33.99%
iter 4930: loss 2.6511, time 2059.68ms, mfu 34.00%
iter 4940: loss 2.8945, time 2070.19ms, mfu 33.99%
iter 4950: loss 2.9020, time 2056.77ms, mfu 34.01%
iter 4960: loss 2.6619, time 2065.04ms, mfu 34.01%
iter 4970: loss 3.1589, time 2063.18ms, mfu 34.01%
iter 4980: loss 2.7789, time 2055.75ms, mfu 34.03%
iter 4990: loss 3.0120, time 2066.03ms, mfu 34.03%
step 5000: train loss 2.8030, val loss 2.8569
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 5000: loss 2.8127, time 9037.83ms, mfu 31.40%
iter 5010: loss 2.7815, time 2060.24ms, mfu 31.67%
iter 5020: loss 2.5290, time 2063.72ms, mfu 31.91%
iter 5030: loss 2.4893, time 2060.45ms, mfu 32.13%
iter 5040: loss 2.9599, time 2052.24ms, mfu 32.34%
iter 5050: loss 2.9941, time 2061.26ms, mfu 32.51%
iter 5060: loss 2.4154, time 2072.39ms, mfu 32.65%
iter 5070: loss 2.5486, time 2064.66ms, mfu 32.79%
iter 5080: loss 2.3005, time 2068.79ms, mfu 32.91%
iter 5090: loss 2.7054, time 2076.96ms, mfu 33.00%
iter 5100: loss 2.6714, time 2059.32ms, mfu 33.11%
iter 5110: loss 3.0298, time 2061.06ms, mfu 33.21%
iter 5120: loss 2.6713, time 2057.03ms, mfu 33.30%
iter 5130: loss 3.1154, time 2051.57ms, mfu 33.40%
iter 5140: loss 3.0023, time 2052.74ms, mfu 33.48%
iter 5150: loss 2.9078, time 2068.32ms, mfu 33.53%
iter 5160: loss 2.8363, time 2063.19ms, mfu 33.58%
iter 5170: loss 2.8335, time 2059.94ms, mfu 33.63%
iter 5180: loss 2.4896, time 2078.49ms, mfu 33.65%
iter 5190: loss 3.0122, time 2061.53ms, mfu 33.69%
iter 5200: loss 2.6836, time 2068.42ms, mfu 33.72%
iter 5210: loss 2.1724, time 2069.70ms, mfu 33.74%
iter 5220: loss 2.6539, time 2058.02ms, mfu 33.78%
iter 5230: loss 2.6254, time 2059.58ms, mfu 33.82%
iter 5240: loss 2.7223, time 2072.75ms, mfu 33.82%
iter 5250: loss 2.6877, time 2071.79ms, mfu 33.83%
iter 5260: loss 2.5440, time 2066.79ms, mfu 33.85%
iter 5270: loss 2.8340, time 2062.85ms, mfu 33.87%
iter 5280: loss 2.6948, time 2075.01ms, mfu 33.87%
iter 5290: loss 2.7700, time 2063.62ms, mfu 33.89%
iter 5300: loss 2.7649, time 2061.02ms, mfu 33.91%
iter 5310: loss 2.4702, time 2062.59ms, mfu 33.92%
iter 5320: loss 2.7136, time 2061.96ms, mfu 33.94%
iter 5330: loss 2.7587, time 2068.73ms, mfu 33.94%
iter 5340: loss 2.8885, time 2061.52ms, mfu 33.95%
iter 5350: loss 2.9620, time 2058.17ms, mfu 33.97%
iter 5360: loss 2.5015, time 2059.90ms, mfu 33.99%
iter 5370: loss 2.6350, time 2052.63ms, mfu 34.01%
iter 5380: loss 2.8041, time 2065.43ms, mfu 34.01%
iter 5390: loss 2.6625, time 2053.61ms, mfu 34.03%
iter 5400: loss 2.9459, time 2060.95ms, mfu 34.04%
iter 5410: loss 2.8226, time 2071.94ms, mfu 34.02%
iter 5420: loss 2.5892, time 2061.57ms, mfu 34.03%
iter 5430: loss 2.8050, time 2059.07ms, mfu 34.04%
iter 5440: loss 2.5097, time 2059.87ms, mfu 34.05%
iter 5450: loss 2.6736, time 2062.97ms, mfu 34.05%
iter 5460: loss 2.8148, time 2062.16ms, mfu 34.05%
iter 5470: loss 2.7791, time 2059.89ms, mfu 34.05%
iter 5480: loss 2.7999, time 2059.54ms, mfu 34.06%
iter 5490: loss 3.2725, time 2068.44ms, mfu 34.05%
iter 5500: loss 2.8687, time 2063.56ms, mfu 34.05%
iter 5510: loss 2.7539, time 2060.16ms, mfu 34.06%
iter 5520: loss 3.0550, time 2068.15ms, mfu 34.05%
iter 5530: loss 2.7933, time 2066.53ms, mfu 34.04%
iter 5540: loss 2.6721, time 2061.05ms, mfu 34.05%
iter 5550: loss 2.9439, time 2073.22ms, mfu 34.03%
iter 5560: loss 2.6863, time 2062.04ms, mfu 34.04%
iter 5570: loss 2.6809, time 2059.43ms, mfu 34.04%
iter 5580: loss 2.5534, time 2058.25ms, mfu 34.05%
iter 5590: loss 2.7018, time 2064.93ms, mfu 34.05%
iter 5600: loss 2.8506, time 2062.74ms, mfu 34.05%
iter 5610: loss 3.0196, time 2082.08ms, mfu 34.02%
iter 5620: loss 2.6995, time 2072.20ms, mfu 34.01%
iter 5630: loss 3.0107, time 2059.27ms, mfu 34.02%
iter 5640: loss 2.4634, time 2059.52ms, mfu 34.03%
iter 5650: loss 2.7511, time 2055.32ms, mfu 34.04%
iter 5660: loss 2.6317, time 2060.02ms, mfu 34.05%
iter 5670: loss 2.8164, time 2064.27ms, mfu 34.05%
iter 5680: loss 2.7845, time 2066.83ms, mfu 34.04%
iter 5690: loss 2.7309, time 2053.46ms, mfu 34.06%
iter 5700: loss 2.6763, time 2064.71ms, mfu 34.06%
iter 5710: loss 2.4668, time 2059.13ms, mfu 34.06%
iter 5720: loss 2.8844, time 2064.02ms, mfu 34.06%
iter 5730: loss 2.5106, time 2058.15ms, mfu 34.07%
iter 5740: loss 2.7284, time 2070.48ms, mfu 34.06%
iter 5750: loss 2.8526, time 2071.04ms, mfu 34.04%
iter 5760: loss 2.8057, time 2062.77ms, mfu 34.04%
iter 5770: loss 2.7825, time 2072.32ms, mfu 34.03%
iter 5780: loss 2.7251, time 2055.06ms, mfu 34.05%
iter 5790: loss 2.7657, time 2066.28ms, mfu 34.04%
iter 5800: loss 2.8040, time 2060.52ms, mfu 34.05%
iter 5810: loss 2.8236, time 2060.50ms, mfu 34.05%
iter 5820: loss 2.5736, time 2074.36ms, mfu 34.03%
iter 5830: loss 2.6721, time 2061.64ms, mfu 34.04%
iter 5840: loss 2.7983, time 2063.89ms, mfu 34.04%
iter 5850: loss 2.8966, time 2061.17ms, mfu 34.04%
iter 5860: loss 2.8243, time 2059.42ms, mfu 34.05%
iter 5870: loss 2.7769, time 2071.73ms, mfu 34.04%
iter 5880: loss 2.8519, time 2064.61ms, mfu 34.04%
iter 5890: loss 2.9137, time 2059.39ms, mfu 34.04%
iter 5900: loss 2.6517, time 2070.95ms, mfu 34.03%
iter 5910: loss 2.9979, time 2076.52ms, mfu 34.01%
iter 5920: loss 2.5328, time 2070.57ms, mfu 34.00%
iter 5930: loss 2.4874, time 2055.01ms, mfu 34.02%
iter 5940: loss 2.6012, time 2069.76ms, mfu 34.02%
iter 5950: loss 2.7926, time 2058.31ms, mfu 34.03%
iter 5960: loss 2.8399, time 2062.83ms, mfu 34.03%
iter 5970: loss 2.4334, time 2063.89ms, mfu 34.03%
iter 5980: loss 2.4871, time 2061.36ms, mfu 34.04%
iter 5990: loss 2.7603, time 2075.80ms, mfu 34.02%
step 6000: train loss 2.7348, val loss 2.7800
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 6000: loss 2.7937, time 9040.42ms, mfu 31.39%
iter 6010: loss 2.8561, time 2051.49ms, mfu 31.68%
iter 6020: loss 2.9252, time 2078.77ms, mfu 31.89%
iter 6030: loss 2.7826, time 2057.91ms, mfu 32.12%
iter 6040: loss 2.6314, time 2064.55ms, mfu 32.31%
iter 6050: loss 2.7288, time 2066.30ms, mfu 32.48%
iter 6060: loss 2.8380, time 2071.90ms, mfu 32.62%
iter 6070: loss 2.7656, time 2062.80ms, mfu 32.76%
iter 6080: loss 2.8201, time 2063.82ms, mfu 32.89%
iter 6090: loss 2.9309, time 2065.50ms, mfu 33.00%
iter 6100: loss 2.7605, time 2068.20ms, mfu 33.10%
iter 6110: loss 2.8659, time 2054.06ms, mfu 33.21%
iter 6120: loss 2.8095, time 2064.98ms, mfu 33.29%
iter 6130: loss 2.6604, time 2065.05ms, mfu 33.37%
iter 6140: loss 2.5700, time 2061.70ms, mfu 33.44%
iter 6150: loss 2.5395, time 2055.06ms, mfu 33.51%
iter 6160: loss 2.7895, time 2062.48ms, mfu 33.57%
iter 6170: loss 2.7476, time 2054.03ms, mfu 33.63%
iter 6180: loss 2.3769, time 2056.50ms, mfu 33.68%
iter 6190: loss 2.7728, time 2062.22ms, mfu 33.72%
iter 6200: loss 2.8410, time 2056.67ms, mfu 33.77%
iter 6210: loss 2.4412, time 2059.55ms, mfu 33.80%
iter 6220: loss 2.8561, time 2063.53ms, mfu 33.83%
iter 6230: loss 2.7499, time 2064.29ms, mfu 33.85%
iter 6240: loss 3.0280, time 2059.13ms, mfu 33.87%
iter 6250: loss 2.8489, time 2060.44ms, mfu 33.90%
iter 6260: loss 2.5052, time 2057.64ms, mfu 33.92%
iter 6270: loss 2.4484, time 2068.48ms, mfu 33.93%
iter 6280: loss 2.8751, time 2059.17ms, mfu 33.95%
iter 6290: loss 2.8386, time 2072.31ms, mfu 33.94%
iter 6300: loss 2.8937, time 2059.09ms, mfu 33.96%
iter 6310: loss 2.9598, time 2074.39ms, mfu 33.95%
iter 6320: loss 2.9603, time 2070.23ms, mfu 33.95%
iter 6330: loss 2.5865, time 2066.53ms, mfu 33.95%
iter 6340: loss 2.8333, time 2059.97ms, mfu 33.97%
iter 6350: loss 3.0807, time 2073.29ms, mfu 33.96%
iter 6360: loss 2.6060, time 2060.75ms, mfu 33.97%
iter 6370: loss 2.7680, time 2062.30ms, mfu 33.98%
iter 6380: loss 2.7461, time 2067.87ms, mfu 33.98%
iter 6390: loss 2.5055, time 2047.10ms, mfu 34.02%
iter 6400: loss 2.8759, time 2060.80ms, mfu 34.02%
iter 6410: loss 2.7861, time 2071.88ms, mfu 34.01%
iter 6420: loss 2.8208, time 2073.05ms, mfu 34.00%
iter 6430: loss 2.4515, time 2065.19ms, mfu 34.00%
iter 6440: loss 2.9032, time 2067.03ms, mfu 34.00%
iter 6450: loss 2.7120, time 2070.47ms, mfu 33.99%
iter 6460: loss 2.6434, time 2064.10ms, mfu 34.00%
iter 6470: loss 2.7141, time 2059.00ms, mfu 34.01%
iter 6480: loss 2.3601, time 2056.58ms, mfu 34.03%
iter 6490: loss 2.7366, time 2065.14ms, mfu 34.03%
iter 6500: loss 2.7211, time 2078.13ms, mfu 34.00%
iter 6510: loss 2.8686, time 2072.11ms, mfu 33.99%
iter 6520: loss 2.8704, time 2073.99ms, mfu 33.98%
iter 6530: loss 2.7231, time 2069.10ms, mfu 33.98%
iter 6540: loss 2.5677, time 2061.74ms, mfu 33.99%
iter 6550: loss 2.9674, time 2061.93ms, mfu 34.00%
iter 6560: loss 2.6548, time 2062.25ms, mfu 34.01%
iter 6570: loss 2.8212, time 2065.32ms, mfu 34.01%
iter 6580: loss 2.8856, time 2069.61ms, mfu 34.00%
iter 6590: loss 2.5839, time 2078.52ms, mfu 33.98%
iter 6600: loss 2.7059, time 2084.07ms, mfu 33.95%
iter 6610: loss 2.8822, time 2072.34ms, mfu 33.95%
iter 6620: loss 2.6400, time 2052.48ms, mfu 33.98%
iter 6630: loss 3.0263, time 2064.04ms, mfu 33.98%
iter 6640: loss 2.8466, time 2076.14ms, mfu 33.97%
iter 6650: loss 2.6849, time 2068.52ms, mfu 33.97%
iter 6660: loss 3.0016, time 2059.73ms, mfu 33.98%
iter 6670: loss 2.7598, time 2058.34ms, mfu 34.00%
iter 6680: loss 2.6723, time 2069.23ms, mfu 33.99%
iter 6690: loss 2.6609, time 2067.45ms, mfu 33.99%
iter 6700: loss 2.7810, time 2072.90ms, mfu 33.98%
iter 6710: loss 2.5192, time 2074.93ms, mfu 33.97%
iter 6720: loss 2.3313, time 2066.86ms, mfu 33.97%
iter 6730: loss 2.8297, time 2079.93ms, mfu 33.95%
iter 6740: loss 2.3553, time 2071.79ms, mfu 33.95%
iter 6750: loss 2.7652, time 2066.70ms, mfu 33.95%
iter 6760: loss 2.7459, time 2057.07ms, mfu 33.97%
iter 6770: loss 2.5465, time 2065.00ms, mfu 33.98%
iter 6780: loss 2.2827, time 2064.57ms, mfu 33.98%
iter 6790: loss 2.7563, time 2060.09ms, mfu 34.00%
iter 6800: loss 2.5942, time 2067.08ms, mfu 34.00%
iter 6810: loss 2.4001, time 2075.86ms, mfu 33.98%
iter 6820: loss 2.7329, time 2087.28ms, mfu 33.95%
iter 6830: loss 2.6670, time 2080.82ms, mfu 33.93%
iter 6840: loss 2.7223, time 2073.69ms, mfu 33.92%
iter 6850: loss 2.4119, time 2067.73ms, mfu 33.93%
iter 6860: loss 2.4829, time 2066.01ms, mfu 33.94%
iter 6870: loss 2.7631, time 2066.09ms, mfu 33.94%
iter 6880: loss 2.6818, time 2068.91ms, mfu 33.95%
iter 6890: loss 2.8367, time 2078.08ms, mfu 33.93%
iter 6900: loss 2.9548, time 2065.18ms, mfu 33.94%
iter 6910: loss 2.7495, time 2058.83ms, mfu 33.96%
iter 6920: loss 2.6917, time 2065.74ms, mfu 33.96%
iter 6930: loss 3.0313, time 2065.94ms, mfu 33.97%
iter 6940: loss 2.9945, time 2071.37ms, mfu 33.96%
iter 6950: loss 2.9768, time 2068.13ms, mfu 33.96%
iter 6960: loss 2.2144, time 2062.39ms, mfu 33.97%
iter 6970: loss 2.5947, time 2068.89ms, mfu 33.97%
iter 6980: loss 2.1534, time 2074.16ms, mfu 33.96%
iter 6990: loss 2.8031, time 2066.51ms, mfu 33.97%
step 7000: train loss 2.7199, val loss 2.7451
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 7000: loss 2.6378, time 8990.41ms, mfu 31.35%
iter 7010: loss 2.3748, time 2058.15ms, mfu 31.63%
iter 7020: loss 2.9583, time 2065.35ms, mfu 31.87%
iter 7030: loss 2.8537, time 2073.01ms, mfu 32.07%
iter 7040: loss 2.6749, time 2068.63ms, mfu 32.26%
iter 7050: loss 2.8980, time 2066.23ms, mfu 32.43%
iter 7060: loss 2.6195, time 2070.61ms, mfu 32.58%
iter 7070: loss 2.6558, time 2060.36ms, mfu 32.74%
iter 7080: loss 2.7641, time 2064.97ms, mfu 32.86%
iter 7090: loss 2.4308, time 2063.20ms, mfu 32.98%
iter 7100: loss 2.4519, time 2074.19ms, mfu 33.07%
iter 7110: loss 2.7545, time 2067.47ms, mfu 33.16%
iter 7120: loss 2.8048, time 2077.71ms, mfu 33.23%
iter 7130: loss 2.9350, time 2064.97ms, mfu 33.31%
iter 7140: loss 2.6718, time 2060.87ms, mfu 33.39%
iter 7150: loss 2.8571, time 2054.19ms, mfu 33.47%
iter 7160: loss 2.7824, time 2063.02ms, mfu 33.53%
iter 7170: loss 2.7879, time 2062.80ms, mfu 33.58%
iter 7180: loss 2.7715, time 2056.40ms, mfu 33.64%
iter 7190: loss 2.6512, time 2047.85ms, mfu 33.71%
iter 7200: loss 2.6282, time 2068.08ms, mfu 33.73%
iter 7210: loss 2.6781, time 2080.54ms, mfu 33.74%
iter 7220: loss 2.3595, time 2061.42ms, mfu 33.77%
iter 7230: loss 2.8317, time 2056.37ms, mfu 33.81%
iter 7240: loss 2.8356, time 2069.20ms, mfu 33.82%
iter 7250: loss 2.6854, time 2068.84ms, mfu 33.84%
iter 7260: loss 3.0189, time 2066.52ms, mfu 33.85%
iter 7270: loss 2.3311, time 2062.97ms, mfu 33.87%
iter 7280: loss 2.6729, time 2073.55ms, mfu 33.88%
iter 7290: loss 2.4065, time 2062.07ms, mfu 33.89%
iter 7300: loss 2.7326, time 2066.48ms, mfu 33.91%
iter 7310: loss 2.8379, time 2071.99ms, mfu 33.91%
iter 7320: loss 2.9087, time 2072.50ms, mfu 33.91%
iter 7330: loss 2.8323, time 2061.21ms, mfu 33.92%
iter 7340: loss 2.5362, time 2066.00ms, mfu 33.93%
iter 7350: loss 2.6603, time 2055.68ms, mfu 33.96%
iter 7360: loss 2.4863, time 2072.29ms, mfu 33.95%
iter 7370: loss 2.6952, time 2059.11ms, mfu 33.97%
iter 7380: loss 2.9313, time 2063.75ms, mfu 33.98%
iter 7390: loss 2.8076, time 2080.66ms, mfu 33.95%
iter 7400: loss 2.7884, time 2062.11ms, mfu 33.97%
iter 7410: loss 2.2030, time 2060.66ms, mfu 33.98%
iter 7420: loss 2.8226, time 2051.75ms, mfu 34.01%
iter 7430: loss 2.7441, time 2056.97ms, mfu 34.02%
iter 7440: loss 2.6438, time 2065.42ms, mfu 34.02%
iter 7450: loss 2.8727, time 2064.33ms, mfu 34.02%
iter 7460: loss 2.3760, time 2063.12ms, mfu 34.02%
iter 7470: loss 2.6915, time 2072.23ms, mfu 34.01%
iter 7480: loss 2.3677, time 2071.93ms, mfu 34.00%
iter 7490: loss 2.8116, time 2063.97ms, mfu 34.01%
iter 7500: loss 2.8886, time 2063.60ms, mfu 34.01%
iter 7510: loss 2.5551, time 2062.11ms, mfu 34.02%
iter 7520: loss 2.5777, time 2056.48ms, mfu 34.03%
iter 7530: loss 2.7493, time 2078.79ms, mfu 34.01%
iter 7540: loss 2.6058, time 2052.31ms, mfu 34.03%
iter 7550: loss 2.7032, time 2065.04ms, mfu 34.03%
iter 7560: loss 2.5870, time 2056.16ms, mfu 34.04%
iter 7570: loss 2.7618, time 2056.58ms, mfu 34.06%
iter 7580: loss 2.6164, time 2076.06ms, mfu 34.03%
iter 7590: loss 2.7801, time 2064.83ms, mfu 34.03%
iter 7600: loss 2.7909, time 2062.50ms, mfu 34.04%
iter 7610: loss 2.8960, time 2058.75ms, mfu 34.05%
iter 7620: loss 2.8551, time 2055.24ms, mfu 34.06%
iter 7630: loss 2.7798, time 2069.51ms, mfu 34.05%
iter 7640: loss 2.6871, time 2072.71ms, mfu 34.03%
iter 7650: loss 3.0158, time 2062.60ms, mfu 34.04%
iter 7660: loss 2.8931, time 2065.47ms, mfu 34.03%
iter 7670: loss 2.3321, time 2062.72ms, mfu 34.04%
iter 7680: loss 2.2162, time 2054.44ms, mfu 34.05%
iter 7690: loss 2.7187, time 2073.70ms, mfu 34.04%
iter 7700: loss 3.0990, time 2075.42ms, mfu 34.02%
iter 7710: loss 3.0998, time 2082.62ms, mfu 33.99%
iter 7720: loss 2.8787, time 2061.16ms, mfu 34.00%
iter 7730: loss 2.7363, time 2062.32ms, mfu 34.01%
iter 7740: loss 2.7613, time 2058.37ms, mfu 34.02%
iter 7750: loss 2.4897, time 2068.29ms, mfu 34.01%
iter 7760: loss 2.8700, time 2069.80ms, mfu 34.01%
iter 7770: loss 2.6892, time 2063.09ms, mfu 34.01%
iter 7780: loss 2.5989, time 2064.17ms, mfu 34.01%
iter 7790: loss 2.5908, time 2050.02ms, mfu 34.04%
iter 7800: loss 2.5284, time 2078.61ms, mfu 34.02%
iter 7810: loss 2.4190, time 2067.64ms, mfu 34.01%
iter 7820: loss 2.7623, time 2060.49ms, mfu 34.02%
iter 7830: loss 2.6298, time 2072.14ms, mfu 34.01%
iter 7840: loss 2.5627, time 2054.42ms, mfu 34.03%
iter 7850: loss 2.6072, time 2069.88ms, mfu 34.02%
iter 7860: loss 2.7139, time 2059.51ms, mfu 34.03%
iter 7870: loss 2.5188, time 2068.34ms, mfu 34.02%
iter 7880: loss 2.2874, time 2070.74ms, mfu 34.01%
iter 7890: loss 2.3243, time 2073.84ms, mfu 34.00%
iter 7900: loss 2.6967, time 2062.62ms, mfu 34.01%
iter 7910: loss 2.6026, time 2062.92ms, mfu 34.01%
iter 7920: loss 2.7556, time 2077.83ms, mfu 33.99%
iter 7930: loss 2.8487, time 2064.57ms, mfu 34.00%
iter 7940: loss 2.5658, time 2071.63ms, mfu 33.99%
iter 7950: loss 2.5790, time 2065.73ms, mfu 33.99%
iter 7960: loss 2.8130, time 2065.42ms, mfu 33.99%
iter 7970: loss 2.5048, time 2064.73ms, mfu 34.00%
iter 7980: loss 2.6489, time 2077.72ms, mfu 33.98%
iter 7990: loss 2.6368, time 2064.54ms, mfu 33.98%
step 8000: train loss 2.6810, val loss 2.7095
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 8000: loss 2.4861, time 12696.51ms, mfu 31.14%
iter 8010: loss 2.7566, time 2065.55ms, mfu 31.43%
iter 8020: loss 2.7151, time 2069.67ms, mfu 31.68%
iter 8030: loss 2.8548, time 2070.45ms, mfu 31.90%
iter 8040: loss 2.7300, time 2069.78ms, mfu 32.11%
iter 8050: loss 2.1881, time 2060.55ms, mfu 32.31%
iter 8060: loss 2.5641, time 2072.43ms, mfu 32.47%
iter 8070: loss 2.7639, time 2056.83ms, mfu 32.64%
iter 8080: loss 2.7717, time 2062.66ms, mfu 32.78%
iter 8090: loss 2.2702, time 2061.05ms, mfu 32.91%
iter 8100: loss 2.4686, time 2060.00ms, mfu 33.03%
iter 8110: loss 2.6002, time 2058.27ms, mfu 33.14%
iter 8120: loss 2.7896, time 2064.73ms, mfu 33.23%
iter 8130: loss 2.5712, time 2077.51ms, mfu 33.29%
iter 8140: loss 2.6600, time 2061.58ms, mfu 33.37%
iter 8150: loss 2.5645, time 2064.34ms, mfu 33.43%
iter 8160: loss 2.7058, time 2070.11ms, mfu 33.48%
iter 8170: loss 2.6112, time 2065.09ms, mfu 33.54%
iter 8180: loss 2.8279, time 2056.39ms, mfu 33.60%
iter 8190: loss 2.7758, time 2061.77ms, mfu 33.65%
iter 8200: loss 2.8485, time 2073.25ms, mfu 33.67%
iter 8210: loss 2.7109, time 2070.40ms, mfu 33.70%
iter 8220: loss 2.7037, time 2065.21ms, mfu 33.73%
iter 8230: loss 2.6159, time 2066.61ms, mfu 33.76%
iter 8240: loss 2.4814, time 2059.59ms, mfu 33.79%
iter 8250: loss 2.5850, time 2065.16ms, mfu 33.82%
iter 8260: loss 2.7522, time 2065.04ms, mfu 33.84%
iter 8270: loss 2.8145, time 2071.00ms, mfu 33.84%
iter 8280: loss 2.7633, time 2061.92ms, mfu 33.87%
iter 8290: loss 2.8925, time 2072.70ms, mfu 33.87%
iter 8300: loss 2.4420, time 2072.57ms, mfu 33.87%
iter 8310: loss 2.5096, time 2062.87ms, mfu 33.89%
iter 8320: loss 2.3306, time 2058.83ms, mfu 33.92%
iter 8330: loss 2.7891, time 2063.07ms, mfu 33.93%
iter 8340: loss 2.4590, time 2068.68ms, mfu 33.93%
iter 8350: loss 2.6073, time 2061.93ms, mfu 33.95%
iter 8360: loss 2.6111, time 2064.37ms, mfu 33.96%
iter 8370: loss 2.9273, time 2060.10ms, mfu 33.97%
iter 8380: loss 2.6358, time 2068.16ms, mfu 33.97%
iter 8390: loss 2.4288, time 2073.60ms, mfu 33.96%
iter 8400: loss 2.5959, time 2066.53ms, mfu 33.97%
iter 8410: loss 2.5588, time 2062.63ms, mfu 33.98%
iter 8420: loss 2.7146, time 2063.75ms, mfu 33.98%
iter 8430: loss 2.7121, time 2070.44ms, mfu 33.98%
iter 8440: loss 2.1057, time 2065.77ms, mfu 33.98%
iter 8450: loss 2.8603, time 2077.68ms, mfu 33.96%
iter 8460: loss 2.6322, time 2067.53ms, mfu 33.97%
iter 8470: loss 2.8530, time 2073.25ms, mfu 33.96%
iter 8480: loss 2.8399, time 2055.41ms, mfu 33.98%
iter 8490: loss 2.8382, time 2066.71ms, mfu 33.98%
iter 8500: loss 2.4481, time 2067.90ms, mfu 33.98%
iter 8510: loss 2.3293, time 2068.23ms, mfu 33.98%
iter 8520: loss 2.5659, time 2056.64ms, mfu 34.00%
iter 8530: loss 2.8179, time 2066.04ms, mfu 34.00%
iter 8540: loss 2.6658, time 2058.01ms, mfu 34.01%
iter 8550: loss 2.7417, time 2056.76ms, mfu 34.03%
iter 8560: loss 2.5201, time 2068.36ms, mfu 34.02%
iter 8570: loss 2.4454, time 2058.80ms, mfu 34.03%
iter 8580: loss 2.7683, time 2066.59ms, mfu 34.03%
iter 8590: loss 2.5503, time 2057.12ms, mfu 34.04%
iter 8600: loss 2.5039, time 2073.37ms, mfu 34.03%
iter 8610: loss 2.6278, time 2069.51ms, mfu 34.02%
iter 8620: loss 2.5305, time 2064.83ms, mfu 34.02%
iter 8630: loss 2.8835, time 2074.82ms, mfu 34.00%
iter 8640: loss 2.6614, time 2068.06ms, mfu 34.00%
iter 8650: loss 2.7330, time 2073.04ms, mfu 33.99%
iter 8660: loss 2.0691, time 2072.65ms, mfu 33.98%
iter 8670: loss 2.8066, time 2068.17ms, mfu 33.98%
iter 8680: loss 2.2184, time 2057.42ms, mfu 34.00%
iter 8690: loss 2.6948, time 2067.88ms, mfu 33.99%
iter 8700: loss 2.6029, time 2067.33ms, mfu 33.99%
iter 8710: loss 2.4013, time 2057.17ms, mfu 34.01%
iter 8720: loss 2.6765, time 2067.40ms, mfu 34.01%
iter 8730: loss 2.5796, time 2074.69ms, mfu 33.99%
iter 8740: loss 2.4355, time 2060.88ms, mfu 34.00%
iter 8750: loss 2.4995, time 2064.73ms, mfu 34.00%
iter 8760: loss 2.7665, time 2073.26ms, mfu 33.99%
iter 8770: loss 3.0022, time 2061.41ms, mfu 34.00%
iter 8780: loss 2.8570, time 2062.83ms, mfu 34.01%
iter 8790: loss 2.4886, time 2067.07ms, mfu 34.01%
iter 8800: loss 2.5721, time 2062.35ms, mfu 34.01%
iter 8810: loss 2.7232, time 2060.39ms, mfu 34.02%
iter 8820: loss 2.6539, time 2063.56ms, mfu 34.02%
iter 8830: loss 2.6905, time 2064.49ms, mfu 34.02%
iter 8840: loss 2.1432, time 2064.24ms, mfu 34.03%
iter 8850: loss 2.6969, time 2067.57ms, mfu 34.02%
iter 8860: loss 2.6095, time 2069.66ms, mfu 34.01%
iter 8870: loss 2.8346, time 2058.43ms, mfu 34.03%
iter 8880: loss 2.2024, time 2060.10ms, mfu 34.03%
iter 8890: loss 2.5171, time 2077.34ms, mfu 34.01%
iter 8900: loss 2.3834, time 2048.69ms, mfu 34.04%
iter 8910: loss 2.4685, time 2065.86ms, mfu 34.04%
iter 8920: loss 2.7455, time 2063.03ms, mfu 34.04%
iter 8930: loss 2.0023, time 2065.21ms, mfu 34.04%
iter 8940: loss 2.5953, time 2061.96ms, mfu 34.04%
iter 8950: loss 2.4106, time 2062.46ms, mfu 34.04%
iter 8960: loss 2.4227, time 2065.19ms, mfu 34.04%
iter 8970: loss 2.8596, time 2071.85ms, mfu 34.03%
iter 8980: loss 2.3502, time 2061.79ms, mfu 34.03%
iter 8990: loss 2.9678, time 2058.52ms, mfu 34.04%
step 9000: train loss 2.6725, val loss 2.7046
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 9000: loss 2.6708, time 9067.21ms, mfu 31.41%
iter 9010: loss 2.3761, time 2059.76ms, mfu 31.68%
iter 9020: loss 2.9535, time 2065.20ms, mfu 31.92%
iter 9030: loss 2.8212, time 2062.28ms, mfu 32.13%
iter 9040: loss 2.7287, time 2067.86ms, mfu 32.32%
iter 9050: loss 2.7450, time 2075.15ms, mfu 32.47%
iter 9060: loss 2.8350, time 2059.45ms, mfu 32.63%
iter 9070: loss 2.8669, time 2062.13ms, mfu 32.78%
iter 9080: loss 2.4298, time 2059.04ms, mfu 32.91%
iter 9090: loss 2.7568, time 2075.61ms, mfu 33.01%
iter 9100: loss 2.7852, time 2067.10ms, mfu 33.10%
iter 9110: loss 2.3842, time 2064.67ms, mfu 33.20%
iter 9120: loss 2.4844, time 2069.41ms, mfu 33.27%
iter 9130: loss 2.7053, time 2053.77ms, mfu 33.37%
iter 9140: loss 2.5114, time 2071.96ms, mfu 33.42%
iter 9150: loss 2.7582, time 2069.12ms, mfu 33.47%
iter 9160: loss 2.8697, time 2052.09ms, mfu 33.55%
iter 9170: loss 2.7488, time 2057.87ms, mfu 33.61%
iter 9180: loss 2.5352, time 2058.42ms, mfu 33.66%
iter 9190: loss 2.7650, time 2071.84ms, mfu 33.69%
iter 9200: loss 2.5589, time 2060.48ms, mfu 33.73%
iter 9210: loss 2.8538, time 2055.66ms, mfu 33.77%
iter 9220: loss 2.6922, time 2073.17ms, mfu 33.78%
iter 9230: loss 2.7160, time 2058.79ms, mfu 33.82%
iter 9240: loss 2.7395, time 2063.07ms, mfu 33.84%
iter 9250: loss 2.5759, time 2068.71ms, mfu 33.85%
iter 9260: loss 2.8481, time 2058.15ms, mfu 33.88%
iter 9270: loss 2.5700, time 2063.53ms, mfu 33.90%
iter 9280: loss 2.7680, time 2060.54ms, mfu 33.92%
iter 9290: loss 2.5290, time 2069.19ms, mfu 33.92%
iter 9300: loss 2.3819, time 2054.04ms, mfu 33.95%
iter 9310: loss 2.4058, time 2082.99ms, mfu 33.93%
iter 9320: loss 2.8427, time 2060.19ms, mfu 33.95%
iter 9330: loss 2.5775, time 2072.85ms, mfu 33.94%
iter 9340: loss 2.5913, time 2067.09ms, mfu 33.95%
iter 9350: loss 3.0575, time 2059.60ms, mfu 33.96%
iter 9360: loss 2.5959, time 2063.73ms, mfu 33.97%
iter 9370: loss 2.9189, time 2061.32ms, mfu 33.98%
iter 9380: loss 3.0476, time 2058.14ms, mfu 34.00%
iter 9390: loss 2.8034, time 2072.76ms, mfu 33.99%
iter 9400: loss 2.6706, time 2068.00ms, mfu 33.99%
iter 9410: loss 2.6080, time 2057.80ms, mfu 34.00%
iter 9420: loss 2.6711, time 2069.01ms, mfu 34.00%
iter 9430: loss 2.5529, time 2058.13ms, mfu 34.01%
iter 9440: loss 2.5636, time 2069.97ms, mfu 34.00%
iter 9450: loss 2.4920, time 2059.26ms, mfu 34.02%
iter 9460: loss 2.5836, time 2079.10ms, mfu 33.99%
iter 9470: loss 2.6364, time 2054.41ms, mfu 34.01%
iter 9480: loss 2.6912, time 2062.91ms, mfu 34.02%
iter 9490: loss 2.6827, time 2054.98ms, mfu 34.04%
iter 9500: loss 2.7192, time 2077.00ms, mfu 34.01%
iter 9510: loss 2.8513, time 2062.92ms, mfu 34.02%
iter 9520: loss 2.7043, time 2068.20ms, mfu 34.01%
iter 9530: loss 2.5075, time 2064.79ms, mfu 34.02%
iter 9540: loss 2.8808, time 2063.65ms, mfu 34.02%
iter 9550: loss 2.6072, time 2062.87ms, mfu 34.02%
iter 9560: loss 2.6233, time 2068.67ms, mfu 34.02%
iter 9570: loss 2.6678, time 2059.40ms, mfu 34.03%
iter 9580: loss 2.3900, time 2059.01ms, mfu 34.04%
iter 9590: loss 2.6057, time 2069.21ms, mfu 34.03%
iter 9600: loss 2.7250, time 2059.72ms, mfu 34.04%
iter 9610: loss 2.7833, time 2067.90ms, mfu 34.03%
iter 9620: loss 2.4287, time 2063.40ms, mfu 34.03%
iter 9630: loss 2.6121, time 2071.65ms, mfu 34.02%
iter 9640: loss 2.6881, time 2063.89ms, mfu 34.02%
iter 9650: loss 2.5513, time 2058.03ms, mfu 34.03%
iter 9660: loss 2.5789, time 2063.16ms, mfu 34.04%
iter 9670: loss 2.5896, time 2058.13ms, mfu 34.05%
iter 9680: loss 2.9033, time 2077.73ms, mfu 34.02%
iter 9690: loss 2.5954, time 2054.36ms, mfu 34.04%
iter 9700: loss 2.7855, time 2065.53ms, mfu 34.04%
iter 9710: loss 2.7659, time 2060.47ms, mfu 34.04%
iter 9720: loss 2.9010, time 2077.33ms, mfu 34.02%
iter 9730: loss 2.5085, time 2060.36ms, mfu 34.03%
iter 9740: loss 2.6956, time 2059.17ms, mfu 34.04%
iter 9750: loss 2.5603, time 2066.01ms, mfu 34.04%
iter 9760: loss 2.2655, time 2061.20ms, mfu 34.04%
iter 9770: loss 2.3644, time 2061.35ms, mfu 34.04%
iter 9780: loss 2.6883, time 2074.92ms, mfu 34.03%
iter 9790: loss 2.6941, time 2060.25ms, mfu 34.03%
iter 9800: loss 2.6390, time 2076.50ms, mfu 34.01%
iter 9810: loss 2.5607, time 2055.54ms, mfu 34.03%
iter 9820: loss 2.5934, time 2065.19ms, mfu 34.03%
iter 9830: loss 2.4772, time 2068.38ms, mfu 34.02%
iter 9840: loss 2.4179, time 2063.82ms, mfu 34.03%
iter 9850: loss 2.5912, time 2060.43ms, mfu 34.03%
iter 9860: loss 2.4204, time 2061.86ms, mfu 34.04%
iter 9870: loss 2.7484, time 2053.80ms, mfu 34.05%
iter 9880: loss 3.0405, time 2063.22ms, mfu 34.05%
iter 9890: loss 2.5587, time 2059.87ms, mfu 34.06%
iter 9900: loss 2.5990, time 2065.75ms, mfu 34.05%
iter 9910: loss 2.6287, time 2058.45ms, mfu 34.06%
iter 9920: loss 2.4566, time 2066.16ms, mfu 34.06%
iter 9930: loss 2.9590, time 2060.24ms, mfu 34.06%
iter 9940: loss 2.5088, time 2070.80ms, mfu 34.05%
iter 9950: loss 2.6071, time 2054.06ms, mfu 34.06%
iter 9960: loss 2.6405, time 2060.80ms, mfu 34.07%
iter 9970: loss 2.6527, time 2069.78ms, mfu 34.05%
iter 9980: loss 2.2369, time 2057.63ms, mfu 34.06%
iter 9990: loss 2.9309, time 2059.60ms, mfu 34.07%
step 10000: train loss 2.6098, val loss 2.6928
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 10000: loss 2.6818, time 9066.47ms, mfu 31.44%
iter 10010: loss 2.4153, time 2060.50ms, mfu 31.70%
iter 10020: loss 2.8921, time 2064.69ms, mfu 31.93%
iter 10030: loss 2.4925, time 2049.52ms, mfu 32.17%
iter 10040: loss 2.7314, time 2056.18ms, mfu 32.37%
iter 10050: loss 2.6600, time 2066.31ms, mfu 32.53%
iter 10060: loss 2.4391, time 2057.98ms, mfu 32.69%
iter 10070: loss 2.8679, time 2079.04ms, mfu 32.80%
iter 10080: loss 2.8293, time 2060.80ms, mfu 32.93%
iter 10090: loss 2.7463, time 2074.05ms, mfu 33.03%
iter 10100: loss 2.4889, time 2059.03ms, mfu 33.14%
iter 10110: loss 2.7040, time 2070.97ms, mfu 33.22%
iter 10120: loss 2.6523, time 2073.87ms, mfu 33.28%
iter 10130: loss 2.8288, time 2058.71ms, mfu 33.37%
iter 10140: loss 2.3883, time 2069.49ms, mfu 33.42%
iter 10150: loss 2.8258, time 2062.30ms, mfu 33.49%
iter 10160: loss 2.4561, time 2067.90ms, mfu 33.54%
iter 10170: loss 2.4972, time 2054.33ms, mfu 33.60%
iter 10180: loss 2.6207, time 2065.53ms, mfu 33.64%
iter 10190: loss 2.7256, time 2056.09ms, mfu 33.70%
iter 10200: loss 2.4992, time 2065.36ms, mfu 33.73%
iter 10210: loss 2.3864, time 2062.48ms, mfu 33.76%
iter 10220: loss 2.3153, time 2059.83ms, mfu 33.80%
iter 10230: loss 2.8782, time 2057.51ms, mfu 33.83%
iter 10240: loss 2.7194, time 2053.70ms, mfu 33.87%
iter 10250: loss 2.5983, time 2052.90ms, mfu 33.91%
iter 10260: loss 2.2427, time 2064.02ms, mfu 33.92%
iter 10270: loss 2.4142, time 2060.09ms, mfu 33.94%
iter 10280: loss 2.7083, time 2062.23ms, mfu 33.95%
iter 10290: loss 2.7991, time 2064.34ms, mfu 33.96%
iter 10300: loss 2.2950, time 2074.24ms, mfu 33.95%
iter 10310: loss 2.7618, time 2064.57ms, mfu 33.96%
iter 10320: loss 2.6323, time 2057.95ms, mfu 33.98%
iter 10330: loss 2.7193, time 2060.09ms, mfu 33.99%
iter 10340: loss 2.3563, time 2061.61ms, mfu 34.00%
iter 10350: loss 2.2564, time 2064.29ms, mfu 34.00%
iter 10360: loss 2.6965, time 2063.59ms, mfu 34.01%
iter 10370: loss 2.4466, time 2058.57ms, mfu 34.02%
iter 10380: loss 2.6256, time 2071.88ms, mfu 34.01%
iter 10390: loss 2.6320, time 2070.80ms, mfu 34.00%
iter 10400: loss 2.7519, time 2053.04ms, mfu 34.02%
iter 10410: loss 2.4551, time 2060.26ms, mfu 34.03%
iter 10420: loss 2.2128, time 2059.74ms, mfu 34.04%
iter 10430: loss 2.6083, time 2055.56ms, mfu 34.05%
iter 10440: loss 2.9739, time 2067.86ms, mfu 34.04%
iter 10450: loss 2.4810, time 2059.90ms, mfu 34.05%
iter 10460: loss 2.6777, time 2047.67ms, mfu 34.08%
iter 10470: loss 2.3237, time 2056.70ms, mfu 34.09%
iter 10480: loss 2.2072, time 2061.85ms, mfu 34.08%
iter 10490: loss 2.5535, time 2055.57ms, mfu 34.09%
iter 10500: loss 2.7508, time 2071.99ms, mfu 34.08%
iter 10510: loss 2.4628, time 2060.19ms, mfu 34.08%
iter 10520: loss 2.7934, time 2058.71ms, mfu 34.08%
iter 10530: loss 2.2086, time 2059.99ms, mfu 34.09%
iter 10540: loss 2.4975, time 2057.92ms, mfu 34.09%
iter 10550: loss 2.6391, time 2068.16ms, mfu 34.08%
iter 10560: loss 2.4180, time 2068.61ms, mfu 34.07%
iter 10570: loss 2.4721, time 2058.13ms, mfu 34.07%
iter 10580: loss 2.5195, time 2072.43ms, mfu 34.06%
iter 10590: loss 2.5897, time 2063.70ms, mfu 34.06%
iter 10600: loss 2.6969, time 2057.46ms, mfu 34.06%
iter 10610: loss 2.7318, time 2062.76ms, mfu 34.06%
iter 10620: loss 2.5522, time 2062.53ms, mfu 34.06%
iter 10630: loss 2.4850, time 2068.16ms, mfu 34.06%
iter 10640: loss 2.5061, time 2059.72ms, mfu 34.06%
iter 10650: loss 2.8947, time 2062.64ms, mfu 34.06%
iter 10660: loss 2.6385, time 2064.40ms, mfu 34.06%
iter 10670: loss 2.3477, time 2070.80ms, mfu 34.05%
iter 10680: loss 2.7086, time 2069.82ms, mfu 34.03%
iter 10690: loss 2.9617, time 2059.70ms, mfu 34.04%
iter 10700: loss 2.8884, time 2053.54ms, mfu 34.06%
iter 10710: loss 2.4545, time 2063.79ms, mfu 34.06%
iter 10720: loss 2.3016, time 2071.36ms, mfu 34.04%
iter 10730: loss 2.4445, time 2069.64ms, mfu 34.03%
iter 10740: loss 2.4914, time 2061.34ms, mfu 34.04%
iter 10750: loss 2.2444, time 2061.84ms, mfu 34.04%
iter 10760: loss 2.8815, time 2059.65ms, mfu 34.05%
iter 10770: loss 2.5330, time 2055.11ms, mfu 34.06%
iter 10780: loss 2.7568, time 2051.29ms, mfu 34.08%
iter 10790: loss 2.4291, time 2055.94ms, mfu 34.09%
iter 10800: loss 2.5251, time 2071.08ms, mfu 34.07%
iter 10810: loss 2.4847, time 2059.14ms, mfu 34.08%
iter 10820: loss 2.7133, time 2073.96ms, mfu 34.06%
iter 10830: loss 2.6447, time 2073.83ms, mfu 34.04%
iter 10840: loss 2.5646, time 2064.62ms, mfu 34.04%
iter 10850: loss 2.5846, time 2056.03ms, mfu 34.05%
iter 10860: loss 2.3809, time 2058.69ms, mfu 34.06%
iter 10870: loss 2.6342, time 2066.15ms, mfu 34.05%
iter 10880: loss 2.5886, time 2062.46ms, mfu 34.06%
iter 10890: loss 2.5468, time 2058.63ms, mfu 34.06%
iter 10900: loss 2.6735, time 2076.87ms, mfu 34.04%
iter 10910: loss 2.8591, time 2071.32ms, mfu 34.03%
iter 10920: loss 2.4669, time 2058.01ms, mfu 34.04%
iter 10930: loss 2.4665, time 2061.06ms, mfu 34.04%
iter 10940: loss 2.2181, time 2079.22ms, mfu 34.02%
iter 10950: loss 2.5276, time 2060.87ms, mfu 34.03%
iter 10960: loss 2.6678, time 2067.74ms, mfu 34.02%
iter 10970: loss 2.5568, time 2060.92ms, mfu 34.03%
iter 10980: loss 2.6901, time 2072.64ms, mfu 34.01%
iter 10990: loss 2.6687, time 2057.86ms, mfu 34.03%
step 11000: train loss 2.6080, val loss 2.6397
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 11000: loss 2.4670, time 9053.47ms, mfu 31.40%
iter 11010: loss 2.6667, time 2060.61ms, mfu 31.67%
iter 11020: loss 2.3796, time 2061.45ms, mfu 31.91%
iter 11030: loss 2.8439, time 2069.49ms, mfu 32.12%
iter 11040: loss 2.5498, time 2060.99ms, mfu 32.31%
iter 11050: loss 2.9064, time 2063.02ms, mfu 32.49%
iter 11060: loss 2.5818, time 2061.54ms, mfu 32.65%
iter 11070: loss 2.8234, time 2069.07ms, mfu 32.78%
iter 11080: loss 2.7696, time 2063.83ms, mfu 32.90%
iter 11090: loss 2.9317, time 2063.52ms, mfu 33.02%
iter 11100: loss 3.0274, time 2064.01ms, mfu 33.12%
iter 11110: loss 2.8279, time 2056.49ms, mfu 33.22%
iter 11120: loss 2.7370, time 2077.21ms, mfu 33.28%
iter 11130: loss 2.5241, time 2052.89ms, mfu 33.38%
iter 11140: loss 2.5037, time 2058.84ms, mfu 33.45%
iter 11150: loss 2.5966, time 2065.71ms, mfu 33.51%
iter 11160: loss 2.6222, time 2052.02ms, mfu 33.58%
iter 11170: loss 2.6213, time 2059.76ms, mfu 33.63%
iter 11180: loss 2.5192, time 2062.17ms, mfu 33.68%
iter 11190: loss 2.2646, time 2062.04ms, mfu 33.72%
iter 11200: loss 2.5076, time 2061.29ms, mfu 33.75%
iter 11210: loss 2.4897, time 2059.35ms, mfu 33.79%
iter 11220: loss 2.4618, time 2052.26ms, mfu 33.83%
iter 11230: loss 2.4812, time 2055.71ms, mfu 33.87%
iter 11240: loss 2.6200, time 2058.84ms, mfu 33.89%
iter 11250: loss 2.6613, time 2049.23ms, mfu 33.93%
iter 11260: loss 2.4928, time 2061.67ms, mfu 33.95%
iter 11270: loss 3.0143, time 2060.88ms, mfu 33.96%
iter 11280: loss 2.6406, time 2065.26ms, mfu 33.97%
iter 11290: loss 2.6204, time 2060.38ms, mfu 33.98%
iter 11300: loss 2.7274, time 2069.37ms, mfu 33.98%
iter 11310: loss 2.8454, time 2054.26ms, mfu 34.00%
iter 11320: loss 2.2316, time 2065.30ms, mfu 34.00%
iter 11330: loss 2.5909, time 2057.67ms, mfu 34.02%
iter 11340: loss 2.7343, time 2062.24ms, mfu 34.02%
iter 11350: loss 2.6025, time 2058.97ms, mfu 34.03%
iter 11360: loss 2.7927, time 2061.03ms, mfu 34.04%
iter 11370: loss 2.5694, time 2060.10ms, mfu 34.04%
iter 11380: loss 2.9913, time 2065.86ms, mfu 34.04%
iter 11390: loss 2.7212, time 2075.13ms, mfu 34.02%
iter 11400: loss 2.3514, time 2065.52ms, mfu 34.02%
iter 11410: loss 2.3657, time 2054.95ms, mfu 34.04%
iter 11420: loss 2.5710, time 2068.00ms, mfu 34.03%
iter 11430: loss 2.6336, time 2065.74ms, mfu 34.03%
iter 11440: loss 2.9689, time 2058.63ms, mfu 34.04%
iter 11450: loss 2.8207, time 2068.54ms, mfu 34.03%
iter 11460: loss 2.8212, time 2060.36ms, mfu 34.04%
iter 11470: loss 2.6242, time 2064.60ms, mfu 34.04%
iter 11480: loss 2.4707, time 2069.76ms, mfu 34.03%
iter 11490: loss 2.5390, time 2058.07ms, mfu 34.04%
iter 11500: loss 2.6867, time 2057.60ms, mfu 34.05%
iter 11510: loss 2.8560, time 2059.79ms, mfu 34.06%
iter 11520: loss 2.4891, time 2054.33ms, mfu 34.07%
iter 11530: loss 2.7639, time 2050.21ms, mfu 34.09%
iter 11540: loss 2.6526, time 2064.52ms, mfu 34.08%
iter 11550: loss 2.7234, time 2058.79ms, mfu 34.09%
iter 11560: loss 2.6222, time 2048.64ms, mfu 34.11%
iter 11570: loss 2.5293, time 2059.88ms, mfu 34.11%
iter 11580: loss 2.5126, time 2062.10ms, mfu 34.11%
iter 11590: loss 2.6322, time 2063.48ms, mfu 34.10%
iter 11600: loss 2.5095, time 2049.66ms, mfu 34.12%
iter 11610: loss 2.9998, time 2071.40ms, mfu 34.10%
iter 11620: loss 2.4878, time 2054.63ms, mfu 34.11%
iter 11630: loss 2.3961, time 2063.01ms, mfu 34.10%
iter 11640: loss 2.5189, time 2065.25ms, mfu 34.09%
iter 11650: loss 2.7439, time 2058.58ms, mfu 34.10%
iter 11660: loss 2.2963, time 2064.10ms, mfu 34.09%
iter 11670: loss 2.2873, time 2066.71ms, mfu 34.08%
iter 11680: loss 2.5790, time 2053.27ms, mfu 34.10%
iter 11690: loss 2.5777, time 2058.24ms, mfu 34.10%
iter 11700: loss 2.5350, time 2064.83ms, mfu 34.09%
iter 11710: loss 2.7743, time 2052.43ms, mfu 34.11%
iter 11720: loss 2.2890, time 2063.76ms, mfu 34.10%
iter 11730: loss 2.7664, time 2073.01ms, mfu 34.08%
iter 11740: loss 2.7230, time 2056.49ms, mfu 34.09%
iter 11750: loss 2.5505, time 2060.73ms, mfu 34.09%
iter 11760: loss 2.4322, time 2067.34ms, mfu 34.08%
iter 11770: loss 2.4931, time 2076.52ms, mfu 34.05%
iter 11780: loss 2.5660, time 2066.01ms, mfu 34.05%
iter 11790: loss 2.6750, time 2060.29ms, mfu 34.05%
iter 11800: loss 3.0233, time 2055.14ms, mfu 34.07%
iter 11810: loss 2.3940, time 2061.12ms, mfu 34.07%
iter 11820: loss 2.5639, time 2065.28ms, mfu 34.06%
iter 11830: loss 2.4781, time 2068.13ms, mfu 34.05%
iter 11840: loss 2.2906, time 2058.33ms, mfu 34.06%
iter 11850: loss 2.6401, time 2051.42ms, mfu 34.08%
iter 11860: loss 2.6692, time 2055.97ms, mfu 34.09%
iter 11870: loss 2.5525, time 2059.25ms, mfu 34.09%
iter 11880: loss 2.6553, time 2065.60ms, mfu 34.09%
iter 11890: loss 2.2409, time 2056.33ms, mfu 34.09%
iter 11900: loss 2.4609, time 2055.12ms, mfu 34.10%
iter 11910: loss 2.6352, time 2058.76ms, mfu 34.11%
iter 11920: loss 2.6475, time 2056.36ms, mfu 34.11%
iter 11930: loss 2.4512, time 2072.67ms, mfu 34.09%
iter 11940: loss 2.5475, time 2062.44ms, mfu 34.09%
iter 11950: loss 2.4088, time 2065.88ms, mfu 34.08%
iter 11960: loss 2.7781, time 2060.43ms, mfu 34.08%
iter 11970: loss 2.7867, time 2079.31ms, mfu 34.05%
iter 11980: loss 2.5822, time 2056.45ms, mfu 34.06%
iter 11990: loss 2.4477, time 2059.48ms, mfu 34.07%
step 12000: train loss 2.5605, val loss 2.6301
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 12000: loss 2.0776, time 9091.03ms, mfu 31.43%
iter 12010: loss 2.3315, time 2054.17ms, mfu 31.71%
iter 12020: loss 2.6457, time 2059.31ms, mfu 31.95%
iter 12030: loss 2.4206, time 2055.05ms, mfu 32.18%
iter 12040: loss 2.6454, time 2064.45ms, mfu 32.36%
iter 12050: loss 2.7072, time 2071.03ms, mfu 32.52%
iter 12060: loss 2.5767, time 2051.99ms, mfu 32.69%
iter 12070: loss 2.7401, time 2055.95ms, mfu 32.84%
iter 12080: loss 2.4669, time 2062.49ms, mfu 32.96%
iter 12090: loss 2.3963, time 2064.36ms, mfu 33.07%
iter 12100: loss 2.7748, time 2060.38ms, mfu 33.17%
iter 12110: loss 2.6146, time 2055.91ms, mfu 33.27%
iter 12120: loss 2.4939, time 2050.86ms, mfu 33.37%
iter 12130: loss 2.6030, time 2050.41ms, mfu 33.46%
iter 12140: loss 2.5353, time 2060.08ms, mfu 33.52%
iter 12150: loss 2.5083, time 2071.09ms, mfu 33.56%
iter 12160: loss 2.5298, time 2060.78ms, mfu 33.62%
iter 12170: loss 2.6107, time 2056.81ms, mfu 33.67%
iter 12180: loss 2.5145, time 2064.40ms, mfu 33.71%
iter 12190: loss 2.8468, time 2063.68ms, mfu 33.74%
iter 12200: loss 2.6968, time 2062.90ms, mfu 33.77%
iter 12210: loss 2.6488, time 2061.54ms, mfu 33.80%
iter 12220: loss 2.3838, time 2065.22ms, mfu 33.83%
iter 12230: loss 2.3998, time 2054.90ms, mfu 33.86%
iter 12240: loss 2.7740, time 2050.77ms, mfu 33.90%
iter 12250: loss 2.6679, time 2063.03ms, mfu 33.92%
iter 12260: loss 2.3619, time 2060.71ms, mfu 33.93%
iter 12270: loss 2.9184, time 2054.22ms, mfu 33.96%
iter 12280: loss 2.2216, time 2066.02ms, mfu 33.97%
iter 12290: loss 2.7737, time 2062.23ms, mfu 33.98%
iter 12300: loss 2.2813, time 2066.06ms, mfu 33.98%
iter 12310: loss 2.7578, time 2054.62ms, mfu 34.00%
iter 12320: loss 2.4254, time 2062.75ms, mfu 34.01%
iter 12330: loss 2.4140, time 2051.90ms, mfu 34.03%
iter 12340: loss 2.3865, time 2059.76ms, mfu 34.04%
iter 12350: loss 2.8641, time 2070.56ms, mfu 34.03%
iter 12360: loss 2.6863, time 2055.26ms, mfu 34.04%
iter 12370: loss 2.3716, time 2052.03ms, mfu 34.06%
iter 12380: loss 2.5410, time 2074.35ms, mfu 34.04%
iter 12390: loss 2.6395, time 2059.74ms, mfu 34.05%
iter 12400: loss 2.3718, time 2059.30ms, mfu 34.06%
iter 12410: loss 2.3765, time 2054.23ms, mfu 34.07%
iter 12420: loss 2.3483, time 2066.84ms, mfu 34.06%
iter 12430: loss 2.5009, time 2058.70ms, mfu 34.07%
iter 12440: loss 2.7279, time 2057.16ms, mfu 34.08%
iter 12450: loss 2.7081, time 2064.47ms, mfu 34.07%
iter 12460: loss 3.0000, time 2056.11ms, mfu 34.08%
iter 12470: loss 2.6775, time 2054.56ms, mfu 34.09%
iter 12480: loss 2.7992, time 2049.64ms, mfu 34.11%
iter 12490: loss 2.7590, time 2065.50ms, mfu 34.10%
iter 12500: loss 2.4465, time 2033.22ms, mfu 34.15%
iter 12510: loss 2.5414, time 2051.11ms, mfu 34.16%
iter 12520: loss 2.7769, time 2053.03ms, mfu 34.17%
iter 12530: loss 2.5233, time 2067.70ms, mfu 34.15%
iter 12540: loss 2.8246, time 2049.87ms, mfu 34.16%
iter 12550: loss 2.5609, time 2060.79ms, mfu 34.15%
iter 12560: loss 2.6976, time 2072.47ms, mfu 34.13%
iter 12570: loss 2.4856, time 2065.39ms, mfu 34.12%
iter 12580: loss 2.8740, time 2073.80ms, mfu 34.09%
iter 12590: loss 2.6110, time 2063.77ms, mfu 34.09%
iter 12600: loss 2.6411, time 2056.00ms, mfu 34.10%
iter 12610: loss 2.5130, time 2063.49ms, mfu 34.09%
iter 12620: loss 2.7267, time 2057.86ms, mfu 34.10%
iter 12630: loss 2.1115, time 2063.43ms, mfu 34.09%
iter 12640: loss 2.5355, time 2063.19ms, mfu 34.09%
iter 12650: loss 2.3345, time 2068.63ms, mfu 34.08%
iter 12660: loss 2.6651, time 2062.40ms, mfu 34.07%
iter 12670: loss 2.2534, time 2048.00ms, mfu 34.10%
iter 12680: loss 2.3468, time 2060.68ms, mfu 34.10%
iter 12690: loss 2.4591, time 2074.64ms, mfu 34.07%
iter 12700: loss 2.5695, time 2053.39ms, mfu 34.09%
iter 12710: loss 2.2450, time 2048.97ms, mfu 34.11%
iter 12720: loss 2.5286, time 2058.98ms, mfu 34.11%
iter 12730: loss 2.6772, time 2075.76ms, mfu 34.08%
iter 12740: loss 2.7052, time 2057.47ms, mfu 34.09%
iter 12750: loss 2.6637, time 2058.86ms, mfu 34.09%
iter 12760: loss 2.5787, time 2070.42ms, mfu 34.08%
iter 12770: loss 2.8703, time 2069.21ms, mfu 34.07%
iter 12780: loss 2.7567, time 2072.70ms, mfu 34.05%
iter 12790: loss 2.6592, time 2053.78ms, mfu 34.06%
iter 12800: loss 2.4989, time 2054.01ms, mfu 34.08%
iter 12810: loss 2.4378, time 2074.32ms, mfu 34.06%
iter 12820: loss 2.7438, time 2057.89ms, mfu 34.07%
iter 12830: loss 2.7508, time 2055.97ms, mfu 34.08%
iter 12840: loss 2.6588, time 2058.76ms, mfu 34.08%
iter 12850: loss 2.4466, time 2074.52ms, mfu 34.06%
iter 12860: loss 2.4130, time 2061.19ms, mfu 34.06%
iter 12870: loss 2.4161, time 2066.53ms, mfu 34.06%
iter 12880: loss 2.6381, time 2053.29ms, mfu 34.07%
iter 12890: loss 2.4429, time 2064.49ms, mfu 34.07%
iter 12900: loss 2.5636, time 2057.16ms, mfu 34.08%
iter 12910: loss 2.5343, time 2071.58ms, mfu 34.06%
iter 12920: loss 2.5449, time 2059.03ms, mfu 34.07%
iter 12930: loss 2.8290, time 2053.00ms, mfu 34.08%
iter 12940: loss 2.3646, time 2067.89ms, mfu 34.07%
iter 12950: loss 2.6786, time 2061.23ms, mfu 34.07%
iter 12960: loss 2.7049, time 2062.13ms, mfu 34.07%
iter 12970: loss 2.4932, time 2068.48ms, mfu 34.06%
iter 12980: loss 2.3409, time 2053.80ms, mfu 34.08%
iter 12990: loss 2.4000, time 2066.20ms, mfu 34.07%
step 13000: train loss 2.5864, val loss 2.6138
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 13000: loss 2.7380, time 9066.45ms, mfu 31.44%
iter 13010: loss 2.3439, time 2049.57ms, mfu 31.72%
iter 13020: loss 2.6481, time 2077.55ms, mfu 31.93%
iter 13030: loss 2.5613, time 2069.23ms, mfu 32.13%
iter 13040: loss 2.3326, time 2054.73ms, mfu 32.34%
iter 13050: loss 2.6436, time 2071.79ms, mfu 32.50%
iter 13060: loss 2.7501, time 2052.48ms, mfu 32.67%
iter 13070: loss 2.6416, time 2046.99ms, mfu 32.84%
iter 13080: loss 2.7759, time 2070.79ms, mfu 32.94%
iter 13090: loss 2.7115, time 2055.40ms, mfu 33.07%
iter 13100: loss 2.6570, time 2058.22ms, mfu 33.18%
iter 13110: loss 2.6462, time 2075.60ms, mfu 33.24%
iter 13120: loss 2.4610, time 2052.49ms, mfu 33.34%
iter 13130: loss 2.4683, time 2061.66ms, mfu 33.42%
iter 13140: loss 2.6229, time 2051.07ms, mfu 33.50%
iter 13150: loss 2.5957, time 2078.38ms, mfu 33.53%
iter 13160: loss 2.6516, time 2057.44ms, mfu 33.59%
iter 13170: loss 2.3856, time 2058.19ms, mfu 33.65%
iter 13180: loss 2.4373, time 2058.86ms, mfu 33.69%
iter 13190: loss 2.5548, time 2065.81ms, mfu 33.73%
iter 13200: loss 2.3040, time 2062.69ms, mfu 33.76%
iter 13210: loss 2.7417, time 2062.70ms, mfu 33.79%
iter 13220: loss 2.4150, time 2053.57ms, mfu 33.83%
iter 13230: loss 2.5343, time 2057.92ms, mfu 33.86%
iter 13240: loss 2.3495, time 2072.88ms, mfu 33.87%
iter 13250: loss 2.4724, time 2058.65ms, mfu 33.89%
iter 13260: loss 2.7842, time 2057.68ms, mfu 33.92%
iter 13270: loss 2.3401, time 2058.67ms, mfu 33.94%
iter 13280: loss 2.6965, time 2073.04ms, mfu 33.93%
iter 13290: loss 2.2799, time 2062.26ms, mfu 33.95%
iter 13300: loss 2.7876, time 2060.75ms, mfu 33.96%
iter 13310: loss 2.4272, time 2067.31ms, mfu 33.96%
iter 13320: loss 2.5021, time 2046.23ms, mfu 34.00%
iter 13330: loss 2.8174, time 2054.58ms, mfu 34.02%
iter 13340: loss 2.4047, time 2057.83ms, mfu 34.03%
iter 13350: loss 2.5204, time 2075.18ms, mfu 34.02%
iter 13360: loss 2.8097, time 2055.31ms, mfu 34.03%
iter 13370: loss 2.6610, time 2050.86ms, mfu 34.05%
iter 13380: loss 2.4978, time 2066.50ms, mfu 34.05%
iter 13390: loss 2.4170, time 2063.99ms, mfu 34.05%
iter 13400: loss 2.8377, time 2066.82ms, mfu 34.04%
iter 13410: loss 2.5655, time 2051.32ms, mfu 34.06%
iter 13420: loss 2.4013, time 2055.35ms, mfu 34.08%
iter 13430: loss 2.7592, time 2065.76ms, mfu 34.07%
iter 13440: loss 2.5101, time 2064.50ms, mfu 34.07%
iter 13450: loss 2.8504, time 2055.05ms, mfu 34.08%
iter 13460: loss 2.6912, time 2080.14ms, mfu 34.05%
iter 13470: loss 2.4065, time 2060.37ms, mfu 34.05%
iter 13480: loss 2.5997, time 2045.47ms, mfu 34.08%
iter 13490: loss 2.8609, time 2071.01ms, mfu 34.07%
iter 13500: loss 2.4951, time 2067.84ms, mfu 34.06%
iter 13510: loss 2.4402, time 2062.77ms, mfu 34.06%
iter 13520: loss 2.6831, time 2064.39ms, mfu 34.06%
iter 13530: loss 2.8145, time 2060.25ms, mfu 34.06%
iter 13540: loss 2.5684, time 2066.46ms, mfu 34.05%
iter 13550: loss 2.6011, time 2057.54ms, mfu 34.06%
iter 13560: loss 2.3702, time 2061.98ms, mfu 34.06%
iter 13570: loss 2.7817, time 2061.32ms, mfu 34.07%
iter 13580: loss 2.6540, time 2052.62ms, mfu 34.08%
iter 13590: loss 2.4682, time 2073.13ms, mfu 34.06%
iter 13600: loss 2.4975, time 2061.92ms, mfu 34.06%
iter 13610: loss 2.6230, time 2063.33ms, mfu 34.06%
iter 13620: loss 2.3349, time 2052.95ms, mfu 34.08%
iter 13630: loss 2.2930, time 2087.13ms, mfu 34.04%
iter 13640: loss 2.5781, time 2061.86ms, mfu 34.04%
iter 13650: loss 2.7901, time 2063.06ms, mfu 34.04%
iter 13660: loss 2.5238, time 2057.47ms, mfu 34.05%
iter 13670: loss 2.2911, time 2063.01ms, mfu 34.05%
iter 13680: loss 2.6762, time 2061.15ms, mfu 34.06%
iter 13690: loss 2.4528, time 2056.75ms, mfu 34.07%
iter 13700: loss 2.9699, time 2067.16ms, mfu 34.06%
iter 13710: loss 2.4235, time 2062.99ms, mfu 34.06%
iter 13720: loss 2.5405, time 2064.34ms, mfu 34.06%
iter 13730: loss 2.8400, time 2064.87ms, mfu 34.05%
iter 13740: loss 2.7712, time 2060.92ms, mfu 34.06%
iter 13750: loss 2.8090, time 2056.13ms, mfu 34.07%
iter 13760: loss 2.4001, time 2056.18ms, mfu 34.08%
iter 13770: loss 2.5907, time 2054.68ms, mfu 34.09%
iter 13780: loss 2.7545, time 2053.94ms, mfu 34.10%
iter 13790: loss 2.4430, time 2053.55ms, mfu 34.11%
iter 13800: loss 2.8071, time 2049.99ms, mfu 34.13%
iter 13810: loss 2.5336, time 2064.05ms, mfu 34.12%
iter 13820: loss 2.3503, time 2048.92ms, mfu 34.14%
iter 13830: loss 2.5504, time 2049.13ms, mfu 34.15%
iter 13840: loss 2.6415, time 2053.36ms, mfu 34.16%
iter 13850: loss 2.8027, time 2055.39ms, mfu 34.16%
iter 13860: loss 2.5493, time 2062.19ms, mfu 34.15%
iter 13870: loss 2.5127, time 2048.00ms, mfu 34.17%
iter 13880: loss 2.4871, time 2059.22ms, mfu 34.16%
iter 13890: loss 2.5747, time 2057.28ms, mfu 34.16%
iter 13900: loss 2.8729, time 2056.06ms, mfu 34.16%
iter 13910: loss 2.7369, time 2049.10ms, mfu 34.17%
iter 13920: loss 2.4248, time 2067.21ms, mfu 34.16%
iter 13930: loss 2.7975, time 2063.91ms, mfu 34.14%
iter 13940: loss 2.3047, time 2061.30ms, mfu 34.14%
iter 13950: loss 2.6183, time 2060.41ms, mfu 34.13%
iter 13960: loss 2.5239, time 2062.41ms, mfu 34.13%
iter 13970: loss 2.3967, time 2057.85ms, mfu 34.13%
iter 13980: loss 2.3231, time 2054.23ms, mfu 34.14%
iter 13990: loss 2.6802, time 2054.23ms, mfu 34.14%
step 14000: train loss 2.5575, val loss 2.6164
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 14000: loss 2.8029, time 9061.69ms, mfu 31.50%
iter 14010: loss 2.2803, time 2058.57ms, mfu 31.77%
iter 14020: loss 2.4520, time 2053.19ms, mfu 32.01%
iter 14030: loss 2.6593, time 2058.83ms, mfu 32.22%
iter 14040: loss 2.6270, time 2062.11ms, mfu 32.41%
iter 14050: loss 2.6535, time 2063.15ms, mfu 32.57%
iter 14060: loss 2.4947, time 2051.61ms, mfu 32.74%
iter 14070: loss 2.5535, time 2059.72ms, mfu 32.88%
iter 14080: loss 2.7918, time 2057.74ms, mfu 33.00%
iter 14090: loss 2.6818, time 2059.67ms, mfu 33.11%
iter 14100: loss 2.4939, time 2055.59ms, mfu 33.22%
iter 14110: loss 2.3787, time 2061.76ms, mfu 33.31%
iter 14120: loss 2.7219, time 2068.15ms, mfu 33.37%
iter 14130: loss 2.7953, time 2050.43ms, mfu 33.46%
iter 14140: loss 2.5326, time 2051.66ms, mfu 33.54%
iter 14150: loss 2.6087, time 2057.54ms, mfu 33.60%
iter 14160: loss 2.0256, time 2054.80ms, mfu 33.66%
iter 14170: loss 2.6912, time 2055.35ms, mfu 33.71%
iter 14180: loss 2.6032, time 2053.06ms, mfu 33.76%
iter 14190: loss 2.3300, time 2058.39ms, mfu 33.80%
iter 14200: loss 2.6319, time 2057.47ms, mfu 33.83%
iter 14210: loss 2.5641, time 2054.45ms, mfu 33.87%
iter 14220: loss 2.2662, time 2059.74ms, mfu 33.90%
iter 14230: loss 2.5128, time 2064.64ms, mfu 33.91%
iter 14240: loss 2.3486, time 2056.29ms, mfu 33.93%
iter 14250: loss 1.9805, time 2056.54ms, mfu 33.96%
iter 14260: loss 2.2195, time 2047.35ms, mfu 33.99%
iter 14270: loss 2.3904, time 2054.91ms, mfu 34.01%
iter 14280: loss 2.6537, time 2052.87ms, mfu 34.03%
iter 14290: loss 2.6837, time 2053.81ms, mfu 34.05%
iter 14300: loss 2.4102, time 2043.07ms, mfu 34.09%
iter 14310: loss 2.7811, time 2054.35ms, mfu 34.10%
iter 14320: loss 2.1844, time 2051.38ms, mfu 34.11%
iter 14330: loss 2.8172, time 2067.50ms, mfu 34.10%
iter 14340: loss 2.6297, time 2051.32ms, mfu 34.11%
iter 14350: loss 2.4724, time 2061.10ms, mfu 34.11%
iter 14360: loss 2.2456, time 2057.96ms, mfu 34.11%
iter 14370: loss 2.7333, time 2061.22ms, mfu 34.11%
iter 14380: loss 2.3020, time 2052.53ms, mfu 34.12%
iter 14390: loss 2.8970, time 2056.63ms, mfu 34.13%
iter 14400: loss 2.5623, time 2059.27ms, mfu 34.13%
iter 14410: loss 2.5717, time 2065.48ms, mfu 34.12%
iter 14420: loss 2.2594, time 2054.26ms, mfu 34.12%
iter 14430: loss 2.7837, time 2051.33ms, mfu 34.14%
iter 14440: loss 2.3503, time 2061.86ms, mfu 34.13%
iter 14450: loss 2.5512, time 2068.87ms, mfu 34.11%
iter 14460: loss 2.4546, time 2067.24ms, mfu 34.10%
iter 14470: loss 2.5551, time 2057.82ms, mfu 34.10%
iter 14480: loss 2.8855, time 2052.69ms, mfu 34.12%
iter 14490: loss 2.6255, time 2053.91ms, mfu 34.13%
iter 14500: loss 2.3376, time 2055.56ms, mfu 34.13%
iter 14510: loss 2.4140, time 2044.11ms, mfu 34.16%
iter 14520: loss 2.5269, time 2054.41ms, mfu 34.16%
iter 14530: loss 2.7492, time 2052.74ms, mfu 34.17%
iter 14540: loss 2.7673, time 2060.31ms, mfu 34.16%
iter 14550: loss 2.4833, time 2051.53ms, mfu 34.17%
iter 14560: loss 2.6350, time 2068.51ms, mfu 34.15%
iter 14570: loss 2.6695, time 2058.86ms, mfu 34.15%
iter 14580: loss 2.3509, time 2054.67ms, mfu 34.15%
iter 14590: loss 2.4199, time 2051.26ms, mfu 34.16%
iter 14600: loss 2.7392, time 2049.65ms, mfu 34.17%
iter 14610: loss 2.4420, time 2059.19ms, mfu 34.17%
iter 14620: loss 2.6778, time 2046.55ms, mfu 34.18%
iter 14630: loss 2.4819, time 2057.19ms, mfu 34.18%
iter 14640: loss 2.3190, time 2052.14ms, mfu 34.19%
iter 14650: loss 2.6133, time 2058.33ms, mfu 34.18%
iter 14660: loss 2.4233, time 2050.75ms, mfu 34.19%
iter 14670: loss 2.3562, time 2057.31ms, mfu 34.18%
iter 14680: loss 2.5500, time 2054.87ms, mfu 34.19%
iter 14690: loss 2.5178, time 2063.88ms, mfu 34.17%
iter 14700: loss 2.3900, time 2055.78ms, mfu 34.17%
iter 14710: loss 2.4925, time 2046.91ms, mfu 34.19%
iter 14720: loss 2.5132, time 2048.60ms, mfu 34.20%
iter 14730: loss 2.5137, time 2055.82ms, mfu 34.20%
iter 14740: loss 2.7357, time 2054.62ms, mfu 34.20%
iter 14750: loss 2.8404, time 2052.81ms, mfu 34.20%
iter 14760: loss 2.3811, time 2050.69ms, mfu 34.20%
iter 14770: loss 2.6199, time 2050.52ms, mfu 34.21%
iter 14780: loss 2.3716, time 2062.08ms, mfu 34.20%
iter 14790: loss 2.2393, time 2047.75ms, mfu 34.21%
iter 14800: loss 2.9103, time 2065.51ms, mfu 34.19%
iter 14810: loss 2.7354, time 2047.65ms, mfu 34.20%
iter 14820: loss 2.2770, time 2050.80ms, mfu 34.21%
iter 14830: loss 2.4562, time 2053.93ms, mfu 34.21%
iter 14840: loss 2.4282, time 2055.70ms, mfu 34.20%
iter 14850: loss 2.7176, time 2055.31ms, mfu 34.20%
iter 14860: loss 2.6067, time 2059.74ms, mfu 34.19%
iter 14870: loss 2.7508, time 2050.48ms, mfu 34.20%
iter 14880: loss 2.7051, time 2052.87ms, mfu 34.20%
iter 14890: loss 2.5682, time 2051.48ms, mfu 34.21%
iter 14900: loss 2.6220, time 2053.99ms, mfu 34.21%
iter 14910: loss 2.6087, time 2052.58ms, mfu 34.21%
iter 14920: loss 2.5139, time 2044.17ms, mfu 34.22%
iter 14930: loss 2.6384, time 2048.19ms, mfu 34.23%
iter 14940: loss 2.3015, time 2049.53ms, mfu 34.24%
iter 14950: loss 2.7948, time 2049.86ms, mfu 34.24%
iter 14960: loss 2.6191, time 2052.49ms, mfu 34.24%
iter 14970: loss 2.6581, time 2053.43ms, mfu 34.24%
iter 14980: loss 2.7812, time 2054.09ms, mfu 34.23%
iter 14990: loss 2.5137, time 2062.48ms, mfu 34.22%
step 15000: train loss 2.5469, val loss 2.6056
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 15000: loss 2.5324, time 9050.78ms, mfu 31.57%
iter 15010: loss 2.3895, time 2047.69ms, mfu 31.85%
iter 15020: loss 2.8188, time 2045.50ms, mfu 32.10%
iter 15030: loss 2.7421, time 2061.15ms, mfu 32.30%
iter 15040: loss 2.6073, time 2057.51ms, mfu 32.48%
iter 15050: loss 2.5282, time 2044.93ms, mfu 32.67%
iter 15060: loss 2.6307, time 2050.35ms, mfu 32.83%
iter 15070: loss 2.3521, time 2041.18ms, mfu 32.99%
iter 15080: loss 2.6511, time 2055.11ms, mfu 33.11%
iter 15090: loss 2.5863, time 2047.23ms, mfu 33.23%
iter 15100: loss 2.3589, time 2053.05ms, mfu 33.33%
iter 15110: loss 2.1197, time 2048.22ms, mfu 33.42%
iter 15120: loss 2.3586, time 2050.31ms, mfu 33.51%
iter 15130: loss 2.1581, time 2057.07ms, mfu 33.57%
iter 15140: loss 2.3600, time 2052.52ms, mfu 33.64%
iter 15150: loss 2.4316, time 2050.59ms, mfu 33.70%
iter 15160: loss 2.3814, time 2052.94ms, mfu 33.75%
iter 15170: loss 2.5108, time 2047.81ms, mfu 33.81%
iter 15180: loss 2.6854, time 2056.53ms, mfu 33.84%
iter 15190: loss 2.4573, time 2052.01ms, mfu 33.88%
iter 15200: loss 2.5847, time 2045.68ms, mfu 33.93%
iter 15210: loss 2.5517, time 2033.83ms, mfu 33.99%
iter 15220: loss 2.6235, time 2062.86ms, mfu 34.00%
iter 15230: loss 2.6293, time 2050.30ms, mfu 34.03%
iter 15240: loss 2.5032, time 2045.70ms, mfu 34.06%
iter 15250: loss 2.5022, time 2047.63ms, mfu 34.08%
iter 15260: loss 2.4612, time 2048.19ms, mfu 34.10%
iter 15270: loss 2.5695, time 2051.40ms, mfu 34.12%
iter 15280: loss 2.5566, time 2058.64ms, mfu 34.12%
iter 15290: loss 2.5804, time 2060.08ms, mfu 34.12%
iter 15300: loss 2.1813, time 2061.19ms, mfu 34.12%
iter 15310: loss 2.6232, time 2057.42ms, mfu 34.12%
iter 15320: loss 2.6546, time 2063.45ms, mfu 34.11%
iter 15330: loss 2.4853, time 2045.05ms, mfu 34.14%
iter 15340: loss 2.3950, time 2054.40ms, mfu 34.14%
iter 15350: loss 2.5932, time 2054.46ms, mfu 34.15%
iter 15360: loss 2.6560, time 2047.94ms, mfu 34.16%
iter 15370: loss 2.6596, time 2048.36ms, mfu 34.18%
iter 15380: loss 2.3597, time 2063.25ms, mfu 34.16%
iter 15390: loss 2.5653, time 2053.59ms, mfu 34.17%
iter 15400: loss 2.5883, time 2052.95ms, mfu 34.17%
iter 15410: loss 2.8086, time 2060.44ms, mfu 34.17%
iter 15420: loss 2.3315, time 2048.61ms, mfu 34.18%
iter 15430: loss 2.3299, time 2055.70ms, mfu 34.18%
iter 15440: loss 2.6439, time 2048.96ms, mfu 34.19%
iter 15450: loss 2.3613, time 2046.56ms, mfu 34.20%
iter 15460: loss 2.4900, time 2049.88ms, mfu 34.21%
iter 15470: loss 2.3410, time 2045.58ms, mfu 34.23%
iter 15480: loss 3.0208, time 2052.42ms, mfu 34.23%
iter 15490: loss 2.1029, time 2049.41ms, mfu 34.23%
iter 15500: loss 2.5235, time 2037.47ms, mfu 34.26%
iter 15510: loss 2.4992, time 2050.61ms, mfu 34.26%
iter 15520: loss 2.7876, time 2052.40ms, mfu 34.25%
iter 15530: loss 2.4952, time 2044.66ms, mfu 34.27%
iter 15540: loss 2.6472, time 2052.98ms, mfu 34.26%
iter 15550: loss 2.7036, time 2049.75ms, mfu 34.26%
iter 15560: loss 2.2222, time 2049.50ms, mfu 34.26%
iter 15570: loss 2.8318, time 2051.03ms, mfu 34.26%
iter 15580: loss 2.5763, time 2065.18ms, mfu 34.24%
iter 15590: loss 2.6690, time 2049.73ms, mfu 34.24%
iter 15600: loss 2.5214, time 2041.67ms, mfu 34.26%
iter 15610: loss 2.3846, time 2057.31ms, mfu 34.25%
iter 15620: loss 2.4082, time 2056.73ms, mfu 34.24%
iter 15630: loss 2.7194, time 2054.98ms, mfu 34.23%
iter 15640: loss 2.3063, time 2056.83ms, mfu 34.23%
iter 15650: loss 2.3059, time 2052.67ms, mfu 34.23%
iter 15660: loss 2.5217, time 2052.38ms, mfu 34.23%
iter 15670: loss 2.3697, time 2057.60ms, mfu 34.22%
iter 15680: loss 2.7433, time 2060.98ms, mfu 34.21%
iter 15690: loss 2.3187, time 2048.51ms, mfu 34.22%
iter 15700: loss 2.6030, time 2053.68ms, mfu 34.22%
iter 15710: loss 2.6533, time 2043.51ms, mfu 34.23%
iter 15720: loss 2.8647, time 2057.78ms, mfu 34.22%
iter 15730: loss 2.2071, time 2062.18ms, mfu 34.21%
iter 15740: loss 2.1813, time 2049.82ms, mfu 34.21%
iter 15750: loss 2.6227, time 2048.93ms, mfu 34.22%
iter 15760: loss 2.7295, time 2058.90ms, mfu 34.21%
iter 15770: loss 2.2505, time 2065.11ms, mfu 34.19%
iter 15780: loss 2.6953, time 2058.72ms, mfu 34.19%
iter 15790: loss 2.5725, time 2048.19ms, mfu 34.20%
iter 15800: loss 2.4904, time 2047.66ms, mfu 34.21%
iter 15810: loss 2.3056, time 2051.80ms, mfu 34.21%
iter 15820: loss 2.2874, time 2060.33ms, mfu 34.20%
iter 15830: loss 2.4724, time 2054.06ms, mfu 34.20%
iter 15840: loss 2.3941, time 2058.38ms, mfu 34.19%
iter 15850: loss 2.5230, time 2053.16ms, mfu 34.20%
iter 15860: loss 2.5530, time 2050.85ms, mfu 34.20%
iter 15870: loss 2.2016, time 2056.78ms, mfu 34.20%
iter 15880: loss 2.4216, time 2060.16ms, mfu 34.19%
iter 15890: loss 2.6163, time 2054.45ms, mfu 34.19%
iter 15900: loss 2.2982, time 2054.87ms, mfu 34.19%
iter 15910: loss 2.2620, time 2049.58ms, mfu 34.20%
iter 15920: loss 2.6602, time 2048.36ms, mfu 34.21%
iter 15930: loss 2.4288, time 2050.69ms, mfu 34.21%
iter 15940: loss 2.6208, time 2045.64ms, mfu 34.23%
iter 15950: loss 2.7025, time 2050.76ms, mfu 34.23%
iter 15960: loss 2.3091, time 2049.07ms, mfu 34.24%
iter 15970: loss 2.6048, time 2051.71ms, mfu 34.24%
iter 15980: loss 2.5497, time 2060.04ms, mfu 34.22%
iter 15990: loss 2.6321, time 2044.96ms, mfu 34.24%
step 16000: train loss 2.5108, val loss 2.5968
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 16000: loss 2.4993, time 9021.45ms, mfu 31.59%
iter 16010: loss 2.8461, time 2053.27ms, mfu 31.85%
iter 16020: loss 2.2652, time 2054.36ms, mfu 32.09%
iter 16030: loss 2.6752, time 2051.28ms, mfu 32.31%
iter 16040: loss 2.6377, time 2046.22ms, mfu 32.51%
iter 16050: loss 2.7656, time 2060.81ms, mfu 32.67%
iter 16060: loss 2.5904, time 2050.13ms, mfu 32.83%
iter 16070: loss 2.5344, time 2050.30ms, mfu 32.97%
iter 16080: loss 2.4649, time 2054.32ms, mfu 33.09%
iter 16090: loss 2.3970, time 2050.45ms, mfu 33.21%
iter 16100: loss 2.7187, time 2045.18ms, mfu 33.33%
iter 16110: loss 2.7439, time 2057.61ms, mfu 33.41%
iter 16120: loss 2.4232, time 2058.49ms, mfu 33.48%
iter 16130: loss 2.7324, time 2056.35ms, mfu 33.55%
iter 16140: loss 2.8502, time 2048.49ms, mfu 33.62%
iter 16150: loss 2.5180, time 2044.42ms, mfu 33.70%
iter 16160: loss 2.5804, time 2053.10ms, mfu 33.75%
iter 16170: loss 2.5761, time 2046.38ms, mfu 33.81%
iter 16180: loss 2.4343, time 2043.24ms, mfu 33.87%
iter 16190: loss 2.3515, time 2047.65ms, mfu 33.91%
iter 16200: loss 2.2920, time 2047.42ms, mfu 33.95%
iter 16210: loss 2.9033, time 2058.19ms, mfu 33.97%
iter 16220: loss 2.3696, time 2049.86ms, mfu 34.00%
iter 16230: loss 2.4057, time 2045.33ms, mfu 34.03%
iter 16240: loss 2.6864, time 2058.83ms, mfu 34.04%
iter 16250: loss 2.5651, time 2052.33ms, mfu 34.06%
iter 16260: loss 2.6559, time 2043.90ms, mfu 34.09%
iter 16270: loss 2.3451, time 2051.73ms, mfu 34.11%
iter 16280: loss 2.6381, time 2053.28ms, mfu 34.12%
iter 16290: loss 2.7059, time 2050.64ms, mfu 34.13%
iter 16300: loss 2.7849, time 2042.69ms, mfu 34.16%
iter 16310: loss 2.5320, time 2043.28ms, mfu 34.18%
iter 16320: loss 2.3190, time 2055.18ms, mfu 34.18%
iter 16330: loss 2.5818, time 2044.84ms, mfu 34.20%
iter 16340: loss 2.7277, time 2055.69ms, mfu 34.20%
iter 16350: loss 2.7512, time 2055.99ms, mfu 34.20%
iter 16360: loss 2.4944, time 2055.38ms, mfu 34.19%
iter 16370: loss 2.7119, time 2056.66ms, mfu 34.19%
iter 16380: loss 2.7095, time 2050.27ms, mfu 34.20%
iter 16390: loss 2.6410, time 2061.16ms, mfu 34.19%
iter 16400: loss 2.5953, time 2053.21ms, mfu 34.19%
iter 16410: loss 2.4840, time 2053.07ms, mfu 34.19%
iter 16420: loss 2.5949, time 2048.99ms, mfu 34.20%
iter 16430: loss 2.7740, time 2047.46ms, mfu 34.21%
iter 16440: loss 2.8673, time 2054.82ms, mfu 34.21%
iter 16450: loss 2.6198, time 2045.64ms, mfu 34.23%
iter 16460: loss 2.8139, time 2047.91ms, mfu 34.23%
iter 16470: loss 2.4340, time 2045.09ms, mfu 34.25%
iter 16480: loss 2.3495, time 2047.67ms, mfu 34.25%
iter 16490: loss 2.4162, time 2051.58ms, mfu 34.25%
iter 16500: loss 2.6461, time 2043.42ms, mfu 34.26%
iter 16510: loss 2.5307, time 2048.68ms, mfu 34.27%
iter 16520: loss 2.4841, time 2042.90ms, mfu 34.28%
iter 16530: loss 2.5116, time 2041.76ms, mfu 34.29%
iter 16540: loss 2.3272, time 2042.52ms, mfu 34.30%
iter 16550: loss 2.2739, time 2053.35ms, mfu 34.29%
iter 16560: loss 2.6956, time 2052.60ms, mfu 34.29%
iter 16570: loss 2.8273, time 2034.91ms, mfu 34.31%
iter 16580: loss 2.5294, time 2044.89ms, mfu 34.32%
iter 16590: loss 2.7706, time 2048.31ms, mfu 34.32%
iter 16600: loss 2.5464, time 2048.27ms, mfu 34.31%
iter 16610: loss 2.6033, time 2044.75ms, mfu 34.32%
iter 16620: loss 2.6606, time 2047.78ms, mfu 34.32%
iter 16630: loss 2.7447, time 2044.96ms, mfu 34.32%
iter 16640: loss 2.6759, time 2058.74ms, mfu 34.30%
iter 16650: loss 2.6810, time 2051.58ms, mfu 34.30%
iter 16660: loss 2.3872, time 2056.85ms, mfu 34.28%
iter 16670: loss 2.2233, time 2044.02ms, mfu 34.29%
iter 16680: loss 2.3473, time 2061.95ms, mfu 34.27%
iter 16690: loss 2.6947, time 2045.75ms, mfu 34.28%
iter 16700: loss 2.4462, time 2053.62ms, mfu 34.27%
iter 16710: loss 2.4761, time 2063.45ms, mfu 34.25%
iter 16720: loss 2.6136, time 2033.21ms, mfu 34.28%
iter 16730: loss 2.4234, time 2043.31ms, mfu 34.29%
iter 16740: loss 2.2819, time 2041.39ms, mfu 34.30%
iter 16750: loss 2.4658, time 2057.88ms, mfu 34.29%
iter 16760: loss 2.5526, time 2048.49ms, mfu 34.29%
iter 16770: loss 2.4635, time 2056.37ms, mfu 34.28%
iter 16780: loss 2.6369, time 2055.27ms, mfu 34.27%
iter 16790: loss 2.4946, time 2046.38ms, mfu 34.27%
iter 16800: loss 2.4449, time 2051.69ms, mfu 34.27%
iter 16810: loss 2.8056, time 2050.97ms, mfu 34.27%
iter 16820: loss 2.6816, time 2050.53ms, mfu 34.27%
iter 16830: loss 2.6181, time 2045.86ms, mfu 34.28%
iter 16840: loss 2.6604, time 2051.87ms, mfu 34.27%
iter 16850: loss 2.6324, time 2044.74ms, mfu 34.28%
iter 16860: loss 2.4470, time 2043.88ms, mfu 34.29%
iter 16870: loss 2.4891, time 2046.77ms, mfu 34.29%
iter 16880: loss 2.2405, time 2050.77ms, mfu 34.29%
iter 16890: loss 2.3522, time 2057.29ms, mfu 34.28%
iter 16900: loss 2.7712, time 2048.22ms, mfu 34.28%
iter 16910: loss 2.5849, time 2062.88ms, mfu 34.26%
iter 16920: loss 2.2291, time 2047.28ms, mfu 34.26%
iter 16930: loss 2.3130, time 2047.19ms, mfu 34.27%
iter 16940: loss 2.6223, time 2047.89ms, mfu 34.27%
iter 16950: loss 2.3072, time 2040.53ms, mfu 34.29%
iter 16960: loss 2.6404, time 2048.33ms, mfu 34.29%
iter 16970: loss 2.5679, time 2051.82ms, mfu 34.28%
iter 16980: loss 2.6659, time 2045.36ms, mfu 34.29%
iter 16990: loss 2.4610, time 2046.12ms, mfu 34.30%
step 17000: train loss 2.5031, val loss 2.6176
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 17000: loss 2.5025, time 9049.82ms, mfu 31.64%
iter 17010: loss 2.4492, time 2046.85ms, mfu 31.91%
iter 17020: loss 2.7681, time 2048.29ms, mfu 32.15%
iter 17030: loss 2.6648, time 2058.91ms, mfu 32.35%
iter 17040: loss 2.6501, time 2043.28ms, mfu 32.55%
iter 17050: loss 2.6715, time 2043.62ms, mfu 32.73%
iter 17060: loss 2.8489, time 2053.03ms, mfu 32.88%
iter 17070: loss 2.3205, time 2046.36ms, mfu 33.03%
iter 17080: loss 2.1749, time 2049.32ms, mfu 33.15%
iter 17090: loss 2.8133, time 2052.82ms, mfu 33.26%
iter 17100: loss 2.6915, time 2047.16ms, mfu 33.37%
iter 17110: loss 2.7509, time 2049.01ms, mfu 33.46%
iter 17120: loss 2.8725, time 2061.15ms, mfu 33.52%
iter 17130: loss 2.7440, time 2050.09ms, mfu 33.60%
iter 17140: loss 2.5501, time 2052.00ms, mfu 33.66%
iter 17150: loss 2.5734, time 2055.80ms, mfu 33.71%
iter 17160: loss 2.3107, time 2051.93ms, mfu 33.76%
iter 17170: loss 2.6111, time 2065.28ms, mfu 33.79%
iter 17180: loss 2.5572, time 2047.61ms, mfu 33.84%
iter 17190: loss 2.2599, time 2059.02ms, mfu 33.87%
iter 17200: loss 2.5295, time 2059.74ms, mfu 33.89%
iter 17210: loss 2.6658, time 2049.26ms, mfu 33.93%
iter 17220: loss 2.3482, time 2055.35ms, mfu 33.96%
iter 17230: loss 2.3992, time 2046.48ms, mfu 34.00%
iter 17240: loss 2.7894, time 2057.81ms, mfu 34.01%
iter 17250: loss 2.6257, time 2055.51ms, mfu 34.03%
iter 17260: loss 2.3506, time 2067.69ms, mfu 34.02%
iter 17270: loss 2.4402, time 2069.62ms, mfu 34.01%
iter 17280: loss 2.5134, time 2048.38ms, mfu 34.04%
iter 17290: loss 2.5276, time 2041.84ms, mfu 34.08%
iter 17300: loss 2.2971, time 2048.46ms, mfu 34.10%
iter 17310: loss 2.6004, time 2064.06ms, mfu 34.10%
iter 17320: loss 2.5300, time 2038.85ms, mfu 34.13%
iter 17330: loss 2.4263, time 2051.50ms, mfu 34.14%
iter 17340: loss 2.3857, time 2048.43ms, mfu 34.16%
iter 17350: loss 2.0726, time 2052.51ms, mfu 34.17%
iter 17360: loss 2.4436, time 2057.21ms, mfu 34.16%
iter 17370: loss 2.5092, time 2059.29ms, mfu 34.16%
iter 17380: loss 2.7875, time 2053.31ms, mfu 34.17%
iter 17390: loss 2.6140, time 2044.99ms, mfu 34.18%
iter 17400: loss 2.6837, time 2034.19ms, mfu 34.22%
iter 17410: loss 2.4962, time 2046.45ms, mfu 34.23%
iter 17420: loss 2.4321, time 2043.09ms, mfu 34.25%
iter 17430: loss 2.7111, time 2051.59ms, mfu 34.25%
iter 17440: loss 2.5206, time 2040.64ms, mfu 34.27%
iter 17450: loss 2.5912, time 2037.67ms, mfu 34.29%
iter 17460: loss 2.7268, time 2053.88ms, mfu 34.28%
iter 17470: loss 2.3265, time 2035.62ms, mfu 34.30%
iter 17480: loss 2.6156, time 2048.18ms, mfu 34.30%
iter 17490: loss 2.4059, time 2049.73ms, mfu 34.30%
iter 17500: loss 2.2469, time 2057.49ms, mfu 34.28%
iter 17510: loss 2.5060, time 2052.37ms, mfu 34.28%
iter 17520: loss 2.7370, time 2059.90ms, mfu 34.26%
iter 17530: loss 2.3551, time 2055.96ms, mfu 34.25%
iter 17540: loss 2.4487, time 2036.45ms, mfu 34.28%
iter 17550: loss 2.8264, time 2050.90ms, mfu 34.28%
iter 17560: loss 2.5364, time 2040.23ms, mfu 34.29%
iter 17570: loss 2.5814, time 2036.22ms, mfu 34.31%
iter 17580: loss 2.5693, time 2041.48ms, mfu 34.32%
iter 17590: loss 2.6920, time 2047.07ms, mfu 34.32%
iter 17600: loss 2.6617, time 2039.36ms, mfu 34.34%
iter 17610: loss 2.3513, time 2029.56ms, mfu 34.36%
iter 17620: loss 2.4541, time 2029.78ms, mfu 34.39%
iter 17630: loss 2.5524, time 2027.45ms, mfu 34.42%
iter 17640: loss 2.5926, time 2043.88ms, mfu 34.41%
iter 17650: loss 2.5012, time 2049.50ms, mfu 34.40%
iter 17660: loss 2.7000, time 2044.81ms, mfu 34.39%
iter 17670: loss 2.4874, time 2045.13ms, mfu 34.39%
iter 17680: loss 2.3603, time 2053.44ms, mfu 34.37%
iter 17690: loss 2.6509, time 2047.91ms, mfu 34.37%
iter 17700: loss 2.2015, time 2057.21ms, mfu 34.34%
iter 17710: loss 2.4685, time 2046.04ms, mfu 34.34%
iter 17720: loss 2.4874, time 2048.66ms, mfu 34.34%
iter 17730: loss 2.7454, time 2047.60ms, mfu 34.34%
iter 17740: loss 2.6389, time 2048.97ms, mfu 34.33%
iter 17750: loss 2.6719, time 2032.72ms, mfu 34.36%
iter 17760: loss 2.3490, time 2048.04ms, mfu 34.35%
iter 17770: loss 2.5283, time 2044.39ms, mfu 34.35%
iter 17780: loss 2.6234, time 2049.51ms, mfu 34.34%
iter 17790: loss 2.9732, time 2043.98ms, mfu 34.35%
iter 17800: loss 2.2693, time 2041.98ms, mfu 34.35%
iter 17810: loss 2.4541, time 2032.48ms, mfu 34.37%
iter 17820: loss 2.5366, time 2041.11ms, mfu 34.38%
iter 17830: loss 2.4183, time 2050.35ms, mfu 34.37%
iter 17840: loss 2.5554, time 2039.38ms, mfu 34.38%
iter 17850: loss 2.6485, time 2060.44ms, mfu 34.35%
iter 17860: loss 2.4161, time 2033.29ms, mfu 34.37%
iter 17870: loss 2.5290, time 2053.53ms, mfu 34.35%
iter 17880: loss 2.5156, time 2052.25ms, mfu 34.34%
iter 17890: loss 2.4510, time 2046.29ms, mfu 34.34%
iter 17900: loss 2.6007, time 2051.44ms, mfu 34.33%
iter 17910: loss 2.4240, time 2049.71ms, mfu 34.33%
iter 17920: loss 2.7542, time 2053.78ms, mfu 34.31%
iter 17930: loss 2.0915, time 2054.13ms, mfu 34.30%
iter 17940: loss 2.5501, time 2037.62ms, mfu 34.32%
iter 17950: loss 2.6879, time 2049.09ms, mfu 34.32%
iter 17960: loss 2.5390, time 2050.53ms, mfu 34.31%
iter 17970: loss 2.1091, time 2037.76ms, mfu 34.33%
iter 17980: loss 2.6417, time 2054.78ms, mfu 34.32%
iter 17990: loss 2.4377, time 2049.14ms, mfu 34.31%
step 18000: train loss 2.5104, val loss 2.5790
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 18000: loss 2.6239, time 9213.51ms, mfu 31.64%
iter 18010: loss 2.5675, time 2042.40ms, mfu 31.92%
iter 18020: loss 2.2252, time 2040.96ms, mfu 32.17%
iter 18030: loss 2.2279, time 2045.03ms, mfu 32.39%
iter 18040: loss 2.5594, time 2035.03ms, mfu 32.60%
iter 18050: loss 2.1532, time 2031.29ms, mfu 32.80%
iter 18060: loss 2.7848, time 2048.00ms, mfu 32.95%
iter 18070: loss 2.4558, time 2055.00ms, mfu 33.07%
iter 18080: loss 2.5108, time 2041.13ms, mfu 33.21%
iter 18090: loss 2.5548, time 2050.17ms, mfu 33.32%
iter 18100: loss 2.6184, time 2035.04ms, mfu 33.44%
iter 18110: loss 2.6283, time 2048.13ms, mfu 33.52%
iter 18120: loss 2.3918, time 2058.51ms, mfu 33.58%
iter 18130: loss 2.7367, time 2038.65ms, mfu 33.67%
iter 18140: loss 2.4815, time 2032.35ms, mfu 33.76%
iter 18150: loss 2.4789, time 2038.03ms, mfu 33.83%
iter 18160: loss 2.3502, time 2043.69ms, mfu 33.89%
iter 18170: loss 2.5408, time 2061.57ms, mfu 33.91%
iter 18180: loss 2.6044, time 2051.94ms, mfu 33.94%
iter 18190: loss 2.4899, time 2030.59ms, mfu 34.01%
iter 18200: loss 2.4462, time 2037.56ms, mfu 34.05%
iter 18210: loss 2.3983, time 2049.02ms, mfu 34.08%
iter 18220: loss 2.4561, time 2051.31ms, mfu 34.09%
iter 18230: loss 2.5420, time 2044.02ms, mfu 34.12%
iter 18240: loss 2.6179, time 2023.95ms, mfu 34.18%
iter 18250: loss 2.6346, time 2048.23ms, mfu 34.19%
iter 18260: loss 2.3404, time 2047.15ms, mfu 34.21%
iter 18270: loss 2.4838, time 2041.89ms, mfu 34.23%
iter 18280: loss 2.4960, time 2044.54ms, mfu 34.24%
iter 18290: loss 2.4660, time 2045.21ms, mfu 34.25%
iter 18300: loss 2.6292, time 2043.23ms, mfu 34.26%
iter 18310: loss 2.3600, time 2042.28ms, mfu 34.28%
iter 18320: loss 2.6103, time 2048.39ms, mfu 34.28%
iter 18330: loss 2.9168, time 2040.72ms, mfu 34.30%
iter 18340: loss 2.3151, time 2048.41ms, mfu 34.30%
iter 18350: loss 2.5969, time 2049.46ms, mfu 34.29%
iter 18360: loss 2.4688, time 2045.08ms, mfu 34.30%
iter 18370: loss 2.3587, time 2025.80ms, mfu 34.34%
iter 18380: loss 2.5908, time 2035.95ms, mfu 34.36%
iter 18390: loss 2.6701, time 2038.69ms, mfu 34.37%
iter 18400: loss 2.3106, time 2020.63ms, mfu 34.41%
iter 18410: loss 2.6747, time 2047.00ms, mfu 34.40%
iter 18420: loss 2.3913, time 2030.54ms, mfu 34.42%
iter 18430: loss 2.6270, time 2035.82ms, mfu 34.43%
iter 18440: loss 1.9692, time 2030.70ms, mfu 34.44%
iter 18450: loss 2.6884, time 2034.75ms, mfu 34.45%
iter 18460: loss 2.3653, time 2022.74ms, mfu 34.48%
iter 18470: loss 2.6160, time 2046.20ms, mfu 34.47%
iter 18480: loss 2.4401, time 2029.32ms, mfu 34.48%
iter 18490: loss 2.4347, time 2050.40ms, mfu 34.46%
iter 18500: loss 2.3109, time 2027.22ms, mfu 34.48%
iter 18510: loss 2.6085, time 2029.05ms, mfu 34.49%
iter 18520: loss 2.5804, time 2045.09ms, mfu 34.48%
iter 18530: loss 2.5533, time 2041.40ms, mfu 34.47%
iter 18540: loss 2.6045, time 2033.03ms, mfu 34.48%
iter 18550: loss 2.5358, time 2041.20ms, mfu 34.48%
iter 18560: loss 2.5098, time 2028.49ms, mfu 34.49%
iter 18570: loss 2.4624, time 2039.25ms, mfu 34.49%
iter 18580: loss 2.5906, time 2034.30ms, mfu 34.49%
iter 18590: loss 2.4067, time 2042.36ms, mfu 34.48%
iter 18600: loss 2.5380, time 2042.66ms, mfu 34.48%
iter 18610: loss 2.4452, time 2032.47ms, mfu 34.48%
iter 18620: loss 2.4393, time 2056.12ms, mfu 34.45%
iter 18630: loss 2.2584, time 2045.95ms, mfu 34.44%
iter 18640: loss 2.6107, time 2045.91ms, mfu 34.43%
iter 18650: loss 2.7307, time 2042.87ms, mfu 34.43%
iter 18660: loss 2.5931, time 2036.80ms, mfu 34.43%
iter 18670: loss 2.3195, time 2046.09ms, mfu 34.42%
iter 18680: loss 2.4437, time 2052.15ms, mfu 34.41%
iter 18690: loss 2.4574, time 2049.43ms, mfu 34.39%
iter 18700: loss 2.4084, time 2036.04ms, mfu 34.40%
iter 18710: loss 2.8515, time 2032.45ms, mfu 34.42%
iter 18720: loss 2.5933, time 2023.79ms, mfu 34.45%
iter 18730: loss 2.4207, time 2053.02ms, mfu 34.43%
iter 18740: loss 2.4620, time 2046.36ms, mfu 34.42%
iter 18750: loss 2.1568, time 2046.31ms, mfu 34.41%
iter 18760: loss 2.5149, time 2039.52ms, mfu 34.41%
iter 18770: loss 2.4324, time 2050.11ms, mfu 34.40%
iter 18780: loss 2.4186, time 2045.70ms, mfu 34.39%
iter 18790: loss 2.5736, time 2055.88ms, mfu 34.37%
iter 18800: loss 2.6184, time 2041.83ms, mfu 34.38%
iter 18810: loss 2.2102, time 2046.84ms, mfu 34.37%
iter 18820: loss 2.4944, time 2051.02ms, mfu 34.36%
iter 18830: loss 2.3915, time 2050.44ms, mfu 34.35%
iter 18840: loss 2.5534, time 2048.20ms, mfu 34.34%
iter 18850: loss 2.8659, time 2051.50ms, mfu 34.34%
iter 18860: loss 2.3966, time 2045.36ms, mfu 34.34%
iter 18870: loss 2.6404, time 2042.98ms, mfu 34.34%
iter 18880: loss 2.7463, time 2040.94ms, mfu 34.35%
iter 18890: loss 2.7846, time 2046.15ms, mfu 34.35%
iter 18900: loss 2.3818, time 2048.30ms, mfu 34.34%
iter 18910: loss 2.5840, time 2053.99ms, mfu 34.33%
iter 18920: loss 2.5176, time 2048.05ms, mfu 34.33%
iter 18930: loss 2.7594, time 2043.07ms, mfu 34.33%
iter 18940: loss 2.4353, time 2034.33ms, mfu 34.35%
iter 18950: loss 2.5070, time 2048.95ms, mfu 34.35%
iter 18960: loss 2.3331, time 2042.46ms, mfu 34.35%
iter 18970: loss 2.2222, time 2039.34ms, mfu 34.36%
iter 18980: loss 2.4629, time 2059.51ms, mfu 34.34%
iter 18990: loss 2.8133, time 2073.84ms, mfu 34.29%
step 19000: train loss 2.4960, val loss 2.5795
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 19000: loss 2.6946, time 9157.05ms, mfu 31.63%
iter 19010: loss 2.6494, time 2067.11ms, mfu 31.87%
iter 19020: loss 2.2805, time 2074.73ms, mfu 32.07%
iter 19030: loss 2.6235, time 2094.20ms, mfu 32.21%
iter 19040: loss 2.5598, time 2070.01ms, mfu 32.39%
iter 19050: loss 2.2728, time 2073.01ms, mfu 32.54%
iter 19060: loss 2.3224, time 2066.86ms, mfu 32.68%
iter 19070: loss 2.2549, time 2048.33ms, mfu 32.84%
iter 19080: loss 2.5840, time 2075.74ms, mfu 32.94%
iter 19090: loss 2.3142, time 2059.30ms, mfu 33.06%
iter 19100: loss 2.4708, time 2078.70ms, mfu 33.14%
iter 19110: loss 2.5081, time 2069.21ms, mfu 33.22%
iter 19120: loss 2.6420, time 2067.26ms, mfu 33.29%
iter 19130: loss 2.4997, time 2087.81ms, mfu 33.33%
iter 19140: loss 2.5713, time 2106.13ms, mfu 33.33%
iter 19150: loss 2.3247, time 2104.61ms, mfu 33.34%
iter 19160: loss 2.5173, time 2090.53ms, mfu 33.36%
iter 19170: loss 2.1094, time 2096.41ms, mfu 33.38%
iter 19180: loss 2.5944, time 2114.51ms, mfu 33.36%
iter 19190: loss 2.4135, time 2113.27ms, mfu 33.35%
iter 19200: loss 2.2291, time 2077.70ms, mfu 33.40%
iter 19210: loss 2.3812, time 2088.56ms, mfu 33.42%
iter 19220: loss 2.3152, time 2115.48ms, mfu 33.40%
iter 19230: loss 2.6011, time 2098.44ms, mfu 33.41%
iter 19240: loss 2.1657, time 2077.72ms, mfu 33.45%
iter 19250: loss 2.3054, time 2076.12ms, mfu 33.49%
iter 19260: loss 2.6090, time 2105.13ms, mfu 33.48%
iter 19270: loss 2.2419, time 2105.72ms, mfu 33.47%
iter 19280: loss 2.2283, time 2125.57ms, mfu 33.43%
iter 19290: loss 2.3726, time 2105.62ms, mfu 33.42%
iter 19300: loss 2.4235, time 2105.86ms, mfu 33.41%
iter 19310: loss 2.7795, time 2076.56ms, mfu 33.46%
iter 19320: loss 2.7724, time 2073.76ms, mfu 33.50%
iter 19330: loss 2.2650, time 2079.17ms, mfu 33.53%
iter 19340: loss 2.6037, time 2106.95ms, mfu 33.51%
iter 19350: loss 2.4660, time 2120.26ms, mfu 33.47%
iter 19360: loss 2.5757, time 2101.19ms, mfu 33.47%
iter 19370: loss 2.2094, time 2082.51ms, mfu 33.50%
iter 19380: loss 2.6828, time 2124.18ms, mfu 33.45%
iter 19390: loss 2.7281, time 2109.57ms, mfu 33.44%
iter 19400: loss 2.4323, time 2078.23ms, mfu 33.48%
iter 19410: loss 2.2160, time 2111.44ms, mfu 33.46%
iter 19420: loss 2.3757, time 2097.52ms, mfu 33.46%
iter 19430: loss 2.6591, time 2083.16ms, mfu 33.49%
iter 19440: loss 2.5653, time 2119.07ms, mfu 33.45%
iter 19450: loss 2.3441, time 2072.10ms, mfu 33.50%
iter 19460: loss 2.6150, time 2081.46ms, mfu 33.52%
iter 19470: loss 2.4495, time 2105.86ms, mfu 33.51%
iter 19480: loss 2.1716, time 2121.41ms, mfu 33.47%
iter 19490: loss 2.7254, time 2068.88ms, mfu 33.52%
iter 19500: loss 2.5382, time 2086.21ms, mfu 33.53%
iter 19510: loss 2.3775, time 2092.54ms, mfu 33.54%
iter 19520: loss 2.4431, time 2124.95ms, mfu 33.49%
iter 19530: loss 2.3789, time 2102.55ms, mfu 33.48%
iter 19540: loss 2.5554, time 2091.72ms, mfu 33.49%
iter 19550: loss 2.2595, time 2123.87ms, mfu 33.45%
iter 19560: loss 2.3262, time 2086.51ms, mfu 33.47%
iter 19570: loss 2.2374, time 2075.02ms, mfu 33.51%
iter 19580: loss 2.4286, time 2079.22ms, mfu 33.54%
iter 19590: loss 2.4578, time 2078.14ms, mfu 33.57%
iter 19600: loss 2.4941, time 2082.04ms, mfu 33.59%
iter 19610: loss 2.2544, time 2109.73ms, mfu 33.56%
iter 19620: loss 2.9105, time 2125.76ms, mfu 33.51%
iter 19630: loss 2.7088, time 2124.45ms, mfu 33.46%
iter 19640: loss 2.4173, time 2110.34ms, mfu 33.45%
iter 19650: loss 2.5965, time 2122.06ms, mfu 33.41%
iter 19660: loss 2.7698, time 2095.87ms, mfu 33.42%
iter 19670: loss 2.6804, time 2106.50ms, mfu 33.42%
iter 19680: loss 2.5901, time 2107.80ms, mfu 33.41%
iter 19690: loss 2.2906, time 2097.26ms, mfu 33.42%
iter 19700: loss 2.5577, time 2069.51ms, mfu 33.47%
iter 19710: loss 2.8713, time 2073.22ms, mfu 33.51%
iter 19720: loss 2.6003, time 2070.65ms, mfu 33.55%
iter 19730: loss 2.4759, time 2066.23ms, mfu 33.60%
iter 19740: loss 2.7348, time 2056.79ms, mfu 33.65%
iter 19750: loss 2.3621, time 2056.41ms, mfu 33.71%
iter 19760: loss 2.6156, time 2045.84ms, mfu 33.77%
iter 19770: loss 2.3359, time 2040.40ms, mfu 33.84%
iter 19780: loss 2.5365, time 2083.39ms, mfu 33.82%
iter 19790: loss 2.7236, time 2097.98ms, mfu 33.79%
iter 19800: loss 2.3659, time 2081.01ms, mfu 33.79%
iter 19810: loss 2.0193, time 2075.69ms, mfu 33.79%
iter 19820: loss 2.4093, time 2061.28ms, mfu 33.82%
iter 19830: loss 2.5405, time 2071.63ms, mfu 33.83%
iter 19840: loss 2.2666, time 2059.57ms, mfu 33.86%
iter 19850: loss 2.1745, time 2078.81ms, mfu 33.85%
iter 19860: loss 2.7285, time 2082.36ms, mfu 33.84%
iter 19870: loss 2.5300, time 2077.24ms, mfu 33.84%
iter 19880: loss 2.5143, time 2110.61ms, mfu 33.79%
iter 19890: loss 2.4780, time 2128.99ms, mfu 33.71%
iter 19900: loss 2.7815, time 2108.06ms, mfu 33.67%
iter 19910: loss 2.9184, time 2086.88ms, mfu 33.67%
iter 19920: loss 2.3199, time 2097.57ms, mfu 33.65%
iter 19930: loss 2.3362, time 2079.79ms, mfu 33.66%
iter 19940: loss 2.4887, time 2091.47ms, mfu 33.66%
iter 19950: loss 2.4897, time 2113.30ms, mfu 33.62%
iter 19960: loss 2.7871, time 2098.47ms, mfu 33.60%
iter 19970: loss 2.4831, time 2113.39ms, mfu 33.57%
iter 19980: loss 2.3700, time 2090.87ms, mfu 33.57%
iter 19990: loss 2.5672, time 2111.66ms, mfu 33.54%
step 20000: train loss 2.4705, val loss 2.5600
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 20000: loss 2.3049, time 9127.17ms, mfu 30.96%
iter 20010: loss 2.5311, time 2082.86ms, mfu 31.23%
iter 20020: loss 2.7243, time 2070.26ms, mfu 31.50%
iter 20030: loss 2.7753, time 2078.03ms, mfu 31.73%
iter 20040: loss 2.1705, time 2072.72ms, mfu 31.95%
iter 20050: loss 2.4184, time 2083.52ms, mfu 32.13%
iter 20060: loss 2.4823, time 2109.43ms, mfu 32.25%
iter 20070: loss 2.3417, time 2107.97ms, mfu 32.35%
iter 20080: loss 2.4189, time 2099.63ms, mfu 32.46%
iter 20090: loss 2.5853, time 2092.77ms, mfu 32.58%
iter 20100: loss 2.5596, time 2096.81ms, mfu 32.67%
iter 20110: loss 2.6586, time 2126.40ms, mfu 32.71%
iter 20120: loss 2.6058, time 2117.85ms, mfu 32.75%
iter 20130: loss 2.2927, time 2080.97ms, mfu 32.85%
iter 20140: loss 2.8337, time 2070.70ms, mfu 32.96%
iter 20150: loss 2.4707, time 2109.90ms, mfu 33.00%
iter 20160: loss 2.6414, time 2122.88ms, mfu 33.01%
iter 20170: loss 2.3587, time 2113.06ms, mfu 33.03%
iter 20180: loss 2.5295, time 2085.31ms, mfu 33.10%
iter 20190: loss 2.5012, time 2062.77ms, mfu 33.19%
iter 20200: loss 2.4359, time 2088.61ms, mfu 33.24%
iter 20210: loss 2.5483, time 2093.39ms, mfu 33.27%
iter 20220: loss 2.6374, time 2075.75ms, mfu 33.33%
iter 20230: loss 2.0785, time 2072.26ms, mfu 33.38%
iter 20240: loss 1.9792, time 2073.14ms, mfu 33.44%
iter 20250: loss 2.4951, time 2077.57ms, mfu 33.47%
iter 20260: loss 2.3644, time 2073.81ms, mfu 33.51%
iter 20270: loss 2.5195, time 2048.74ms, mfu 33.59%
iter 20280: loss 2.7400, time 2078.72ms, mfu 33.61%
iter 20290: loss 2.7078, time 2056.61ms, mfu 33.67%
iter 20300: loss 2.3227, time 2051.42ms, mfu 33.73%
iter 20310: loss 2.5757, time 2076.06ms, mfu 33.74%
iter 20320: loss 2.6827, time 2079.06ms, mfu 33.74%
iter 20330: loss 2.5830, time 2090.59ms, mfu 33.73%
iter 20340: loss 2.6910, time 2079.09ms, mfu 33.74%
iter 20350: loss 2.4816, time 2069.08ms, mfu 33.76%
iter 20360: loss 2.4796, time 2085.73ms, mfu 33.75%
iter 20370: loss 2.5446, time 2064.10ms, mfu 33.78%
iter 20380: loss 2.3412, time 2065.52ms, mfu 33.80%
iter 20390: loss 2.5178, time 2047.76ms, mfu 33.85%
iter 20400: loss 2.7551, time 2061.34ms, mfu 33.88%
iter 20410: loss 2.5763, time 2073.31ms, mfu 33.88%
iter 20420: loss 2.7298, time 2076.27ms, mfu 33.87%
iter 20430: loss 2.3009, time 2065.80ms, mfu 33.89%
iter 20440: loss 2.3277, time 2098.70ms, mfu 33.85%
iter 20450: loss 2.1856, time 2098.19ms, mfu 33.81%
iter 20460: loss 2.4148, time 2092.47ms, mfu 33.79%
iter 20470: loss 2.8183, time 2112.99ms, mfu 33.73%
iter 20480: loss 2.6513, time 2095.29ms, mfu 33.71%
iter 20490: loss 2.5901, time 2095.69ms, mfu 33.69%
iter 20500: loss 2.3461, time 2101.42ms, mfu 33.67%
iter 20510: loss 2.0833, time 2112.99ms, mfu 33.63%
iter 20520: loss 2.2939, time 2115.17ms, mfu 33.59%
iter 20530: loss 2.5580, time 2096.05ms, mfu 33.58%
iter 20540: loss 2.7201, time 2099.78ms, mfu 33.57%
iter 20550: loss 2.4388, time 2110.35ms, mfu 33.54%
iter 20560: loss 2.5888, time 2111.72ms, mfu 33.51%
iter 20570: loss 2.0955, time 2076.86ms, mfu 33.54%
iter 20580: loss 2.6397, time 2109.87ms, mfu 33.52%
iter 20590: loss 2.6569, time 2084.79ms, mfu 33.54%
iter 20600: loss 2.1208, time 2110.78ms, mfu 33.51%
iter 20610: loss 2.3751, time 2116.68ms, mfu 33.48%
iter 20620: loss 2.4391, time 2078.04ms, mfu 33.51%
iter 20630: loss 2.3905, time 2083.16ms, mfu 33.53%
iter 20640: loss 2.6307, time 2086.27ms, mfu 33.55%
iter 20650: loss 2.5851, time 2084.46ms, mfu 33.56%
iter 20660: loss 2.6328, time 2118.20ms, mfu 33.53%
iter 20670: loss 2.7644, time 2079.58ms, mfu 33.55%
iter 20680: loss 2.4270, time 2067.97ms, mfu 33.59%
iter 20690: loss 2.5857, time 2076.95ms, mfu 33.62%
iter 20700: loss 2.5030, time 2092.26ms, mfu 33.61%
iter 20710: loss 2.2983, time 2079.15ms, mfu 33.63%
iter 20720: loss 2.7840, time 2073.03ms, mfu 33.66%
iter 20730: loss 2.7703, time 2072.05ms, mfu 33.68%
iter 20740: loss 2.3082, time 2115.14ms, mfu 33.64%
iter 20750: loss 2.5007, time 2090.94ms, mfu 33.63%
iter 20760: loss 2.3180, time 2090.88ms, mfu 33.63%
iter 20770: loss 2.3490, time 2074.07ms, mfu 33.65%
iter 20780: loss 2.7984, time 2130.72ms, mfu 33.59%
iter 20790: loss 2.5980, time 2119.87ms, mfu 33.54%
iter 20800: loss 2.3243, time 2106.92ms, mfu 33.52%
iter 20810: loss 2.6890, time 2102.07ms, mfu 33.51%
iter 20820: loss 2.3951, time 2086.62ms, mfu 33.53%
iter 20830: loss 2.3490, time 2079.88ms, mfu 33.55%
iter 20840: loss 2.1059, time 2085.54ms, mfu 33.57%
iter 20850: loss 2.5789, time 2090.74ms, mfu 33.57%
iter 20860: loss 2.5478, time 2110.53ms, mfu 33.54%
iter 20870: loss 2.7613, time 2082.73ms, mfu 33.56%
iter 20880: loss 2.5769, time 2107.56ms, mfu 33.54%
iter 20890: loss 2.2841, time 2092.34ms, mfu 33.54%
iter 20900: loss 2.6098, time 2091.61ms, mfu 33.55%
iter 20910: loss 2.9469, time 2090.03ms, mfu 33.55%
iter 20920: loss 2.4519, time 2090.17ms, mfu 33.56%
iter 20930: loss 2.1891, time 2088.59ms, mfu 33.57%
iter 20940: loss 2.4173, time 2101.59ms, mfu 33.55%
iter 20950: loss 2.5453, time 2101.07ms, mfu 33.54%
iter 20960: loss 2.3601, time 2097.66ms, mfu 33.54%
iter 20970: loss 2.4965, time 2076.41ms, mfu 33.57%
iter 20980: loss 2.3732, time 2082.07ms, mfu 33.59%
iter 20990: loss 2.5255, time 2081.84ms, mfu 33.60%
step 21000: train loss 2.4877, val loss 2.5653
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 21000: loss 2.3033, time 9065.20ms, mfu 31.02%
iter 21010: loss 2.7702, time 2081.10ms, mfu 31.29%
iter 21020: loss 2.2506, time 2122.56ms, mfu 31.47%
iter 21030: loss 2.4540, time 2108.63ms, mfu 31.66%
iter 21040: loss 2.5078, time 2084.83ms, mfu 31.86%
iter 21050: loss 2.4593, time 2083.02ms, mfu 32.05%
iter 21060: loss 2.2905, time 2111.12ms, mfu 32.17%
iter 21070: loss 2.6290, time 2090.70ms, mfu 32.31%
iter 21080: loss 2.4797, time 2077.13ms, mfu 32.47%
iter 21090: loss 2.3769, time 2074.53ms, mfu 32.61%
iter 21100: loss 2.4844, time 2087.15ms, mfu 32.71%
iter 21110: loss 2.1747, time 2074.18ms, mfu 32.83%
iter 21120: loss 2.5464, time 2059.90ms, mfu 32.96%
iter 21130: loss 2.6820, time 2064.25ms, mfu 33.06%
iter 21140: loss 2.6739, time 2054.59ms, mfu 33.18%
iter 21150: loss 2.6393, time 2069.52ms, mfu 33.25%
iter 21160: loss 2.7563, time 2065.32ms, mfu 33.33%
iter 21170: loss 2.2612, time 2048.20ms, mfu 33.43%
iter 21180: loss 2.5129, time 2059.80ms, mfu 33.50%
iter 21190: loss 2.6525, time 2048.66ms, mfu 33.58%
iter 21200: loss 2.4726, time 2044.12ms, mfu 33.66%
iter 21210: loss 2.5289, time 2064.42ms, mfu 33.69%
iter 21220: loss 2.7360, time 2047.37ms, mfu 33.76%
iter 21230: loss 2.4160, time 2047.87ms, mfu 33.81%
iter 21240: loss 2.3951, time 2065.11ms, mfu 33.83%
iter 21250: loss 2.5662, time 2046.70ms, mfu 33.88%
iter 21260: loss 2.3221, time 2054.57ms, mfu 33.91%
iter 21270: loss 2.5679, time 2052.95ms, mfu 33.94%
iter 21280: loss 2.6328, time 2045.00ms, mfu 33.98%
iter 21290: loss 2.6171, time 2042.36ms, mfu 34.03%
iter 21300: loss 1.9673, time 2047.65ms, mfu 34.05%
iter 21310: loss 2.4585, time 2055.41ms, mfu 34.07%
iter 21320: loss 2.2799, time 2057.06ms, mfu 34.08%
iter 21330: loss 2.3813, time 2045.73ms, mfu 34.10%
iter 21340: loss 2.5388, time 2043.79ms, mfu 34.13%
iter 21350: loss 2.4416, time 2049.61ms, mfu 34.15%
iter 21360: loss 2.2244, time 2053.65ms, mfu 34.15%
iter 21370: loss 2.4727, time 2044.09ms, mfu 34.17%
iter 21380: loss 2.4452, time 2045.23ms, mfu 34.19%
iter 21390: loss 2.3373, time 2046.00ms, mfu 34.21%
iter 21400: loss 2.5141, time 2048.81ms, mfu 34.21%
iter 21410: loss 2.5042, time 2054.62ms, mfu 34.21%
iter 21420: loss 2.3967, time 2048.00ms, mfu 34.22%
iter 21430: loss 2.3168, time 2046.91ms, mfu 34.23%
iter 21440: loss 2.5505, time 2044.47ms, mfu 34.25%
iter 21450: loss 2.6172, time 2045.61ms, mfu 34.26%
iter 21460: loss 2.1205, time 2042.59ms, mfu 34.27%
iter 21470: loss 2.3121, time 2047.03ms, mfu 34.28%
iter 21480: loss 2.6566, time 2039.52ms, mfu 34.29%
iter 21490: loss 2.3667, time 2051.38ms, mfu 34.29%
iter 21500: loss 2.5575, time 2044.32ms, mfu 34.30%
iter 21510: loss 2.4720, time 2046.98ms, mfu 34.30%
iter 21520: loss 2.5616, time 2030.81ms, mfu 34.33%
iter 21530: loss 2.3708, time 2025.87ms, mfu 34.36%
iter 21540: loss 2.6299, time 2053.08ms, mfu 34.35%
iter 21550: loss 2.3262, time 2058.04ms, mfu 34.33%
iter 21560: loss 2.3968, time 2025.53ms, mfu 34.36%
iter 21570: loss 2.5163, time 2037.70ms, mfu 34.38%
iter 21580: loss 2.3235, time 2044.13ms, mfu 34.38%
iter 21590: loss 2.5240, time 2062.71ms, mfu 34.34%
iter 21600: loss 2.6779, time 2048.24ms, mfu 34.34%
iter 21610: loss 2.4958, time 2041.75ms, mfu 34.35%
iter 21620: loss 2.1344, time 2048.44ms, mfu 34.34%
iter 21630: loss 2.7206, time 2035.28ms, mfu 34.36%
iter 21640: loss 2.7461, time 2049.99ms, mfu 34.35%
iter 21650: loss 2.6700, time 2051.29ms, mfu 34.34%
iter 21660: loss 2.4399, time 2033.07ms, mfu 34.36%
iter 21670: loss 2.6212, time 2052.43ms, mfu 34.35%
iter 21680: loss 2.7213, time 2046.05ms, mfu 34.35%
iter 21690: loss 2.5390, time 2053.74ms, mfu 34.33%
iter 21700: loss 2.6351, time 2035.97ms, mfu 34.35%
iter 21710: loss 2.4779, time 2042.10ms, mfu 34.36%
iter 21720: loss 2.3776, time 2054.87ms, mfu 34.34%
iter 21730: loss 2.3918, time 2048.39ms, mfu 34.34%
iter 21740: loss 2.4109, time 2034.45ms, mfu 34.36%
iter 21750: loss 2.3127, time 2041.94ms, mfu 34.36%
iter 21760: loss 2.4471, time 2051.70ms, mfu 34.35%
iter 21770: loss 2.3663, time 2039.56ms, mfu 34.36%
iter 21780: loss 2.7221, time 2055.04ms, mfu 34.34%
iter 21790: loss 2.6982, time 2039.90ms, mfu 34.35%
iter 21800: loss 2.4300, time 2056.98ms, mfu 34.33%
iter 21810: loss 2.6521, time 2058.72ms, mfu 34.31%
iter 21820: loss 2.5549, time 2062.07ms, mfu 34.29%
iter 21830: loss 2.4077, time 2044.30ms, mfu 34.30%
iter 21840: loss 2.4835, time 2047.11ms, mfu 34.30%
iter 21850: loss 2.5915, time 2043.43ms, mfu 34.31%
iter 21860: loss 2.4817, time 2046.34ms, mfu 34.31%
iter 21870: loss 2.5733, time 2049.69ms, mfu 34.31%
iter 21880: loss 2.4169, time 2035.20ms, mfu 34.33%
iter 21890: loss 2.7048, time 2037.58ms, mfu 34.34%
iter 21900: loss 2.1925, time 2051.87ms, mfu 34.33%
iter 21910: loss 2.6805, time 2058.14ms, mfu 34.31%
iter 21920: loss 2.5104, time 2054.06ms, mfu 34.30%
iter 21930: loss 2.3913, time 2040.68ms, mfu 34.32%
iter 21940: loss 2.4296, time 2046.36ms, mfu 34.32%
iter 21950: loss 2.3566, time 2027.26ms, mfu 34.35%
iter 21960: loss 2.5261, time 2042.07ms, mfu 34.36%
iter 21970: loss 2.7742, time 2060.12ms, mfu 34.33%
iter 21980: loss 2.9902, time 2060.84ms, mfu 34.31%
iter 21990: loss 2.8476, time 2034.09ms, mfu 34.33%
step 22000: train loss 2.4545, val loss 2.5775
saving checkpoint to ../runs/minimal-en-125m-4ep
iter 22000: loss 2.6384, time 9025.38ms, mfu 31.68%
saving final checkpoint to ../runs/minimal-en-125m-4ep
[1;34mwandb[0m:
[1;34mwandb[0m: 🚀 View run [33m125m-12k-4ep[0m at: [34mhttps://wandb.ai/dawidmajewski-ezotic/minimal-en/runs/hlbpqpoe[0m
[1;34mwandb[0m: Find logs at: [1;35m../logs/wandb/wandb/run-20260824_231331-hlbpqpoe/logs[0m
|