Upload training.log with huggingface_hub
Browse files- training.log +247 -0
training.log
ADDED
|
@@ -0,0 +1,247 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[19:47:48] === PHASE 1 SUPERVISED PRE-TRAINING START ===
|
| 2 |
+
[19:47:48] Baixando dataset do HuggingFace...
|
| 3 |
+
[19:47:48] (isso pode levar 15-30 min para 10 GB)
|
| 4 |
+
[19:52:15] Dataset em: /kaggle/working/dataset
|
| 5 |
+
[19:52:15] Binance 1s: 2304 arquivos
|
| 6 |
+
[19:52:15] Binance 1m: 0 arquivos (fallback)
|
| 7 |
+
[19:52:15] NovaDax trades: 2304 arquivos
|
| 8 |
+
[19:52:15] NovaDax klines: 1 arquivos
|
| 9 |
+
[19:52:15] NovaDax indexado: 2304 datas de trades, 1 de klines
|
| 10 |
+
[19:52:15] Split: train=1461 | val=366 | test=477 dias
|
| 11 |
+
[19:52:15] Testando pipeline de dados com o primeiro arquivo...
|
| 12 |
+
[19:52:16] OK: 86400 rows | colunas: ['ts_ms', 'bin_ret', 'bin_ret_10', 'bin_ret_30', 'vol_z', 'trades_z']...
|
| 13 |
+
[19:52:16] y30 dist: flat=68476 up=8737 dn=9187
|
| 14 |
+
[19:52:16] Feats: mean=0.0361 | std=0.3507
|
| 15 |
+
[19:52:17] DataLoaders criados
|
| 16 |
+
[19:52:17] Modelo: 4,906,507 params (4.91M) | trainable: 4,906,507
|
| 17 |
+
[19:52:17] DataParallel em 2 GPUs
|
| 18 |
+
[19:52:17] Steps estimados: 4,400 | Steps/época: 24,654
|
| 19 |
+
[19:52:24] Tentando baixar checkpoint do HuggingFace...
|
| 20 |
+
[19:52:24] Nenhum checkpoint encontrado, iniciando do zero.
|
| 21 |
+
[19:52:24] Iniciando treino | step=0 | best_val=inf
|
| 22 |
+
[19:52:24] Máximo: 11h | Checkpoint a cada 1h
|
| 23 |
+
[19:52:24] === Época 1 | step=0 | elapsed=0.00h ===
|
| 24 |
+
[19:52:46] step= 100 | loss=2.5517 | acc30=0.184 | lr=2.61e-04 | 0.01h
|
| 25 |
+
[19:53:03] step= 200 | loss=2.1935 | acc30=0.260 | lr=3.00e-04 | 0.01h
|
| 26 |
+
[19:53:20] step= 300 | loss=2.3798 | acc30=0.439 | lr=2.99e-04 | 0.02h
|
| 27 |
+
[19:53:37] step= 400 | loss=2.7090 | acc30=0.989 | lr=2.97e-04 | 0.02h
|
| 28 |
+
[19:53:54] step= 500 | loss=2.8053 | acc30=0.583 | lr=2.95e-04 | 0.03h
|
| 29 |
+
[19:54:12] step= 600 | loss=2.5192 | acc30=0.263 | lr=2.91e-04 | 0.03h
|
| 30 |
+
[19:54:31] step= 700 | loss=2.2992 | acc30=0.212 | lr=2.87e-04 | 0.04h
|
| 31 |
+
[19:54:48] step= 800 | loss=2.2822 | acc30=0.213 | lr=2.82e-04 | 0.04h
|
| 32 |
+
[19:55:06] step= 900 | loss=2.3126 | acc30=0.226 | lr=2.77e-04 | 0.04h
|
| 33 |
+
[19:55:23] step= 1000 | loss=2.6052 | acc30=0.505 | lr=2.70e-04 | 0.05h
|
| 34 |
+
[19:55:23] Validando...
|
| 35 |
+
[19:55:40] VAL: loss=2.6031 acc30=0.601 acc60=0.277 acc120=0.342
|
| 36 |
+
[19:55:58] step= 1100 | loss=2.7995 | acc30=0.363 | lr=2.63e-04 | 0.06h
|
| 37 |
+
[19:56:15] step= 1200 | loss=2.9147 | acc30=0.927 | lr=2.56e-04 | 0.06h
|
| 38 |
+
[19:56:33] step= 1300 | loss=2.4326 | acc30=0.359 | lr=2.48e-04 | 0.07h
|
| 39 |
+
[19:56:51] step= 1400 | loss=2.3332 | acc30=0.215 | lr=2.39e-04 | 0.07h
|
| 40 |
+
[19:57:09] step= 1500 | loss=2.4511 | acc30=0.167 | lr=2.30e-04 | 0.08h
|
| 41 |
+
[19:57:26] step= 1600 | loss=2.4280 | acc30=0.173 | lr=2.21e-04 | 0.08h
|
| 42 |
+
[19:57:44] step= 1700 | loss=2.3409 | acc30=0.185 | lr=2.11e-04 | 0.09h
|
| 43 |
+
[19:58:02] step= 1800 | loss=2.3322 | acc30=0.207 | lr=2.00e-04 | 0.09h
|
| 44 |
+
[19:58:19] step= 1900 | loss=2.1135 | acc30=0.278 | lr=1.90e-04 | 0.10h
|
| 45 |
+
[19:58:36] step= 2000 | loss=2.1493 | acc30=0.271 | lr=1.79e-04 | 0.10h
|
| 46 |
+
[19:58:36] Validando...
|
| 47 |
+
[19:58:53] VAL: loss=2.3881 acc30=0.202 acc60=0.277 acc120=0.342
|
| 48 |
+
[19:59:11] step= 2100 | loss=2.1894 | acc30=0.257 | lr=1.68e-04 | 0.11h
|
| 49 |
+
[19:59:29] step= 2200 | loss=2.2146 | acc30=0.248 | lr=1.57e-04 | 0.12h
|
| 50 |
+
[19:59:47] step= 2300 | loss=2.3106 | acc30=0.216 | lr=1.46e-04 | 0.12h
|
| 51 |
+
[20:00:04] step= 2400 | loss=2.1249 | acc30=0.286 | lr=1.35e-04 | 0.13h
|
| 52 |
+
[20:00:22] step= 2500 | loss=2.4719 | acc30=0.178 | lr=1.24e-04 | 0.13h
|
| 53 |
+
[20:00:39] step= 2600 | loss=2.3971 | acc30=0.191 | lr=1.13e-04 | 0.14h
|
| 54 |
+
[20:00:57] step= 2700 | loss=2.2963 | acc30=0.210 | lr=1.03e-04 | 0.14h
|
| 55 |
+
[20:01:14] step= 2800 | loss=2.6621 | acc30=0.121 | lr=9.25e-05 | 0.15h
|
| 56 |
+
[20:01:32] step= 2900 | loss=2.8470 | acc30=0.359 | lr=8.24e-05 | 0.15h
|
| 57 |
+
[20:01:50] step= 3000 | loss=2.5497 | acc30=0.147 | lr=7.28e-05 | 0.16h
|
| 58 |
+
[20:01:50] Validando...
|
| 59 |
+
[20:02:06] VAL: loss=2.3954 acc30=0.197 acc60=0.268 acc120=0.324
|
| 60 |
+
[20:02:25] step= 3100 | loss=2.3512 | acc30=0.239 | lr=6.35e-05 | 0.17h
|
| 61 |
+
[20:02:42] step= 3200 | loss=2.0462 | acc30=0.324 | lr=5.47e-05 | 0.17h
|
| 62 |
+
[20:03:00] step= 3300 | loss=3.5350 | acc30=0.198 | lr=4.65e-05 | 0.18h
|
| 63 |
+
[20:03:18] step= 3400 | loss=2.7692 | acc30=0.233 | lr=3.88e-05 | 0.18h
|
| 64 |
+
[20:03:36] step= 3500 | loss=2.2814 | acc30=0.249 | lr=3.17e-05 | 0.19h
|
| 65 |
+
[20:03:53] step= 3600 | loss=2.8433 | acc30=0.103 | lr=2.52e-05 | 0.19h
|
| 66 |
+
[20:04:11] step= 3700 | loss=2.3994 | acc30=0.195 | lr=1.94e-05 | 0.20h
|
| 67 |
+
[20:04:28] step= 3800 | loss=2.4279 | acc30=0.175 | lr=1.44e-05 | 0.20h
|
| 68 |
+
[20:04:46] step= 3900 | loss=2.5048 | acc30=0.157 | lr=1.00e-05 | 0.21h
|
| 69 |
+
[20:05:04] step= 4000 | loss=2.4662 | acc30=0.160 | lr=6.43e-06 | 0.21h
|
| 70 |
+
[20:05:04] Validando...
|
| 71 |
+
[20:05:20] VAL: loss=2.3599 acc30=0.200 acc60=0.273 acc120=0.337
|
| 72 |
+
[20:05:38] step= 4100 | loss=2.3305 | acc30=0.229 | lr=3.63e-06 | 0.22h
|
| 73 |
+
[20:05:56] step= 4200 | loss=2.3756 | acc30=0.214 | lr=1.62e-06 | 0.23h
|
| 74 |
+
[20:06:14] step= 4300 | loss=2.6299 | acc30=0.112 | lr=4.08e-07 | 0.23h
|
| 75 |
+
[20:06:31] step= 4400 | loss=3.2240 | acc30=0.053 | lr=1.00e-08 | 0.24h
|
| 76 |
+
[20:06:49] step= 4500 | loss=2.4437 | acc30=0.290 | lr=1.00e-08 | 0.24h
|
| 77 |
+
[20:07:07] step= 4600 | loss=2.1073 | acc30=0.389 | lr=1.00e-08 | 0.25h
|
| 78 |
+
[20:07:24] step= 4700 | loss=2.2409 | acc30=0.248 | lr=1.00e-08 | 0.25h
|
| 79 |
+
[20:07:42] step= 4800 | loss=2.0776 | acc30=0.385 | lr=1.00e-08 | 0.26h
|
| 80 |
+
[20:08:00] step= 4900 | loss=2.3494 | acc30=0.258 | lr=1.00e-08 | 0.26h
|
| 81 |
+
[20:08:18] step= 5000 | loss=2.5491 | acc30=0.167 | lr=1.00e-08 | 0.27h
|
| 82 |
+
[20:08:18] Validando...
|
| 83 |
+
[20:08:34] VAL: loss=2.3569 acc30=0.200 acc60=0.272 acc120=0.337
|
| 84 |
+
[20:08:52] step= 5100 | loss=2.8098 | acc30=0.088 | lr=1.00e-08 | 0.27h
|
| 85 |
+
[20:09:10] step= 5200 | loss=2.3844 | acc30=0.192 | lr=1.00e-08 | 0.28h
|
| 86 |
+
[20:09:28] step= 5300 | loss=2.4574 | acc30=0.163 | lr=1.00e-08 | 0.28h
|
| 87 |
+
[20:09:45] step= 5400 | loss=2.4677 | acc30=0.173 | lr=1.00e-08 | 0.29h
|
| 88 |
+
[20:10:03] step= 5500 | loss=2.1160 | acc30=0.358 | lr=1.00e-08 | 0.29h
|
| 89 |
+
[20:10:21] step= 5600 | loss=2.2874 | acc30=0.226 | lr=1.00e-08 | 0.30h
|
| 90 |
+
[20:10:39] step= 5700 | loss=2.2475 | acc30=0.276 | lr=1.00e-08 | 0.30h
|
| 91 |
+
[20:10:57] step= 5800 | loss=2.4170 | acc30=0.293 | lr=1.00e-08 | 0.31h
|
| 92 |
+
[20:11:14] step= 5900 | loss=4.3855 | acc30=0.010 | lr=1.00e-08 | 0.31h
|
| 93 |
+
[20:11:32] step= 6000 | loss=3.4007 | acc30=0.065 | lr=1.00e-08 | 0.32h
|
| 94 |
+
[20:11:32] Validando...
|
| 95 |
+
[20:11:48] VAL: loss=2.3568 acc30=0.200 acc60=0.272 acc120=0.337
|
| 96 |
+
[20:12:06] step= 6100 | loss=2.9336 | acc30=0.081 | lr=1.00e-08 | 0.33h
|
| 97 |
+
[20:12:24] step= 6200 | loss=2.4307 | acc30=0.189 | lr=1.00e-08 | 0.33h
|
| 98 |
+
[20:12:41] step= 6300 | loss=3.0773 | acc30=0.060 | lr=1.00e-08 | 0.34h
|
| 99 |
+
[20:12:59] step= 6400 | loss=2.3944 | acc30=0.218 | lr=1.00e-08 | 0.34h
|
| 100 |
+
[20:13:17] step= 6500 | loss=2.7768 | acc30=0.123 | lr=1.00e-08 | 0.35h
|
| 101 |
+
[20:13:35] step= 6600 | loss=3.0216 | acc30=0.074 | lr=1.00e-08 | 0.35h
|
| 102 |
+
[20:13:52] step= 6700 | loss=2.3582 | acc30=0.162 | lr=1.00e-08 | 0.36h
|
| 103 |
+
[20:14:10] step= 6800 | loss=2.7971 | acc30=0.096 | lr=1.00e-08 | 0.36h
|
| 104 |
+
[20:14:28] step= 6900 | loss=2.4053 | acc30=0.172 | lr=1.00e-08 | 0.37h
|
| 105 |
+
[20:14:47] step= 7000 | loss=2.5172 | acc30=0.146 | lr=1.00e-08 | 0.37h
|
| 106 |
+
[20:14:47] Validando...
|
| 107 |
+
[20:15:03] VAL: loss=2.3568 acc30=0.200 acc60=0.272 acc120=0.337
|
| 108 |
+
[20:15:20] step= 7100 | loss=2.2227 | acc30=0.267 | lr=1.00e-08 | 0.38h
|
| 109 |
+
[20:15:38] step= 7200 | loss=2.1562 | acc30=0.295 | lr=1.00e-08 | 0.39h
|
| 110 |
+
[20:15:56] step= 7300 | loss=2.4936 | acc30=0.211 | lr=1.00e-08 | 0.39h
|
| 111 |
+
[20:16:14] step= 7400 | loss=2.2936 | acc30=0.218 | lr=1.00e-08 | 0.40h
|
| 112 |
+
[20:16:31] step= 7500 | loss=2.8560 | acc30=0.088 | lr=1.00e-08 | 0.40h
|
| 113 |
+
[20:16:49] step= 7600 | loss=2.4172 | acc30=0.177 | lr=1.00e-08 | 0.41h
|
| 114 |
+
[20:17:07] step= 7700 | loss=2.3952 | acc30=0.214 | lr=1.00e-08 | 0.41h
|
| 115 |
+
[20:17:25] step= 7800 | loss=2.7299 | acc30=0.109 | lr=1.00e-08 | 0.42h
|
| 116 |
+
[20:17:42] step= 7900 | loss=2.3964 | acc30=0.184 | lr=1.00e-08 | 0.42h
|
| 117 |
+
[20:18:00] step= 8000 | loss=3.2202 | acc30=0.096 | lr=1.00e-08 | 0.43h
|
| 118 |
+
[20:18:00] Validando...
|
| 119 |
+
[20:18:16] VAL: loss=2.3568 acc30=0.200 acc60=0.272 acc120=0.337
|
| 120 |
+
[20:18:34] step= 8100 | loss=2.9130 | acc30=0.117 | lr=1.00e-08 | 0.44h
|
| 121 |
+
[20:18:51] step= 8200 | loss=3.6459 | acc30=0.049 | lr=1.00e-08 | 0.44h
|
| 122 |
+
[20:19:09] step= 8300 | loss=2.1714 | acc30=0.289 | lr=1.00e-08 | 0.45h
|
| 123 |
+
[20:19:27] step= 8400 | loss=2.1737 | acc30=0.303 | lr=1.00e-08 | 0.45h
|
| 124 |
+
[20:19:45] step= 8500 | loss=2.3622 | acc30=0.230 | lr=1.00e-08 | 0.46h
|
| 125 |
+
[20:20:02] step= 8600 | loss=3.8097 | acc30=0.022 | lr=1.00e-08 | 0.46h
|
| 126 |
+
[20:20:20] step= 8700 | loss=4.7551 | acc30=0.011 | lr=1.00e-08 | 0.47h
|
| 127 |
+
[20:20:38] step= 8800 | loss=2.5744 | acc30=0.135 | lr=1.00e-08 | 0.47h
|
| 128 |
+
[20:20:56] step= 8900 | loss=2.2570 | acc30=0.266 | lr=1.00e-08 | 0.48h
|
| 129 |
+
[20:21:13] step= 9000 | loss=2.0862 | acc30=0.397 | lr=1.00e-08 | 0.48h
|
| 130 |
+
[20:21:13] Validando...
|
| 131 |
+
[20:21:31] VAL: loss=2.3567 acc30=0.200 acc60=0.272 acc120=0.337
|
| 132 |
+
[20:21:48] step= 9100 | loss=2.8084 | acc30=0.129 | lr=1.00e-08 | 0.49h
|
| 133 |
+
[20:22:06] step= 9200 | loss=2.4362 | acc30=0.148 | lr=1.00e-08 | 0.50h
|
| 134 |
+
[20:22:24] step= 9300 | loss=2.2917 | acc30=0.217 | lr=1.00e-08 | 0.50h
|
| 135 |
+
[20:22:41] step= 9400 | loss=2.0908 | acc30=0.378 | lr=1.00e-08 | 0.50h
|
| 136 |
+
[20:22:59] step= 9500 | loss=2.1227 | acc30=0.321 | lr=1.00e-08 | 0.51h
|
| 137 |
+
[20:23:17] step= 9600 | loss=2.8909 | acc30=0.176 | lr=1.00e-08 | 0.51h
|
| 138 |
+
[20:23:35] step= 9700 | loss=2.9897 | acc30=0.115 | lr=1.00e-08 | 0.52h
|
| 139 |
+
[20:23:52] step= 9800 | loss=2.1571 | acc30=0.311 | lr=1.00e-08 | 0.52h
|
| 140 |
+
[20:24:10] step= 9900 | loss=2.2028 | acc30=0.266 | lr=1.00e-08 | 0.53h
|
| 141 |
+
[20:24:28] step= 10000 | loss=2.2150 | acc30=0.270 | lr=1.00e-08 | 0.53h
|
| 142 |
+
[20:24:28] Validando...
|
| 143 |
+
[20:24:44] VAL: loss=2.3566 acc30=0.200 acc60=0.272 acc120=0.337
|
| 144 |
+
[20:25:02] step= 10100 | loss=2.2938 | acc30=0.205 | lr=1.00e-08 | 0.54h
|
| 145 |
+
[20:25:19] step= 10200 | loss=2.3379 | acc30=0.220 | lr=1.00e-08 | 0.55h
|
| 146 |
+
[20:25:37] step= 10300 | loss=2.2540 | acc30=0.234 | lr=1.00e-08 | 0.55h
|
| 147 |
+
[20:25:55] step= 10400 | loss=2.1680 | acc30=0.299 | lr=1.00e-08 | 0.56h
|
| 148 |
+
[20:26:13] step= 10500 | loss=2.1737 | acc30=0.288 | lr=1.00e-08 | 0.56h
|
| 149 |
+
[20:26:30] step= 10600 | loss=2.2804 | acc30=0.209 | lr=1.00e-08 | 0.57h
|
| 150 |
+
[20:26:48] step= 10700 | loss=2.2547 | acc30=0.218 | lr=1.00e-08 | 0.57h
|
| 151 |
+
[20:27:06] step= 10800 | loss=2.7201 | acc30=0.160 | lr=1.00e-08 | 0.58h
|
| 152 |
+
[20:27:24] step= 10900 | loss=3.1726 | acc30=0.065 | lr=1.00e-08 | 0.58h
|
| 153 |
+
[20:27:41] step= 11000 | loss=3.3902 | acc30=0.045 | lr=1.00e-08 | 0.59h
|
| 154 |
+
[20:27:41] Validando...
|
| 155 |
+
[20:27:57] VAL: loss=2.3565 acc30=0.200 acc60=0.272 acc120=0.337
|
| 156 |
+
[20:28:15] step= 11100 | loss=2.7160 | acc30=0.150 | lr=1.00e-08 | 0.60h
|
| 157 |
+
[20:28:33] step= 11200 | loss=2.3661 | acc30=0.193 | lr=1.00e-08 | 0.60h
|
| 158 |
+
[20:28:51] step= 11300 | loss=2.5008 | acc30=0.194 | lr=1.00e-08 | 0.61h
|
| 159 |
+
[20:29:09] step= 11400 | loss=3.8704 | acc30=0.027 | lr=1.00e-08 | 0.61h
|
| 160 |
+
[20:29:26] step= 11500 | loss=2.7116 | acc30=0.140 | lr=1.00e-08 | 0.62h
|
| 161 |
+
[20:29:44] step= 11600 | loss=2.2880 | acc30=0.243 | lr=1.00e-08 | 0.62h
|
| 162 |
+
[20:30:02] step= 11700 | loss=2.1555 | acc30=0.309 | lr=1.00e-08 | 0.63h
|
| 163 |
+
[20:30:19] step= 11800 | loss=2.2513 | acc30=0.252 | lr=1.00e-08 | 0.63h
|
| 164 |
+
[20:30:37] step= 11900 | loss=2.1663 | acc30=0.290 | lr=1.00e-08 | 0.64h
|
| 165 |
+
[20:30:55] step= 12000 | loss=2.1389 | acc30=0.330 | lr=1.00e-08 | 0.64h
|
| 166 |
+
[20:30:55] Validando...
|
| 167 |
+
[20:31:11] VAL: loss=2.3565 acc30=0.200 acc60=0.272 acc120=0.337
|
| 168 |
+
[20:31:29] step= 12100 | loss=2.2540 | acc30=0.261 | lr=1.00e-08 | 0.65h
|
| 169 |
+
[20:31:46] step= 12200 | loss=3.5529 | acc30=0.050 | lr=1.00e-08 | 0.66h
|
| 170 |
+
[20:32:04] step= 12300 | loss=2.2384 | acc30=0.274 | lr=1.00e-08 | 0.66h
|
| 171 |
+
[20:32:22] step= 12400 | loss=2.1568 | acc30=0.307 | lr=1.00e-08 | 0.67h
|
| 172 |
+
[20:32:40] step= 12500 | loss=2.0656 | acc30=0.405 | lr=1.00e-08 | 0.67h
|
| 173 |
+
[20:32:57] step= 12600 | loss=2.2727 | acc30=0.237 | lr=1.00e-08 | 0.68h
|
| 174 |
+
[20:33:15] step= 12700 | loss=2.2428 | acc30=0.243 | lr=1.00e-08 | 0.68h
|
| 175 |
+
[20:33:33] step= 12800 | loss=2.5467 | acc30=0.206 | lr=1.00e-08 | 0.69h
|
| 176 |
+
[20:33:51] step= 12900 | loss=3.3666 | acc30=0.058 | lr=1.00e-08 | 0.69h
|
| 177 |
+
[20:34:08] step= 13000 | loss=2.1594 | acc30=0.297 | lr=1.00e-08 | 0.70h
|
| 178 |
+
[20:34:08] Validando...
|
| 179 |
+
[20:34:25] VAL: loss=2.3564 acc30=0.200 acc60=0.272 acc120=0.337
|
| 180 |
+
[20:34:43] step= 13100 | loss=2.1094 | acc30=0.344 | lr=1.00e-08 | 0.71h
|
| 181 |
+
[20:35:01] step= 13200 | loss=2.2251 | acc30=0.279 | lr=1.00e-08 | 0.71h
|
| 182 |
+
[20:35:18] step= 13300 | loss=2.3688 | acc30=0.197 | lr=1.00e-08 | 0.72h
|
| 183 |
+
[20:35:36] step= 13400 | loss=2.2561 | acc30=0.242 | lr=1.00e-08 | 0.72h
|
| 184 |
+
[20:35:54] step= 13500 | loss=2.3765 | acc30=0.180 | lr=1.00e-08 | 0.73h
|
| 185 |
+
[20:36:12] step= 13600 | loss=2.4212 | acc30=0.171 | lr=1.00e-08 | 0.73h
|
| 186 |
+
[20:36:29] step= 13700 | loss=2.4924 | acc30=0.158 | lr=1.00e-08 | 0.73h
|
| 187 |
+
[20:36:47] step= 13800 | loss=2.3785 | acc30=0.198 | lr=1.00e-08 | 0.74h
|
| 188 |
+
[20:37:05] step= 13900 | loss=2.3181 | acc30=0.195 | lr=1.00e-08 | 0.74h
|
| 189 |
+
[20:37:23] step= 14000 | loss=2.1844 | acc30=0.263 | lr=1.00e-08 | 0.75h
|
| 190 |
+
[20:37:23] Validando...
|
| 191 |
+
[20:37:39] VAL: loss=2.3563 acc30=0.200 acc60=0.271 acc120=0.337
|
| 192 |
+
[20:37:57] step= 14100 | loss=2.2146 | acc30=0.258 | lr=1.00e-08 | 0.76h
|
| 193 |
+
[20:38:15] step= 14200 | loss=2.0913 | acc30=0.388 | lr=1.00e-08 | 0.76h
|
| 194 |
+
[20:38:32] step= 14300 | loss=3.8682 | acc30=0.156 | lr=1.00e-08 | 0.77h
|
| 195 |
+
[20:38:50] step= 14400 | loss=4.0869 | acc30=0.049 | lr=1.00e-08 | 0.77h
|
| 196 |
+
[20:39:07] step= 14500 | loss=2.3883 | acc30=0.184 | lr=1.00e-08 | 0.78h
|
| 197 |
+
[20:39:25] step= 14600 | loss=2.6680 | acc30=0.115 | lr=1.00e-08 | 0.78h
|
| 198 |
+
[20:39:43] step= 14700 | loss=2.5179 | acc30=0.137 | lr=1.00e-08 | 0.79h
|
| 199 |
+
[20:40:01] step= 14800 | loss=2.4570 | acc30=0.157 | lr=1.00e-08 | 0.79h
|
| 200 |
+
[20:40:18] step= 14900 | loss=2.5993 | acc30=0.135 | lr=1.00e-08 | 0.80h
|
| 201 |
+
[20:40:36] step= 15000 | loss=2.3919 | acc30=0.185 | lr=1.00e-08 | 0.80h
|
| 202 |
+
[20:40:36] Validando...
|
| 203 |
+
[20:40:53] VAL: loss=2.3562 acc30=0.200 acc60=0.272 acc120=0.337
|
| 204 |
+
[20:41:11] step= 15100 | loss=2.1487 | acc30=0.298 | lr=1.00e-08 | 0.81h
|
| 205 |
+
[20:41:29] step= 15200 | loss=2.1501 | acc30=0.312 | lr=1.00e-08 | 0.82h
|
| 206 |
+
[20:41:46] step= 15300 | loss=2.3261 | acc30=0.211 | lr=1.00e-08 | 0.82h
|
| 207 |
+
[20:42:04] step= 15400 | loss=2.1218 | acc30=0.383 | lr=1.00e-08 | 0.83h
|
| 208 |
+
[20:42:22] step= 15500 | loss=2.3623 | acc30=0.280 | lr=1.00e-08 | 0.83h
|
| 209 |
+
[20:42:40] step= 15600 | loss=2.5723 | acc30=0.124 | lr=1.00e-08 | 0.84h
|
| 210 |
+
[20:42:57] step= 15700 | loss=2.4892 | acc30=0.155 | lr=1.00e-08 | 0.84h
|
| 211 |
+
[20:43:15] step= 15800 | loss=2.4106 | acc30=0.187 | lr=1.00e-08 | 0.85h
|
| 212 |
+
[20:43:33] step= 15900 | loss=2.4061 | acc30=0.168 | lr=1.00e-08 | 0.85h
|
| 213 |
+
[20:43:51] step= 16000 | loss=2.2963 | acc30=0.221 | lr=1.00e-08 | 0.86h
|
| 214 |
+
[20:43:51] Validando...
|
| 215 |
+
[20:44:07] VAL: loss=2.3562 acc30=0.200 acc60=0.272 acc120=0.337
|
| 216 |
+
[20:44:24] step= 16100 | loss=2.1279 | acc30=0.334 | lr=1.00e-08 | 0.87h
|
| 217 |
+
[20:44:42] step= 16200 | loss=4.1668 | acc30=0.086 | lr=1.00e-08 | 0.87h
|
| 218 |
+
[20:45:00] step= 16300 | loss=3.8789 | acc30=0.062 | lr=1.00e-08 | 0.88h
|
| 219 |
+
[20:45:18] step= 16400 | loss=2.2729 | acc30=0.221 | lr=1.00e-08 | 0.88h
|
| 220 |
+
[20:45:35] step= 16500 | loss=2.1995 | acc30=0.280 | lr=1.00e-08 | 0.89h
|
| 221 |
+
[20:45:53] step= 16600 | loss=2.2381 | acc30=0.237 | lr=1.00e-08 | 0.89h
|
| 222 |
+
[20:46:11] step= 16700 | loss=2.1802 | acc30=0.273 | lr=1.00e-08 | 0.90h
|
| 223 |
+
[20:46:29] step= 16800 | loss=2.1204 | acc30=0.316 | lr=1.00e-08 | 0.90h
|
| 224 |
+
[20:46:46] step= 16900 | loss=2.3939 | acc30=0.181 | lr=1.00e-08 | 0.91h
|
| 225 |
+
[20:47:04] step= 17000 | loss=2.2572 | acc30=0.222 | lr=1.00e-08 | 0.91h
|
| 226 |
+
[20:47:04] Validando...
|
| 227 |
+
[20:47:20] VAL: loss=2.3561 acc30=0.200 acc60=0.272 acc120=0.337
|
| 228 |
+
[20:47:38] step= 17100 | loss=2.2856 | acc30=0.219 | lr=1.00e-08 | 0.92h
|
| 229 |
+
[20:47:56] step= 17200 | loss=2.4377 | acc30=0.175 | lr=1.00e-08 | 0.93h
|
| 230 |
+
[20:48:13] step= 17300 | loss=2.5572 | acc30=0.148 | lr=1.00e-08 | 0.93h
|
| 231 |
+
[20:48:31] step= 17400 | loss=2.1858 | acc30=0.307 | lr=1.00e-08 | 0.94h
|
| 232 |
+
[20:48:49] step= 17500 | loss=2.3156 | acc30=0.289 | lr=1.00e-08 | 0.94h
|
| 233 |
+
[20:49:06] step= 17600 | loss=2.6825 | acc30=0.131 | lr=1.00e-08 | 0.95h
|
| 234 |
+
[20:49:24] step= 17700 | loss=2.5440 | acc30=0.152 | lr=1.00e-08 | 0.95h
|
| 235 |
+
[20:49:42] step= 17800 | loss=2.2803 | acc30=0.222 | lr=1.00e-08 | 0.96h
|
| 236 |
+
[20:50:00] step= 17900 | loss=2.2970 | acc30=0.229 | lr=1.00e-08 | 0.96h
|
| 237 |
+
[20:50:17] step= 18000 | loss=2.1674 | acc30=0.288 | lr=1.00e-08 | 0.96h
|
| 238 |
+
[20:50:17] Validando...
|
| 239 |
+
[20:50:35] VAL: loss=2.3560 acc30=0.200 acc60=0.272 acc120=0.337
|
| 240 |
+
[20:50:52] step= 18100 | loss=3.2475 | acc30=0.071 | lr=1.00e-08 | 0.97h
|
| 241 |
+
[20:51:10] step= 18200 | loss=2.5619 | acc30=0.143 | lr=1.00e-08 | 0.98h
|
| 242 |
+
[20:51:28] step= 18300 | loss=2.1799 | acc30=0.278 | lr=1.00e-08 | 0.98h
|
| 243 |
+
[20:51:45] step= 18400 | loss=2.1331 | acc30=0.299 | lr=1.00e-08 | 0.99h
|
| 244 |
+
[20:52:03] step= 18500 | loss=2.2125 | acc30=0.269 | lr=1.00e-08 | 0.99h
|
| 245 |
+
[20:52:21] step= 18600 | loss=2.1454 | acc30=0.307 | lr=1.00e-08 | 1.00h
|
| 246 |
+
[20:52:24] --- CHECKPOINT (hora 1.0) ---
|
| 247 |
+
[20:52:24] Checkpoint salvo: checkpoint_step00018616.pt (59.0 MB)
|