Hugging Face's logo

YuqianFu
/
CausalStyleAdv

TensorBoard
Model card Files
xet
Metrics Community
CausalStyleAdv / Meta-causal /code-withStyleAttack /64945.log
YuqianFu's picture
YuqianFu
Upload folder using huggingface_hub
197d4ca verified
raw
history blame contribute delete
114 kB
 /home/yuqian_fu
here1
here2
{'gpu': '0', 'data': 'cartoon', 'ntr': None, 'translate': None, 'autoaug': 'CA_multiple', 'n': 3, 'stride': 5, 'factor_num': 16, 'epochs': 70, 'nbatch': 100, 'batchsize': 6, 'lr': 0.01, 'lr_scheduler': 'cosine', 'svroot': '/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/saved-PACS//cartoon/CA_multiple_16fa_v2_ep70_lr0.01_cosine_base0.01_bs6_lamCa_1_lamRe1_adt4_cls1_EW2_70_rmTrue_rnTrue_str5_WithStyleAttackExp1', 'clsadapt': True, 'lambda_causal': 1.0, 'lambda_re': 1.0, 'randm': True, 'randn': True, 'network': 'resnet18'}
stride: 5
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/cartoon_train.hdf5 torch.Size([2107, 3, 227, 227]) torch.Size([2107])
--------------------------CA_multiple--------------------------
---------------------------16 factors-----------------
randm: True
randn: True
n: 3
randm: False
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/cartoon_val.hdf5 torch.Size([237, 3, 227, 227]) torch.Size([237])
-------------------------------------loading pretrain weights----------------------------------
Epoch 1, weight, value: tensor([[-0.0100, 0.0028, 0.0031, ..., -0.0029, -0.0129, 0.0067],
[ 0.0146, 0.0136, 0.0209, ..., 0.0182, -0.0162, -0.0063],
[ 0.0144, -0.0044, -0.0032, ..., 0.0196, 0.0067, 0.0059],
...,
[-0.0163, -0.0022, 0.0082, ..., 0.0012, 0.0086, -0.0212],
[-0.0167, -0.0119, 0.0066, ..., -0.0117, 0.0125, -0.0117],
[-0.0118, -0.0128, -0.0087, ..., -0.0020, 0.0197, 0.0048]],
device='cuda:0'), grad: None
Epoch 1, bias, value: tensor([-0.0072, -0.0193, 0.0095, 0.0119, 0.0171, 0.0063, -0.0017],
device='cuda:0'), grad: None
351
0.01
changing lr
---------------------saving model at epoch 0----------------------------------------------------
epoch 0, time 485.47, cls_loss 16.9782 cls_loss_mapping 2.0665 cls_loss_causal 2.0978 re_mapping 0.2883 re_causal 0.2878 /// teacc 12.66 lr 0.00999497
Epoch 2, weight, value: tensor([[ 0.0028, -0.0005, 0.0340, ..., -0.0002, 0.0233, 0.0462],
[ 0.0658, 0.0861, 0.0585, ..., 0.0947, 0.0172, 0.0346],
[ 0.0262, -0.0145, 0.0102, ..., 0.0781, 0.0733, 0.0588],
...,
[-0.1058, -0.0852, -0.0944, ..., 0.0447, 0.0601, 0.0102],
[ 0.0011, 0.0218, 0.0048, ..., -0.0536, -0.0391, -0.0657],
[-0.0314, -0.0301, 0.0041, ..., -0.0928, -0.0691, -0.0766]],
device='cuda:0'), grad: tensor([[-0.0698, -0.0076, -0.0194, ..., -0.0092, -0.0196, -0.0026],
[-0.0486, -0.0072, -0.0131, ..., -0.0160, -0.0171, -0.0081],
[ 0.0616, 0.0076, 0.0175, ..., 0.0109, 0.0186, 0.0045],
...,
[ 0.0679, 0.0086, 0.0195, ..., 0.0142, 0.0205, 0.0061],
[ 0.0363, 0.0044, 0.0101, ..., 0.0059, 0.0107, 0.0021],
[ 0.0342, 0.0043, 0.0096, ..., 0.0069, 0.0106, 0.0030]],
device='cuda:0')
Epoch 2, bias, value: tensor([-0.0115, -0.2892, 0.0229, -0.1414, -0.0596, 0.3315, 0.1640],
device='cuda:0'), grad: tensor([-0.1210, -0.2117, 0.1382, -0.1489, 0.1758, 0.0772, 0.0903],
device='cuda:0')
351
0.009994965332706574
changing lr
---------------------saving model at epoch 1----------------------------------------------------
epoch 1, time 485.94, cls_loss 2.0030 cls_loss_mapping 1.9396 cls_loss_causal 1.9387 re_mapping 0.0647 re_causal 0.0644 /// teacc 41.77 lr 0.00997987
Epoch 3, weight, value: tensor([[ 3.4078e-04, 1.6000e-03, 3.1084e-02, ..., -1.0942e-03,
2.9201e-02, 5.4861e-02],
[ 6.9312e-02, 8.8419e-02, 5.7617e-02, ..., 9.5143e-02,
1.9121e-02, 3.5263e-02],
[ 2.8979e-02, -1.5773e-02, 2.5921e-02, ..., 6.5189e-02,
6.9842e-02, 5.3385e-02],
...,
[-1.1333e-01, -9.1322e-02, -1.0055e-01, ..., 5.0898e-02,
6.5492e-02, 1.1030e-02],
[-5.0384e-03, 1.4584e-02, 4.9783e-04, ..., -4.5509e-02,
-4.2445e-02, -7.5379e-02],
[-2.9434e-02, -2.5451e-02, 6.7909e-05, ..., -9.6225e-02,
-7.2842e-02, -7.1301e-02]], device='cuda:0'), grad: tensor([[ 0.0200, 0.0067, 0.0079, ..., 0.0014, 0.0034, -0.0002],
[ 0.0065, 0.0008, 0.0016, ..., 0.0011, 0.0014, 0.0007],
[ 0.0329, 0.0065, 0.0102, ..., 0.0055, 0.0080, 0.0035],
...,
[ 0.0395, 0.0060, 0.0104, ..., 0.0067, 0.0091, 0.0043],
[-0.0082, 0.0013, -0.0012, ..., -0.0020, -0.0016, -0.0008],
[-0.0192, -0.0043, -0.0041, ..., -0.0010, -0.0025, -0.0002]],
device='cuda:0')
Epoch 3, bias, value: tensor([-0.0612, -0.3493, 0.0748, -0.2091, -0.0425, 0.3946, 0.2087],
device='cuda:0'), grad: tensor([ 0.0385, 0.0165, 0.0712, -0.1410, 0.0951, -0.0314, -0.0487],
device='cuda:0')
351
0.009979871469976196
changing lr
epoch 2, time 484.44, cls_loss 1.5550 cls_loss_mapping 1.8738 cls_loss_causal 1.9254 re_mapping 0.0608 re_causal 0.0606 /// teacc 13.08 lr 0.00995475
Epoch 4, weight, value: tensor([[-0.0137, -0.0117, 0.0201, ..., -0.0003, 0.0411, 0.0727],
[ 0.0566, 0.0884, 0.0561, ..., 0.0828, 0.0134, 0.0221],
[ 0.0523, 0.0052, 0.0546, ..., 0.0551, 0.0651, 0.0509],
...,
[-0.1301, -0.1026, -0.1092, ..., 0.0596, 0.0749, 0.0162],
[-0.0134, 0.0029, -0.0080, ..., -0.0267, -0.0355, -0.0760],
[-0.0125, -0.0185, -0.0061, ..., -0.1114, -0.0908, -0.0794]],
device='cuda:0'), grad: tensor([[-0.1074, -0.0163, -0.0219, ..., -0.0164, -0.0173, -0.0177],
[-0.0030, -0.0003, -0.0006, ..., -0.0013, -0.0013, -0.0013],
[ 0.0052, 0.0004, 0.0009, ..., 0.0014, 0.0014, 0.0014],
...,
[ 0.0270, 0.0045, 0.0053, ..., 0.0024, 0.0023, 0.0031],
[ 0.0175, 0.0033, 0.0036, ..., 0.0030, 0.0029, 0.0029],
[ 0.0384, 0.0056, 0.0080, ..., 0.0066, 0.0071, 0.0070]],
device='cuda:0')
Epoch 4, bias, value: tensor([-0.0258, -0.3557, 0.0860, -0.2310, -0.0076, 0.3735, 0.1763],
device='cuda:0'), grad: tensor([-0.2710, -0.0077, 0.0140, 0.0579, 0.0644, 0.0446, 0.0978],
device='cuda:0')
351
0.009954748808839675
changing lr
---------------------saving model at epoch 3----------------------------------------------------
epoch 3, time 488.33, cls_loss 1.3546 cls_loss_mapping 1.7404 cls_loss_causal 1.8961 re_mapping 0.0588 re_causal 0.0585 /// teacc 53.59 lr 0.00991965
Epoch 5, weight, value: tensor([[-0.0074, -0.0123, 0.0209, ..., -0.0032, 0.0541, 0.0888],
[ 0.0611, 0.0895, 0.0580, ..., 0.0690, 0.0110, 0.0133],
[ 0.0503, 0.0057, 0.0587, ..., 0.0498, 0.0610, 0.0490],
...,
[-0.1420, -0.1151, -0.1146, ..., 0.0570, 0.0713, 0.0154],
[-0.0224, -0.0038, -0.0130, ..., -0.0028, -0.0191, -0.0638],
[-0.0048, -0.0111, -0.0072, ..., -0.1203, -0.1105, -0.0966]],
device='cuda:0'), grad: tensor([[ 1.4026e-01, 1.3710e-02, 3.0167e-02, ..., 3.7689e-02,
3.3051e-02, 3.6926e-02],
[ 8.0729e-04, 8.0884e-05, 1.7869e-04, ..., 1.8716e-04,
1.3447e-04, 1.5807e-04],
[ 2.3224e-02, 2.2202e-03, 5.0621e-03, ..., 5.9090e-03,
5.0125e-03, 6.0768e-03],
...,
[-8.1116e-02, -8.6365e-03, -1.6846e-02, ..., -1.8463e-02,
-1.6403e-02, -1.9104e-02],
[ 4.2725e-03, 4.3392e-04, 9.1553e-04, ..., 1.0796e-03,
7.8964e-04, 9.8324e-04],
[-9.1919e-02, -8.2321e-03, -2.0401e-02, ..., -2.7817e-02,
-2.3865e-02, -2.6306e-02]], device='cuda:0')
Epoch 5, bias, value: tensor([-0.0100, -0.3700, 0.0890, -0.2639, 0.0169, 0.3545, 0.1994],
device='cuda:0'), grad: tensor([ 0.4014, 0.0023, 0.0675, 0.0128, -0.2286, 0.0120, -0.2673],
device='cuda:0')
351
0.009919647942993149
changing lr
epoch 4, time 490.81, cls_loss 1.1636 cls_loss_mapping 1.6025 cls_loss_causal 1.8435 re_mapping 0.0581 re_causal 0.0578 /// teacc 47.26 lr 0.00987464
Epoch 6, weight, value: tensor([[-0.0092, -0.0115, 0.0200, ..., -0.0068, 0.0601, 0.0934],
[ 0.0657, 0.0900, 0.0604, ..., 0.0629, 0.0111, 0.0086],
[ 0.0585, 0.0162, 0.0694, ..., 0.0436, 0.0533, 0.0450],
...,
[-0.1502, -0.1233, -0.1191, ..., 0.0550, 0.0668, 0.0162],
[-0.0251, -0.0028, -0.0127, ..., 0.0180, -0.0031, -0.0522],
[-0.0082, -0.0168, -0.0138, ..., -0.1336, -0.1261, -0.1108]],
device='cuda:0'), grad: tensor([[ 0.0056, 0.0005, 0.0013, ..., 0.0016, 0.0015, 0.0011],
[-0.0229, -0.0027, -0.0048, ..., -0.0075, -0.0068, -0.0052],
[ 0.0082, 0.0008, 0.0021, ..., 0.0037, 0.0028, 0.0022],
...,
[-0.0790, -0.0061, -0.0199, ..., -0.0227, -0.0231, -0.0172],
[ 0.0002, 0.0001, 0.0004, ..., -0.0002, 0.0006, 0.0005],
[ 0.0731, 0.0060, 0.0168, ..., 0.0200, 0.0208, 0.0149]],
device='cuda:0')
Epoch 6, bias, value: tensor([-0.0260, -0.3347, 0.0891, -0.2536, 0.0157, 0.3505, 0.1747],
device='cuda:0'), grad: tensor([ 0.0153, -0.0533, 0.0237, 0.0438, -0.2416, 0.0005, 0.2114],
device='cuda:0')
351
0.009874639560909117
changing lr
epoch 5, time 491.16, cls_loss 0.9659 cls_loss_mapping 1.4805 cls_loss_causal 1.8159 re_mapping 0.0580 re_causal 0.0577 /// teacc 44.30 lr 0.00981981
Epoch 7, weight, value: tensor([[-0.0087, -0.0102, 0.0192, ..., -0.0007, 0.0825, 0.1130],
[ 0.0642, 0.0912, 0.0624, ..., 0.0528, 0.0038, -0.0031],
[ 0.0583, 0.0211, 0.0742, ..., 0.0440, 0.0475, 0.0432],
...,
[-0.1536, -0.1256, -0.1215, ..., 0.0537, 0.0628, 0.0130],
[-0.0222, -0.0026, -0.0132, ..., 0.0380, 0.0139, -0.0401],
[-0.0087, -0.0225, -0.0157, ..., -0.1544, -0.1509, -0.1300]],
device='cuda:0'), grad: tensor([[ 2.9419e-02, 3.1052e-03, 8.2626e-03, ..., 1.6357e-02,
1.6663e-02, 1.5762e-02],
[ 1.7862e-03, 1.0294e-04, 4.2748e-04, ..., 4.0174e-04,
4.5538e-04, 3.9983e-04],
[ 6.8115e-02, 1.1337e-02, 2.0950e-02, ..., 1.4877e-02,
1.6281e-02, 1.6724e-02],
...,
[-7.9285e-02, -1.3260e-02, -2.5253e-02, ..., -2.6566e-02,
-2.7786e-02, -2.7908e-02],
[-8.7678e-05, 6.4820e-07, -4.7654e-05, ..., -4.4513e-04,
-1.5175e-04, -1.6284e-04],
[-4.0710e-02, -2.8210e-03, -9.1019e-03, ..., -1.0178e-02,
-1.1513e-02, -1.0269e-02]], device='cuda:0')
Epoch 7, bias, value: tensor([-0.0170, -0.3173, 0.0795, -0.2454, 0.0139, 0.3444, 0.1572],
device='cuda:0'), grad: tensor([ 0.0901, 0.0056, 0.1676, 0.0627, -0.2017, -0.0006, -0.1240],
device='cuda:0')
351
0.009819814303479266
changing lr
epoch 6, time 485.87, cls_loss 0.8556 cls_loss_mapping 1.3801 cls_loss_causal 1.7626 re_mapping 0.0591 re_causal 0.0588 /// teacc 16.46 lr 0.00975528
Epoch 8, weight, value: tensor([[-0.0206, -0.0165, 0.0098, ..., -0.0012, 0.0856, 0.1165],
[ 0.0669, 0.0905, 0.0646, ..., 0.0561, 0.0145, 0.0017],
[ 0.0613, 0.0221, 0.0739, ..., 0.0347, 0.0348, 0.0349],
...,
[-0.1449, -0.1215, -0.1127, ..., 0.0389, 0.0514, 0.0033],
[-0.0210, -0.0022, -0.0147, ..., 0.0576, 0.0264, -0.0269],
[-0.0144, -0.0258, -0.0177, ..., -0.1550, -0.1571, -0.1350]],
device='cuda:0'), grad: tensor([[-1.8387e-02, 2.7108e-04, -4.0817e-03, ..., 1.6880e-03,
-1.1263e-03, 6.9761e-04],
[ 2.3499e-02, 4.3144e-03, 4.4899e-03, ..., 8.8425e-03,
8.3237e-03, 5.8784e-03],
[ 1.4435e-02, 1.4095e-03, 2.9545e-03, ..., 3.0537e-03,
3.5877e-03, 2.1877e-03],
...,
[-4.3671e-02, -9.4910e-03, -8.1940e-03, ..., -1.9760e-02,
-1.8127e-02, -1.3351e-02],
[-3.3302e-03, -8.7172e-06, -5.5170e-04, ..., -8.2636e-04,
-2.4056e-04, -1.7405e-04],
[ 1.0490e-02, 1.2846e-03, 1.9875e-03, ..., 2.7065e-03,
2.8782e-03, 1.7948e-03]], device='cuda:0')
Epoch 8, bias, value: tensor([-0.0110, -0.3280, 0.0902, -0.2517, 0.0217, 0.3492, 0.1446],
device='cuda:0'), grad: tensor([-0.0444, 0.0637, 0.0409, 0.0447, -0.1157, -0.0180, 0.0287],
device='cuda:0')
351
0.009755282581475767
changing lr
epoch 7, time 487.93, cls_loss 0.7146 cls_loss_mapping 1.2877 cls_loss_causal 1.6994 re_mapping 0.0594 re_causal 0.0592 /// teacc 34.60 lr 0.00968117
Epoch 9, weight, value: tensor([[-0.0208, -0.0210, 0.0043, ..., -0.0036, 0.0879, 0.1206],
[ 0.0655, 0.0872, 0.0628, ..., 0.0514, 0.0141, -0.0047],
[ 0.0660, 0.0297, 0.0799, ..., 0.0356, 0.0282, 0.0334],
...,
[-0.1432, -0.1176, -0.1069, ..., 0.0358, 0.0539, 0.0041],
[-0.0209, -0.0021, -0.0152, ..., 0.0649, 0.0277, -0.0243],
[-0.0217, -0.0287, -0.0192, ..., -0.1577, -0.1638, -0.1406]],
device='cuda:0'), grad: tensor([[-7.3004e-04, -3.0255e-04, -2.6321e-04, ..., 3.6263e-04,
1.9729e-04, 2.9969e-04],
[ 4.7493e-04, 2.4632e-05, 8.4996e-05, ..., 2.2411e-04,
2.1279e-04, 2.3365e-04],
[-1.7941e-05, -1.2837e-05, -1.4558e-05, ..., 2.2948e-06,
-4.7162e-06, -4.2170e-06],
...,
[-1.8625e-03, -1.7643e-05, -2.9802e-04, ..., -1.2522e-03,
-1.2255e-03, -1.3466e-03],
[ 4.2328e-02, 1.7605e-03, 7.6981e-03, ..., 1.1963e-02,
7.0267e-03, 8.6975e-03],
[-4.2358e-02, -1.7443e-03, -7.6790e-03, ..., -1.1917e-02,
-6.9313e-03, -8.6136e-03]], device='cuda:0')
Epoch 9, bias, value: tensor([-0.0195, -0.3211, 0.0706, -0.2405, 0.0008, 0.3643, 0.1601],
device='cuda:0'), grad: tensor([-1.0853e-03, 1.5802e-03, -1.3411e-05, 6.2561e-03, -6.5804e-03,
1.4355e-01, -1.4368e-01], device='cuda:0')
351
0.009681174353198686
changing lr
epoch 8, time 487.83, cls_loss 0.6065 cls_loss_mapping 1.1835 cls_loss_causal 1.6608 re_mapping 0.0617 re_causal 0.0615 /// teacc 49.37 lr 0.00959764
Epoch 10, weight, value: tensor([[-2.4050e-02, -1.7761e-02, 3.4237e-03, ..., -1.7390e-03,
8.9277e-02, 1.2650e-01],
[ 6.3247e-02, 8.4007e-02, 6.1289e-02, ..., 5.2398e-02,
1.3658e-02, -8.1415e-03],
[ 6.9869e-02, 3.1879e-02, 8.0394e-02, ..., 2.8270e-02,
1.8491e-02, 2.4165e-02],
...,
[-1.4282e-01, -1.2039e-01, -1.0256e-01, ..., 2.5484e-02,
5.2426e-02, 1.4157e-04],
[-2.1257e-02, -1.3096e-03, -1.7762e-02, ..., 7.3701e-02,
2.8826e-02, -2.2097e-02],
[-2.1198e-02, -2.8250e-02, -1.8889e-02, ..., -1.6167e-01,
-1.6683e-01, -1.4144e-01]], device='cuda:0'), grad: tensor([[ 1.7500e-03, 3.7360e-04, 5.3549e-04, ..., 3.4451e-04,
4.9019e-04, 4.7493e-04],
[ 2.2078e-04, 5.2124e-05, 7.2539e-05, ..., 4.3780e-05,
6.1393e-05, 5.5730e-05],
[-4.3671e-02, -8.8120e-03, -1.5808e-02, ..., -1.7059e-02,
-1.6403e-02, -1.5350e-02],
...,
[ 3.6896e-02, 7.3662e-03, 1.3657e-02, ..., 1.5488e-02,
1.4389e-02, 1.3420e-02],
[-5.0688e-04, 7.3314e-06, -6.0052e-05, ..., -1.6499e-04,
-7.8738e-05, -2.9728e-05],
[ 1.5955e-03, 2.3997e-04, 4.1461e-04, ..., 3.8505e-04,
3.8671e-04, 3.2830e-04]], device='cuda:0')
Epoch 10, bias, value: tensor([-0.0185, -0.3178, 0.0688, -0.2319, 0.0088, 0.3629, 0.1423],
device='cuda:0'), grad: tensor([ 0.0046, 0.0006, -0.1209, 0.0099, 0.1028, -0.0015, 0.0044],
device='cuda:0')
351
0.009597638862757255
changing lr
---------------------saving model at epoch 9----------------------------------------------------
epoch 9, time 496.62, cls_loss 0.5204 cls_loss_mapping 1.0647 cls_loss_causal 1.5735 re_mapping 0.0619 re_causal 0.0617 /// teacc 78.48 lr 0.00950484
Epoch 11, weight, value: tensor([[-0.0143, -0.0127, 0.0037, ..., 0.0025, 0.0978, 0.1330],
[ 0.0605, 0.0804, 0.0595, ..., 0.0587, 0.0218, -0.0022],
[ 0.0698, 0.0287, 0.0795, ..., 0.0239, 0.0185, 0.0256],
...,
[-0.1494, -0.1267, -0.1035, ..., 0.0184, 0.0450, -0.0050],
[-0.0252, 0.0017, -0.0171, ..., 0.0766, 0.0278, -0.0212],
[-0.0184, -0.0282, -0.0203, ..., -0.1655, -0.1755, -0.1504]],
device='cuda:0'), grad: tensor([[ 1.2459e-02, 1.3876e-03, 2.9373e-03, ..., 2.5539e-03,
2.7657e-03, 3.2463e-03],
[ 1.9388e-03, 1.8775e-04, 4.2844e-04, ..., 4.3082e-04,
4.6372e-04, 5.4216e-04],
[ 5.2612e-02, 1.4328e-02, 2.0798e-02, ..., 4.3564e-03,
5.9853e-03, 6.3286e-03],
...,
[ 2.6886e-02, 2.6340e-03, 6.0768e-03, ..., 5.0621e-03,
5.5580e-03, 6.7787e-03],
[ 6.6996e-05, 9.0525e-06, 1.7419e-05, ..., 3.0063e-06,
5.5954e-06, 9.2536e-06],
[-9.5337e-02, -1.8707e-02, -3.0579e-02, ..., -1.2718e-02,
-1.5114e-02, -1.7288e-02]], device='cuda:0')
Epoch 11, bias, value: tensor([ 0.0015, -0.3192, 0.0555, -0.2433, 0.0173, 0.3439, 0.1586],
device='cuda:0'), grad: tensor([ 4.2358e-02, 6.6528e-03, 1.5479e-01, 4.8218e-03, 9.2896e-02,
2.2638e-04, -3.0176e-01], device='cuda:0')
351
0.009504844339512096
changing lr
epoch 10, time 488.71, cls_loss 0.4416 cls_loss_mapping 0.9614 cls_loss_causal 1.5411 re_mapping 0.0639 re_causal 0.0637 /// teacc 43.88 lr 0.00940298
Epoch 12, weight, value: tensor([[-0.0199, -0.0105, 0.0022, ..., 0.0044, 0.1020, 0.1346],
[ 0.0602, 0.0779, 0.0589, ..., 0.0587, 0.0236, -0.0019],
[ 0.0649, 0.0265, 0.0755, ..., 0.0258, 0.0215, 0.0291],
...,
[-0.1561, -0.1310, -0.1052, ..., 0.0121, 0.0383, -0.0080],
[-0.0179, 0.0020, -0.0169, ..., 0.0814, 0.0301, -0.0190],
[-0.0118, -0.0252, -0.0170, ..., -0.1677, -0.1806, -0.1532]],
device='cuda:0'), grad: tensor([[ 6.2752e-04, 8.8215e-05, 1.4699e-04, ..., 1.3494e-04,
8.3327e-05, 4.4316e-05],
[ 3.3450e-04, 4.9382e-05, 9.3460e-05, ..., 3.2276e-05,
-1.6272e-05, -3.8147e-05],
[-1.0910e-02, -1.5612e-03, -2.5730e-03, ..., -2.1000e-03,
-1.0977e-03, -4.0007e-04],
...,
[ 5.4455e-04, 8.6308e-05, 1.0788e-04, ..., 6.6996e-05,
9.3579e-06, -3.2425e-05],
[ 3.1796e-03, 4.5466e-04, 7.5006e-04, ..., 6.1369e-04,
3.2234e-04, 1.1897e-04],
[ 3.2673e-03, 4.6039e-04, 7.7772e-04, ..., 6.8188e-04,
4.0054e-04, 1.9753e-04]], device='cuda:0')
Epoch 12, bias, value: tensor([-0.0175, -0.3137, 0.0387, -0.2354, -0.0062, 0.3748, 0.1735],
device='cuda:0'), grad: tensor([ 0.0018, 0.0009, -0.0317, 0.0086, 0.0015, 0.0092, 0.0096],
device='cuda:0')
351
0.009402977659283692
changing lr
---------------------saving model at epoch 11----------------------------------------------------
epoch 11, time 494.18, cls_loss 0.3891 cls_loss_mapping 0.8157 cls_loss_causal 1.4446 re_mapping 0.0666 re_causal 0.0663 /// teacc 78.90 lr 0.00929224
Epoch 13, weight, value: tensor([[-0.0114, -0.0099, 0.0014, ..., 0.0046, 0.1073, 0.1377],
[ 0.0535, 0.0753, 0.0569, ..., 0.0620, 0.0234, -0.0039],
[ 0.0735, 0.0337, 0.0822, ..., 0.0240, 0.0212, 0.0305],
...,
[-0.1620, -0.1316, -0.1037, ..., 0.0099, 0.0368, -0.0077],
[-0.0194, 0.0019, -0.0192, ..., 0.0848, 0.0300, -0.0185],
[-0.0162, -0.0297, -0.0210, ..., -0.1704, -0.1856, -0.1569]],
device='cuda:0'), grad: tensor([[ 4.6611e-05, 1.1183e-05, 1.9923e-05, ..., -2.2575e-05,
-2.5004e-05, -2.1055e-05],
[ 1.0766e-05, 3.0249e-06, 4.0717e-06, ..., 2.8312e-06,
2.3656e-06, 2.8126e-06],
[-1.6193e-03, -5.1022e-04, -6.5613e-04, ..., -5.1498e-04,
-3.5930e-04, -4.0555e-04],
...,
[ 1.3876e-04, 3.0369e-05, 4.1813e-05, ..., 6.6936e-05,
7.4625e-05, 8.1360e-05],
[ 2.7239e-05, 7.3612e-06, 1.0043e-05, ..., 7.8529e-06,
6.9402e-06, 8.0913e-06],
[ 6.2734e-06, 2.1681e-05, 1.8269e-05, ..., 1.8656e-05,
-9.0450e-06, -1.5110e-05]], device='cuda:0')
Epoch 13, bias, value: tensor([-9.4110e-05, -3.2179e-01, 3.0043e-02, -2.2424e-01, -2.7306e-02,
3.7682e-01, 1.8037e-01], device='cuda:0'), grad: tensor([ 1.3185e-04, 2.7537e-05, -3.9215e-03, 3.3684e-03, 3.8481e-04,
7.0870e-05, -6.4135e-05], device='cuda:0')
351
0.009292243968009333
changing lr
epoch 12, time 487.57, cls_loss 0.3192 cls_loss_mapping 0.7209 cls_loss_causal 1.3775 re_mapping 0.0656 re_causal 0.0653 /// teacc 63.71 lr 0.00917287
Epoch 14, weight, value: tensor([[-0.0128, -0.0094, 0.0006, ..., 0.0067, 0.1094, 0.1410],
[ 0.0535, 0.0740, 0.0570, ..., 0.0602, 0.0213, -0.0071],
[ 0.0651, 0.0308, 0.0764, ..., 0.0198, 0.0181, 0.0274],
...,
[-0.1485, -0.1289, -0.0992, ..., 0.0107, 0.0365, -0.0067],
[-0.0197, 0.0018, -0.0197, ..., 0.0855, 0.0319, -0.0186],
[-0.0218, -0.0307, -0.0218, ..., -0.1687, -0.1871, -0.1557]],
device='cuda:0'), grad: tensor([[ 3.2940e-03, 4.0150e-04, 5.7077e-04, ..., 5.7507e-04,
1.0185e-03, 9.1982e-04],
[ 1.8632e-04, 2.0161e-05, 5.4568e-05, ..., 1.0991e-04,
1.2201e-04, 1.3220e-04],
[ 1.4639e-04, 1.1936e-05, 3.6836e-05, ..., 5.2154e-05,
4.9412e-05, 5.9545e-05],
...,
[ 3.3045e-04, 9.2089e-06, 7.9036e-05, ..., 4.6194e-05,
-1.2025e-05, 2.9653e-05],
[ 4.0016e-03, 2.8110e-04, 1.0900e-03, ..., 1.6613e-03,
1.4162e-03, 1.8120e-03],
[-8.0948e-03, -7.3528e-04, -1.8692e-03, ..., -2.5005e-03,
-2.6455e-03, -3.0155e-03]], device='cuda:0')
Epoch 14, bias, value: tensor([-0.0019, -0.3214, 0.0035, -0.2192, 0.0107, 0.3626, 0.1791],
device='cuda:0'), grad: tensor([ 0.0094, 0.0007, 0.0006, 0.0006, 0.0016, 0.0171, -0.0299],
device='cuda:0')
351
0.009172866268606516
changing lr
epoch 13, time 489.06, cls_loss 0.2306 cls_loss_mapping 0.6268 cls_loss_causal 1.2624 re_mapping 0.0687 re_causal 0.0684 /// teacc 16.46 lr 0.00904508
Epoch 15, weight, value: tensor([[-0.0146, -0.0088, -0.0035, ..., 0.0025, 0.1056, 0.1395],
[ 0.0602, 0.0743, 0.0604, ..., 0.0627, 0.0234, -0.0063],
[ 0.0683, 0.0323, 0.0766, ..., 0.0169, 0.0138, 0.0250],
...,
[-0.1461, -0.1274, -0.0961, ..., 0.0092, 0.0416, -0.0031],
[-0.0188, 0.0011, -0.0206, ..., 0.0915, 0.0352, -0.0144],
[-0.0272, -0.0327, -0.0228, ..., -0.1682, -0.1902, -0.1596]],
device='cuda:0'), grad: tensor([[ 0.0088, 0.0010, 0.0025, ..., 0.0017, 0.0029, 0.0036],
[ 0.0007, 0.0002, 0.0003, ..., 0.0004, 0.0003, 0.0004],
[-0.0638, -0.0165, -0.0286, ..., -0.0233, -0.0222, -0.0276],
...,
[ 0.0265, 0.0029, 0.0075, ..., 0.0056, 0.0092, 0.0112],
[ 0.0006, 0.0003, 0.0004, ..., 0.0004, 0.0002, 0.0003],
[ 0.0212, 0.0095, 0.0139, ..., 0.0119, 0.0074, 0.0095]],
device='cuda:0')
Epoch 15, bias, value: tensor([-0.0048, -0.3123, 0.0152, -0.2279, 0.0035, 0.3635, 0.1762],
device='cuda:0'), grad: tensor([ 0.0277, 0.0026, -0.2307, 0.0250, 0.0837, 0.0027, 0.0891],
device='cuda:0')
351
0.00904508497187474
changing lr
epoch 14, time 487.91, cls_loss 0.2457 cls_loss_mapping 0.5984 cls_loss_causal 1.2677 re_mapping 0.0668 re_causal 0.0666 /// teacc 56.12 lr 0.00890916
Epoch 16, weight, value: tensor([[-0.0190, -0.0126, -0.0079, ..., -0.0027, 0.1032, 0.1370],
[ 0.0610, 0.0744, 0.0602, ..., 0.0630, 0.0232, -0.0067],
[ 0.0704, 0.0346, 0.0775, ..., 0.0177, 0.0127, 0.0243],
...,
[-0.1485, -0.1284, -0.0956, ..., 0.0113, 0.0454, 0.0003],
[-0.0218, 0.0008, -0.0216, ..., 0.0921, 0.0343, -0.0144],
[-0.0231, -0.0305, -0.0202, ..., -0.1639, -0.1877, -0.1558]],
device='cuda:0'), grad: tensor([[ 3.6216e-04, 6.3956e-05, 7.0810e-05, ..., 1.5664e-04,
1.2243e-04, 1.6391e-04],
[ 3.8025e-02, 7.5226e-03, 1.1467e-02, ..., 1.0361e-02,
1.5282e-02, 1.5373e-02],
[-3.8666e-02, -7.6523e-03, -1.1673e-02, ..., -1.0529e-02,
-1.5549e-02, -1.5640e-02],
...,
[ 1.2600e-04, 4.2140e-05, 8.7678e-05, ..., -1.9401e-05,
7.9334e-05, 3.8594e-05],
[ 1.6558e-04, 3.2425e-05, 4.8608e-05, ..., 4.7058e-05,
6.5863e-05, 6.7592e-05],
[ 6.8665e-05, 1.3933e-05, 2.0951e-05, ..., 1.8701e-05,
2.7299e-05, 2.7537e-05]], device='cuda:0')
Epoch 16, bias, value: tensor([-0.0068, -0.2992, 0.0154, -0.2199, -0.0004, 0.3446, 0.1795],
device='cuda:0'), grad: tensor([ 1.2054e-03, 1.0278e-01, -1.0455e-01, -1.5152e-04, 6.4135e-05,
4.5562e-04, 1.8466e-04], device='cuda:0')
351
0.008909157412340152
changing lr
epoch 15, time 487.09, cls_loss 0.1644 cls_loss_mapping 0.5142 cls_loss_causal 1.1892 re_mapping 0.0672 re_causal 0.0669 /// teacc 66.67 lr 0.00876536
Epoch 17, weight, value: tensor([[-0.0198, -0.0133, -0.0073, ..., -0.0034, 0.1047, 0.1385],
[ 0.0540, 0.0720, 0.0575, ..., 0.0566, 0.0184, -0.0133],
[ 0.0707, 0.0354, 0.0767, ..., 0.0200, 0.0152, 0.0280],
...,
[-0.1433, -0.1255, -0.0916, ..., 0.0082, 0.0417, -0.0034],
[-0.0192, 0.0005, -0.0220, ..., 0.0945, 0.0362, -0.0125],
[-0.0222, -0.0315, -0.0223, ..., -0.1588, -0.1861, -0.1524]],
device='cuda:0'), grad: tensor([[ 5.0049e-03, 6.1369e-04, 1.1215e-03, ..., 2.3670e-03,
2.7428e-03, 3.0746e-03],
[-5.3482e-03, -6.4516e-04, -1.1797e-03, ..., -2.5311e-03,
-2.9335e-03, -3.2787e-03],
[ 1.3359e-05, 2.4792e-06, 3.2727e-06, ..., 4.5523e-06,
4.6417e-06, 5.2229e-06],
...,
[ 4.4912e-05, 8.4862e-06, 1.0416e-05, ..., 1.3746e-05,
1.2174e-05, 1.3910e-05],
[ 2.6345e-04, 5.5492e-05, 6.3419e-05, ..., 6.0797e-05,
4.9591e-05, 5.6654e-05],
[ 1.0535e-05, -3.6955e-05, -2.1502e-05, ..., 8.2850e-05,
1.2177e-04, 1.2362e-04]], device='cuda:0')
Epoch 17, bias, value: tensor([-0.0110, -0.3091, 0.0176, -0.2245, 0.0073, 0.3497, 0.1830],
device='cuda:0'), grad: tensor([ 1.5442e-02, -1.6510e-02, 3.9279e-05, 2.3291e-05, 1.3268e-04,
7.6675e-04, 9.7513e-05], device='cuda:0')
351
0.00876535733001806
changing lr
epoch 16, time 486.79, cls_loss 0.1451 cls_loss_mapping 0.4625 cls_loss_causal 1.1703 re_mapping 0.0665 re_causal 0.0663 /// teacc 75.95 lr 0.00861397
Epoch 18, weight, value: tensor([[-0.0251, -0.0154, -0.0094, ..., -0.0082, 0.1041, 0.1376],
[ 0.0554, 0.0698, 0.0561, ..., 0.0582, 0.0230, -0.0094],
[ 0.0726, 0.0386, 0.0782, ..., 0.0188, 0.0155, 0.0284],
...,
[-0.1426, -0.1242, -0.0895, ..., 0.0081, 0.0359, -0.0078],
[-0.0184, 0.0003, -0.0225, ..., 0.0962, 0.0364, -0.0117],
[-0.0208, -0.0312, -0.0229, ..., -0.1566, -0.1876, -0.1535]],
device='cuda:0'), grad: tensor([[-3.5076e-03, -3.7932e-04, -9.4175e-04, ..., -8.6021e-04,
-2.0180e-03, -2.1801e-03],
[ 3.5357e-04, 4.5151e-05, 9.7215e-05, ..., 8.0526e-05,
1.9038e-04, 2.0647e-04],
[-2.5787e-03, -8.8978e-04, -1.0452e-03, ..., -4.4012e-04,
-3.0732e-04, -2.9087e-04],
...,
[ 3.7956e-04, 7.5400e-05, 1.2279e-04, ..., 8.1301e-05,
1.5175e-04, 1.6046e-04],
[ 4.0770e-04, 1.2398e-04, 1.5557e-04, ..., 7.5758e-05,
7.9870e-05, 8.0645e-05],
[ 3.2368e-03, 4.5490e-04, 9.3269e-04, ..., 7.6056e-04,
1.6651e-03, 1.7910e-03]], device='cuda:0')
Epoch 18, bias, value: tensor([-0.0241, -0.2958, 0.0095, -0.2232, 0.0036, 0.3491, 0.1936],
device='cuda:0'), grad: tensor([-0.0110, 0.0011, -0.0058, 0.0039, 0.0011, 0.0010, 0.0097],
device='cuda:0')
351
0.008613974319136962
changing lr
epoch 17, time 490.87, cls_loss 0.1359 cls_loss_mapping 0.4107 cls_loss_causal 1.1271 re_mapping 0.0690 re_causal 0.0689 /// teacc 31.65 lr 0.00845531
Epoch 19, weight, value: tensor([[-0.0262, -0.0149, -0.0089, ..., -0.0061, 0.1036, 0.1387],
[ 0.0530, 0.0664, 0.0550, ..., 0.0562, 0.0261, -0.0083],
[ 0.0759, 0.0417, 0.0804, ..., 0.0192, 0.0141, 0.0275],
...,
[-0.1386, -0.1222, -0.0873, ..., 0.0064, 0.0339, -0.0098],
[-0.0211, -0.0006, -0.0238, ..., 0.0952, 0.0354, -0.0127],
[-0.0220, -0.0315, -0.0255, ..., -0.1548, -0.1872, -0.1521]],
device='cuda:0'), grad: tensor([[ 2.1145e-05, 1.6410e-06, 2.7213e-06, ..., 1.2532e-05,
9.9614e-06, 1.2040e-05],
[ 4.3586e-06, 5.0291e-07, 8.1584e-07, ..., 2.3693e-06,
1.9353e-06, 2.2948e-06],
[-1.2871e-06, -4.6566e-07, -8.1956e-07, ..., 7.4506e-09,
-4.6380e-07, -6.3702e-07],
...,
[-3.9369e-05, -3.1348e-06, -5.4277e-06, ..., -2.2262e-05,
-1.7211e-05, -2.0772e-05],
[ 7.0855e-06, 7.5996e-07, 1.2480e-06, ..., 3.7327e-06,
2.9169e-06, 3.5390e-06],
[ 8.8438e-06, 1.6782e-06, 2.3842e-06, ..., 4.1127e-06,
3.3025e-06, 4.1015e-06]], device='cuda:0')
Epoch 19, bias, value: tensor([-0.0223, -0.2936, 0.0166, -0.2197, 0.0082, 0.3351, 0.1881],
device='cuda:0'), grad: tensor([ 6.7830e-05, 1.3463e-05, -3.2894e-06, 3.5204e-07, -1.2648e-04,
2.2292e-05, 2.5839e-05], device='cuda:0')
351
0.008455313244934327
changing lr
epoch 18, time 489.60, cls_loss 0.0976 cls_loss_mapping 0.3568 cls_loss_causal 1.0910 re_mapping 0.0673 re_causal 0.0673 /// teacc 77.22 lr 0.00828969
Epoch 20, weight, value: tensor([[-0.0217, -0.0144, -0.0075, ..., -0.0047, 0.1070, 0.1415],
[ 0.0498, 0.0659, 0.0546, ..., 0.0559, 0.0258, -0.0093],
[ 0.0745, 0.0411, 0.0786, ..., 0.0179, 0.0144, 0.0282],
...,
[-0.1379, -0.1209, -0.0862, ..., 0.0031, 0.0304, -0.0127],
[-0.0152, 0.0002, -0.0226, ..., 0.0979, 0.0375, -0.0094],
[-0.0275, -0.0336, -0.0282, ..., -0.1550, -0.1903, -0.1554]],
device='cuda:0'), grad: tensor([[ 6.8140e-04, 9.6321e-05, 2.1148e-04, ..., 1.2398e-04,
2.8491e-04, 2.6059e-04],
[ 8.2016e-05, 1.1437e-05, 2.5108e-05, ..., 1.6108e-05,
3.4273e-05, 3.1531e-05],
[ 7.6714e-03, 1.0843e-03, 2.3823e-03, ..., 1.3924e-03,
3.2101e-03, 2.9316e-03],
...,
[-8.6060e-03, -1.2188e-03, -2.6779e-03, ..., -1.5488e-03,
-3.6030e-03, -3.2883e-03],
[ 8.0839e-06, 2.9299e-06, 6.7875e-06, ..., -1.3940e-05,
3.6377e-06, 3.3900e-07],
[ 8.2791e-05, 1.1660e-05, 2.5585e-05, ..., 1.5527e-05,
3.4660e-05, 3.1769e-05]], device='cuda:0')
Epoch 20, bias, value: tensor([-0.0104, -0.3015, 0.0106, -0.2195, 0.0064, 0.3531, 0.1737],
device='cuda:0'), grad: tensor([ 2.2831e-03, 2.7514e-04, 2.5711e-02, 2.7728e-04, -2.8854e-02,
2.4706e-05, 2.7752e-04], device='cuda:0')
351
0.008289693629698565
changing lr
---------------------saving model at epoch 19----------------------------------------------------
epoch 19, time 497.06, cls_loss 0.1102 cls_loss_mapping 0.3219 cls_loss_causal 1.0679 re_mapping 0.0690 re_causal 0.0690 /// teacc 85.65 lr 0.00811745
Epoch 21, weight, value: tensor([[-0.0224, -0.0154, -0.0073, ..., -0.0084, 0.1079, 0.1397],
[ 0.0464, 0.0648, 0.0531, ..., 0.0535, 0.0237, -0.0116],
[ 0.0759, 0.0387, 0.0755, ..., 0.0178, 0.0120, 0.0267],
...,
[-0.1408, -0.1212, -0.0868, ..., 0.0026, 0.0276, -0.0143],
[-0.0176, -0.0003, -0.0230, ..., 0.0976, 0.0375, -0.0091],
[-0.0218, -0.0310, -0.0260, ..., -0.1495, -0.1864, -0.1501]],
device='cuda:0'), grad: tensor([[ 4.0054e-05, 2.4103e-06, 7.0930e-06, ..., 1.6317e-05,
9.4175e-06, 1.6287e-05],
[ 2.0787e-06, 9.4995e-08, 3.2783e-07, ..., 8.7358e-07,
4.6566e-07, 8.5682e-07],
[ 8.4378e-07, 3.7253e-08, 1.4342e-07, ..., 3.5577e-07,
2.1048e-07, 3.4459e-07],
...,
[-1.4752e-06, -2.1793e-07, -4.3400e-07, ..., -1.2107e-06,
-1.3504e-06, -1.4361e-06],
[ 1.5274e-06, 6.7055e-08, 2.4587e-07, ..., 6.0722e-07,
3.1851e-07, 6.0722e-07],
[-4.6432e-05, -3.0883e-06, -8.5682e-06, ..., -1.7300e-05,
-9.3505e-06, -1.7017e-05]], device='cuda:0')
Epoch 21, bias, value: tensor([-0.0157, -0.3083, 0.0252, -0.2177, -0.0058, 0.3422, 0.1923],
device='cuda:0'), grad: tensor([ 1.3244e-04, 6.9551e-06, 2.8443e-06, 9.5814e-06, -4.6417e-06,
5.1148e-06, -1.5259e-04], device='cuda:0')
351
0.00811744900929367
changing lr
---------------------saving model at epoch 20----------------------------------------------------
epoch 20, time 492.87, cls_loss 0.0760 cls_loss_mapping 0.2840 cls_loss_causal 1.0343 re_mapping 0.0687 re_causal 0.0688 /// teacc 88.61 lr 0.00793893
Epoch 22, weight, value: tensor([[-2.5705e-02, -1.5634e-02, -7.1052e-03, ..., -1.0747e-02,
1.0806e-01, 1.3788e-01],
[ 5.1844e-02, 6.5237e-02, 5.4339e-02, ..., 5.6280e-02,
2.6188e-02, -9.1537e-03],
[ 6.8620e-02, 3.6508e-02, 7.1406e-02, ..., 1.6478e-02,
9.5231e-03, 2.5444e-02],
...,
[-1.3768e-01, -1.1991e-01, -8.5010e-02, ..., 2.3305e-03,
2.6403e-02, -1.5035e-02],
[-1.5751e-02, -7.5948e-05, -2.3335e-02, ..., 9.7384e-02,
3.7156e-02, -9.2214e-03],
[-2.1356e-02, -3.0200e-02, -2.5406e-02, ..., -1.4869e-01,
-1.8599e-01, -1.4877e-01]], device='cuda:0'), grad: tensor([[ 2.1867e-06, 2.5891e-07, 4.9360e-07, ..., 2.1048e-07,
5.0291e-08, 2.0303e-07],
[ 5.4315e-06, 6.2026e-07, 1.2163e-06, ..., 5.7369e-07,
2.3656e-07, 6.0908e-07],
[ 5.8375e-06, 2.1812e-06, 3.0119e-06, ..., 1.9372e-06,
2.6189e-06, 2.4084e-06],
...,
[ 5.4896e-05, 2.3603e-05, 3.2067e-05, ..., 1.9833e-05,
2.8804e-05, 2.5585e-05],
[ 2.3600e-06, 1.9185e-07, 4.9174e-07, ..., -3.5390e-08,
-1.6019e-07, 3.3528e-08],
[-1.3016e-05, -1.3337e-06, -2.7455e-06, ..., -1.2256e-06,
-3.1479e-07, -1.2722e-06]], device='cuda:0')
Epoch 22, bias, value: tensor([-0.0246, -0.2882, 0.0090, -0.2145, -0.0027, 0.3421, 0.1908],
device='cuda:0'), grad: tensor([ 7.1079e-06, 1.7703e-05, 1.3880e-05, -1.2386e-04, 1.2010e-04,
7.9721e-06, -4.2945e-05], device='cuda:0')
351
0.007938926261462368
changing lr
---------------------saving model at epoch 21----------------------------------------------------
epoch 21, time 495.37, cls_loss 0.0555 cls_loss_mapping 0.2618 cls_loss_causal 1.0042 re_mapping 0.0691 re_causal 0.0694 /// teacc 89.03 lr 0.00775448
Epoch 23, weight, value: tensor([[-0.0250, -0.0152, -0.0076, ..., -0.0115, 0.1060, 0.1355],
[ 0.0525, 0.0647, 0.0542, ..., 0.0553, 0.0270, -0.0090],
[ 0.0669, 0.0349, 0.0696, ..., 0.0153, 0.0090, 0.0249],
...,
[-0.1370, -0.1186, -0.0835, ..., 0.0030, 0.0262, -0.0145],
[-0.0132, -0.0003, -0.0236, ..., 0.0985, 0.0381, -0.0077],
[-0.0240, -0.0302, -0.0255, ..., -0.1479, -0.1849, -0.1475]],
device='cuda:0'), grad: tensor([[-5.6386e-05, -6.1654e-06, -1.6123e-05, ..., -5.9873e-05,
-7.1406e-05, -7.0572e-05],
[ 2.3186e-04, 3.1978e-05, 7.4446e-05, ..., 1.0973e-04,
9.6023e-05, 1.0985e-04],
[-6.2808e-06, -3.4869e-06, -7.7672e-07, ..., 2.3004e-06,
1.0543e-06, 1.2200e-06],
...,
[ 5.5641e-05, 1.2450e-05, 1.6242e-05, ..., 1.5393e-05,
9.5591e-06, 1.1526e-05],
[ 5.9242e-03, 7.5436e-04, 2.0351e-03, ..., 1.7452e-03,
8.3017e-04, 1.3838e-03],
[-6.1455e-03, -7.8821e-04, -2.1076e-03, ..., -1.8110e-03,
-8.6451e-04, -1.4343e-03]], device='cuda:0')
Epoch 23, bias, value: tensor([-0.0237, -0.2811, 0.0093, -0.2139, -0.0032, 0.3464, 0.1781],
device='cuda:0'), grad: tensor([-1.8227e-04, 7.1716e-04, -3.7491e-05, -1.1344e-06, 1.5235e-04,
1.8356e-02, -1.8997e-02], device='cuda:0')
351
0.007754484907260515
changing lr
epoch 22, time 489.20, cls_loss 0.0471 cls_loss_mapping 0.2469 cls_loss_causal 0.9975 re_mapping 0.0680 re_causal 0.0683 /// teacc 82.70 lr 0.00756450
Epoch 24, weight, value: tensor([[-0.0221, -0.0143, -0.0068, ..., -0.0097, 0.1085, 0.1372],
[ 0.0452, 0.0622, 0.0522, ..., 0.0519, 0.0224, -0.0141],
[ 0.0691, 0.0355, 0.0695, ..., 0.0148, 0.0087, 0.0250],
...,
[-0.1330, -0.1165, -0.0813, ..., 0.0051, 0.0275, -0.0122],
[-0.0186, -0.0018, -0.0254, ..., 0.0961, 0.0369, -0.0089],
[-0.0209, -0.0295, -0.0248, ..., -0.1462, -0.1833, -0.1452]],
device='cuda:0'), grad: tensor([[ 1.4174e-04, 1.3284e-05, 2.9057e-05, ..., 1.0335e-04,
7.4446e-05, 7.3075e-05],
[ 1.5712e-04, 1.6555e-05, 2.9877e-05, ..., 8.7261e-05,
7.5936e-05, 7.2062e-05],
[ 1.4429e-03, 1.5485e-04, 2.7990e-04, ..., 8.6403e-04,
7.5531e-04, 7.2527e-04],
...,
[ 1.9875e-03, 2.1029e-04, 3.7670e-04, ..., 1.0900e-03,
9.5558e-04, 9.0551e-04],
[-4.4212e-03, -4.6921e-04, -8.4877e-04, ..., -2.5463e-03,
-2.2163e-03, -2.1152e-03],
[ 4.1676e-04, 4.4823e-05, 7.9811e-05, ..., 2.3985e-04,
2.1195e-04, 2.0242e-04]], device='cuda:0')
Epoch 24, bias, value: tensor([-0.0165, -0.2987, 0.0153, -0.2091, 0.0057, 0.3272, 0.1878],
device='cuda:0'), grad: tensor([ 0.0006, 0.0006, 0.0057, 0.0011, 0.0080, -0.0178, 0.0017],
device='cuda:0')
351
0.007564496387029534
changing lr
epoch 23, time 487.18, cls_loss 0.0468 cls_loss_mapping 0.2215 cls_loss_causal 0.9253 re_mapping 0.0658 re_causal 0.0661 /// teacc 74.68 lr 0.00736934
Epoch 25, weight, value: tensor([[-0.0234, -0.0146, -0.0073, ..., -0.0096, 0.1084, 0.1368],
[ 0.0481, 0.0616, 0.0525, ..., 0.0520, 0.0238, -0.0126],
[ 0.0674, 0.0362, 0.0695, ..., 0.0141, 0.0085, 0.0248],
...,
[-0.1308, -0.1160, -0.0815, ..., 0.0056, 0.0258, -0.0134],
[-0.0168, -0.0017, -0.0253, ..., 0.0961, 0.0370, -0.0083],
[-0.0243, -0.0299, -0.0251, ..., -0.1461, -0.1824, -0.1447]],
device='cuda:0'), grad: tensor([[ 6.6817e-05, 1.1541e-05, 1.8194e-05, ..., 4.5657e-05,
4.5270e-05, 4.9025e-05],
[ 6.1035e-05, 1.0975e-05, 1.7136e-05, ..., 4.0889e-05,
4.0948e-05, 4.4376e-05],
[-1.2368e-05, -4.3958e-06, -4.6641e-06, ..., -1.9595e-06,
-1.4678e-06, -2.7642e-06],
...,
[-5.5218e-04, -9.5844e-05, -1.5271e-04, ..., -3.7932e-04,
-3.8004e-04, -4.1032e-04],
[ 3.8296e-05, 7.4282e-06, 1.0990e-05, ..., 2.3633e-05,
2.3440e-05, 2.5749e-05],
[ 4.2021e-06, 1.1772e-06, 1.3858e-06, ..., 1.8775e-06,
1.8701e-06, 2.2650e-06]], device='cuda:0')
Epoch 25, bias, value: tensor([-0.0207, -0.2837, 0.0064, -0.2068, 0.0094, 0.3279, 0.1792],
device='cuda:0'), grad: tensor([ 1.8096e-04, 1.6356e-04, -3.0279e-05, 1.0567e-03, -1.4839e-03,
1.0246e-04, 1.0207e-05], device='cuda:0')
351
0.007369343312364995
changing lr
epoch 24, time 489.43, cls_loss 0.0561 cls_loss_mapping 0.2156 cls_loss_causal 0.9517 re_mapping 0.0632 re_causal 0.0636 /// teacc 79.75 lr 0.00716942
Epoch 26, weight, value: tensor([[-0.0244, -0.0149, -0.0076, ..., -0.0097, 0.1082, 0.1363],
[ 0.0462, 0.0608, 0.0517, ..., 0.0509, 0.0229, -0.0135],
[ 0.0703, 0.0368, 0.0696, ..., 0.0148, 0.0095, 0.0267],
...,
[-0.1327, -0.1161, -0.0814, ..., 0.0048, 0.0241, -0.0150],
[-0.0165, -0.0017, -0.0252, ..., 0.0958, 0.0370, -0.0079],
[-0.0249, -0.0295, -0.0252, ..., -0.1451, -0.1814, -0.1440]],
device='cuda:0'), grad: tensor([[ 5.2631e-05, 7.4133e-06, 9.4920e-06, ..., 2.3797e-05,
2.7403e-05, 2.1726e-05],
[ 1.2308e-05, 2.1905e-06, 8.9407e-08, ..., 5.4166e-06,
1.6391e-06, -1.0058e-06],
[ 6.5804e-04, 8.4043e-05, 1.1456e-04, ..., 1.8108e-04,
1.8704e-04, 1.4186e-04],
...,
[ 1.3752e-03, 2.1374e-04, 1.9753e-04, ..., 6.8903e-04,
6.8665e-04, 4.8661e-04],
[-1.8311e-03, -2.8539e-04, -2.6178e-04, ..., -9.2459e-04,
-9.2125e-04, -6.5231e-04],
[-4.1652e-04, -4.5180e-05, -8.1360e-05, ..., -4.9084e-05,
-5.5760e-05, -4.9591e-05]], device='cuda:0')
Epoch 26, bias, value: tensor([-0.0235, -0.2851, 0.0169, -0.1963, 0.0014, 0.3240, 0.1741],
device='cuda:0'), grad: tensor([ 1.5664e-04, 3.3677e-05, 2.0599e-03, 4.2963e-04, 3.9406e-03,
-5.2376e-03, -1.3800e-03], device='cuda:0')
351
0.0071694186955877925
changing lr
epoch 25, time 486.77, cls_loss 0.0445 cls_loss_mapping 0.2002 cls_loss_causal 0.9541 re_mapping 0.0637 re_causal 0.0641 /// teacc 86.08 lr 0.00696513
Epoch 27, weight, value: tensor([[-0.0224, -0.0148, -0.0069, ..., -0.0090, 0.1102, 0.1378],
[ 0.0452, 0.0599, 0.0507, ..., 0.0500, 0.0225, -0.0140],
[ 0.0656, 0.0353, 0.0671, ..., 0.0134, 0.0080, 0.0254],
...,
[-0.1328, -0.1142, -0.0798, ..., 0.0032, 0.0210, -0.0175],
[-0.0156, -0.0017, -0.0253, ..., 0.0967, 0.0379, -0.0066],
[-0.0209, -0.0287, -0.0242, ..., -0.1424, -0.1791, -0.1420]],
device='cuda:0'), grad: tensor([[ 2.1248e-03, 3.3641e-04, 6.0511e-04, ..., 6.1703e-04,
1.2503e-03, 1.2083e-03],
[-2.7657e-03, -4.6110e-04, -7.8583e-04, ..., -7.3004e-04,
-1.3189e-03, -1.2741e-03],
[ 3.1161e-04, 5.1141e-05, 9.0480e-05, ..., 1.1814e-04,
1.1945e-04, 1.3459e-04],
...,
[-6.3467e-04, -1.0312e-04, -1.8442e-04, ..., -2.4652e-04,
-2.5082e-04, -2.8372e-04],
[ 2.0123e-04, 3.3110e-05, 5.8502e-05, ..., 7.1347e-05,
7.2539e-05, 8.2135e-05],
[ 7.2813e-04, 1.3781e-04, 2.0635e-04, ..., 1.5700e-04,
1.1438e-04, 1.1861e-04]], device='cuda:0')
Epoch 27, bias, value: tensor([-0.0163, -0.2837, 0.0050, -0.1952, -0.0045, 0.3244, 0.1816],
device='cuda:0'), grad: tensor([ 0.0065, -0.0087, 0.0010, 0.0001, -0.0021, 0.0007, 0.0025],
device='cuda:0')
351
0.0069651251582696205
changing lr
epoch 26, time 487.73, cls_loss 0.0368 cls_loss_mapping 0.1828 cls_loss_causal 0.8843 re_mapping 0.0628 re_causal 0.0632 /// teacc 86.92 lr 0.00675687
Epoch 28, weight, value: tensor([[-0.0236, -0.0151, -0.0075, ..., -0.0090, 0.1096, 0.1365],
[ 0.0429, 0.0591, 0.0491, ..., 0.0491, 0.0215, -0.0149],
[ 0.0693, 0.0359, 0.0674, ..., 0.0138, 0.0074, 0.0253],
...,
[-0.1272, -0.1129, -0.0777, ..., 0.0041, 0.0222, -0.0156],
[-0.0171, -0.0017, -0.0255, ..., 0.0952, 0.0373, -0.0070],
[-0.0240, -0.0291, -0.0241, ..., -0.1416, -0.1779, -0.1408]],
device='cuda:0'), grad: tensor([[ 6.0856e-05, 2.1964e-05, 2.7269e-05, ..., 6.5342e-06,
-1.4685e-05, -5.9530e-06],
[-6.1631e-05, -9.1717e-06, -6.6832e-06, ..., -4.4763e-05,
-1.7956e-05, -2.1353e-05],
[ 9.1195e-05, 2.1666e-05, 5.9724e-05, ..., 3.2812e-05,
1.0960e-05, 2.4229e-05],
...,
[-7.8297e-04, -1.6737e-04, -3.1304e-04, ..., -1.6844e-04,
-1.1462e-04, -1.4091e-04],
[ 7.5996e-05, 1.4573e-05, 2.2784e-05, ..., 1.7464e-05,
1.4804e-05, 1.4916e-05],
[ 2.5916e-04, 5.2959e-05, 8.9407e-05, ..., 5.7548e-05,
4.3720e-05, 4.9055e-05]], device='cuda:0')
Epoch 28, bias, value: tensor([-0.0193, -0.2885, 0.0172, -0.1947, 0.0115, 0.3121, 0.1729],
device='cuda:0'), grad: tensor([ 0.0004, -0.0001, 0.0008, 0.0014, -0.0038, 0.0003, 0.0011],
device='cuda:0')
351
0.006756874120406716
changing lr
epoch 27, time 487.15, cls_loss 0.0301 cls_loss_mapping 0.1562 cls_loss_causal 0.8664 re_mapping 0.0620 re_causal 0.0625 /// teacc 86.50 lr 0.00654508
Epoch 29, weight, value: tensor([[-0.0220, -0.0148, -0.0074, ..., -0.0088, 0.1090, 0.1355],
[ 0.0451, 0.0592, 0.0495, ..., 0.0491, 0.0212, -0.0149],
[ 0.0643, 0.0347, 0.0653, ..., 0.0122, 0.0056, 0.0231],
...,
[-0.1302, -0.1122, -0.0772, ..., 0.0024, 0.0208, -0.0162],
[-0.0137, -0.0015, -0.0252, ..., 0.0964, 0.0387, -0.0057],
[-0.0215, -0.0287, -0.0234, ..., -0.1391, -0.1752, -0.1381]],
device='cuda:0'), grad: tensor([[-3.0112e-04, -3.0011e-05, -8.4877e-05, ..., -1.0002e-04,
-1.4138e-04, -1.4341e-04],
[-2.8610e-05, -2.1160e-06, -7.0333e-06, ..., -5.9530e-06,
-9.5963e-06, -8.3521e-06],
[-8.0168e-05, -2.8774e-05, -3.0190e-05, ..., -7.9945e-06,
-3.1590e-06, -7.7263e-06],
...,
[ 1.4508e-04, 1.4298e-05, 4.1276e-05, ..., 4.0680e-05,
6.0260e-05, 6.1095e-05],
[ 4.0799e-05, 8.3372e-06, 1.2673e-05, ..., 9.0152e-06,
1.1228e-05, 1.1653e-05],
[ 1.9717e-04, 3.2246e-05, 5.9605e-05, ..., 5.6714e-05,
7.3969e-05, 7.7367e-05]], device='cuda:0')
Epoch 29, bias, value: tensor([-0.0145, -0.2783, 0.0044, -0.1967, -0.0009, 0.3187, 0.1783],
device='cuda:0'), grad: tensor([-1.0118e-03, -8.4460e-05, -1.9419e-04, 7.6890e-05, 4.8494e-04,
1.1617e-04, 6.1226e-04], device='cuda:0')
351
0.00654508497187474
changing lr
epoch 28, time 488.45, cls_loss 0.0259 cls_loss_mapping 0.1594 cls_loss_causal 0.8755 re_mapping 0.0618 re_causal 0.0624 /// teacc 83.54 lr 0.00633018
Epoch 30, weight, value: tensor([[-0.0226, -0.0141, -0.0070, ..., -0.0097, 0.1078, 0.1339],
[ 0.0451, 0.0585, 0.0489, ..., 0.0486, 0.0208, -0.0149],
[ 0.0660, 0.0352, 0.0655, ..., 0.0120, 0.0055, 0.0230],
...,
[-0.1285, -0.1112, -0.0761, ..., 0.0028, 0.0212, -0.0155],
[-0.0149, -0.0017, -0.0254, ..., 0.0952, 0.0380, -0.0061],
[-0.0242, -0.0300, -0.0249, ..., -0.1378, -0.1735, -0.1366]],
device='cuda:0'), grad: tensor([[-2.3043e-04, -2.5898e-05, -3.6359e-05, ..., -6.9618e-05,
-6.9380e-05, -6.5029e-05],
[ 1.2040e-05, 9.2015e-07, 2.3283e-06, ..., 4.3623e-06,
7.4543e-06, 6.0536e-06],
[ 6.5751e-06, 5.9977e-07, 1.3076e-06, ..., 2.2091e-06,
3.5539e-06, 2.9653e-06],
...,
[-2.4462e-04, -1.8463e-05, -5.0098e-05, ..., -1.2875e-04,
-9.0241e-05, -9.0182e-05],
[ 1.3925e-05, 1.1101e-06, 2.8498e-06, ..., 6.5938e-06,
5.9046e-06, 5.5246e-06],
[ 2.1410e-04, 2.4378e-05, 3.3259e-05, ..., 6.6340e-05,
5.7548e-05, 5.6148e-05]], device='cuda:0')
Epoch 30, bias, value: tensor([-0.0197, -0.2738, 0.0092, -0.1897, 0.0017, 0.3110, 0.1723],
device='cuda:0'), grad: tensor([-7.7391e-04, 4.0770e-05, 2.1964e-05, 8.1396e-04, -8.7309e-04,
4.8757e-05, 7.2145e-04], device='cuda:0')
351
0.006330184227833378
changing lr
epoch 29, time 490.75, cls_loss 0.0292 cls_loss_mapping 0.1531 cls_loss_causal 0.8828 re_mapping 0.0614 re_causal 0.0621 /// teacc 86.92 lr 0.00611260
Epoch 31, weight, value: tensor([[-0.0218, -0.0140, -0.0068, ..., -0.0089, 0.1091, 0.1349],
[ 0.0433, 0.0577, 0.0479, ..., 0.0471, 0.0192, -0.0167],
[ 0.0632, 0.0351, 0.0644, ..., 0.0113, 0.0043, 0.0219],
...,
[-0.1234, -0.1095, -0.0737, ..., 0.0036, 0.0218, -0.0143],
[-0.0140, -0.0017, -0.0254, ..., 0.0953, 0.0381, -0.0057],
[-0.0247, -0.0296, -0.0247, ..., -0.1371, -0.1730, -0.1360]],
device='cuda:0'), grad: tensor([[ 1.1892e-03, 1.4639e-04, 3.5691e-04, ..., 1.6296e-04,
1.0908e-04, 1.8024e-04],
[ 1.6034e-04, 2.6032e-05, 4.6492e-05, ..., 1.7181e-05,
9.0823e-06, 1.5661e-05],
[ 2.4438e-04, 3.5912e-05, 7.3314e-05, ..., 3.0607e-05,
1.9833e-05, 3.1680e-05],
...,
[ 1.6680e-03, 2.5249e-04, 4.9782e-04, ..., 1.9598e-04,
1.2219e-04, 1.9646e-04],
[ 1.1700e-04, 1.9327e-05, 3.4690e-05, ..., 1.2390e-05,
7.2531e-06, 1.1504e-05],
[-3.3855e-03, -4.7874e-04, -1.0099e-03, ..., -4.1747e-04,
-2.6464e-04, -4.3321e-04]], device='cuda:0')
Epoch 31, bias, value: tensor([-0.0185, -0.2735, 0.0005, -0.1896, 0.0133, 0.3104, 0.1683],
device='cuda:0'), grad: tensor([ 3.7479e-03, 4.8637e-04, 7.4959e-04, 2.9132e-05, 5.0888e-03,
3.5143e-04, -1.0460e-02], device='cuda:0')
351
0.006112604669781575
changing lr
epoch 30, time 493.20, cls_loss 0.0291 cls_loss_mapping 0.1489 cls_loss_causal 0.8701 re_mapping 0.0617 re_causal 0.0625 /// teacc 74.68 lr 0.00589278
Epoch 32, weight, value: tensor([[-0.0220, -0.0139, -0.0067, ..., -0.0089, 0.1086, 0.1337],
[ 0.0467, 0.0574, 0.0485, ..., 0.0477, 0.0197, -0.0158],
[ 0.0662, 0.0358, 0.0649, ..., 0.0117, 0.0051, 0.0228],
...,
[-0.1264, -0.1089, -0.0743, ..., 0.0026, 0.0201, -0.0159],
[-0.0151, -0.0019, -0.0255, ..., 0.0943, 0.0375, -0.0061],
[-0.0254, -0.0300, -0.0251, ..., -0.1358, -0.1713, -0.1342]],
device='cuda:0'), grad: tensor([[-1.3389e-05, -1.6242e-06, -1.9707e-06, ..., -2.7418e-06,
-9.4846e-06, -9.3952e-06],
[-1.7524e-05, -2.8014e-06, -3.5129e-06, ..., -1.0431e-06,
4.0978e-07, 1.3411e-07],
[ 6.8881e-06, 1.2890e-06, 1.4454e-06, ..., 2.3954e-06,
2.4922e-06, 2.6710e-06],
...,
[ 1.1548e-05, 1.7248e-06, 1.8440e-06, ..., 2.1644e-06,
5.9679e-06, 5.9269e-06],
[-3.1888e-06, -1.3709e-06, -8.5682e-07, ..., -4.1500e-06,
-3.0473e-06, -3.2410e-06],
[ 1.1280e-05, 1.9930e-06, 2.2314e-06, ..., 2.0228e-06,
1.6540e-06, 1.8477e-06]], device='cuda:0')
Epoch 32, bias, value: tensor([-0.0184, -0.2584, 0.0100, -0.1898, -0.0010, 0.3036, 0.1646],
device='cuda:0'), grad: tensor([-4.1336e-05, -5.4091e-05, 2.0459e-05, 1.2487e-05, 3.4273e-05,
-5.7705e-06, 3.3677e-05], device='cuda:0')
351
0.005892784473993186
changing lr
epoch 31, time 488.22, cls_loss 0.0271 cls_loss_mapping 0.1293 cls_loss_causal 0.8446 re_mapping 0.0604 re_causal 0.0613 /// teacc 82.70 lr 0.00567117
Epoch 33, weight, value: tensor([[-0.0197, -0.0135, -0.0061, ..., -0.0079, 0.1092, 0.1343],
[ 0.0433, 0.0561, 0.0470, ..., 0.0464, 0.0185, -0.0169],
[ 0.0662, 0.0359, 0.0648, ..., 0.0113, 0.0046, 0.0225],
...,
[-0.1251, -0.1077, -0.0734, ..., 0.0020, 0.0190, -0.0164],
[-0.0145, -0.0019, -0.0255, ..., 0.0947, 0.0381, -0.0053],
[-0.0254, -0.0299, -0.0248, ..., -0.1351, -0.1700, -0.1333]],
device='cuda:0'), grad: tensor([[ 7.5288e-06, 2.6189e-06, 3.0771e-06, ..., 2.4885e-06,
1.8999e-06, 2.2314e-06],
[-1.2450e-05, -1.6689e-06, -3.1479e-06, ..., -3.0622e-06,
-2.6710e-06, -4.0866e-06],
[ 1.7472e-06, 7.2643e-07, 7.4133e-07, ..., 7.0408e-07,
4.8056e-07, 4.6566e-07],
...,
[ 4.3586e-07, 2.6822e-07, 2.6822e-07, ..., -1.4901e-08,
6.3330e-08, 4.8429e-08],
[ 2.8573e-06, 9.9838e-07, 1.1586e-06, ..., 9.1642e-07,
7.1153e-07, 8.3074e-07],
[ 1.7941e-05, 4.2692e-06, 5.8673e-06, ..., 5.0589e-06,
4.1910e-06, 5.6028e-06]], device='cuda:0')
Epoch 33, bias, value: tensor([-1.1186e-02, -2.6577e-01, 1.0681e-02, -1.8894e-01, -2.5201e-04,
3.0202e-01, 1.6403e-01], device='cuda:0'), grad: tensor([ 1.7270e-05, -3.9309e-05, 3.2783e-06, -3.6538e-05, 1.3039e-07,
6.4075e-06, 4.8697e-05], device='cuda:0')
351
0.00567116632908828
changing lr
epoch 32, time 494.20, cls_loss 0.0140 cls_loss_mapping 0.1146 cls_loss_causal 0.8057 re_mapping 0.0611 re_causal 0.0621 /// teacc 83.12 lr 0.00544820
Epoch 34, weight, value: tensor([[-0.0195, -0.0134, -0.0060, ..., -0.0085, 0.1079, 0.1328],
[ 0.0439, 0.0556, 0.0467, ..., 0.0464, 0.0186, -0.0164],
[ 0.0643, 0.0355, 0.0638, ..., 0.0109, 0.0038, 0.0215],
...,
[-0.1249, -0.1069, -0.0729, ..., 0.0022, 0.0192, -0.0160],
[-0.0141, -0.0020, -0.0255, ..., 0.0944, 0.0382, -0.0049],
[-0.0244, -0.0296, -0.0242, ..., -0.1340, -0.1684, -0.1319]],
device='cuda:0'), grad: tensor([[ 1.2890e-06, 1.9372e-07, 2.7195e-07, ..., 2.6450e-07,
2.2352e-08, 1.7509e-07],
[ 9.8720e-07, 5.5879e-08, 1.4529e-07, ..., 1.4156e-07,
2.5705e-07, 3.3528e-07],
[ 1.4901e-07, -1.4901e-08, -7.4506e-09, ..., 7.4506e-08,
5.9605e-08, 6.7055e-08],
...,
[ 1.3672e-06, 4.4703e-07, 5.2899e-07, ..., 6.1840e-07,
5.4017e-07, 6.1095e-07],
[ 7.1228e-06, 6.7800e-07, 1.1809e-06, ..., 1.3076e-06,
9.6112e-07, 1.6615e-06],
[-9.7528e-06, -8.9779e-07, -1.5907e-06, ..., -1.7695e-06,
-1.2815e-06, -2.2501e-06]], device='cuda:0')
Epoch 34, bias, value: tensor([-0.0111, -0.2602, 0.0059, -0.1873, -0.0030, 0.3006, 0.1656],
device='cuda:0'), grad: tensor([ 4.3325e-06, 3.6284e-06, 6.5938e-07, -2.7381e-06, 3.4980e-06,
2.5198e-05, -3.4690e-05], device='cuda:0')
351
0.00544819654451717
changing lr
epoch 33, time 487.32, cls_loss 0.0128 cls_loss_mapping 0.1002 cls_loss_causal 0.7890 re_mapping 0.0586 re_causal 0.0596 /// teacc 80.17 lr 0.00522432
Epoch 35, weight, value: tensor([[-0.0213, -0.0138, -0.0065, ..., -0.0094, 0.1059, 0.1304],
[ 0.0420, 0.0549, 0.0458, ..., 0.0458, 0.0181, -0.0168],
[ 0.0661, 0.0362, 0.0639, ..., 0.0114, 0.0046, 0.0225],
...,
[-0.1238, -0.1061, -0.0723, ..., 0.0023, 0.0190, -0.0159],
[-0.0145, -0.0020, -0.0256, ..., 0.0938, 0.0380, -0.0048],
[-0.0227, -0.0296, -0.0236, ..., -0.1327, -0.1667, -0.1304]],
device='cuda:0'), grad: tensor([[ 1.6081e-04, 1.3858e-05, 3.4720e-05, ..., 4.6670e-05,
2.1726e-05, 4.5151e-05],
[ 1.0207e-05, 5.5134e-07, 2.1979e-06, ..., 2.7604e-06,
1.0394e-06, 2.5928e-06],
[ 2.6911e-05, 1.4007e-06, 5.7444e-06, ..., 7.3723e-06,
2.7679e-06, 6.8769e-06],
...,
[-6.6936e-05, -9.1121e-06, -1.4573e-05, ..., -2.1294e-05,
-1.2442e-05, -2.1517e-05],
[ 1.9655e-05, 1.0766e-06, 4.2394e-06, ..., 5.3495e-06,
2.0340e-06, 5.0180e-06],
[-1.6916e-04, -8.7768e-06, -3.6329e-05, ..., -4.5896e-05,
-1.7032e-05, -4.2856e-05]], device='cuda:0')
Epoch 35, bias, value: tensor([-0.0173, -0.2628, 0.0116, -0.1854, -0.0026, 0.2957, 0.1711],
device='cuda:0'), grad: tensor([ 5.9175e-04, 4.2528e-05, 1.1235e-04, 7.7784e-05, -1.9562e-04,
8.1778e-05, -7.1049e-04], device='cuda:0')
351
0.005224324151752577
changing lr
epoch 34, time 489.98, cls_loss 0.0186 cls_loss_mapping 0.1047 cls_loss_causal 0.7736 re_mapping 0.0585 re_causal 0.0595 /// teacc 83.54 lr 0.00500000
Epoch 36, weight, value: tensor([[-0.0216, -0.0141, -0.0069, ..., -0.0092, 0.1056, 0.1298],
[ 0.0414, 0.0545, 0.0456, ..., 0.0451, 0.0177, -0.0170],
[ 0.0655, 0.0358, 0.0632, ..., 0.0109, 0.0041, 0.0221],
...,
[-0.1214, -0.1049, -0.0712, ..., 0.0022, 0.0184, -0.0160],
[-0.0141, -0.0021, -0.0256, ..., 0.0938, 0.0382, -0.0044],
[-0.0250, -0.0298, -0.0240, ..., -0.1324, -0.1659, -0.1300]],
device='cuda:0'), grad: tensor([[ 3.3408e-05, 6.4932e-06, 9.7007e-06, ..., 1.7822e-05,
2.0593e-05, 2.3276e-05],
[-1.0021e-06, -6.2026e-07, -3.3528e-07, ..., 6.7614e-07,
3.2391e-06, 3.2559e-06],
[ 2.3395e-05, 4.2990e-06, 7.3910e-06, ..., 1.0505e-05,
1.4037e-05, 1.5646e-05],
...,
[-7.9870e-05, -1.5073e-05, -2.4453e-05, ..., -3.7611e-05,
-5.0575e-05, -5.6446e-05],
[-1.4920e-06, -2.2538e-07, -2.7567e-07, ..., -1.7807e-06,
-7.6927e-07, -5.2899e-07],
[ 1.7151e-05, 3.1851e-06, 5.3979e-06, ..., 7.2606e-06,
9.5218e-06, 1.0468e-05]], device='cuda:0')
Epoch 36, bias, value: tensor([-0.0171, -0.2617, 0.0107, -0.1806, 0.0014, 0.2941, 0.1635],
device='cuda:0'), grad: tensor([ 1.2863e-04, 1.5218e-06, 8.2374e-05, 2.6941e-05, -2.9230e-04,
-4.2953e-06, 5.7399e-05], device='cuda:0')
351
0.005000000000000003
changing lr
epoch 35, time 489.59, cls_loss 0.0186 cls_loss_mapping 0.0992 cls_loss_causal 0.7708 re_mapping 0.0567 re_causal 0.0578 /// teacc 86.08 lr 0.00477568
Epoch 37, weight, value: tensor([[-0.0225, -0.0142, -0.0072, ..., -0.0097, 0.1046, 0.1286],
[ 0.0419, 0.0541, 0.0454, ..., 0.0452, 0.0181, -0.0164],
[ 0.0645, 0.0356, 0.0626, ..., 0.0108, 0.0036, 0.0217],
...,
[-0.1206, -0.1042, -0.0707, ..., 0.0022, 0.0184, -0.0158],
[-0.0141, -0.0022, -0.0255, ..., 0.0934, 0.0381, -0.0043],
[-0.0259, -0.0298, -0.0239, ..., -0.1321, -0.1653, -0.1296]],
device='cuda:0'), grad: tensor([[ 7.0870e-05, 1.1154e-05, 2.2084e-05, ..., 2.9728e-05,
2.9624e-05, 3.3706e-05],
[-1.6823e-05, -2.8890e-06, -5.2638e-06, ..., -1.3612e-05,
-1.8016e-05, -1.7941e-05],
[-8.9824e-05, -2.4319e-05, -2.8491e-05, ..., -6.5714e-06,
-1.2629e-05, -1.4260e-05],
...,
[ 4.0799e-05, 7.8306e-06, 1.2830e-05, ..., 1.1452e-05,
1.1101e-05, 1.3262e-05],
[ 4.5776e-05, 7.6741e-06, 1.4372e-05, ..., 1.4342e-05,
1.1854e-05, 1.5013e-05],
[-1.1605e-04, -1.4335e-05, -3.6120e-05, ..., -4.6641e-05,
-3.4481e-05, -4.4852e-05]], device='cuda:0')
Epoch 37, bias, value: tensor([-0.0201, -0.2563, 0.0100, -0.1743, 0.0009, 0.2915, 0.1585],
device='cuda:0'), grad: tensor([ 2.4629e-04, -4.9651e-05, -2.0874e-04, 1.8287e-04, 1.3006e-04,
1.5986e-04, -4.6062e-04], device='cuda:0')
351
0.004775675848247429
changing lr
epoch 36, time 488.84, cls_loss 0.0166 cls_loss_mapping 0.0919 cls_loss_causal 0.7734 re_mapping 0.0554 re_causal 0.0565 /// teacc 65.40 lr 0.00455180
Epoch 38, weight, value: tensor([[-0.0223, -0.0141, -0.0071, ..., -0.0098, 0.1041, 0.1278],
[ 0.0414, 0.0536, 0.0449, ..., 0.0448, 0.0178, -0.0165],
[ 0.0643, 0.0358, 0.0624, ..., 0.0106, 0.0035, 0.0215],
...,
[-0.1181, -0.1033, -0.0697, ..., 0.0027, 0.0188, -0.0149],
[-0.0145, -0.0024, -0.0256, ..., 0.0929, 0.0379, -0.0042],
[-0.0246, -0.0293, -0.0234, ..., -0.1310, -0.1640, -0.1285]],
device='cuda:0'), grad: tensor([[ 1.3316e-04, 1.7837e-05, 2.2829e-05, ..., 1.4164e-05,
1.2748e-05, 2.1756e-05],
[ 5.2564e-06, 4.0606e-07, 8.4192e-07, ..., 5.1595e-07,
6.2212e-07, 8.6613e-07],
[ 3.3081e-05, 2.6152e-06, 5.2229e-06, ..., 3.0957e-06,
3.8035e-06, 5.3719e-06],
...,
[-5.8189e-06, -4.7125e-07, -1.5218e-06, ..., -1.4678e-06,
-1.4100e-06, -1.4715e-06],
[ 2.6207e-06, 1.7136e-07, 3.7812e-07, ..., 3.9116e-08,
2.0675e-07, 3.4086e-07],
[-1.6868e-04, -2.0489e-05, -2.7731e-05, ..., -1.6347e-05,
-1.6004e-05, -2.6926e-05]], device='cuda:0')
Epoch 38, bias, value: tensor([-0.0192, -0.2548, 0.0095, -0.1794, 0.0065, 0.2877, 0.1599],
device='cuda:0'), grad: tensor([ 4.4608e-04, 1.7881e-05, 1.1235e-04, 1.7788e-06, -1.9386e-05,
9.0301e-06, -5.6791e-04], device='cuda:0')
351
0.004551803455482836
changing lr
epoch 37, time 487.50, cls_loss 0.0129 cls_loss_mapping 0.0849 cls_loss_causal 0.7819 re_mapping 0.0550 re_causal 0.0562 /// teacc 89.03 lr 0.00432883
Epoch 39, weight, value: tensor([[-0.0228, -0.0142, -0.0072, ..., -0.0100, 0.1035, 0.1270],
[ 0.0410, 0.0531, 0.0445, ..., 0.0444, 0.0176, -0.0165],
[ 0.0628, 0.0354, 0.0617, ..., 0.0099, 0.0029, 0.0208],
...,
[-0.1189, -0.1028, -0.0695, ..., 0.0021, 0.0180, -0.0154],
[-0.0129, -0.0021, -0.0254, ..., 0.0936, 0.0385, -0.0033],
[-0.0235, -0.0291, -0.0230, ..., -0.1298, -0.1629, -0.1274]],
device='cuda:0'), grad: tensor([[ 1.1024e-03, 1.6773e-04, 1.4496e-04, ..., 3.4833e-04,
2.5558e-04, 3.3259e-04],
[ 4.5925e-05, 1.2770e-05, 1.7360e-06, ..., 1.1012e-05,
1.0118e-05, 1.5013e-05],
[-1.8396e-03, -5.5504e-04, -3.8934e-04, ..., -4.4203e-04,
-5.5504e-04, -6.4707e-04],
...,
[-9.4032e-04, -4.8190e-05, -7.2837e-05, ..., -3.4523e-04,
-1.7154e-04, -2.5058e-04],
[ 2.1040e-04, 2.8461e-05, 2.7418e-05, ..., 6.7711e-05,
4.3333e-05, 5.9068e-05],
[ 8.0061e-04, 2.2531e-04, 1.6654e-04, ..., 2.0170e-04,
2.3818e-04, 2.7800e-04]], device='cuda:0')
Epoch 39, bias, value: tensor([-0.0210, -0.2532, 0.0062, -0.1753, 0.0016, 0.2897, 0.1620],
device='cuda:0'), grad: tensor([ 0.0037, 0.0001, -0.0050, 0.0018, -0.0036, 0.0007, 0.0022],
device='cuda:0')
351
0.004328833670911726
changing lr
epoch 38, time 486.95, cls_loss 0.0092 cls_loss_mapping 0.0883 cls_loss_causal 0.7727 re_mapping 0.0537 re_causal 0.0550 /// teacc 78.06 lr 0.00410722
Epoch 40, weight, value: tensor([[-0.0224, -0.0141, -0.0071, ..., -0.0100, 0.1032, 0.1266],
[ 0.0395, 0.0524, 0.0439, ..., 0.0437, 0.0169, -0.0171],
[ 0.0637, 0.0356, 0.0616, ..., 0.0100, 0.0029, 0.0208],
...,
[-0.1182, -0.1022, -0.0690, ..., 0.0022, 0.0181, -0.0152],
[-0.0136, -0.0023, -0.0255, ..., 0.0932, 0.0385, -0.0032],
[-0.0221, -0.0287, -0.0226, ..., -0.1290, -0.1619, -0.1264]],
device='cuda:0'), grad: tensor([[ 1.0431e-04, 8.6427e-06, 8.2403e-06, ..., 3.6299e-05,
2.9549e-05, 2.5302e-05],
[ 2.6047e-05, 1.6242e-06, 3.4273e-07, ..., 8.7395e-06,
7.0557e-06, 4.9658e-06],
[ 9.1696e-04, 7.5698e-05, 5.7697e-05, ..., 3.6097e-04,
2.8181e-04, 2.4402e-04],
...,
[ 1.0424e-03, 7.0632e-05, 5.5134e-05, ..., 3.4261e-04,
2.7657e-04, 2.2483e-04],
[-2.4719e-03, -1.8394e-04, -1.4257e-04, ..., -8.9836e-04,
-7.1335e-04, -6.0034e-04],
[ 2.6083e-04, 1.5914e-05, 1.1899e-05, ..., 9.9719e-05,
7.9751e-05, 6.6876e-05]], device='cuda:0')
Epoch 40, bias, value: tensor([-0.0200, -0.2550, 0.0099, -0.1766, 0.0015, 0.2851, 0.1650],
device='cuda:0'), grad: tensor([ 3.3784e-04, 8.6427e-05, 2.9964e-03, 3.9721e-04, 3.4218e-03,
-8.0948e-03, 8.5974e-04], device='cuda:0')
351
0.0041072155260068206
changing lr
epoch 39, time 489.22, cls_loss 0.0097 cls_loss_mapping 0.0775 cls_loss_causal 0.7592 re_mapping 0.0543 re_causal 0.0557 /// teacc 82.28 lr 0.00388740
Epoch 41, weight, value: tensor([[-0.0227, -0.0141, -0.0072, ..., -0.0101, 0.1026, 0.1259],
[ 0.0403, 0.0520, 0.0439, ..., 0.0437, 0.0173, -0.0165],
[ 0.0628, 0.0355, 0.0611, ..., 0.0095, 0.0024, 0.0202],
...,
[-0.1180, -0.1017, -0.0687, ..., 0.0020, 0.0177, -0.0154],
[-0.0123, -0.0022, -0.0253, ..., 0.0937, 0.0388, -0.0026],
[-0.0233, -0.0287, -0.0227, ..., -0.1287, -0.1614, -0.1262]],
device='cuda:0'), grad: tensor([[ 4.6196e-03, 4.9496e-04, 5.7077e-04, ..., 1.1282e-03,
2.0866e-03, 2.3994e-03],
[ 2.5436e-05, 1.1943e-05, 7.2420e-06, ..., -7.5214e-06,
-7.6517e-06, -4.2208e-06],
[-5.7106e-03, -7.9346e-04, -8.9979e-04, ..., -1.3742e-03,
-2.3251e-03, -2.7313e-03],
...,
[ 2.3162e-04, 5.5492e-05, 6.3598e-05, ..., 5.3585e-05,
6.6578e-05, 8.4639e-05],
[ 5.7983e-04, 1.6105e-04, 1.8013e-04, ..., 1.3936e-04,
1.2082e-04, 1.7250e-04],
[ 1.0353e-04, 2.8998e-05, 3.2276e-05, ..., 2.4587e-05,
2.3708e-05, 3.2336e-05]], device='cuda:0')
Epoch 41, bias, value: tensor([-2.1788e-02, -2.5003e-01, 8.0240e-03, -1.7373e-01, -1.4076e-04,
2.8711e-01, 1.6039e-01], device='cuda:0'), grad: tensor([ 1.5137e-02, 5.2959e-05, -1.8082e-02, 4.0960e-04, 6.5422e-04,
1.5545e-03, 2.7752e-04], device='cuda:0')
351
0.0038873953302184317
changing lr
epoch 40, time 485.05, cls_loss 0.0115 cls_loss_mapping 0.0751 cls_loss_causal 0.7536 re_mapping 0.0549 re_causal 0.0564 /// teacc 88.61 lr 0.00366982
Epoch 42, weight, value: tensor([[-0.0213, -0.0140, -0.0069, ..., -0.0095, 0.1029, 0.1261],
[ 0.0404, 0.0518, 0.0437, ..., 0.0434, 0.0173, -0.0165],
[ 0.0623, 0.0354, 0.0608, ..., 0.0093, 0.0022, 0.0200],
...,
[-0.1180, -0.1012, -0.0685, ..., 0.0017, 0.0173, -0.0155],
[-0.0132, -0.0024, -0.0255, ..., 0.0930, 0.0385, -0.0028],
[-0.0228, -0.0285, -0.0225, ..., -0.1279, -0.1605, -0.1255]],
device='cuda:0'), grad: tensor([[ 5.6076e-03, 1.2617e-03, 1.4315e-03, ..., 2.1935e-03,
1.7796e-03, 3.0479e-03],
[ 4.9362e-03, 2.6488e-04, 7.0047e-04, ..., 1.4935e-03,
1.3771e-03, 1.4277e-03],
[ 1.0651e-04, -7.0781e-07, 8.5086e-06, ..., 3.8654e-05,
3.2961e-05, 3.8892e-05],
...,
[ 3.0041e-04, 2.2113e-05, 4.7088e-05, ..., 9.2387e-05,
8.4519e-05, 9.2983e-05],
[ 9.2316e-03, 3.9482e-04, 1.2436e-03, ..., 2.7428e-03,
2.5539e-03, 2.5234e-03],
[-2.0370e-02, -1.9531e-03, -3.4580e-03, ..., -6.6147e-03,
-5.8784e-03, -7.1831e-03]], device='cuda:0')
Epoch 42, bias, value: tensor([-0.0175, -0.2477, 0.0069, -0.1734, -0.0019, 0.2824, 0.1610],
device='cuda:0'), grad: tensor([ 0.0323, 0.0152, 0.0005, 0.0005, 0.0010, 0.0268, -0.0762],
device='cuda:0')
351
0.003669815772166629
changing lr
epoch 41, time 487.61, cls_loss 0.0074 cls_loss_mapping 0.0727 cls_loss_causal 0.7325 re_mapping 0.0550 re_causal 0.0565 /// teacc 85.23 lr 0.00345492
Epoch 43, weight, value: tensor([[-0.0212, -0.0139, -0.0070, ..., -0.0095, 0.1025, 0.1256],
[ 0.0400, 0.0514, 0.0434, ..., 0.0431, 0.0171, -0.0166],
[ 0.0619, 0.0353, 0.0605, ..., 0.0092, 0.0022, 0.0200],
...,
[-0.1162, -0.1007, -0.0678, ..., 0.0020, 0.0174, -0.0152],
[-0.0136, -0.0025, -0.0255, ..., 0.0928, 0.0385, -0.0026],
[-0.0236, -0.0285, -0.0226, ..., -0.1276, -0.1601, -0.1253]],
device='cuda:0'), grad: tensor([[ 1.8328e-05, 3.5167e-06, 4.0494e-06, ..., 6.2659e-06,
3.9861e-06, 5.5842e-06],
[ 3.2261e-06, 8.4192e-07, 1.1250e-06, ..., 1.3001e-06,
1.0841e-06, 1.3933e-06],
[ 2.5794e-05, 9.8720e-06, 1.1310e-05, ..., 9.2685e-06,
9.9540e-06, 1.1086e-05],
...,
[ 2.9549e-05, 1.0736e-05, 1.2115e-05, ..., 1.0125e-05,
1.0498e-05, 1.1921e-05],
[ 1.4000e-05, 2.1495e-06, 3.7774e-06, ..., 5.8562e-06,
3.5763e-06, 5.4203e-06],
[-3.0845e-05, -3.8296e-06, -5.8822e-06, ..., -1.1764e-05,
-6.1207e-06, -9.7081e-06]], device='cuda:0')
Epoch 43, bias, value: tensor([-0.0182, -0.2470, 0.0065, -0.1719, 0.0034, 0.2799, 0.1570],
device='cuda:0'), grad: tensor([ 5.5403e-05, 8.9854e-06, 5.5552e-05, -1.2755e-04, 6.5923e-05,
4.7505e-05, -1.0574e-04], device='cuda:0')
351
0.0034549150281252667
changing lr
epoch 42, time 486.22, cls_loss 0.0069 cls_loss_mapping 0.0656 cls_loss_causal 0.7226 re_mapping 0.0538 re_causal 0.0554 /// teacc 68.35 lr 0.00324313
Epoch 44, weight, value: tensor([[-0.0216, -0.0140, -0.0071, ..., -0.0096, 0.1021, 0.1251],
[ 0.0402, 0.0512, 0.0433, ..., 0.0430, 0.0172, -0.0164],
[ 0.0620, 0.0353, 0.0603, ..., 0.0091, 0.0021, 0.0199],
...,
[-0.1160, -0.1002, -0.0676, ..., 0.0017, 0.0170, -0.0154],
[-0.0131, -0.0025, -0.0254, ..., 0.0929, 0.0387, -0.0023],
[-0.0234, -0.0283, -0.0224, ..., -0.1271, -0.1595, -0.1248]],
device='cuda:0'), grad: tensor([[ 1.6963e-04, 2.2471e-05, 2.5943e-05, ..., 8.0049e-05,
6.7055e-05, 7.7963e-05],
[-3.7432e-05, -6.4336e-06, -1.2316e-05, ..., -4.7833e-05,
-5.9873e-05, -6.4135e-05],
[-1.2279e-04, -5.3316e-05, -5.8621e-05, ..., -1.4551e-05,
-3.0726e-05, -4.2588e-05],
...,
[ 3.2067e-04, 7.4446e-05, 8.3625e-05, ..., 9.1612e-05,
9.7215e-05, 1.2577e-04],
[ 5.6219e-04, 5.9366e-05, 6.4909e-05, ..., 1.5295e-04,
1.1712e-04, 1.6892e-04],
[-9.2030e-04, -9.9599e-05, -1.0693e-04, ..., -2.6965e-04,
-1.9670e-04, -2.7466e-04]], device='cuda:0')
Epoch 44, bias, value: tensor([-0.0191, -0.2443, 0.0069, -0.1719, 0.0013, 0.2796, 0.1572],
device='cuda:0'), grad: tensor([ 6.8665e-04, 2.4974e-05, -2.1517e-04, 9.7215e-05, 9.1743e-04,
2.0123e-03, -3.5248e-03], device='cuda:0')
351
0.0032431258795932905
changing lr
epoch 43, time 489.72, cls_loss 0.0105 cls_loss_mapping 0.0591 cls_loss_causal 0.7122 re_mapping 0.0533 re_causal 0.0549 /// teacc 88.61 lr 0.00303487
Epoch 45, weight, value: tensor([[-0.0222, -0.0140, -0.0073, ..., -0.0099, 0.1017, 0.1245],
[ 0.0400, 0.0509, 0.0431, ..., 0.0428, 0.0171, -0.0163],
[ 0.0616, 0.0353, 0.0601, ..., 0.0087, 0.0019, 0.0196],
...,
[-0.1159, -0.0999, -0.0675, ..., 0.0017, 0.0166, -0.0157],
[-0.0133, -0.0026, -0.0255, ..., 0.0928, 0.0388, -0.0020],
[-0.0227, -0.0282, -0.0221, ..., -0.1265, -0.1588, -0.1241]],
device='cuda:0'), grad: tensor([[-8.6203e-06, -1.0803e-06, -2.7604e-06, ..., -4.3027e-06,
-4.8839e-06, -7.2680e-06],
[-3.8981e-04, -2.3216e-05, -9.8407e-05, ..., -1.4544e-04,
-2.0409e-04, -1.9073e-04],
[-3.0082e-06, -1.6894e-06, -1.5181e-06, ..., 3.5763e-07,
3.6880e-07, 1.2107e-07],
...,
[ 3.8028e-04, 2.3350e-05, 9.6440e-05, ..., 1.4150e-04,
1.9836e-04, 1.8644e-04],
[ 6.8918e-06, 5.0664e-07, 1.7155e-06, ..., 2.4550e-06,
3.3509e-06, 3.2224e-06],
[ 8.7097e-06, 1.1623e-06, 2.7977e-06, ..., 3.7644e-06,
4.5709e-06, 5.8189e-06]], device='cuda:0')
Epoch 45, bias, value: tensor([-2.0831e-02, -2.4265e-01, 5.8777e-03, -1.6848e-01, 1.9903e-05,
2.7736e-01, 1.5830e-01], device='cuda:0'), grad: tensor([-3.1412e-05, -1.2970e-03, -5.6811e-06, 1.4350e-05, 1.2646e-03,
2.2843e-05, 2.9579e-05], device='cuda:0')
351
0.0030348748417303863
changing lr
epoch 44, time 487.09, cls_loss 0.0084 cls_loss_mapping 0.0627 cls_loss_causal 0.7363 re_mapping 0.0523 re_causal 0.0540 /// teacc 72.15 lr 0.00283058
Epoch 46, weight, value: tensor([[-0.0226, -0.0141, -0.0073, ..., -0.0101, 0.1011, 0.1239],
[ 0.0396, 0.0506, 0.0428, ..., 0.0426, 0.0171, -0.0163],
[ 0.0621, 0.0355, 0.0601, ..., 0.0086, 0.0018, 0.0196],
...,
[-0.1147, -0.0993, -0.0669, ..., 0.0018, 0.0167, -0.0154],
[-0.0123, -0.0025, -0.0252, ..., 0.0933, 0.0392, -0.0013],
[-0.0237, -0.0283, -0.0223, ..., -0.1264, -0.1586, -0.1242]],
device='cuda:0'), grad: tensor([[ 6.1005e-05, 8.5235e-06, 2.0206e-05, ..., 3.7134e-05,
4.8548e-05, 5.1439e-05],
[-5.2303e-05, -7.2159e-06, -1.8224e-05, ..., -3.3587e-05,
-4.5151e-05, -4.7475e-05],
[-2.4978e-06, -8.5682e-07, -1.0654e-06, ..., 8.8662e-07,
-8.7544e-08, -2.1793e-07],
...,
[ 2.4624e-06, 2.3469e-07, 5.1782e-07, ..., 1.7583e-06,
1.6093e-06, 1.7621e-06],
[-6.5379e-07, -3.5018e-07, 9.3132e-08, ..., -8.7172e-06,
-6.4522e-06, -6.6496e-06],
[-8.6203e-06, -1.7323e-07, -1.4603e-06, ..., 1.7472e-06,
8.6240e-07, 4.0419e-07]], device='cuda:0')
Epoch 46, bias, value: tensor([-0.0229, -0.2422, 0.0083, -0.1695, 0.0023, 0.2799, 0.1536],
device='cuda:0'), grad: tensor([ 2.0087e-04, -1.7345e-04, -5.4091e-06, 2.8592e-06, 8.5533e-06,
-5.4836e-06, -2.7969e-05], device='cuda:0')
351
0.0028305813044122124
changing lr
---------------------saving model at epoch 45----------------------------------------------------
epoch 45, time 492.10, cls_loss 0.0091 cls_loss_mapping 0.0608 cls_loss_causal 0.7182 re_mapping 0.0522 re_causal 0.0539 /// teacc 90.30 lr 0.00263066
Epoch 47, weight, value: tensor([[-0.0212, -0.0139, -0.0069, ..., -0.0095, 0.1019, 0.1245],
[ 0.0388, 0.0502, 0.0424, ..., 0.0422, 0.0167, -0.0167],
[ 0.0622, 0.0355, 0.0600, ..., 0.0085, 0.0017, 0.0195],
...,
[-0.1149, -0.0990, -0.0669, ..., 0.0016, 0.0161, -0.0159],
[-0.0125, -0.0025, -0.0252, ..., 0.0931, 0.0391, -0.0013],
[-0.0236, -0.0282, -0.0222, ..., -0.1260, -0.1581, -0.1238]],
device='cuda:0'), grad: tensor([[-2.3961e-04, -1.5020e-05, -4.5061e-05, ..., -7.1526e-05,
-1.4293e-04, -1.5366e-04],
[ 4.7952e-05, 6.9402e-06, 1.2204e-05, ..., 1.8135e-05,
2.4453e-05, 2.5079e-05],
[-3.4273e-06, -3.3509e-06, -2.6412e-06, ..., 1.6391e-06,
3.9525e-06, 3.6862e-06],
...,
[ 1.8501e-04, 1.4961e-05, 3.5077e-05, ..., 5.6356e-05,
1.0079e-04, 1.0961e-04],
[-6.5863e-05, -2.5541e-05, -2.4706e-05, ..., -6.4194e-05,
-4.3005e-05, -4.3362e-05],
[-2.0619e-06, 1.1697e-06, -4.8615e-07, ..., 8.0764e-06,
8.3670e-06, 9.6112e-06]], device='cuda:0')
Epoch 47, bias, value: tensor([-1.8068e-02, -2.4327e-01, 9.5686e-03, -1.6941e-01, -7.8014e-05,
2.7772e-01, 1.5303e-01], device='cuda:0'), grad: tensor([-8.3208e-04, 1.5223e-04, 4.2468e-07, 2.0671e-04, 6.2895e-04,
-1.4079e-04, -1.5587e-05], device='cuda:0')
351
0.0026306566876350096
changing lr
epoch 46, time 489.43, cls_loss 0.0071 cls_loss_mapping 0.0539 cls_loss_causal 0.6687 re_mapping 0.0523 re_causal 0.0540 /// teacc 81.86 lr 0.00243550
Epoch 48, weight, value: tensor([[-0.0214, -0.0139, -0.0070, ..., -0.0097, 0.1014, 0.1239],
[ 0.0393, 0.0501, 0.0423, ..., 0.0422, 0.0167, -0.0165],
[ 0.0616, 0.0354, 0.0597, ..., 0.0082, 0.0015, 0.0192],
...,
[-0.1144, -0.0986, -0.0665, ..., 0.0016, 0.0162, -0.0156],
[-0.0122, -0.0024, -0.0251, ..., 0.0932, 0.0393, -0.0009],
[-0.0237, -0.0282, -0.0221, ..., -0.1258, -0.1577, -0.1235]],
device='cuda:0'), grad: tensor([[ 4.4298e-04, 6.1333e-05, 1.1182e-04, ..., 2.1458e-04,
1.8668e-04, 1.7023e-04],
[ 1.8826e-03, 2.6131e-04, 4.8971e-04, ..., 1.0023e-03,
8.5163e-04, 7.7057e-04],
[ 2.8515e-04, 3.9458e-05, 6.5804e-05, ..., 7.5281e-05,
7.8142e-05, 8.3566e-05],
...,
[-2.8276e-04, -4.5151e-05, -1.1039e-04, ..., -2.2113e-04,
-1.5962e-04, -2.0111e-04],
[ 5.2691e-04, 7.3135e-05, 1.3399e-04, ..., 2.4748e-04,
2.1160e-04, 2.0254e-04],
[-2.9583e-03, -4.0483e-04, -7.1955e-04, ..., -1.3676e-03,
-1.2093e-03, -1.0691e-03]], device='cuda:0')
Epoch 48, bias, value: tensor([-1.8857e-02, -2.4018e-01, 8.1123e-03, -1.6882e-01, -1.7916e-04,
2.7774e-01, 1.5164e-01], device='cuda:0'), grad: tensor([ 0.0014, 0.0060, 0.0009, 0.0003, -0.0010, 0.0017, -0.0095],
device='cuda:0')
351
0.0024355036129704724
changing lr
epoch 47, time 487.96, cls_loss 0.0077 cls_loss_mapping 0.0550 cls_loss_causal 0.6829 re_mapping 0.0517 re_causal 0.0534 /// teacc 81.01 lr 0.00224552
Epoch 49, weight, value: tensor([[-0.0218, -0.0139, -0.0071, ..., -0.0099, 0.1009, 0.1234],
[ 0.0387, 0.0498, 0.0420, ..., 0.0420, 0.0165, -0.0166],
[ 0.0617, 0.0354, 0.0596, ..., 0.0082, 0.0014, 0.0191],
...,
[-0.1136, -0.0983, -0.0662, ..., 0.0017, 0.0162, -0.0154],
[-0.0129, -0.0026, -0.0253, ..., 0.0928, 0.0391, -0.0010],
[-0.0224, -0.0280, -0.0218, ..., -0.1249, -0.1569, -0.1228]],
device='cuda:0'), grad: tensor([[ 1.1139e-05, 3.9563e-06, 4.2059e-06, ..., 5.1297e-06,
-3.5092e-06, -1.6186e-06],
[ 1.8522e-05, 5.1521e-06, 6.6422e-06, ..., 5.1931e-06,
4.4443e-06, 4.9062e-06],
[-1.5345e-03, -5.2452e-04, -6.5041e-04, ..., -2.7299e-04,
-2.6488e-04, -3.1328e-04],
...,
[ 2.2805e-04, 7.3135e-05, 9.2268e-05, ..., 4.5002e-05,
4.8876e-05, 5.4628e-05],
[ 1.2875e-05, 2.3752e-05, 2.5123e-05, ..., -2.8685e-05,
-1.7419e-05, -1.6347e-05],
[ 1.1272e-03, 3.7432e-04, 4.6635e-04, ..., 2.1875e-04,
2.0671e-04, 2.4176e-04]], device='cuda:0')
Epoch 49, bias, value: tensor([-0.0203, -0.2404, 0.0085, -0.1690, 0.0013, 0.2743, 0.1549],
device='cuda:0'), grad: tensor([ 2.9311e-05, 4.7684e-05, -3.3817e-03, 3.1281e-04, 5.2595e-04,
-8.5831e-05, 2.5520e-03], device='cuda:0')
351
0.00224551509273949
changing lr
epoch 48, time 489.20, cls_loss 0.0051 cls_loss_mapping 0.0480 cls_loss_causal 0.6975 re_mapping 0.0510 re_causal 0.0528 /// teacc 82.28 lr 0.00206107
Epoch 50, weight, value: tensor([[-0.0211, -0.0138, -0.0069, ..., -0.0097, 0.1009, 0.1233],
[ 0.0382, 0.0495, 0.0417, ..., 0.0417, 0.0164, -0.0167],
[ 0.0627, 0.0359, 0.0599, ..., 0.0083, 0.0016, 0.0193],
...,
[-0.1137, -0.0980, -0.0661, ..., 0.0015, 0.0159, -0.0155],
[-0.0126, -0.0026, -0.0252, ..., 0.0929, 0.0392, -0.0008],
[-0.0227, -0.0280, -0.0217, ..., -0.1247, -0.1566, -0.1226]],
device='cuda:0'), grad: tensor([[ 1.1253e-03, 1.8346e-04, 4.4370e-04, ..., 3.2568e-04,
3.2568e-04, 3.5405e-04],
[-1.6794e-03, -2.7108e-04, -6.5708e-04, ..., -4.8637e-04,
-4.8971e-04, -5.3120e-04],
[ 1.3448e-05, 1.6382e-06, 4.7870e-06, ..., 4.1053e-06,
4.4927e-06, 4.7088e-06],
...,
[ 6.8322e-06, 7.2643e-07, 1.7742e-06, ..., 1.8915e-06,
2.3209e-06, 2.2016e-06],
[ 1.9342e-05, 3.0510e-06, 7.3537e-06, ..., 5.6028e-06,
5.7891e-06, 6.2510e-06],
[ 5.0974e-04, 8.1480e-05, 1.9753e-04, ..., 1.4758e-04,
1.5008e-04, 1.6248e-04]], device='cuda:0')
Epoch 50, bias, value: tensor([-0.0180, -0.2408, 0.0113, -0.1703, -0.0003, 0.2742, 0.1533],
device='cuda:0'), grad: tensor([ 3.3379e-03, -4.9934e-03, 4.1991e-05, 8.9183e-06, 2.1189e-05,
5.7757e-05, 1.5182e-03], device='cuda:0')
351
0.002061073738537637
changing lr
epoch 49, time 484.91, cls_loss 0.0058 cls_loss_mapping 0.0522 cls_loss_causal 0.6878 re_mapping 0.0512 re_causal 0.0531 /// teacc 75.53 lr 0.00188255
Epoch 51, weight, value: tensor([[-0.0213, -0.0139, -0.0069, ..., -0.0098, 0.1006, 0.1229],
[ 0.0383, 0.0494, 0.0416, ..., 0.0417, 0.0165, -0.0165],
[ 0.0622, 0.0358, 0.0597, ..., 0.0082, 0.0014, 0.0190],
...,
[-0.1135, -0.0978, -0.0659, ..., 0.0015, 0.0158, -0.0155],
[-0.0126, -0.0026, -0.0252, ..., 0.0928, 0.0393, -0.0006],
[-0.0230, -0.0279, -0.0217, ..., -0.1246, -0.1563, -0.1224]],
device='cuda:0'), grad: tensor([[ 3.5453e-04, 4.2230e-05, 8.2910e-05, ..., 1.2863e-04,
1.9825e-04, 1.6248e-04],
[ 4.3213e-05, 5.1521e-06, 8.0839e-06, ..., 2.7746e-05,
2.8342e-05, 2.6956e-05],
[ 2.9758e-05, 3.7849e-06, 5.3681e-06, ..., 1.8895e-05,
1.9521e-05, 1.8239e-05],
...,
[-3.7861e-04, -3.9101e-05, -9.9599e-05, ..., -7.7963e-05,
-2.0134e-04, -1.4389e-04],
[-1.8530e-03, -3.1042e-04, -3.1281e-04, ..., -1.2970e-03,
-1.1034e-03, -1.1406e-03],
[ 1.6336e-03, 2.7657e-04, 2.7704e-04, ..., 1.1320e-03,
9.6369e-04, 9.9659e-04]], device='cuda:0')
Epoch 51, bias, value: tensor([-0.0185, -0.2392, 0.0103, -0.1683, -0.0005, 0.2735, 0.1519],
device='cuda:0'), grad: tensor([ 1.2074e-03, 1.3626e-04, 9.5785e-05, 5.7364e-04, -1.3552e-03,
-5.5962e-03, 4.9400e-03], device='cuda:0')
351
0.0018825509907063344
changing lr
epoch 50, time 487.79, cls_loss 0.0047 cls_loss_mapping 0.0466 cls_loss_causal 0.6943 re_mapping 0.0511 re_causal 0.0530 /// teacc 78.90 lr 0.00171031
Epoch 52, weight, value: tensor([[-0.0216, -0.0139, -0.0070, ..., -0.0099, 0.1002, 0.1225],
[ 0.0385, 0.0493, 0.0415, ..., 0.0417, 0.0165, -0.0164],
[ 0.0622, 0.0358, 0.0596, ..., 0.0081, 0.0013, 0.0190],
...,
[-0.1131, -0.0975, -0.0657, ..., 0.0015, 0.0157, -0.0155],
[-0.0127, -0.0027, -0.0252, ..., 0.0927, 0.0393, -0.0006],
[-0.0229, -0.0279, -0.0216, ..., -0.1242, -0.1559, -0.1221]],
device='cuda:0'), grad: tensor([[-1.2982e-04, -1.2673e-05, -4.6909e-05, ..., -6.3837e-05,
-1.1736e-04, -1.3304e-04],
[-2.3592e-04, -4.2081e-05, -3.9279e-05, ..., -8.7023e-05,
-8.8811e-05, -6.4790e-05],
[-3.4034e-05, -2.2024e-05, -2.2888e-05, ..., 6.2622e-06,
9.6485e-06, 3.5074e-06],
...,
[ 2.4235e-04, 3.5316e-05, 6.6459e-05, ..., 1.0478e-04,
1.4913e-04, 1.5426e-04],
[-9.1434e-05, 1.8887e-06, -5.2340e-06, ..., -7.2837e-05,
-4.9442e-05, -4.6045e-05],
[ 1.8239e-04, 2.4110e-05, 3.0428e-05, ..., 9.0301e-05,
7.6056e-05, 6.6817e-05]], device='cuda:0')
Epoch 52, bias, value: tensor([-0.0198, -0.2375, 0.0106, -0.1674, -0.0003, 0.2722, 0.1516],
device='cuda:0'), grad: tensor([-5.4979e-04, -6.8712e-04, -4.6283e-05, 1.8716e-04, 8.5592e-04,
-3.2711e-04, 5.6744e-04], device='cuda:0')
351
0.0017103063703014388
changing lr
epoch 51, time 488.35, cls_loss 0.0062 cls_loss_mapping 0.0457 cls_loss_causal 0.6795 re_mapping 0.0503 re_causal 0.0522 /// teacc 78.48 lr 0.00154469
Epoch 53, weight, value: tensor([[-0.0217, -0.0139, -0.0071, ..., -0.0100, 0.1000, 0.1222],
[ 0.0387, 0.0492, 0.0415, ..., 0.0417, 0.0166, -0.0162],
[ 0.0617, 0.0357, 0.0594, ..., 0.0078, 0.0011, 0.0187],
...,
[-0.1128, -0.0972, -0.0655, ..., 0.0015, 0.0156, -0.0155],
[-0.0125, -0.0027, -0.0252, ..., 0.0927, 0.0394, -0.0004],
[-0.0228, -0.0278, -0.0215, ..., -0.1240, -0.1556, -0.1218]],
device='cuda:0'), grad: tensor([[-4.3184e-05, -7.6443e-06, -1.2971e-05, ..., -8.9481e-06,
-2.2054e-05, -2.4438e-05],
[ 3.1084e-05, 5.4389e-06, 8.3521e-06, ..., 8.5458e-06,
1.3798e-05, 1.4573e-05],
[ 1.3590e-04, 1.5318e-05, 2.5660e-05, ..., 6.3658e-05,
4.5538e-05, 4.2319e-05],
...,
[-1.6212e-04, -1.7196e-05, -2.8029e-05, ..., -8.2672e-05,
-5.1290e-05, -4.5180e-05],
[ 2.1487e-05, 2.5854e-06, 4.3064e-06, ..., 9.0301e-06,
6.7316e-06, 6.4820e-06],
[-1.1288e-05, -1.6820e-06, -2.6543e-06, ..., -2.7064e-06,
-1.9800e-06, -2.3786e-06]], device='cuda:0')
Epoch 53, bias, value: tensor([-2.0178e-02, -2.3575e-01, 9.2648e-03, -1.6716e-01, -2.3270e-04,
2.7194e-01, 1.5140e-01], device='cuda:0'), grad: tensor([-1.4365e-04, 9.9480e-05, 4.1962e-04, 8.7023e-05, -4.9305e-04,
6.7174e-05, -3.6746e-05], device='cuda:0')
351
0.0015446867550656784
changing lr
epoch 52, time 486.84, cls_loss 0.0055 cls_loss_mapping 0.0471 cls_loss_causal 0.6569 re_mapping 0.0497 re_causal 0.0515 /// teacc 85.23 lr 0.00138603
Epoch 54, weight, value: tensor([[-2.1269e-02, -1.3865e-02, -6.9552e-03, ..., -9.8699e-03,
1.0012e-01, 1.2224e-01],
[ 3.8476e-02, 4.8979e-02, 4.1318e-02, ..., 4.1490e-02,
1.6448e-02, -1.6356e-02],
[ 6.1428e-02, 3.5670e-02, 5.9260e-02, ..., 7.6740e-03,
9.0968e-04, 1.8529e-02],
...,
[-1.1242e-01, -9.6968e-02, -6.5334e-02, ..., 1.5697e-03,
1.5542e-02, -1.5468e-02],
[-1.2210e-02, -2.6671e-03, -2.5152e-02, ..., 9.2897e-02,
3.9566e-02, -1.3846e-04],
[-2.3079e-02, -2.7743e-02, -2.1526e-02, ..., -1.2388e-01,
-1.5544e-01, -1.2169e-01]], device='cuda:0'), grad: tensor([[-6.1512e-04, -2.2143e-05, -1.5163e-04, ..., -1.7321e-04,
-3.0017e-04, -3.2377e-04],
[ 3.2759e-04, 6.2287e-05, 9.6262e-05, ..., 7.9751e-05,
8.0824e-05, 9.2447e-05],
[ 1.1148e-03, 3.7837e-04, 4.0412e-04, ..., 3.4690e-04,
2.2399e-04, 3.0065e-04],
...,
[ 6.1750e-04, 8.4043e-05, 1.8024e-04, ..., 1.6785e-04,
2.0778e-04, 2.3592e-04],
[ 6.5422e-04, 2.2161e-04, 2.3806e-04, ..., 1.9085e-04,
1.0508e-04, 1.4663e-04],
[ 2.6894e-04, 1.5521e-04, 1.3745e-04, ..., 1.1659e-04,
3.2067e-05, 6.8426e-05]], device='cuda:0')
Epoch 54, bias, value: tensor([-1.8857e-02, -2.3560e-01, 8.6756e-03, -1.6740e-01, 2.6472e-05,
2.7247e-01, 1.4994e-01], device='cuda:0'), grad: tensor([-0.0022, 0.0010, 0.0029, -0.0059, 0.0020, 0.0017, 0.0005],
device='cuda:0')
351
0.001386025680863044
changing lr
---------------------saving model at epoch 53----------------------------------------------------
epoch 53, time 490.61, cls_loss 0.0052 cls_loss_mapping 0.0430 cls_loss_causal 0.7053 re_mapping 0.0491 re_causal 0.0511 /// teacc 90.72 lr 0.00123464
Epoch 55, weight, value: tensor([[-2.1294e-02, -1.3864e-02, -6.9845e-03, ..., -9.8950e-03,
9.9935e-02, 1.2198e-01],
[ 3.8158e-02, 4.8834e-02, 4.1167e-02, ..., 4.1333e-02,
1.6349e-02, -1.6418e-02],
[ 6.1134e-02, 3.5595e-02, 5.9100e-02, ..., 7.4893e-03,
7.9670e-04, 1.8389e-02],
...,
[-1.1226e-01, -9.6775e-02, -6.5204e-02, ..., 1.5158e-03,
1.5462e-02, -1.5474e-02],
[-1.2280e-02, -2.6849e-03, -2.5158e-02, ..., 9.2897e-02,
3.9609e-02, -5.1898e-05],
[-2.2534e-02, -2.7644e-02, -2.1351e-02, ..., -1.2353e-01,
-1.5505e-01, -1.2130e-01]], device='cuda:0'), grad: tensor([[ 3.6740e-04, 1.1533e-04, 1.4138e-04, ..., 7.1287e-05,
6.3181e-05, 8.7380e-05],
[ 1.0300e-04, 1.9923e-05, 2.7791e-05, ..., 2.7940e-05,
3.5048e-05, 3.9577e-05],
[-4.7874e-03, -1.3676e-03, -1.7347e-03, ..., -1.0529e-03,
-1.1044e-03, -1.3723e-03],
...,
[ 2.4045e-04, 5.3853e-05, 6.8605e-05, ..., 5.7757e-05,
6.1929e-05, 7.4923e-05],
[ 7.4482e-04, 1.7750e-04, 2.0897e-04, ..., 1.5354e-04,
1.3340e-04, 1.7190e-04],
[ 1.6146e-03, 5.3740e-04, 6.7186e-04, ..., 3.2306e-04,
3.2139e-04, 4.0817e-04]], device='cuda:0')
Epoch 55, bias, value: tensor([-1.9001e-02, -2.3573e-01, 8.0842e-03, -1.6670e-01, -2.0496e-04,
2.7154e-01, 1.5125e-01], device='cuda:0'), grad: tensor([ 0.0010, 0.0003, -0.0144, 0.0048, 0.0007, 0.0022, 0.0053],
device='cuda:0')
351
0.0012346426699819469
changing lr
epoch 54, time 488.59, cls_loss 0.0049 cls_loss_mapping 0.0387 cls_loss_causal 0.6318 re_mapping 0.0490 re_causal 0.0508 /// teacc 81.86 lr 0.00109084
Epoch 56, weight, value: tensor([[-2.1449e-02, -1.3913e-02, -7.0644e-03, ..., -9.9782e-03,
9.9706e-02, 1.2171e-01],
[ 3.8039e-02, 4.8727e-02, 4.1080e-02, ..., 4.1237e-02,
1.6312e-02, -1.6407e-02],
[ 6.1227e-02, 3.5646e-02, 5.9084e-02, ..., 7.4554e-03,
7.6650e-04, 1.8350e-02],
...,
[-1.1180e-01, -9.6573e-02, -6.5017e-02, ..., 1.5741e-03,
1.5466e-02, -1.5377e-02],
[-1.2142e-02, -2.6935e-03, -2.5135e-02, ..., 9.2940e-02,
3.9678e-02, 7.4000e-05],
[-2.2755e-02, -2.7644e-02, -2.1360e-02, ..., -1.2344e-01,
-1.5488e-01, -1.2119e-01]], device='cuda:0'), grad: tensor([[ 1.8072e-04, 2.8282e-05, 2.8521e-05, ..., 1.1289e-04,
1.1140e-04, 1.0514e-04],
[ 8.2701e-06, 2.1160e-06, -5.6066e-07, ..., 1.0535e-05,
9.0897e-06, 9.0003e-06],
[ 2.6870e-04, 4.2528e-05, 4.1068e-05, ..., 1.8346e-04,
1.8668e-04, 1.6940e-04],
...,
[ 1.0824e-04, 2.4393e-05, 2.8685e-05, ..., 4.8369e-05,
4.9680e-05, 4.8935e-05],
[-6.2132e-04, -8.6784e-05, -7.6830e-05, ..., -4.2725e-04,
-4.1485e-04, -3.9148e-04],
[ 1.1188e-04, 1.4149e-05, 1.2986e-05, ..., 7.1883e-05,
6.5327e-05, 6.6519e-05]], device='cuda:0')
Epoch 56, bias, value: tensor([-0.0195, -0.2354, 0.0085, -0.1666, 0.0006, 0.2715, 0.1501],
device='cuda:0'), grad: tensor([ 5.4550e-04, 2.2292e-05, 7.8392e-04, -1.4389e-04, 3.1972e-04,
-1.8892e-03, 3.6216e-04], device='cuda:0')
351
0.0010908425876598518
changing lr
epoch 55, time 487.74, cls_loss 0.0051 cls_loss_mapping 0.0425 cls_loss_causal 0.6995 re_mapping 0.0486 re_causal 0.0505 /// teacc 75.95 lr 0.00095492
Epoch 57, weight, value: tensor([[-2.1218e-02, -1.3897e-02, -7.0272e-03, ..., -9.9669e-03,
9.9634e-02, 1.2159e-01],
[ 3.7903e-02, 4.8628e-02, 4.0986e-02, ..., 4.1183e-02,
1.6288e-02, -1.6388e-02],
[ 6.1276e-02, 3.5676e-02, 5.9051e-02, ..., 7.4110e-03,
7.6182e-04, 1.8332e-02],
...,
[-1.1156e-01, -9.6415e-02, -6.4905e-02, ..., 1.6178e-03,
1.5434e-02, -1.5334e-02],
[-1.2345e-02, -2.7407e-03, -2.5176e-02, ..., 9.2830e-02,
3.9633e-02, 7.1650e-05],
[-2.2691e-02, -2.7605e-02, -2.1297e-02, ..., -1.2326e-01,
-1.5467e-01, -1.2102e-01]], device='cuda:0'), grad: tensor([[ 3.3617e-04, 3.0011e-05, 6.1452e-05, ..., 1.2374e-04,
1.1009e-04, 1.2720e-04],
[-1.7416e-04, -1.3128e-05, -3.3319e-05, ..., -8.8274e-05,
-1.1045e-04, -1.1855e-04],
[-4.5091e-05, -2.1130e-05, -1.8314e-05, ..., -1.2778e-05,
-1.1884e-05, -1.6809e-05],
...,
[-3.4451e-04, -2.5943e-05, -5.5701e-05, ..., -9.2685e-05,
-3.5495e-05, -5.2899e-05],
[ 5.7518e-05, 9.0301e-06, 1.2822e-05, ..., 1.7747e-05,
1.5348e-05, 1.8820e-05],
[ 1.1241e-04, 1.0774e-05, 1.9923e-05, ..., 3.4451e-05,
2.0042e-05, 2.6032e-05]], device='cuda:0')
Epoch 57, bias, value: tensor([-0.0187, -0.2352, 0.0087, -0.1668, 0.0009, 0.2703, 0.1501],
device='cuda:0'), grad: tensor([ 0.0011, -0.0005, -0.0001, 0.0002, -0.0012, 0.0002, 0.0004],
device='cuda:0')
351
0.000954915028125264
changing lr
epoch 56, time 487.53, cls_loss 0.0046 cls_loss_mapping 0.0386 cls_loss_causal 0.6371 re_mapping 0.0485 re_causal 0.0503 /// teacc 89.45 lr 0.00082713
Epoch 58, weight, value: tensor([[-2.1349e-02, -1.3977e-02, -7.1024e-03, ..., -1.0061e-02,
9.9467e-02, 1.2139e-01],
[ 3.7762e-02, 4.8537e-02, 4.0907e-02, ..., 4.1115e-02,
1.6256e-02, -1.6382e-02],
[ 6.0902e-02, 3.5573e-02, 5.8886e-02, ..., 7.3153e-03,
6.9126e-04, 1.8243e-02],
...,
[-1.1138e-01, -9.6263e-02, -6.4804e-02, ..., 1.6073e-03,
1.5345e-02, -1.5362e-02],
[-1.2528e-02, -2.7840e-03, -2.5215e-02, ..., 9.2736e-02,
3.9597e-02, 7.3678e-05],
[-2.2541e-02, -2.7571e-02, -2.1249e-02, ..., -1.2309e-01,
-1.5450e-01, -1.2087e-01]], device='cuda:0'), grad: tensor([[ 8.9228e-05, 8.5235e-06, 1.4402e-05, ..., 4.6790e-05,
3.9130e-05, 4.3839e-05],
[-4.0841e-04, -3.3736e-05, -6.2108e-05, ..., -1.9312e-04,
-1.0115e-04, -1.4365e-04],
[ 2.2307e-05, 1.9856e-06, 3.4813e-06, ..., 1.3821e-05,
1.5780e-05, 1.5527e-05],
...,
[ 5.1230e-05, 4.4182e-06, 7.8455e-06, ..., 2.4691e-05,
1.4320e-05, 1.9193e-05],
[-8.1837e-05, -8.5905e-06, -1.2361e-05, ..., -6.3300e-05,
-8.9049e-05, -8.0585e-05],
[ 3.2496e-04, 2.7806e-05, 4.8935e-05, ..., 1.6892e-04,
1.1927e-04, 1.4389e-04]], device='cuda:0')
Epoch 58, bias, value: tensor([-0.0190, -0.2351, 0.0078, -0.1651, 0.0010, 0.2693, 0.1504],
device='cuda:0'), grad: tensor([ 3.0637e-04, -1.2379e-03, 9.0182e-05, 1.1995e-05, 1.5855e-04,
-3.9029e-04, 1.0614e-03], device='cuda:0')
351
0.0008271337313934874
changing lr
epoch 57, time 488.10, cls_loss 0.0042 cls_loss_mapping 0.0381 cls_loss_causal 0.6417 re_mapping 0.0488 re_causal 0.0507 /// teacc 79.32 lr 0.00070776
Epoch 59, weight, value: tensor([[-2.1408e-02, -1.3989e-02, -7.1333e-03, ..., -1.0104e-02,
9.9326e-02, 1.2121e-01],
[ 3.7841e-02, 4.8474e-02, 4.0876e-02, ..., 4.1118e-02,
1.6304e-02, -1.6304e-02],
[ 6.1134e-02, 3.5647e-02, 5.8916e-02, ..., 7.3222e-03,
6.9682e-04, 1.8255e-02],
...,
[-1.1137e-01, -9.6154e-02, -6.4747e-02, ..., 1.5936e-03,
1.5274e-02, -1.5383e-02],
[-1.2516e-02, -2.8023e-03, -2.5213e-02, ..., 9.2714e-02,
3.9614e-02, 1.2627e-04],
[-2.2560e-02, -2.7561e-02, -2.1224e-02, ..., -1.2300e-01,
-1.5434e-01, -1.2075e-01]], device='cuda:0'), grad: tensor([[-6.8173e-06, 4.6566e-09, -1.0632e-05, ..., -4.0606e-07,
-1.4566e-05, -1.8001e-05],
[ 2.6083e-04, 4.6819e-05, 7.5758e-05, ..., 6.0946e-05,
6.7711e-05, 6.5386e-05],
[ 6.2656e-04, 6.6280e-05, 7.1764e-05, ..., 1.4627e-04,
1.2958e-04, 1.4138e-04],
...,
[ 2.2799e-05, 2.8126e-06, 5.0887e-06, ..., 3.4329e-06,
6.0424e-06, 6.2250e-06],
[ 1.7595e-04, 2.1204e-05, 2.5839e-05, ..., 4.1038e-05,
3.7640e-05, 3.9846e-05],
[-1.1568e-03, -1.4627e-04, -1.7905e-04, ..., -2.6917e-04,
-2.4295e-04, -2.5249e-04]], device='cuda:0')
Epoch 59, bias, value: tensor([-0.0193, -0.2344, 0.0084, -0.1653, 0.0006, 0.2690, 0.1501],
device='cuda:0'), grad: tensor([-1.3560e-05, 7.8678e-04, 1.9627e-03, 2.4247e-04, 6.6400e-05,
5.4741e-04, -3.5896e-03], device='cuda:0')
351
0.00070775603199067
changing lr
epoch 58, time 484.91, cls_loss 0.0032 cls_loss_mapping 0.0359 cls_loss_causal 0.6249 re_mapping 0.0487 re_causal 0.0506 /// teacc 86.92 lr 0.00059702
Epoch 60, weight, value: tensor([[-0.0212, -0.0140, -0.0071, ..., -0.0101, 0.0993, 0.1212],
[ 0.0379, 0.0484, 0.0408, ..., 0.0411, 0.0163, -0.0163],
[ 0.0611, 0.0356, 0.0589, ..., 0.0073, 0.0007, 0.0182],
...,
[-0.1116, -0.0961, -0.0648, ..., 0.0015, 0.0151, -0.0155],
[-0.0125, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0002],
[-0.0226, -0.0275, -0.0212, ..., -0.1229, -0.1542, -0.1207]],
device='cuda:0'), grad: tensor([[ 4.3243e-05, 3.6228e-06, 9.9689e-06, ..., 4.7050e-06,
9.7156e-06, 7.1637e-06],
[ 1.1969e-04, 9.7975e-06, 2.7657e-05, ..., 1.1198e-05,
2.6122e-05, 1.8969e-05],
[ 1.9163e-05, 1.2685e-06, 4.2096e-06, ..., 2.0973e-06,
4.4741e-06, 3.3099e-06],
...,
[ 1.0237e-05, 1.1064e-06, 2.3302e-06, ..., 2.5518e-06,
3.0324e-06, 2.6748e-06],
[ 6.7830e-05, 5.7332e-06, 1.5661e-05, ..., 7.5512e-06,
1.5438e-05, 1.1593e-05],
[-2.8563e-04, -2.3916e-05, -6.5863e-05, ..., -3.0905e-05,
-6.4492e-05, -4.7982e-05]], device='cuda:0')
Epoch 60, bias, value: tensor([-0.0185, -0.2338, 0.0084, -0.1651, -0.0005, 0.2689, 0.1499],
device='cuda:0'), grad: tensor([ 1.4329e-04, 3.9768e-04, 6.4731e-05, 8.4817e-05, 3.3200e-05,
2.2471e-04, -9.4748e-04], device='cuda:0')
351
0.0005970223407163104
changing lr
epoch 59, time 492.38, cls_loss 0.0040 cls_loss_mapping 0.0384 cls_loss_causal 0.6520 re_mapping 0.0485 re_causal 0.0505 /// teacc 82.70 lr 0.00049516
Epoch 61, weight, value: tensor([[-0.0212, -0.0140, -0.0071, ..., -0.0101, 0.0992, 0.1211],
[ 0.0379, 0.0484, 0.0408, ..., 0.0411, 0.0163, -0.0162],
[ 0.0611, 0.0357, 0.0589, ..., 0.0073, 0.0006, 0.0182],
...,
[-0.1115, -0.0960, -0.0647, ..., 0.0015, 0.0151, -0.0155],
[-0.0122, -0.0028, -0.0252, ..., 0.0928, 0.0397, 0.0003],
[-0.0227, -0.0275, -0.0212, ..., -0.1229, -0.1541, -0.1206]],
device='cuda:0'), grad: tensor([[-2.4581e-04, -2.5958e-05, -5.0008e-05, ..., -4.5031e-05,
-1.1921e-04, -8.9407e-05],
[ 8.8871e-05, 5.2899e-06, 7.9200e-06, ..., 3.9667e-05,
2.7984e-05, 2.2709e-05],
[ 2.7013e-04, 3.2693e-05, 3.7163e-05, ..., 1.3626e-04,
1.0562e-04, 8.9467e-05],
...,
[ 5.0497e-04, 4.4256e-05, 7.9632e-05, ..., 1.6642e-04,
2.0587e-04, 1.6856e-04],
[-3.8967e-03, -2.2078e-04, -4.4274e-04, ..., -1.4009e-03,
-1.1187e-03, -9.2793e-04],
[ 3.1357e-03, 1.4865e-04, 3.4857e-04, ..., 1.0357e-03,
8.4352e-04, 6.8998e-04]], device='cuda:0')
Epoch 61, bias, value: tensor([-0.0187, -0.2335, 0.0085, -0.1652, -0.0006, 0.2693, 0.1494],
device='cuda:0'), grad: tensor([-0.0008, 0.0003, 0.0008, 0.0005, 0.0016, -0.0127, 0.0103],
device='cuda:0')
351
0.0004951556604879052
changing lr
epoch 60, time 489.26, cls_loss 0.0038 cls_loss_mapping 0.0370 cls_loss_causal 0.6870 re_mapping 0.0484 re_causal 0.0504 /// teacc 82.28 lr 0.00040236
Epoch 62, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0991, 0.1210],
[ 0.0379, 0.0483, 0.0408, ..., 0.0411, 0.0163, -0.0162],
[ 0.0610, 0.0357, 0.0588, ..., 0.0072, 0.0006, 0.0182],
...,
[-0.1114, -0.0959, -0.0647, ..., 0.0015, 0.0151, -0.0155],
[-0.0122, -0.0028, -0.0252, ..., 0.0928, 0.0397, 0.0003],
[-0.0226, -0.0275, -0.0212, ..., -0.1228, -0.1540, -0.1205]],
device='cuda:0'), grad: tensor([[ 2.2188e-05, 8.2105e-06, 8.0541e-06, ..., 3.5763e-06,
9.4622e-07, 1.9614e-06],
[-4.1723e-05, -2.1998e-06, -4.1761e-06, ..., -1.7449e-05,
-1.4827e-05, -1.3500e-05],
[-4.8637e-04, -1.8167e-04, -1.8668e-04, ..., -9.2447e-05,
-6.3539e-05, -8.6546e-05],
...,
[ 7.5512e-06, 1.9222e-06, 2.1514e-06, ..., 2.1532e-06,
1.8720e-06, 1.9912e-06],
[ 1.2040e-04, 4.2289e-05, 4.3929e-05, ..., 2.5019e-05,
1.8135e-05, 2.3142e-05],
[ 1.1921e-04, 3.4183e-05, 3.6806e-05, ..., 2.8491e-05,
2.1860e-05, 2.5377e-05]], device='cuda:0')
Epoch 62, bias, value: tensor([-0.0192, -0.2331, 0.0083, -0.1652, -0.0005, 0.2692, 0.1497],
device='cuda:0'), grad: tensor([ 5.2631e-05, -1.3030e-04, -1.1740e-03, 6.2513e-04, 2.0266e-05,
2.9707e-04, 3.0971e-04], device='cuda:0')
351
0.00040236113724274745
changing lr
epoch 61, time 489.43, cls_loss 0.0045 cls_loss_mapping 0.0367 cls_loss_causal 0.6441 re_mapping 0.0480 re_causal 0.0499 /// teacc 78.48 lr 0.00031883
Epoch 63, weight, value: tensor([[-0.0213, -0.0140, -0.0071, ..., -0.0101, 0.0991, 0.1209],
[ 0.0378, 0.0483, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0609, 0.0356, 0.0588, ..., 0.0072, 0.0006, 0.0181],
...,
[-0.1114, -0.0959, -0.0646, ..., 0.0015, 0.0151, -0.0155],
[-0.0123, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1227, -0.1539, -0.1204]],
device='cuda:0'), grad: tensor([[ 1.2529e-04, 2.2218e-05, 2.8625e-05, ..., 3.4153e-05,
2.4438e-05, 2.4959e-05],
[ 1.5825e-05, 1.6261e-06, 1.8962e-06, ..., 8.0094e-06,
6.0126e-06, 6.2659e-06],
[-2.7156e-04, -8.5413e-05, -9.2268e-05, ..., -2.8670e-05,
-1.1556e-05, -9.0450e-06],
...,
[-2.2966e-06, 7.2680e-06, 4.5262e-06, ..., -5.1968e-07,
2.3246e-06, -6.3702e-07],
[-3.3855e-05, 2.4676e-05, 1.8314e-05, ..., -9.7752e-05,
-9.1434e-05, -9.2506e-05],
[ 4.2379e-05, 4.2804e-06, 8.4341e-06, ..., 4.6492e-05,
4.1276e-05, 4.2289e-05]], device='cuda:0')
Epoch 63, bias, value: tensor([-0.0190, -0.2331, 0.0080, -0.1650, -0.0006, 0.2687, 0.1501],
device='cuda:0'), grad: tensor([ 3.8457e-04, 5.1320e-05, -7.3528e-04, 3.6812e-04, -5.8979e-05,
-1.7655e-04, 1.6725e-04], device='cuda:0')
351
0.00031882564680131423
changing lr
epoch 62, time 483.86, cls_loss 0.0042 cls_loss_mapping 0.0371 cls_loss_causal 0.6606 re_mapping 0.0480 re_causal 0.0500 /// teacc 87.76 lr 0.00024472
Epoch 64, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0483, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0608, 0.0356, 0.0587, ..., 0.0072, 0.0006, 0.0181],
...,
[-0.1112, -0.0958, -0.0646, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1227, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 2.7698e-06, 5.7667e-06, 3.1590e-06, ..., -3.6303e-06,
-1.3135e-05, -1.1802e-05],
[ 4.9829e-05, 7.4059e-06, 1.1750e-05, ..., 9.0599e-06,
1.1452e-05, 1.4067e-05],
[-1.3173e-04, -5.1230e-05, -5.5671e-05, ..., -1.5497e-05,
-5.6326e-06, -1.2428e-05],
...,
[ 2.7090e-05, 5.1446e-06, 6.8285e-06, ..., 3.5875e-06,
2.9411e-06, 4.6305e-06],
[ 5.3972e-05, 1.4357e-05, 1.7211e-05, ..., 3.2391e-06,
7.2457e-07, 4.1276e-06],
[-5.3346e-05, 9.4622e-07, -2.8946e-06, ..., -4.3772e-06,
-1.2368e-06, -6.2324e-06]], device='cuda:0')
Epoch 64, bias, value: tensor([-1.9339e-02, -2.3265e-01, 7.7978e-03, -1.6499e-01, -7.8075e-05,
2.6835e-01, 1.5003e-01], device='cuda:0'), grad: tensor([-2.4382e-06, 1.5533e-04, -3.2139e-04, 1.3471e-04, 8.2672e-05,
1.5342e-04, -2.0194e-04], device='cuda:0')
351
0.0002447174185242325
changing lr
epoch 63, time 490.22, cls_loss 0.0036 cls_loss_mapping 0.0347 cls_loss_causal 0.6470 re_mapping 0.0480 re_causal 0.0500 /// teacc 83.12 lr 0.00018019
Epoch 65, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0483, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0608, 0.0356, 0.0587, ..., 0.0071, 0.0006, 0.0181],
...,
[-0.1111, -0.0958, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 5.2881e-04, 4.0114e-05, 1.0240e-04, ..., 1.3661e-04,
1.6046e-04, 1.8704e-04],
[-8.0919e-04, -6.5684e-05, -7.5638e-05, ..., -4.0150e-04,
-2.9635e-04, -3.4237e-04],
[ 7.9060e-04, 5.5760e-05, 1.4138e-04, ..., 2.0337e-04,
2.2817e-04, 2.6584e-04],
...,
[-1.2274e-03, -8.4937e-05, -2.8634e-04, ..., -2.2125e-04,
-3.6216e-04, -4.2987e-04],
[ 3.4070e-04, 2.7388e-05, 4.6611e-05, ..., 1.2219e-04,
1.0574e-04, 1.2124e-04],
[ 1.5497e-04, 1.1295e-05, 2.6509e-05, ..., 1.0812e-04,
9.7275e-05, 1.2004e-04]], device='cuda:0')
Epoch 65, bias, value: tensor([-1.9334e-02, -2.3244e-01, 7.7977e-03, -1.6502e-01, 1.9134e-05,
2.6818e-01, 1.4991e-01], device='cuda:0'), grad: tensor([ 0.0018, -0.0027, 0.0026, 0.0007, -0.0042, 0.0011, 0.0006],
device='cuda:0')
351
0.0001801856965207339
changing lr
epoch 64, time 492.97, cls_loss 0.0040 cls_loss_mapping 0.0365 cls_loss_causal 0.6452 re_mapping 0.0477 re_causal 0.0498 /// teacc 82.28 lr 0.00012536
Epoch 66, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0482, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0608, 0.0356, 0.0587, ..., 0.0071, 0.0006, 0.0181],
...,
[-0.1110, -0.0957, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 4.5419e-05, 6.3926e-06, 1.3947e-05, ..., 1.1124e-05,
1.1191e-05, 1.2450e-05],
[-1.0870e-05, -2.7269e-06, -2.4606e-06, ..., -3.9451e-06,
-4.3772e-06, -5.6140e-06],
[ 4.3422e-05, 5.9567e-06, 1.4521e-05, ..., 7.3612e-06,
5.9381e-06, 7.0743e-06],
...,
[ 1.1817e-05, 2.1253e-06, 4.8801e-06, ..., -5.6345e-07,
-2.2799e-06, -1.7304e-06],
[ 2.2992e-05, 3.1963e-06, 7.7263e-06, ..., 3.9227e-06,
3.1628e-06, 3.7048e-06],
[-1.1885e-04, -1.5810e-05, -4.0680e-05, ..., -1.8969e-05,
-1.4514e-05, -1.6913e-05]], device='cuda:0')
Epoch 66, bias, value: tensor([-1.9370e-02, -2.3244e-01, 7.9069e-03, -1.6501e-01, 9.5384e-05,
2.6809e-01, 1.4984e-01], device='cuda:0'), grad: tensor([ 1.8620e-04, -2.0117e-05, 1.9896e-04, 2.7701e-05, 6.4075e-05,
1.0514e-04, -5.6219e-04], device='cuda:0')
351
0.000125360439090882
changing lr
epoch 65, time 491.54, cls_loss 0.0036 cls_loss_mapping 0.0321 cls_loss_causal 0.6499 re_mapping 0.0478 re_causal 0.0498 /// teacc 85.65 lr 0.00008035
Epoch 67, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0482, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0608, 0.0356, 0.0587, ..., 0.0071, 0.0005, 0.0181],
...,
[-0.1110, -0.0957, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 7.2360e-05, 5.2638e-06, 1.7837e-05, ..., 1.5587e-05,
2.2396e-05, 2.0936e-05],
[ 9.2804e-05, 4.1537e-06, 2.0787e-05, ..., 2.1264e-05,
3.0354e-05, 2.8640e-05],
[-9.5293e-06, -6.4522e-06, -6.5379e-06, ..., -1.0915e-06,
-2.5313e-06, -3.1143e-06],
...,
[ 9.6187e-06, 5.1688e-07, 2.1979e-06, ..., 1.9427e-06,
3.2876e-06, 2.9653e-06],
[ 7.5519e-05, 4.3809e-06, 1.7673e-05, ..., 1.7107e-05,
2.4602e-05, 2.3276e-05],
[-2.6488e-04, -9.6783e-06, -5.7906e-05, ..., -6.0499e-05,
-8.6486e-05, -8.0824e-05]], device='cuda:0')
Epoch 67, bias, value: tensor([-1.9302e-02, -2.3238e-01, 7.8917e-03, -1.6504e-01, 1.5820e-04,
2.6802e-01, 1.4977e-01], device='cuda:0'), grad: tensor([ 2.1708e-04, 2.8634e-04, -1.5043e-05, 7.3016e-05, 2.8566e-05,
2.3031e-04, -8.1968e-04], device='cuda:0')
351
8.03520570068517e-05
changing lr
epoch 66, time 488.28, cls_loss 0.0043 cls_loss_mapping 0.0363 cls_loss_causal 0.6060 re_mapping 0.0477 re_causal 0.0496 /// teacc 80.17 lr 0.00004525
Epoch 68, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0482, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0609, 0.0356, 0.0587, ..., 0.0071, 0.0006, 0.0181],
...,
[-0.1110, -0.0957, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0225, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 1.7738e-04, 3.6508e-05, 4.7892e-05, ..., 3.3021e-05,
5.1409e-05, 5.8830e-05],
[-1.1176e-04, -1.2986e-05, -2.6107e-05, ..., -6.1005e-05,
-7.1406e-05, -7.7963e-05],
[-1.8620e-04, -4.3780e-05, -5.1647e-05, ..., -1.6481e-05,
-4.3839e-05, -4.8906e-05],
...,
[ 9.0122e-05, 1.1779e-05, 2.0489e-05, ..., 2.9355e-05,
3.2455e-05, 3.6657e-05],
[ 1.6510e-04, 2.1622e-05, 3.7253e-05, ..., 5.1230e-05,
5.6177e-05, 6.3777e-05],
[-2.3258e-04, -2.8268e-05, -5.0306e-05, ..., -5.5730e-05,
-4.7892e-05, -5.9724e-05]], device='cuda:0')
Epoch 68, bias, value: tensor([-1.9269e-02, -2.3232e-01, 8.0016e-03, -1.6504e-01, 1.6669e-04,
2.6794e-01, 1.4963e-01], device='cuda:0'), grad: tensor([ 0.0005, -0.0004, -0.0005, 0.0003, 0.0003, 0.0005, -0.0007],
device='cuda:0')
351
4.5251191160326525e-05
changing lr
epoch 67, time 486.30, cls_loss 0.0037 cls_loss_mapping 0.0329 cls_loss_causal 0.6237 re_mapping 0.0478 re_causal 0.0498 /// teacc 78.48 lr 0.00002013
Epoch 69, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0482, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0609, 0.0356, 0.0587, ..., 0.0071, 0.0006, 0.0181],
...,
[-0.1110, -0.0957, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0225, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 8.3113e-04, 7.4923e-05, 1.7071e-04, ..., 2.6250e-04,
3.9792e-04, 3.2210e-04],
[-9.2602e-04, -8.4460e-05, -1.9145e-04, ..., -2.9302e-04,
-4.3583e-04, -3.5167e-04],
[-5.4128e-06, -2.5220e-06, -2.1048e-06, ..., 1.5944e-06,
1.8477e-06, 1.1027e-06],
...,
[ 2.8223e-05, 2.8014e-06, 6.0685e-06, ..., 8.5831e-06,
1.2398e-05, 1.0081e-05],
[ 2.3380e-05, 2.6226e-06, 5.3160e-06, ..., 7.4133e-06,
8.4266e-06, 6.3777e-06],
[ 4.0412e-05, 4.8652e-06, 9.2611e-06, ..., 1.1846e-05,
1.4052e-05, 1.0982e-05]], device='cuda:0')
Epoch 69, bias, value: tensor([-1.9275e-02, -2.3228e-01, 8.0206e-03, -1.6505e-01, 1.7925e-04,
2.6789e-01, 1.4963e-01], device='cuda:0'), grad: tensor([ 2.4948e-03, -2.7771e-03, -1.3143e-05, 2.4006e-05, 8.4937e-05,
6.9320e-05, 1.1951e-04], device='cuda:0')
351
2.0128530023804673e-05
changing lr
epoch 68, time 488.59, cls_loss 0.0038 cls_loss_mapping 0.0348 cls_loss_causal 0.6464 re_mapping 0.0477 re_causal 0.0497 /// teacc 81.01 lr 0.00000503
Epoch 70, weight, value: tensor([[-0.0214, -0.0140, -0.0071, ..., -0.0102, 0.0990, 0.1208],
[ 0.0379, 0.0482, 0.0407, ..., 0.0410, 0.0163, -0.0162],
[ 0.0609, 0.0356, 0.0587, ..., 0.0071, 0.0006, 0.0181],
...,
[-0.1110, -0.0957, -0.0645, ..., 0.0016, 0.0151, -0.0154],
[-0.0124, -0.0028, -0.0252, ..., 0.0927, 0.0397, 0.0003],
[-0.0224, -0.0275, -0.0211, ..., -0.1226, -0.1538, -0.1203]],
device='cuda:0'), grad: tensor([[ 2.3559e-05, 6.6012e-06, 1.9409e-06, ..., -3.4779e-05,
-7.4029e-05, -6.5267e-05],
[ 2.2388e-04, 2.8431e-05, 4.1455e-05, ..., 3.1710e-05,
5.0277e-05, 5.2512e-05],
[-5.5850e-05, -4.7028e-05, -3.0845e-05, ..., -2.1219e-05,
1.6883e-05, 5.6773e-06],
...,
[ 7.6115e-05, 1.3202e-05, 1.9178e-05, ..., 2.7522e-05,
4.1485e-05, 4.3154e-05],
[ 4.0293e-04, 5.9634e-05, 8.0347e-05, ..., 5.4657e-05,
7.3731e-05, 7.9453e-05],
[-8.0967e-04, -9.9838e-05, -1.5163e-04, ..., -9.5963e-05,
-1.4210e-04, -1.5724e-04]], device='cuda:0')
Epoch 70, bias, value: tensor([-1.9308e-02, -2.3226e-01, 8.0159e-03, -1.6505e-01, 1.7581e-04,
2.6787e-01, 1.4967e-01], device='cuda:0'), grad: tensor([ 5.3346e-05, 7.1144e-04, -1.0407e-04, 4.0460e-04, 2.4176e-04,
1.2579e-03, -2.5673e-03], device='cuda:0')
351
5.034667293427056e-06
changing lr
epoch 69, time 490.70, cls_loss 0.0033 cls_loss_mapping 0.0336 cls_loss_causal 0.6347 re_mapping 0.0477 re_causal 0.0497 /// teacc 83.54 lr 0.00000000
---------------------saving last model at epoch 69----------------------------------------------------
/home/yuqian_fu
{'gpu': '0', 'svroot': '/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/saved-PACS//cartoon/CA_multiple_16fa_v2_ep70_lr0.01_cosine_base0.01_bs6_lamCa_1_lamRe1_adt4_cls1_EW2_70_rmTrue_rnTrue_str5_WithStyleAttackExp1', 'source_domain': 'cartoon', 'svpath': '/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/saved-PACS//cartoon/CA_multiple_16fa_v2_ep70_lr0.01_cosine_base0.01_bs6_lamCa_1_lamRe1_adt4_cls1_EW2_70_rmTrue_rnTrue_str5_WithStyleAttackExp1/cartoon_16factor_best_test_check.csv', 'factor_num': 16, 'epoch': 'best', 'stride': 5, 'eval_mapping': False, 'network': 'resnet18'}
-------------------------------------loading pretrain weights----------------------------------
loading weight of best
randm: False
stride: 5
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
loading weight of best
columns: ['cartoon', 'art_painting', 'photo', 'sketch']
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/cartoon_test.hdf5 torch.Size([2344, 3, 227, 227]) torch.Size([2344])
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/art_painting_test.hdf5 torch.Size([2048, 3, 227, 227]) torch.Size([2048])
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/photo_test.hdf5 torch.Size([1670, 3, 227, 227]) torch.Size([1670])
/data/work-gcp-europe-west4-a/yuqian_fu/datasets/SingleSourceDG/data/PACS/sketch_test.hdf5 torch.Size([3929, 3, 227, 227]) torch.Size([3929])
cartoon art_painting photo sketch Avg
w/o do (original x) 99.061433 54.6875 66.047904 60.066175 60.267193
cartoon art_painting photo sketch Avg
do 99.146758 55.712891 72.45509 63.425808 63.864596