File size: 7,857 Bytes
48a7106 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | 2021-10-09 12:51:14,515 : ***** Transfer task : STS12 *****
2021-10-09 12:51:17,767 : MSRpar : pearson = 0.6532, spearman = 0.6454, align_loss = 0.1535, uniform_loss = -2.0307
2021-10-09 12:51:18,984 : MSRvid : pearson = 0.8689, spearman = 0.8689, align_loss = 0.2007, uniform_loss = -1.8306
2021-10-09 12:51:20,049 : SMTeuroparl : pearson = 0.5416, spearman = 0.5946, align_loss = 0.2092, uniform_loss = -1.4600
2021-10-09 12:51:21,965 : surprise.OnWN : pearson = 0.7591, spearman = 0.7194, align_loss = 0.2337, uniform_loss = -2.0349
2021-10-09 12:51:22,994 : surprise.SMTnews : pearson = 0.7206, spearman = 0.6276, align_loss = 0.1864, uniform_loss = -1.4955
2021-10-09 12:51:22,997 : ALL : Pearson = 0.7804, Spearman = 0.7079, align_loss = 0.1965, uniform_loss = -1.8154
2021-10-09 12:51:22,997 : ALL (weighted average) : Pearson = 0.7230, Spearman = 0.7074, align_loss = 0.1967, uniform_loss = -1.8304
2021-10-09 12:51:22,997 : ALL (average) : Pearson = 0.7087, Spearman = 0.6912, align_loss = 0.1967, uniform_loss = -1.7703
2021-10-09 12:51:23,000 : ***** Transfer task : STS13 (-SMT) *****
2021-10-09 12:51:23,971 : FNWN : pearson = 0.6360, spearman = 0.6639, align_loss = 0.3311, uniform_loss = -1.8316
2021-10-09 12:51:25,584 : headlines : pearson = 0.8088, spearman = 0.8115, align_loss = 0.1909, uniform_loss = -1.9906
2021-10-09 12:51:26,659 : OnWN : pearson = 0.8599, spearman = 0.8422, align_loss = 0.2473, uniform_loss = -1.8941
2021-10-09 12:51:26,661 : ALL : Pearson = 0.8231, Spearman = 0.8281, align_loss = 0.2342, uniform_loss = -1.9290
2021-10-09 12:51:26,661 : ALL (weighted average) : Pearson = 0.8062, Spearman = 0.8044, align_loss = 0.2297, uniform_loss = -1.9344
2021-10-09 12:51:26,661 : ALL (average) : Pearson = 0.7683, Spearman = 0.7726, align_loss = 0.2564, uniform_loss = -1.9054
2021-10-09 12:51:26,662 : ***** Transfer task : STS14 *****
2021-10-09 12:51:27,841 : deft-forum : pearson = 0.5746, spearman = 0.5652, align_loss = 0.2150, uniform_loss = -1.9485
2021-10-09 12:51:29,101 : deft-news : pearson = 0.8208, spearman = 0.7740, align_loss = 0.1514, uniform_loss = -1.8488
2021-10-09 12:51:30,803 : headlines : pearson = 0.7927, spearman = 0.7746, align_loss = 0.1893, uniform_loss = -2.0195
2021-10-09 12:51:32,323 : images : pearson = 0.8899, spearman = 0.8514, align_loss = 0.2120, uniform_loss = -2.0607
2021-10-09 12:51:33,885 : OnWN : pearson = 0.8645, spearman = 0.8547, align_loss = 0.2589, uniform_loss = -1.9353
2021-10-09 12:51:36,001 : tweet-news : pearson = 0.7930, spearman = 0.7280, align_loss = 0.2927, uniform_loss = -1.9504
2021-10-09 12:51:36,005 : ALL : Pearson = 0.8028, Spearman = 0.7616, align_loss = 0.2268, uniform_loss = -1.9721
2021-10-09 12:51:36,005 : ALL (weighted average) : Pearson = 0.8026, Spearman = 0.7715, align_loss = 0.2285, uniform_loss = -1.9749
2021-10-09 12:51:36,005 : ALL (average) : Pearson = 0.7893, Spearman = 0.7580, align_loss = 0.2199, uniform_loss = -1.9605
2021-10-09 12:51:36,008 : ***** Transfer task : STS15 *****
2021-10-09 12:51:37,508 : answers-forums : pearson = 0.7354, spearman = 0.7407, align_loss = 0.4058, uniform_loss = -2.0225
2021-10-09 12:51:39,060 : answers-students : pearson = 0.7466, spearman = 0.7530, align_loss = 0.2678, uniform_loss = -1.4429
2021-10-09 12:51:40,625 : belief : pearson = 0.8251, spearman = 0.8323, align_loss = 0.2794, uniform_loss = -1.9058
2021-10-09 12:51:42,490 : headlines : pearson = 0.8350, spearman = 0.8412, align_loss = 0.1893, uniform_loss = -2.0135
2021-10-09 12:51:44,133 : images : pearson = 0.8972, spearman = 0.9090, align_loss = 0.1812, uniform_loss = -1.8359
2021-10-09 12:51:44,137 : ALL : Pearson = 0.8239, Spearman = 0.8313, align_loss = 0.2452, uniform_loss = -1.8141
2021-10-09 12:51:44,138 : ALL (weighted average) : Pearson = 0.8147, Spearman = 0.8224, align_loss = 0.2452, uniform_loss = -1.8141
2021-10-09 12:51:44,138 : ALL (average) : Pearson = 0.8078, Spearman = 0.8152, align_loss = 0.2647, uniform_loss = -1.8441
2021-10-09 12:51:44,141 : ***** Transfer task : STS16 *****
2021-10-09 12:51:44,870 : answer-answer : pearson = 0.7683, spearman = 0.7647, align_loss = 0.2618, uniform_loss = -1.6292
2021-10-09 12:51:45,422 : headlines : pearson = 0.8241, spearman = 0.8471, align_loss = 0.1570, uniform_loss = -2.0297
2021-10-09 12:51:46,104 : plagiarism : pearson = 0.8587, spearman = 0.8670, align_loss = 0.1583, uniform_loss = -1.6840
2021-10-09 12:51:47,451 : postediting : pearson = 0.8617, spearman = 0.8771, align_loss = 0.1205, uniform_loss = -1.8880
2021-10-09 12:51:48,001 : question-question : pearson = 0.7415, spearman = 0.7430, align_loss = 0.2224, uniform_loss = -1.9116
2021-10-09 12:51:48,003 : ALL : Pearson = 0.8069, Spearman = 0.8176, align_loss = 0.1840, uniform_loss = -1.8285
2021-10-09 12:51:48,003 : ALL (weighted average) : Pearson = 0.8120, Spearman = 0.8211, align_loss = 0.1837, uniform_loss = -1.8269
2021-10-09 12:51:48,003 : ALL (average) : Pearson = 0.8108, Spearman = 0.8198, align_loss = 0.1840, uniform_loss = -1.8285
2021-10-09 12:51:48,004 :
***** Transfer task : STSBenchmark*****
2021-10-09 12:52:07,123 : train : pearson = 0.8266, spearman = 0.8037, align_loss = 0.1921, uniform_loss = -1.9833
2021-10-09 12:52:12,465 : dev : pearson = 0.8510, spearman = 0.8524, align_loss = 0.2155, uniform_loss = -1.9961
2021-10-09 12:52:17,090 : test : pearson = 0.8202, spearman = 0.8172, align_loss = 0.1894, uniform_loss = -1.9345
2021-10-09 12:52:17,096 : ALL : Pearson = 0.8307, Spearman = 0.8176, align_loss = 0.1958, uniform_loss = -1.9776
2021-10-09 12:52:17,096 : ALL (weighted average) : Pearson = 0.8298, Spearman = 0.8143, align_loss = 0.1957, uniform_loss = -1.9777
2021-10-09 12:52:17,096 : ALL (average) : Pearson = 0.8326, Spearman = 0.8244, align_loss = 0.1990, uniform_loss = -1.9713
2021-10-09 12:52:17,105 :
***** Transfer task : SICKRelatedness*****
2021-10-09 12:52:28,392 : train : pearson = 0.8009, spearman = 0.7122, align_loss = 0.1779, uniform_loss = -1.9437
2021-10-09 12:52:29,814 : dev : pearson = 0.8048, spearman = 0.7328, align_loss = 0.1817, uniform_loss = -2.0617
2021-10-09 12:52:42,160 : test : pearson = 0.7955, spearman = 0.7081, align_loss = 0.1763, uniform_loss = -1.9393
2021-10-09 12:52:42,167 : ALL : Pearson = 0.7984, Spearman = 0.7113, align_loss = 0.1773, uniform_loss = -1.9475
2021-10-09 12:52:42,167 : ALL (weighted average) : Pearson = 0.7984, Spearman = 0.7112, align_loss = 0.1773, uniform_loss = -1.9475
2021-10-09 12:52:42,167 : ALL (average) : Pearson = 0.8004, Spearman = 0.7177, align_loss = 0.1787, uniform_loss = -1.9816
2021-10-09 12:52:42,167 : ------ test ------
2021-10-09 12:52:42,168 : +--------+--------+--------+--------+--------+--------------+-----------------+--------+
| STS12 | STS13 | STS14 | STS15 | STS16 | STSBenchmark | SICKRelatedness | Avg. |
+--------+--------+--------+--------+--------+--------------+-----------------+--------+
| 70.79 | 82.81 | 76.16 | 83.13 | 81.76 | 81.72 | 70.81 | 78.17 |
| 0.197 | 0.234 | 0.227 | 0.245 | 0.184 | 0.189 | 0.176 | 0.207 |
| -1.815 | -1.929 | -1.972 | -1.814 | -1.828 | -1.935 | -1.939 | -1.890 |
+--------+--------+--------+--------+--------+--------------+-----------------+--------+
2021-10-09 12:52:42,170 : +------+------+------+------+------+------+------+------+
| MR | CR | SUBJ | MPQA | SST2 | TREC | MRPC | Avg. |
+------+------+------+------+------+------+------+------+
| 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
+------+------+------+------+------+------+------+------+
|