| Loading data... | |
| Test set: 1406 articles from 59 laws | |
| Queries: 104 | |
| Corpus: 1406 documents | |
| Test: 104 queries, 1406 docs | |
| Loading TELEN... | |
| Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu130). | |
| W0514 16:51:50.221000 21488 site-packages\torch\distributed\elastic\multiprocessing\redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs. | |
| Loading weights: 0%| | 0/199 [00:00<?, ?it/s] Loading weights: 100%|##########| 199/199 [00:00<00:00, 6405.58it/s] | |
| Encoding corpus... | |
| Loading cross-encoder... | |
| Loading weights: 0%| | 0/197 [00:00<?, ?it/s] Loading weights: 100%|##########| 197/197 [00:00<00:00, 35724.76it/s] | |
| [1mRobertaModel LOAD REPORT[0m from: vinai/phobert-base-v2 | |
| Key | Status | | |
| --------------------------------+------------+- | |
| roberta.embeddings.position_ids | UNEXPECTED | | |
| lm_head.layer_norm.bias | UNEXPECTED | | |
| lm_head.bias | UNEXPECTED | | |
| lm_head.dense.bias | UNEXPECTED | | |
| lm_head.dense.weight | UNEXPECTED | | |
| lm_head.layer_norm.weight | UNEXPECTED | | |
| pooler.dense.bias | MISSING | | |
| pooler.dense.weight | MISSING | | |
| Notes: | |
| - UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch. | |
| - MISSING: those params were newly initialized because missing from the checkpoint. Consider training on your downstream task. | |
| --- TELEN standalone --- | |
| TELEN: 0%| | 0/104 [00:00<?, ?it/s] TELEN: 9%|8 | 9/104 [00:00<00:01, 89.03it/s] TELEN: 17%|#7 | 18/104 [00:00<00:01, 80.80it/s] TELEN: 26%|##5 | 27/104 [00:00<00:00, 79.92it/s] TELEN: 36%|###5 | 37/104 [00:00<00:00, 87.24it/s] TELEN: 44%|####4 | 46/104 [00:00<00:00, 85.09it/s] TELEN: 54%|#####3 | 56/104 [00:00<00:00, 89.14it/s] TELEN: 64%|######4 | 67/104 [00:00<00:00, 94.46it/s] TELEN: 75%|#######5 | 78/104 [00:00<00:00, 97.55it/s] TELEN: 86%|########5 | 89/104 [00:00<00:00, 99.19it/s] TELEN: 96%|#########6| 100/104 [00:01<00:00, 102.07it/s] TELEN: 100%|##########| 104/104 [00:01<00:00, 94.47it/s] | |
| --- TELEN + CE rerank top-5 --- | |
| CE-5: 0%| | 0/104 [00:00<?, ?it/s] CE-5: 4%|3 | 4/104 [00:00<00:03, 31.60it/s] CE-5: 8%|7 | 8/104 [00:00<00:02, 32.05it/s] CE-5: 12%|#1 | 12/104 [00:00<00:02, 34.14it/s] CE-5: 15%|#5 | 16/104 [00:00<00:02, 34.15it/s] CE-5: 19%|#9 | 20/104 [00:00<00:02, 33.23it/s] CE-5: 23%|##3 | 24/104 [00:00<00:02, 33.03it/s] CE-5: 27%|##6 | 28/104 [00:00<00:02, 33.16it/s] CE-5: 31%|### | 32/104 [00:00<00:02, 32.03it/s] CE-5: 35%|###4 | 36/104 [00:01<00:02, 33.48it/s] CE-5: 38%|###8 | 40/104 [00:01<00:01, 33.86it/s] CE-5: 42%|####2 | 44/104 [00:01<00:01, 33.35it/s] CE-5: 46%|####6 | 48/104 [00:01<00:01, 33.81it/s] CE-5: 50%|##### | 52/104 [00:01<00:01, 32.82it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-5: 54%|#####3 | 56/104 [00:01<00:01, 32.17it/s] CE-5: 58%|#####7 | 60/104 [00:01<00:01, 32.19it/s] CE-5: 62%|######2 | 65/104 [00:01<00:01, 34.63it/s] CE-5: 66%|######6 | 69/104 [00:02<00:01, 34.92it/s] CE-5: 71%|#######1 | 74/104 [00:02<00:00, 36.61it/s] CE-5: 75%|#######5 | 78/104 [00:02<00:00, 36.82it/s] CE-5: 79%|#######8 | 82/104 [00:02<00:00, 36.49it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-5: 83%|########2 | 86/104 [00:02<00:00, 34.66it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-5: 87%|########6 | 90/104 [00:02<00:00, 34.30it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-5: 90%|######### | 94/104 [00:02<00:00, 33.58it/s] CE-5: 94%|#########4| 98/104 [00:02<00:00, 33.21it/s] CE-5: 98%|#########8| 102/104 [00:03<00:00, 34.61it/s] CE-5: 100%|##########| 104/104 [00:03<00:00, 34.10it/s] | |
| --- TELEN + CE rerank top-10 --- | |
| CE-10: 0%| | 0/104 [00:00<?, ?it/s] CE-10: 3%|2 | 3/104 [00:00<00:04, 23.79it/s] CE-10: 6%|5 | 6/104 [00:00<00:04, 20.58it/s] CE-10: 9%|8 | 9/104 [00:00<00:03, 23.75it/s] CE-10: 12%|#1 | 12/104 [00:00<00:03, 24.22it/s] CE-10: 14%|#4 | 15/104 [00:00<00:03, 25.63it/s] CE-10: 17%|#7 | 18/104 [00:00<00:03, 25.22it/s] CE-10: 20%|## | 21/104 [00:00<00:03, 24.64it/s] CE-10: 23%|##3 | 24/104 [00:00<00:03, 24.30it/s] CE-10: 26%|##5 | 27/104 [00:01<00:03, 24.57it/s] CE-10: 29%|##8 | 30/104 [00:01<00:03, 23.97it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-10: 32%|###1 | 33/104 [00:01<00:03, 22.49it/s] CE-10: 35%|###4 | 36/104 [00:01<00:02, 23.55it/s] CE-10: 38%|###7 | 39/104 [00:01<00:02, 23.29it/s] CE-10: 40%|#### | 42/104 [00:01<00:02, 23.83it/s] CE-10: 43%|####3 | 45/104 [00:01<00:02, 22.92it/s] CE-10: 47%|####7 | 49/104 [00:02<00:02, 24.69it/s] CE-10: 50%|##### | 52/104 [00:02<00:02, 23.78it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-10: 53%|#####2 | 55/104 [00:02<00:02, 22.55it/s] CE-10: 56%|#####5 | 58/104 [00:02<00:01, 23.53it/s] CE-10: 59%|#####8 | 61/104 [00:02<00:01, 23.52it/s] CE-10: 62%|######1 | 64/104 [00:02<00:01, 22.88it/s] CE-10: 64%|######4 | 67/104 [00:02<00:01, 23.33it/s] CE-10: 67%|######7 | 70/104 [00:02<00:01, 22.73it/s] CE-10: 70%|####### | 73/104 [00:03<00:01, 22.52it/s] CE-10: 73%|#######3 | 76/104 [00:03<00:01, 22.40it/s] CE-10: 76%|#######5 | 79/104 [00:03<00:01, 23.74it/s] CE-10: 79%|#######8 | 82/104 [00:03<00:00, 23.42it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-10: 82%|########1 | 85/104 [00:03<00:00, 22.35it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-10: 85%|########4 | 88/104 [00:03<00:00, 20.82it/s] CE-10: 88%|########7 | 91/104 [00:03<00:00, 22.61it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-10: 90%|######### | 94/104 [00:04<00:00, 22.77it/s] CE-10: 93%|#########3| 97/104 [00:04<00:00, 23.73it/s] CE-10: 96%|#########6| 100/104 [00:04<00:00, 23.77it/s] CE-10: 99%|#########9| 103/104 [00:04<00:00, 25.00it/s] CE-10: 100%|##########| 104/104 [00:04<00:00, 23.51it/s] | |
| --- TELEN + CE rerank top-20 --- | |
| CE-20: 0%| | 0/104 [00:00<?, ?it/s] CE-20: 2%|1 | 2/104 [00:00<00:07, 12.81it/s] CE-20: 4%|3 | 4/104 [00:00<00:08, 12.37it/s] CE-20: 6%|5 | 6/104 [00:00<00:08, 12.12it/s] CE-20: 8%|7 | 8/104 [00:00<00:06, 13.74it/s] CE-20: 10%|9 | 10/104 [00:00<00:06, 13.86it/s] CE-20: 12%|#1 | 12/104 [00:00<00:06, 14.79it/s] CE-20: 13%|#3 | 14/104 [00:01<00:06, 14.26it/s] CE-20: 15%|#5 | 16/104 [00:01<00:05, 14.87it/s] CE-20: 17%|#7 | 18/104 [00:01<00:05, 14.87it/s] CE-20: 19%|#9 | 20/104 [00:01<00:05, 14.08it/s] CE-20: 21%|##1 | 22/104 [00:01<00:05, 14.48it/s] CE-20: 23%|##3 | 24/104 [00:01<00:05, 14.71it/s] CE-20: 25%|##5 | 26/104 [00:01<00:05, 14.95it/s] CE-20: 27%|##6 | 28/104 [00:01<00:05, 14.84it/s] CE-20: 29%|##8 | 30/104 [00:02<00:05, 14.67it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 31%|### | 32/104 [00:02<00:05, 13.96it/s] CE-20: 33%|###2 | 34/104 [00:02<00:04, 14.24it/s] CE-20: 35%|###4 | 36/104 [00:02<00:04, 13.78it/s] CE-20: 37%|###6 | 38/104 [00:02<00:04, 13.92it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 38%|###8 | 40/104 [00:02<00:04, 13.54it/s] CE-20: 40%|#### | 42/104 [00:02<00:04, 14.06it/s] CE-20: 42%|####2 | 44/104 [00:03<00:04, 14.18it/s] CE-20: 44%|####4 | 46/104 [00:03<00:04, 14.11it/s] CE-20: 46%|####6 | 48/104 [00:03<00:03, 15.29it/s] CE-20: 48%|####8 | 50/104 [00:03<00:03, 14.92it/s] CE-20: 50%|##### | 52/104 [00:03<00:03, 14.62it/s] CE-20: 52%|#####1 | 54/104 [00:03<00:03, 13.91it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 54%|#####3 | 56/104 [00:03<00:03, 14.12it/s] CE-20: 56%|#####5 | 58/104 [00:04<00:03, 13.99it/s] CE-20: 58%|#####7 | 60/104 [00:04<00:03, 13.81it/s] CE-20: 60%|#####9 | 62/104 [00:04<00:02, 14.78it/s] CE-20: 62%|######1 | 64/104 [00:04<00:02, 14.50it/s] CE-20: 63%|######3 | 66/104 [00:04<00:02, 14.38it/s] CE-20: 65%|######5 | 68/104 [00:04<00:02, 15.15it/s] CE-20: 67%|######7 | 70/104 [00:04<00:02, 14.86it/s] CE-20: 69%|######9 | 72/104 [00:05<00:02, 14.46it/s] CE-20: 71%|#######1 | 74/104 [00:05<00:02, 14.66it/s] CE-20: 73%|#######3 | 76/104 [00:05<00:02, 13.58it/s] CE-20: 75%|#######5 | 78/104 [00:05<00:01, 14.07it/s] CE-20: 77%|#######6 | 80/104 [00:05<00:01, 14.45it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 79%|#######8 | 82/104 [00:05<00:01, 13.51it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 81%|######## | 84/104 [00:05<00:01, 12.86it/s] CE-20: 83%|########2 | 86/104 [00:06<00:01, 12.90it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 85%|########4 | 88/104 [00:06<00:01, 11.80it/s] CE-20: 87%|########6 | 90/104 [00:06<00:01, 13.26it/s] CE-20: 88%|########8 | 92/104 [00:06<00:00, 14.27it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed. | |
| CE-20: 90%|######### | 94/104 [00:06<00:00, 13.78it/s] CE-20: 92%|#########2| 96/104 [00:06<00:00, 14.09it/s] CE-20: 94%|#########4| 98/104 [00:06<00:00, 14.96it/s] CE-20: 96%|#########6| 100/104 [00:07<00:00, 14.98it/s] CE-20: 98%|#########8| 102/104 [00:07<00:00, 15.52it/s] CE-20: 100%|##########| 104/104 [00:07<00:00, 16.29it/s] CE-20: 100%|##########| 104/104 [00:07<00:00, 14.26it/s] | |
| =========================================================================== | |
| MRR BOOST RESULTS | |
| =========================================================================== | |
| Method NDCG@3 MRR@3 NDCG@5 MRR@5 NDCG@10 MRR@10 | |
| -------------------------------------------------------------------------------------- | |
| TELEN 0.9036 0.8830 0.9138 0.8878 0.9132 0.8878 | |
| TELEN+CE@5 0.9346 0.9199 0.9339 0.9223 0.9238 0.9223 | |
| TELEN+CE@10 0.8766 0.8510 0.8887 0.8577 0.8942 0.8609 | |
| TELEN+CE@20 0.7653 0.7212 0.7897 0.7303 0.8128 0.7349 | |
| --- MRR Gain vs TELEN standalone --- | |
| TELEN+CE@5: MRR@3: +4.2% | MRR@5: +3.9% | MRR@10: +3.9% | |
| TELEN+CE@10: MRR@3: -3.6% | MRR@5: -3.4% | MRR@10: -3.0% | |
| TELEN+CE@20: MRR@3: -18.3% | MRR@5: -17.7% | MRR@10: -17.2% | |
| Done! | |