Loading data...
  Test set: 1406 articles from 59 laws
  Queries: 104
  Corpus: 1406 documents
Test: 104 queries, 1406 docs
Loading TELEN...
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu130).
W0514 16:51:50.221000 21488 site-packages\torch\distributed\elastic\multiprocessing\redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.

Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]
Loading weights: 100%|##########| 199/199 [00:00<00:00, 6405.58it/s]
Encoding corpus...
Loading cross-encoder...

Loading weights:   0%|          | 0/197 [00:00<?, ?it/s]
Loading weights: 100%|##########| 197/197 [00:00<00:00, 35724.76it/s]
RobertaModel LOAD REPORT from: vinai/phobert-base-v2
Key                             | Status     | 
--------------------------------+------------+-
roberta.embeddings.position_ids | UNEXPECTED | 
lm_head.layer_norm.bias         | UNEXPECTED | 
lm_head.bias                    | UNEXPECTED | 
lm_head.dense.bias              | UNEXPECTED | 
lm_head.dense.weight            | UNEXPECTED | 
lm_head.layer_norm.weight       | UNEXPECTED | 
pooler.dense.bias               | MISSING    | 
pooler.dense.weight             | MISSING    | 

Notes:
- UNEXPECTED:	can be ignored when loading from different task/architecture; not ok if you expect identical arch.
- MISSING:	those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.

--- TELEN standalone ---

TELEN:   0%|          | 0/104 [00:00<?, ?it/s]
TELEN:   9%|8         | 9/104 [00:00<00:01, 89.03it/s]
TELEN:  17%|#7        | 18/104 [00:00<00:01, 80.80it/s]
TELEN:  26%|##5       | 27/104 [00:00<00:00, 79.92it/s]
TELEN:  36%|###5      | 37/104 [00:00<00:00, 87.24it/s]
TELEN:  44%|####4     | 46/104 [00:00<00:00, 85.09it/s]
TELEN:  54%|#####3    | 56/104 [00:00<00:00, 89.14it/s]
TELEN:  64%|######4   | 67/104 [00:00<00:00, 94.46it/s]
TELEN:  75%|#######5  | 78/104 [00:00<00:00, 97.55it/s]
TELEN:  86%|########5 | 89/104 [00:00<00:00, 99.19it/s]
TELEN:  96%|#########6| 100/104 [00:01<00:00, 102.07it/s]
TELEN: 100%|##########| 104/104 [00:01<00:00, 94.47it/s] 

--- TELEN + CE rerank top-5 ---

CE-5:   0%|          | 0/104 [00:00<?, ?it/s]
CE-5:   4%|3         | 4/104 [00:00<00:03, 31.60it/s]
CE-5:   8%|7         | 8/104 [00:00<00:02, 32.05it/s]
CE-5:  12%|#1        | 12/104 [00:00<00:02, 34.14it/s]
CE-5:  15%|#5        | 16/104 [00:00<00:02, 34.15it/s]
CE-5:  19%|#9        | 20/104 [00:00<00:02, 33.23it/s]
CE-5:  23%|##3       | 24/104 [00:00<00:02, 33.03it/s]
CE-5:  27%|##6       | 28/104 [00:00<00:02, 33.16it/s]
CE-5:  31%|###       | 32/104 [00:00<00:02, 32.03it/s]
CE-5:  35%|###4      | 36/104 [00:01<00:02, 33.48it/s]
CE-5:  38%|###8      | 40/104 [00:01<00:01, 33.86it/s]
CE-5:  42%|####2     | 44/104 [00:01<00:01, 33.35it/s]
CE-5:  46%|####6     | 48/104 [00:01<00:01, 33.81it/s]
CE-5:  50%|#####     | 52/104 [00:01<00:01, 32.82it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-5:  54%|#####3    | 56/104 [00:01<00:01, 32.17it/s]
CE-5:  58%|#####7    | 60/104 [00:01<00:01, 32.19it/s]
CE-5:  62%|######2   | 65/104 [00:01<00:01, 34.63it/s]
CE-5:  66%|######6   | 69/104 [00:02<00:01, 34.92it/s]
CE-5:  71%|#######1  | 74/104 [00:02<00:00, 36.61it/s]
CE-5:  75%|#######5  | 78/104 [00:02<00:00, 36.82it/s]
CE-5:  79%|#######8  | 82/104 [00:02<00:00, 36.49it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-5:  83%|########2 | 86/104 [00:02<00:00, 34.66it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-5:  87%|########6 | 90/104 [00:02<00:00, 34.30it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-5:  90%|######### | 94/104 [00:02<00:00, 33.58it/s]
CE-5:  94%|#########4| 98/104 [00:02<00:00, 33.21it/s]
CE-5:  98%|#########8| 102/104 [00:03<00:00, 34.61it/s]
CE-5: 100%|##########| 104/104 [00:03<00:00, 34.10it/s]

--- TELEN + CE rerank top-10 ---

CE-10:   0%|          | 0/104 [00:00<?, ?it/s]
CE-10:   3%|2         | 3/104 [00:00<00:04, 23.79it/s]
CE-10:   6%|5         | 6/104 [00:00<00:04, 20.58it/s]
CE-10:   9%|8         | 9/104 [00:00<00:03, 23.75it/s]
CE-10:  12%|#1        | 12/104 [00:00<00:03, 24.22it/s]
CE-10:  14%|#4        | 15/104 [00:00<00:03, 25.63it/s]
CE-10:  17%|#7        | 18/104 [00:00<00:03, 25.22it/s]
CE-10:  20%|##        | 21/104 [00:00<00:03, 24.64it/s]
CE-10:  23%|##3       | 24/104 [00:00<00:03, 24.30it/s]
CE-10:  26%|##5       | 27/104 [00:01<00:03, 24.57it/s]
CE-10:  29%|##8       | 30/104 [00:01<00:03, 23.97it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-10:  32%|###1      | 33/104 [00:01<00:03, 22.49it/s]
CE-10:  35%|###4      | 36/104 [00:01<00:02, 23.55it/s]
CE-10:  38%|###7      | 39/104 [00:01<00:02, 23.29it/s]
CE-10:  40%|####      | 42/104 [00:01<00:02, 23.83it/s]
CE-10:  43%|####3     | 45/104 [00:01<00:02, 22.92it/s]
CE-10:  47%|####7     | 49/104 [00:02<00:02, 24.69it/s]
CE-10:  50%|#####     | 52/104 [00:02<00:02, 23.78it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-10:  53%|#####2    | 55/104 [00:02<00:02, 22.55it/s]
CE-10:  56%|#####5    | 58/104 [00:02<00:01, 23.53it/s]
CE-10:  59%|#####8    | 61/104 [00:02<00:01, 23.52it/s]
CE-10:  62%|######1   | 64/104 [00:02<00:01, 22.88it/s]
CE-10:  64%|######4   | 67/104 [00:02<00:01, 23.33it/s]
CE-10:  67%|######7   | 70/104 [00:02<00:01, 22.73it/s]
CE-10:  70%|#######   | 73/104 [00:03<00:01, 22.52it/s]
CE-10:  73%|#######3  | 76/104 [00:03<00:01, 22.40it/s]
CE-10:  76%|#######5  | 79/104 [00:03<00:01, 23.74it/s]
CE-10:  79%|#######8  | 82/104 [00:03<00:00, 23.42it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-10:  82%|########1 | 85/104 [00:03<00:00, 22.35it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-10:  85%|########4 | 88/104 [00:03<00:00, 20.82it/s]
CE-10:  88%|########7 | 91/104 [00:03<00:00, 22.61it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-10:  90%|######### | 94/104 [00:04<00:00, 22.77it/s]
CE-10:  93%|#########3| 97/104 [00:04<00:00, 23.73it/s]
CE-10:  96%|#########6| 100/104 [00:04<00:00, 23.77it/s]
CE-10:  99%|#########9| 103/104 [00:04<00:00, 25.00it/s]
CE-10: 100%|##########| 104/104 [00:04<00:00, 23.51it/s]

--- TELEN + CE rerank top-20 ---

CE-20:   0%|          | 0/104 [00:00<?, ?it/s]
CE-20:   2%|1         | 2/104 [00:00<00:07, 12.81it/s]
CE-20:   4%|3         | 4/104 [00:00<00:08, 12.37it/s]
CE-20:   6%|5         | 6/104 [00:00<00:08, 12.12it/s]
CE-20:   8%|7         | 8/104 [00:00<00:06, 13.74it/s]
CE-20:  10%|9         | 10/104 [00:00<00:06, 13.86it/s]
CE-20:  12%|#1        | 12/104 [00:00<00:06, 14.79it/s]
CE-20:  13%|#3        | 14/104 [00:01<00:06, 14.26it/s]
CE-20:  15%|#5        | 16/104 [00:01<00:05, 14.87it/s]
CE-20:  17%|#7        | 18/104 [00:01<00:05, 14.87it/s]
CE-20:  19%|#9        | 20/104 [00:01<00:05, 14.08it/s]
CE-20:  21%|##1       | 22/104 [00:01<00:05, 14.48it/s]
CE-20:  23%|##3       | 24/104 [00:01<00:05, 14.71it/s]
CE-20:  25%|##5       | 26/104 [00:01<00:05, 14.95it/s]
CE-20:  27%|##6       | 28/104 [00:01<00:05, 14.84it/s]
CE-20:  29%|##8       | 30/104 [00:02<00:05, 14.67it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  31%|###       | 32/104 [00:02<00:05, 13.96it/s]
CE-20:  33%|###2      | 34/104 [00:02<00:04, 14.24it/s]
CE-20:  35%|###4      | 36/104 [00:02<00:04, 13.78it/s]
CE-20:  37%|###6      | 38/104 [00:02<00:04, 13.92it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  38%|###8      | 40/104 [00:02<00:04, 13.54it/s]
CE-20:  40%|####      | 42/104 [00:02<00:04, 14.06it/s]
CE-20:  42%|####2     | 44/104 [00:03<00:04, 14.18it/s]
CE-20:  44%|####4     | 46/104 [00:03<00:04, 14.11it/s]
CE-20:  46%|####6     | 48/104 [00:03<00:03, 15.29it/s]
CE-20:  48%|####8     | 50/104 [00:03<00:03, 14.92it/s]
CE-20:  50%|#####     | 52/104 [00:03<00:03, 14.62it/s]
CE-20:  52%|#####1    | 54/104 [00:03<00:03, 13.91it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  54%|#####3    | 56/104 [00:03<00:03, 14.12it/s]
CE-20:  56%|#####5    | 58/104 [00:04<00:03, 13.99it/s]
CE-20:  58%|#####7    | 60/104 [00:04<00:03, 13.81it/s]
CE-20:  60%|#####9    | 62/104 [00:04<00:02, 14.78it/s]
CE-20:  62%|######1   | 64/104 [00:04<00:02, 14.50it/s]
CE-20:  63%|######3   | 66/104 [00:04<00:02, 14.38it/s]
CE-20:  65%|######5   | 68/104 [00:04<00:02, 15.15it/s]
CE-20:  67%|######7   | 70/104 [00:04<00:02, 14.86it/s]
CE-20:  69%|######9   | 72/104 [00:05<00:02, 14.46it/s]
CE-20:  71%|#######1  | 74/104 [00:05<00:02, 14.66it/s]
CE-20:  73%|#######3  | 76/104 [00:05<00:02, 13.58it/s]
CE-20:  75%|#######5  | 78/104 [00:05<00:01, 14.07it/s]
CE-20:  77%|#######6  | 80/104 [00:05<00:01, 14.45it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  79%|#######8  | 82/104 [00:05<00:01, 13.51it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  81%|########  | 84/104 [00:05<00:01, 12.86it/s]
CE-20:  83%|########2 | 86/104 [00:06<00:01, 12.90it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.
Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  85%|########4 | 88/104 [00:06<00:01, 11.80it/s]
CE-20:  87%|########6 | 90/104 [00:06<00:01, 13.26it/s]
CE-20:  88%|########8 | 92/104 [00:06<00:00, 14.27it/s]Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.

CE-20:  90%|######### | 94/104 [00:06<00:00, 13.78it/s]
CE-20:  92%|#########2| 96/104 [00:06<00:00, 14.09it/s]
CE-20:  94%|#########4| 98/104 [00:06<00:00, 14.96it/s]
CE-20:  96%|#########6| 100/104 [00:07<00:00, 14.98it/s]
CE-20:  98%|#########8| 102/104 [00:07<00:00, 15.52it/s]
CE-20: 100%|##########| 104/104 [00:07<00:00, 16.29it/s]
CE-20: 100%|##########| 104/104 [00:07<00:00, 14.26it/s]

===========================================================================
MRR BOOST RESULTS
===========================================================================
Method                   NDCG@3      MRR@3     NDCG@5      MRR@5    NDCG@10     MRR@10
--------------------------------------------------------------------------------------
TELEN                    0.9036     0.8830     0.9138     0.8878     0.9132     0.8878
TELEN+CE@5               0.9346     0.9199     0.9339     0.9223     0.9238     0.9223
TELEN+CE@10              0.8766     0.8510     0.8887     0.8577     0.8942     0.8609
TELEN+CE@20              0.7653     0.7212     0.7897     0.7303     0.8128     0.7349

--- MRR Gain vs TELEN standalone ---
  TELEN+CE@5: MRR@3: +4.2% | MRR@5: +3.9% | MRR@10: +3.9%
  TELEN+CE@10: MRR@3: -3.6% | MRR@5: -3.4% | MRR@10: -3.0%
  TELEN+CE@20: MRR@3: -18.3% | MRR@5: -17.7% | MRR@10: -17.2%
Done!