| Total anchors available: 50 |
|
|
| === emotion === |
| /usr/local/lib/python3.12/site-packages/transformers/generation/configuration_utils.py:590: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. |
| warnings.warn( |
| /usr/local/lib/python3.12/site-packages/transformers/generation/configuration_utils.py:595: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. |
| warnings.warn( |
| base=0.3367 oracle=0.5467 |
| --- N=25 anchors --- |
| [MLP] training on 1600 (block,task) pairs of dim 8 |
| ep 0 mse=1.87716 |
| ep 50 mse=1.12429 |
| ep 100 mse=0.89985 |
| ep 150 mse=0.87197 |
| ep 200 mse=0.85733 |
| ep 250 mse=0.84491 |
| ep 299 mse=0.83222 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.48985 |
| ep 50 mse=0.85983 |
| ep 100 mse=0.72952 |
| ep 150 mse=0.70668 |
| ep 200 mse=0.68993 |
| ep 250 mse=0.67272 |
| ep 299 mse=0.65372 |
| N25_mean cos=0.9839 acc=0.3500 |
| N25_global_ridge cos=0.9845 acc=0.4133 |
| N25_pertensor_ridge cos=0.9835 acc=0.4267 |
| N25_pertensor_pca cos=0.9840 acc=0.3900 |
| N25_procrustes cos=0.9830 acc=0.3767 |
| N25_topk5_global_ridge cos=0.9819 acc=0.3767 |
| N25_topk5_pertensor_ridge cos=0.9813 acc=0.3733 |
| N25_topk8_global_ridge cos=0.9830 acc=0.3633 |
| N25_topk8_pertensor_ridge cos=0.9824 acc=0.3667 |
| N25_topk12_global_ridge cos=0.9838 acc=0.3733 |
| N25_topk12_pertensor_ridge cos=0.9831 acc=0.3667 |
| N25_pertensor_mlp cos=0.9836 acc=0.3500 |
| N25_topk12_pertensor_mlp cos=0.9828 acc=0.3700 |
| --- N=50 anchors --- |
| topk5 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR'] |
| topk8 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR', 'sst2', 'tweet_sentiment', 'setfit_movie_reviews'] |
| topk12 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR', 'sst2', 'tweet_sentiment', 'setfit_movie_reviews', 'rotten', 'rotten_alt', 'tweet_stance_atheism', 'subj'] |
| [MLP] training on 3200 (block,task) pairs of dim 8 |
| ep 0 mse=1.94874 |
| ep 50 mse=1.18617 |
| ep 100 mse=0.95605 |
| ep 150 mse=0.93347 |
| ep 200 mse=0.92086 |
| ep 250 mse=0.91021 |
| ep 299 mse=0.89942 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.20788 |
| ep 50 mse=0.71581 |
| ep 100 mse=0.65391 |
| ep 150 mse=0.63086 |
| ep 200 mse=0.61476 |
| ep 250 mse=0.59829 |
| ep 299 mse=0.57920 |
| N50_mean cos=0.9860 acc=0.3900 |
| N50_global_ridge cos=1.0000 acc=0.5500 |
| N50_pertensor_ridge cos=0.9999 acc=0.5667 |
| N50_pertensor_pca cos=0.9957 acc=0.5300 |
| N50_procrustes cos=0.9832 acc=0.4067 |
| N50_topk5_global_ridge cos=1.0000 acc=0.5567 |
| N50_topk5_pertensor_ridge cos=1.0000 acc=0.5667 |
| N50_topk8_global_ridge cos=1.0000 acc=0.5533 |
| N50_topk8_pertensor_ridge cos=0.9999 acc=0.5600 |
| N50_topk12_global_ridge cos=1.0000 acc=0.5500 |
| N50_topk12_pertensor_ridge cos=0.9999 acc=0.5567 |
| N50_pertensor_mlp cos=0.9877 acc=0.4233 |
| N50_topk12_pertensor_mlp cos=0.9991 acc=0.5733 |
|
|
| === tweet_emotion === |
| base=0.4667 oracle=0.7267 |
| --- N=25 anchors --- |
| [MLP] training on 1600 (block,task) pairs of dim 8 |
| ep 0 mse=1.87150 |
| ep 50 mse=1.13916 |
| ep 100 mse=0.89801 |
| ep 150 mse=0.87051 |
| ep 200 mse=0.85534 |
| ep 250 mse=0.84204 |
| ep 299 mse=0.82850 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.50460 |
| ep 50 mse=0.85653 |
| ep 100 mse=0.75918 |
| ep 150 mse=0.73987 |
| ep 200 mse=0.72282 |
| ep 250 mse=0.70314 |
| ep 299 mse=0.67960 |
| N25_mean cos=0.9865 acc=0.2700 |
| N25_global_ridge cos=0.9873 acc=0.2633 |
| N25_pertensor_ridge cos=0.9861 acc=0.2700 |
| N25_pertensor_pca cos=0.9866 acc=0.2833 |
| N25_procrustes cos=0.9857 acc=0.2500 |
| N25_topk5_global_ridge cos=0.9855 acc=0.2433 |
| N25_topk5_pertensor_ridge cos=0.9851 acc=0.2400 |
| N25_topk8_global_ridge cos=0.9865 acc=0.2467 |
| N25_topk8_pertensor_ridge cos=0.9860 acc=0.2600 |
| N25_topk12_global_ridge cos=0.9866 acc=0.2600 |
| N25_topk12_pertensor_ridge cos=0.9859 acc=0.2600 |
| N25_pertensor_mlp cos=0.9864 acc=0.2700 |
| N25_topk12_pertensor_mlp cos=0.9856 acc=0.2533 |
| --- N=50 anchors --- |
| topk5 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent'] |
| topk8 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent', 'tweet_stance_atheism', 'tweet_stance_feminist', 'hate_offensive_lang'] |
| topk12 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent', 'tweet_stance_atheism', 'tweet_stance_feminist', 'hate_offensive_lang', 'tweet_stance_abortion', 'hate_speech_off', 'CR', 'tweet_irony'] |
| [MLP] training on 3200 (block,task) pairs of dim 8 |
| ep 0 mse=1.94153 |
| ep 50 mse=1.20591 |
| ep 100 mse=0.95057 |
| ep 150 mse=0.92638 |
| ep 200 mse=0.91371 |
| ep 250 mse=0.90208 |
| ep 299 mse=0.88982 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.26717 |
| ep 50 mse=0.77724 |
| ep 100 mse=0.71221 |
| ep 150 mse=0.69267 |
| ep 200 mse=0.68008 |
| ep 250 mse=0.66781 |
| ep 299 mse=0.65347 |
| N50_mean cos=0.9871 acc=0.2667 |
| N50_global_ridge cos=0.9889 acc=0.2733 |
| N50_pertensor_ridge cos=0.9874 acc=0.2967 |
| N50_pertensor_pca cos=0.9878 acc=0.2633 |
| N50_procrustes cos=0.9859 acc=0.2767 |
| N50_topk5_global_ridge cos=0.9868 acc=0.2467 |
| N50_topk5_pertensor_ridge cos=0.9865 acc=0.2500 |
| N50_topk8_global_ridge cos=0.9882 acc=0.2467 |
| N50_topk8_pertensor_ridge cos=0.9877 acc=0.2567 |
| N50_topk12_global_ridge cos=0.9885 acc=0.2600 |
| N50_topk12_pertensor_ridge cos=0.9879 acc=0.2633 |
| N50_pertensor_mlp cos=0.9871 acc=0.2633 |
| N50_topk12_pertensor_mlp cos=0.9871 acc=0.2700 |
|
|
| === bbc_news === |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| base=0.0633 oracle=0.1033 |
| --- N=25 anchors --- |
| [MLP] training on 1600 (block,task) pairs of dim 8 |
| ep 0 mse=1.87275 |
| ep 50 mse=1.14325 |
| ep 100 mse=0.90648 |
| ep 150 mse=0.87594 |
| ep 200 mse=0.85976 |
| ep 250 mse=0.84697 |
| ep 299 mse=0.83458 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.67284 |
| ep 50 mse=0.92815 |
| ep 100 mse=0.80815 |
| ep 150 mse=0.77849 |
| ep 200 mse=0.76004 |
| ep 250 mse=0.74054 |
| ep 299 mse=0.71906 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_mean cos=0.9766 acc=0.0100 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_global_ridge cos=0.9806 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_pertensor_ridge cos=0.9791 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_pertensor_pca cos=0.9791 acc=0.0033 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_procrustes cos=0.9754 acc=0.0167 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk5_global_ridge cos=0.9798 acc=0.0133 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk5_pertensor_ridge cos=0.9793 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk8_global_ridge cos=0.9799 acc=0.0100 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk8_pertensor_ridge cos=0.9792 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk12_global_ridge cos=0.9801 acc=0.0200 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk12_pertensor_ridge cos=0.9794 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_pertensor_mlp cos=0.9763 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N25_topk12_pertensor_mlp cos=0.9763 acc=0.0267 |
| --- N=50 anchors --- |
| topk5 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate'] |
| topk8 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate', 'yelp_polarity_test', 'setfit_yelp_full', 'toxic_conv'] |
| topk12 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate', 'yelp_polarity_test', 'setfit_yelp_full', 'toxic_conv', 'mteb_toxic_conv', 'mteb_tweet_sent', 'tweet_sentiment', 'subj'] |
| [MLP] training on 3200 (block,task) pairs of dim 8 |
| ep 0 mse=1.96590 |
| ep 50 mse=1.20540 |
| ep 100 mse=0.95742 |
| ep 150 mse=0.93158 |
| ep 200 mse=0.91872 |
| ep 250 mse=0.90798 |
| ep 299 mse=0.89726 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.79321 |
| ep 50 mse=1.00772 |
| ep 100 mse=0.86443 |
| ep 150 mse=0.84261 |
| ep 200 mse=0.82799 |
| ep 250 mse=0.81242 |
| ep 299 mse=0.79380 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_mean cos=0.9773 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_global_ridge cos=0.9816 acc=0.0000 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_pertensor_ridge cos=0.9795 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_pertensor_pca cos=0.9785 acc=0.0033 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_procrustes cos=0.9757 acc=0.0133 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk5_global_ridge cos=0.9798 acc=0.0133 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk5_pertensor_ridge cos=0.9793 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk8_global_ridge cos=0.9802 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk8_pertensor_ridge cos=0.9795 acc=0.0033 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk12_global_ridge cos=0.9802 acc=0.0067 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk12_pertensor_ridge cos=0.9793 acc=0.0033 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_pertensor_mlp cos=0.9771 acc=0.0033 |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. |
| N50_topk12_pertensor_mlp cos=0.9773 acc=0.0067 |
|
|
| === ethos_binary === |
| Repo card metadata block was not found. Setting CardData to empty. |
| base=0.5033 oracle=0.7033 |
| --- N=25 anchors --- |
| [MLP] training on 1600 (block,task) pairs of dim 8 |
| ep 0 mse=1.89516 |
| ep 50 mse=1.19785 |
| ep 100 mse=0.90556 |
| ep 150 mse=0.87049 |
| ep 200 mse=0.85404 |
| ep 250 mse=0.84108 |
| ep 299 mse=0.82826 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.30273 |
| ep 50 mse=0.76208 |
| ep 100 mse=0.65198 |
| ep 150 mse=0.62906 |
| ep 200 mse=0.61299 |
| ep 250 mse=0.59656 |
| ep 299 mse=0.57808 |
| N25_mean cos=0.9907 acc=0.6933 |
| N25_global_ridge cos=0.9912 acc=0.7367 |
| N25_pertensor_ridge cos=0.9902 acc=0.6867 |
| N25_pertensor_pca cos=0.9907 acc=0.7167 |
| N25_procrustes cos=0.9902 acc=0.6600 |
| N25_topk5_global_ridge cos=0.9873 acc=0.7567 |
| N25_topk5_pertensor_ridge cos=0.9870 acc=0.7600 |
| N25_topk8_global_ridge cos=0.9897 acc=0.7633 |
| N25_topk8_pertensor_ridge cos=0.9892 acc=0.7500 |
| N25_topk12_global_ridge cos=0.9902 acc=0.7800 |
| N25_topk12_pertensor_ridge cos=0.9896 acc=0.7833 |
| N25_pertensor_mlp cos=0.9906 acc=0.7600 |
| N25_topk12_pertensor_mlp cos=0.9876 acc=0.6933 |
| --- N=50 anchors --- |
| topk5 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist'] |
| topk8 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist', 'mteb_toxic_conv', 'tweet_sentiment', 'toxic_chat'] |
| topk12 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist', 'mteb_toxic_conv', 'tweet_sentiment', 'toxic_chat', 'toxic_conv', 'sst2', 'setfit_movie_reviews', 'CR'] |
| [MLP] training on 3200 (block,task) pairs of dim 8 |
| ep 0 mse=1.95357 |
| ep 50 mse=1.19555 |
| ep 100 mse=0.95406 |
| ep 150 mse=0.92955 |
| ep 200 mse=0.91713 |
| ep 250 mse=0.90648 |
| ep 299 mse=0.89542 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.48788 |
| ep 50 mse=0.86157 |
| ep 100 mse=0.77278 |
| ep 150 mse=0.74893 |
| ep 200 mse=0.73417 |
| ep 250 mse=0.71961 |
| ep 299 mse=0.70219 |
| N50_mean cos=0.9908 acc=0.6700 |
| N50_global_ridge cos=0.9915 acc=0.6967 |
| N50_pertensor_ridge cos=0.9903 acc=0.6533 |
| N50_pertensor_pca cos=0.9910 acc=0.7100 |
| N50_procrustes cos=0.9901 acc=0.6800 |
| N50_topk5_global_ridge cos=0.9876 acc=0.7267 |
| N50_topk5_pertensor_ridge cos=0.9873 acc=0.7467 |
| N50_topk8_global_ridge cos=0.9899 acc=0.7800 |
| N50_topk8_pertensor_ridge cos=0.9893 acc=0.7667 |
| N50_topk12_global_ridge cos=0.9900 acc=0.7600 |
| N50_topk12_pertensor_ridge cos=0.9893 acc=0.7533 |
| N50_pertensor_mlp cos=0.9909 acc=0.7133 |
| N50_topk12_pertensor_mlp cos=0.9889 acc=0.7267 |
|
|
| === trec === |
| Repo card metadata block was not found. Setting CardData to empty. |
| base=0.1933 oracle=0.4533 |
| --- N=25 anchors --- |
| [MLP] training on 1600 (block,task) pairs of dim 8 |
| ep 0 mse=1.90518 |
| ep 50 mse=1.20990 |
| ep 100 mse=0.90590 |
| ep 150 mse=0.86995 |
| ep 200 mse=0.85345 |
| ep 250 mse=0.84018 |
| ep 299 mse=0.82716 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.42190 |
| ep 50 mse=0.79943 |
| ep 100 mse=0.68162 |
| ep 150 mse=0.65786 |
| ep 200 mse=0.64349 |
| ep 250 mse=0.63000 |
| ep 299 mse=0.61511 |
| N25_mean cos=0.9728 acc=0.2000 |
| N25_global_ridge cos=0.9737 acc=0.2167 |
| N25_pertensor_ridge cos=0.9714 acc=0.2100 |
| N25_pertensor_pca cos=0.9727 acc=0.2133 |
| N25_procrustes cos=0.9715 acc=0.2000 |
| N25_topk5_global_ridge cos=0.9643 acc=0.1933 |
| N25_topk5_pertensor_ridge cos=0.9639 acc=0.1933 |
| N25_topk8_global_ridge cos=0.9710 acc=0.2467 |
| N25_topk8_pertensor_ridge cos=0.9702 acc=0.3600 |
| N25_topk12_global_ridge cos=0.9721 acc=0.2033 |
| N25_topk12_pertensor_ridge cos=0.9710 acc=0.2233 |
| N25_pertensor_mlp cos=0.9723 acc=0.2000 |
| N25_topk12_pertensor_mlp cos=0.9705 acc=0.2367 |
| --- N=50 anchors --- |
| topk5 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary'] |
| topk8 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary', 'subj', 'mteb_emo', 'setfit_emotion'] |
| topk12 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary', 'subj', 'mteb_emo', 'setfit_emotion', 'dair_emo_unsplit', 'mteb_tweet_sent', 'sst5', 'setfit_sst5_alt'] |
| [MLP] training on 3200 (block,task) pairs of dim 8 |
| ep 0 mse=1.93786 |
| ep 50 mse=1.17834 |
| ep 100 mse=0.95673 |
| ep 150 mse=0.93213 |
| ep 200 mse=0.91975 |
| ep 250 mse=0.90997 |
| ep 299 mse=0.90022 |
| [MLP] training on 768 (block,task) pairs of dim 6 |
| ep 0 mse=1.03834 |
| ep 50 mse=0.65581 |
| ep 100 mse=0.57337 |
| ep 150 mse=0.54587 |
| ep 200 mse=0.52850 |
| ep 250 mse=0.51384 |
| ep 299 mse=0.49936 |
| N50_mean cos=0.9738 acc=0.1933 |
| N50_global_ridge cos=0.9753 acc=0.2000 |
| N50_pertensor_ridge cos=0.9723 acc=0.2333 |
| N50_pertensor_pca cos=0.9739 acc=0.1967 |
| N50_procrustes cos=0.9720 acc=0.1900 |
| N50_topk5_global_ridge cos=0.9643 acc=0.1933 |
| N50_topk5_pertensor_ridge cos=0.9639 acc=0.1933 |
| N50_topk8_global_ridge cos=0.9715 acc=0.1967 |
| N50_topk8_pertensor_ridge cos=0.9705 acc=0.2000 |
| N50_topk12_global_ridge cos=0.9721 acc=0.2100 |
| N50_topk12_pertensor_ridge cos=0.9708 acc=0.2067 |
| N50_pertensor_mlp cos=0.9735 acc=0.1933 |
| N50_topk12_pertensor_mlp cos=0.9710 acc=0.1967 |
|
|
| === AVG === |
| oracle_Y 0.5067 |
| N50_topk12_global_ridge 0.3573 |
| N50_topk8_pertensor_ridge 0.3573 |
| N50_topk8_global_ridge 0.3567 |
| N50_topk12_pertensor_ridge 0.3567 |
| N50_topk12_pertensor_mlp 0.3547 |
| N50_topk5_pertensor_ridge 0.3527 |
| N50_pertensor_ridge 0.3513 |
| N25_topk8_pertensor_ridge 0.3487 |
| N50_topk5_global_ridge 0.3473 |
| N50_global_ridge 0.3440 |
| N50_pertensor_pca 0.3407 |
| N25_topk12_pertensor_ridge 0.3280 |
| N25_global_ridge 0.3273 |
| N25_topk12_global_ridge 0.3273 |
| N25_topk8_global_ridge 0.3260 |
| N25_pertensor_pca 0.3213 |
| N25_pertensor_ridge 0.3200 |
| N50_pertensor_mlp 0.3193 |
| N25_pertensor_mlp 0.3173 |
| N25_topk5_global_ridge 0.3167 |
| N25_topk12_pertensor_mlp 0.3160 |
| N25_topk5_pertensor_ridge 0.3147 |
| N50_procrustes 0.3133 |
| base_Y 0.3127 |
| N50_mean 0.3053 |
| N25_mean 0.3047 |
| N25_procrustes 0.3007 |
|
|
