Total anchors available: 50 === emotion === /usr/local/lib/python3.12/site-packages/transformers/generation/configuration_utils.py:590: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. warnings.warn( /usr/local/lib/python3.12/site-packages/transformers/generation/configuration_utils.py:595: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. warnings.warn( base=0.3367 oracle=0.5467 --- N=25 anchors --- [MLP] training on 1600 (block,task) pairs of dim 8 ep 0 mse=1.87716 ep 50 mse=1.12429 ep 100 mse=0.89985 ep 150 mse=0.87197 ep 200 mse=0.85733 ep 250 mse=0.84491 ep 299 mse=0.83222 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.48985 ep 50 mse=0.85983 ep 100 mse=0.72952 ep 150 mse=0.70668 ep 200 mse=0.68993 ep 250 mse=0.67272 ep 299 mse=0.65372 N25_mean cos=0.9839 acc=0.3500 N25_global_ridge cos=0.9845 acc=0.4133 N25_pertensor_ridge cos=0.9835 acc=0.4267 N25_pertensor_pca cos=0.9840 acc=0.3900 N25_procrustes cos=0.9830 acc=0.3767 N25_topk5_global_ridge cos=0.9819 acc=0.3767 N25_topk5_pertensor_ridge cos=0.9813 acc=0.3733 N25_topk8_global_ridge cos=0.9830 acc=0.3633 N25_topk8_pertensor_ridge cos=0.9824 acc=0.3667 N25_topk12_global_ridge cos=0.9838 acc=0.3733 N25_topk12_pertensor_ridge cos=0.9831 acc=0.3667 N25_pertensor_mlp cos=0.9836 acc=0.3500 N25_topk12_pertensor_mlp cos=0.9828 acc=0.3700 --- N=50 anchors --- topk5 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR'] topk8 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR', 'sst2', 'tweet_sentiment', 'setfit_movie_reviews'] topk12 selected: ['setfit_emotion', 'dair_emo_unsplit', 'mteb_emo', 'mteb_tweet_sent', 'CR', 'sst2', 'tweet_sentiment', 'setfit_movie_reviews', 'rotten', 'rotten_alt', 'tweet_stance_atheism', 'subj'] [MLP] training on 3200 (block,task) pairs of dim 8 ep 0 mse=1.94874 ep 50 mse=1.18617 ep 100 mse=0.95605 ep 150 mse=0.93347 ep 200 mse=0.92086 ep 250 mse=0.91021 ep 299 mse=0.89942 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.20788 ep 50 mse=0.71581 ep 100 mse=0.65391 ep 150 mse=0.63086 ep 200 mse=0.61476 ep 250 mse=0.59829 ep 299 mse=0.57920 N50_mean cos=0.9860 acc=0.3900 N50_global_ridge cos=1.0000 acc=0.5500 N50_pertensor_ridge cos=0.9999 acc=0.5667 N50_pertensor_pca cos=0.9957 acc=0.5300 N50_procrustes cos=0.9832 acc=0.4067 N50_topk5_global_ridge cos=1.0000 acc=0.5567 N50_topk5_pertensor_ridge cos=1.0000 acc=0.5667 N50_topk8_global_ridge cos=1.0000 acc=0.5533 N50_topk8_pertensor_ridge cos=0.9999 acc=0.5600 N50_topk12_global_ridge cos=1.0000 acc=0.5500 N50_topk12_pertensor_ridge cos=0.9999 acc=0.5567 N50_pertensor_mlp cos=0.9877 acc=0.4233 N50_topk12_pertensor_mlp cos=0.9991 acc=0.5733 === tweet_emotion === base=0.4667 oracle=0.7267 --- N=25 anchors --- [MLP] training on 1600 (block,task) pairs of dim 8 ep 0 mse=1.87150 ep 50 mse=1.13916 ep 100 mse=0.89801 ep 150 mse=0.87051 ep 200 mse=0.85534 ep 250 mse=0.84204 ep 299 mse=0.82850 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.50460 ep 50 mse=0.85653 ep 100 mse=0.75918 ep 150 mse=0.73987 ep 200 mse=0.72282 ep 250 mse=0.70314 ep 299 mse=0.67960 N25_mean cos=0.9865 acc=0.2700 N25_global_ridge cos=0.9873 acc=0.2633 N25_pertensor_ridge cos=0.9861 acc=0.2700 N25_pertensor_pca cos=0.9866 acc=0.2833 N25_procrustes cos=0.9857 acc=0.2500 N25_topk5_global_ridge cos=0.9855 acc=0.2433 N25_topk5_pertensor_ridge cos=0.9851 acc=0.2400 N25_topk8_global_ridge cos=0.9865 acc=0.2467 N25_topk8_pertensor_ridge cos=0.9860 acc=0.2600 N25_topk12_global_ridge cos=0.9866 acc=0.2600 N25_topk12_pertensor_ridge cos=0.9859 acc=0.2600 N25_pertensor_mlp cos=0.9864 acc=0.2700 N25_topk12_pertensor_mlp cos=0.9856 acc=0.2533 --- N=50 anchors --- topk5 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent'] topk8 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent', 'tweet_stance_atheism', 'tweet_stance_feminist', 'hate_offensive_lang'] topk12 selected: ['tweet_sentiment', 'mteb_emo', 'dair_emo_unsplit', 'setfit_emotion', 'mteb_tweet_sent', 'tweet_stance_atheism', 'tweet_stance_feminist', 'hate_offensive_lang', 'tweet_stance_abortion', 'hate_speech_off', 'CR', 'tweet_irony'] [MLP] training on 3200 (block,task) pairs of dim 8 ep 0 mse=1.94153 ep 50 mse=1.20591 ep 100 mse=0.95057 ep 150 mse=0.92638 ep 200 mse=0.91371 ep 250 mse=0.90208 ep 299 mse=0.88982 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.26717 ep 50 mse=0.77724 ep 100 mse=0.71221 ep 150 mse=0.69267 ep 200 mse=0.68008 ep 250 mse=0.66781 ep 299 mse=0.65347 N50_mean cos=0.9871 acc=0.2667 N50_global_ridge cos=0.9889 acc=0.2733 N50_pertensor_ridge cos=0.9874 acc=0.2967 N50_pertensor_pca cos=0.9878 acc=0.2633 N50_procrustes cos=0.9859 acc=0.2767 N50_topk5_global_ridge cos=0.9868 acc=0.2467 N50_topk5_pertensor_ridge cos=0.9865 acc=0.2500 N50_topk8_global_ridge cos=0.9882 acc=0.2467 N50_topk8_pertensor_ridge cos=0.9877 acc=0.2567 N50_topk12_global_ridge cos=0.9885 acc=0.2600 N50_topk12_pertensor_ridge cos=0.9879 acc=0.2633 N50_pertensor_mlp cos=0.9871 acc=0.2633 N50_topk12_pertensor_mlp cos=0.9871 acc=0.2700 === bbc_news === A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. base=0.0633 oracle=0.1033 --- N=25 anchors --- [MLP] training on 1600 (block,task) pairs of dim 8 ep 0 mse=1.87275 ep 50 mse=1.14325 ep 100 mse=0.90648 ep 150 mse=0.87594 ep 200 mse=0.85976 ep 250 mse=0.84697 ep 299 mse=0.83458 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.67284 ep 50 mse=0.92815 ep 100 mse=0.80815 ep 150 mse=0.77849 ep 200 mse=0.76004 ep 250 mse=0.74054 ep 299 mse=0.71906 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_mean cos=0.9766 acc=0.0100 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_global_ridge cos=0.9806 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_pertensor_ridge cos=0.9791 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_pertensor_pca cos=0.9791 acc=0.0033 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_procrustes cos=0.9754 acc=0.0167 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk5_global_ridge cos=0.9798 acc=0.0133 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk5_pertensor_ridge cos=0.9793 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk8_global_ridge cos=0.9799 acc=0.0100 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk8_pertensor_ridge cos=0.9792 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk12_global_ridge cos=0.9801 acc=0.0200 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk12_pertensor_ridge cos=0.9794 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_pertensor_mlp cos=0.9763 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N25_topk12_pertensor_mlp cos=0.9763 acc=0.0267 --- N=50 anchors --- topk5 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate'] topk8 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate', 'yelp_polarity_test', 'setfit_yelp_full', 'toxic_conv'] topk12 selected: ['ag_news', 'imdb', 'enron_spam', 'tweet_stance_atheism', 'tweet_stance_climate', 'yelp_polarity_test', 'setfit_yelp_full', 'toxic_conv', 'mteb_toxic_conv', 'mteb_tweet_sent', 'tweet_sentiment', 'subj'] [MLP] training on 3200 (block,task) pairs of dim 8 ep 0 mse=1.96590 ep 50 mse=1.20540 ep 100 mse=0.95742 ep 150 mse=0.93158 ep 200 mse=0.91872 ep 250 mse=0.90798 ep 299 mse=0.89726 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.79321 ep 50 mse=1.00772 ep 100 mse=0.86443 ep 150 mse=0.84261 ep 200 mse=0.82799 ep 250 mse=0.81242 ep 299 mse=0.79380 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_mean cos=0.9773 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_global_ridge cos=0.9816 acc=0.0000 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_pertensor_ridge cos=0.9795 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_pertensor_pca cos=0.9785 acc=0.0033 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_procrustes cos=0.9757 acc=0.0133 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk5_global_ridge cos=0.9798 acc=0.0133 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk5_pertensor_ridge cos=0.9793 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk8_global_ridge cos=0.9802 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk8_pertensor_ridge cos=0.9795 acc=0.0033 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk12_global_ridge cos=0.9802 acc=0.0067 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk12_pertensor_ridge cos=0.9793 acc=0.0033 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_pertensor_mlp cos=0.9771 acc=0.0033 A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. N50_topk12_pertensor_mlp cos=0.9773 acc=0.0067 === ethos_binary === Repo card metadata block was not found. Setting CardData to empty. base=0.5033 oracle=0.7033 --- N=25 anchors --- [MLP] training on 1600 (block,task) pairs of dim 8 ep 0 mse=1.89516 ep 50 mse=1.19785 ep 100 mse=0.90556 ep 150 mse=0.87049 ep 200 mse=0.85404 ep 250 mse=0.84108 ep 299 mse=0.82826 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.30273 ep 50 mse=0.76208 ep 100 mse=0.65198 ep 150 mse=0.62906 ep 200 mse=0.61299 ep 250 mse=0.59656 ep 299 mse=0.57808 N25_mean cos=0.9907 acc=0.6933 N25_global_ridge cos=0.9912 acc=0.7367 N25_pertensor_ridge cos=0.9902 acc=0.6867 N25_pertensor_pca cos=0.9907 acc=0.7167 N25_procrustes cos=0.9902 acc=0.6600 N25_topk5_global_ridge cos=0.9873 acc=0.7567 N25_topk5_pertensor_ridge cos=0.9870 acc=0.7600 N25_topk8_global_ridge cos=0.9897 acc=0.7633 N25_topk8_pertensor_ridge cos=0.9892 acc=0.7500 N25_topk12_global_ridge cos=0.9902 acc=0.7800 N25_topk12_pertensor_ridge cos=0.9896 acc=0.7833 N25_pertensor_mlp cos=0.9906 acc=0.7600 N25_topk12_pertensor_mlp cos=0.9876 acc=0.6933 --- N=50 anchors --- topk5 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist'] topk8 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist', 'mteb_toxic_conv', 'tweet_sentiment', 'toxic_chat'] topk12 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'mteb_tweet_sent', 'tweet_stance_abortion', 'tweet_stance_feminist', 'mteb_toxic_conv', 'tweet_sentiment', 'toxic_chat', 'toxic_conv', 'sst2', 'setfit_movie_reviews', 'CR'] [MLP] training on 3200 (block,task) pairs of dim 8 ep 0 mse=1.95357 ep 50 mse=1.19555 ep 100 mse=0.95406 ep 150 mse=0.92955 ep 200 mse=0.91713 ep 250 mse=0.90648 ep 299 mse=0.89542 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.48788 ep 50 mse=0.86157 ep 100 mse=0.77278 ep 150 mse=0.74893 ep 200 mse=0.73417 ep 250 mse=0.71961 ep 299 mse=0.70219 N50_mean cos=0.9908 acc=0.6700 N50_global_ridge cos=0.9915 acc=0.6967 N50_pertensor_ridge cos=0.9903 acc=0.6533 N50_pertensor_pca cos=0.9910 acc=0.7100 N50_procrustes cos=0.9901 acc=0.6800 N50_topk5_global_ridge cos=0.9876 acc=0.7267 N50_topk5_pertensor_ridge cos=0.9873 acc=0.7467 N50_topk8_global_ridge cos=0.9899 acc=0.7800 N50_topk8_pertensor_ridge cos=0.9893 acc=0.7667 N50_topk12_global_ridge cos=0.9900 acc=0.7600 N50_topk12_pertensor_ridge cos=0.9893 acc=0.7533 N50_pertensor_mlp cos=0.9909 acc=0.7133 N50_topk12_pertensor_mlp cos=0.9889 acc=0.7267 === trec === Repo card metadata block was not found. Setting CardData to empty. base=0.1933 oracle=0.4533 --- N=25 anchors --- [MLP] training on 1600 (block,task) pairs of dim 8 ep 0 mse=1.90518 ep 50 mse=1.20990 ep 100 mse=0.90590 ep 150 mse=0.86995 ep 200 mse=0.85345 ep 250 mse=0.84018 ep 299 mse=0.82716 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.42190 ep 50 mse=0.79943 ep 100 mse=0.68162 ep 150 mse=0.65786 ep 200 mse=0.64349 ep 250 mse=0.63000 ep 299 mse=0.61511 N25_mean cos=0.9728 acc=0.2000 N25_global_ridge cos=0.9737 acc=0.2167 N25_pertensor_ridge cos=0.9714 acc=0.2100 N25_pertensor_pca cos=0.9727 acc=0.2133 N25_procrustes cos=0.9715 acc=0.2000 N25_topk5_global_ridge cos=0.9643 acc=0.1933 N25_topk5_pertensor_ridge cos=0.9639 acc=0.1933 N25_topk8_global_ridge cos=0.9710 acc=0.2467 N25_topk8_pertensor_ridge cos=0.9702 acc=0.3600 N25_topk12_global_ridge cos=0.9721 acc=0.2033 N25_topk12_pertensor_ridge cos=0.9710 acc=0.2233 N25_pertensor_mlp cos=0.9723 acc=0.2000 N25_topk12_pertensor_mlp cos=0.9705 acc=0.2367 --- N=50 anchors --- topk5 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary'] topk8 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary', 'subj', 'mteb_emo', 'setfit_emotion'] topk12 selected: ['tweet_stance_atheism', 'tweet_stance_climate', 'tweet_stance_abortion', 'tweet_stance_feminist', 'tweet_stance_hillary', 'subj', 'mteb_emo', 'setfit_emotion', 'dair_emo_unsplit', 'mteb_tweet_sent', 'sst5', 'setfit_sst5_alt'] [MLP] training on 3200 (block,task) pairs of dim 8 ep 0 mse=1.93786 ep 50 mse=1.17834 ep 100 mse=0.95673 ep 150 mse=0.93213 ep 200 mse=0.91975 ep 250 mse=0.90997 ep 299 mse=0.90022 [MLP] training on 768 (block,task) pairs of dim 6 ep 0 mse=1.03834 ep 50 mse=0.65581 ep 100 mse=0.57337 ep 150 mse=0.54587 ep 200 mse=0.52850 ep 250 mse=0.51384 ep 299 mse=0.49936 N50_mean cos=0.9738 acc=0.1933 N50_global_ridge cos=0.9753 acc=0.2000 N50_pertensor_ridge cos=0.9723 acc=0.2333 N50_pertensor_pca cos=0.9739 acc=0.1967 N50_procrustes cos=0.9720 acc=0.1900 N50_topk5_global_ridge cos=0.9643 acc=0.1933 N50_topk5_pertensor_ridge cos=0.9639 acc=0.1933 N50_topk8_global_ridge cos=0.9715 acc=0.1967 N50_topk8_pertensor_ridge cos=0.9705 acc=0.2000 N50_topk12_global_ridge cos=0.9721 acc=0.2100 N50_topk12_pertensor_ridge cos=0.9708 acc=0.2067 N50_pertensor_mlp cos=0.9735 acc=0.1933 N50_topk12_pertensor_mlp cos=0.9710 acc=0.1967 === AVG === oracle_Y 0.5067 N50_topk12_global_ridge 0.3573 N50_topk8_pertensor_ridge 0.3573 N50_topk8_global_ridge 0.3567 N50_topk12_pertensor_ridge 0.3567 N50_topk12_pertensor_mlp 0.3547 N50_topk5_pertensor_ridge 0.3527 N50_pertensor_ridge 0.3513 N25_topk8_pertensor_ridge 0.3487 N50_topk5_global_ridge 0.3473 N50_global_ridge 0.3440 N50_pertensor_pca 0.3407 N25_topk12_pertensor_ridge 0.3280 N25_global_ridge 0.3273 N25_topk12_global_ridge 0.3273 N25_topk8_global_ridge 0.3260 N25_pertensor_pca 0.3213 N25_pertensor_ridge 0.3200 N50_pertensor_mlp 0.3193 N25_pertensor_mlp 0.3173 N25_topk5_global_ridge 0.3167 N25_topk12_pertensor_mlp 0.3160 N25_topk5_pertensor_ridge 0.3147 N50_procrustes 0.3133 base_Y 0.3127 N50_mean 0.3053 N25_mean 0.3047 N25_procrustes 0.3007